New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[cluster-test] Add experiment to automatically generate CPU FlameGraph #2387
[cluster-test] Add experiment to automatically generate CPU FlameGraph #2387
Conversation
fn apply(&self) -> BoxFuture<Result<()>> { | ||
async move { | ||
info!("GenerateCpuFlamegraph {}", self.instance); | ||
let cmd = format!(r#" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need some sort of set -e
to make it fail if any command fails?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that'll be a good idea
impl ExperimentParam for CpuFlamegraphParams { | ||
type E = CpuFlamegraph; | ||
fn build(self, cluster: &Cluster) -> Self::E { | ||
let (perf_instance, _) = cluster.split_n_validators_random(1); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we just make random_validator
function in Cluster? Probably going to be used a lot.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It already exists :-/
) -> BoxFuture<'a, anyhow::Result<Option<String>>> { | ||
async move { | ||
let buffer = Duration::from_secs(60); | ||
let mut tx_emitter = context.tx_emitter.clone(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can be rewritten in one task, so that you don't need to clone tx_emitter. Something like that:
let emit_future = tx_emitter.emit_txn_for(tx_emitter_duration, validator_instances);
let flame_graph_future = GenerateCpuFlamegraph::new(self.perf_instance.clone(), self.duration_secs).apply();
let flame_graph_future = tokio::time::delay_for(buffer).then(flame_graph_future);
join_all(emit_future, flame_graph_future).await?
Might need some .map
to make sure both futures return same type, which is required by join_all
About usage - do you plan to include flamegraph link to perf report in |
Yes, once the s3 setup is complete |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Summary We generate load on the cluster and pick one validator and run perf against it to generate the FlameGraph This FlameGraph is then copied locally to cluster-test. The next step would be to upload this local file to S3 Test Plan Tested on my cluster
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Thank you for your submission. We require all contributors to Have you signed the CLA already, but your status is still pending? Recheck CLA |
Summary
perf
against it to generate the FlameGraphTest Plan
Tested on my cluster