mirror of https://github.com/valkey-io/valkey
c90e634f11
This adds the workflow improvements for PR and Release benchmark where it runs on `c8g.metal-48xl` for `ARM64` and `c7i.metal-48xl` for `X86`. ``` Cluster mode: disabled TLS: disabled io-threads: 1, 9 Pipelining: 1, 10 Clients: 1600 Benchmark Treads: 90 Data size: 16 ,96 Commands: SET, GET ``` c8g.metal-48xl Spec: https://aws.amazon.com/ec2/instance-types/c8g/ c7i.metal.48xl Spec: https://aws.amazon.com/ec2/instance-types/c7i/ ``` vCPU: 192 NUMA nodes: 2 Memory (GiB): 384 Network Bandwidth (Gbps): 50 ``` PR benchmarking will be executed on **ARM64** machine as it has been seen to be more consistent. Additionally, it runs 5 iterations for each tests and posts the average and other statistical metrics like - CI99%: 99% Confidence Interval - range where the true population mean is likely to fall - PI99%: 99% Prediction Interval - range where a single future observation is likely to fall - CV: Coefficient of Variation - relative variability (σ/μ × 100%) _Note: Values with (n=X, σ=Y, CV=Z%, CI99%=±W%, PI99%=±V%) indicate averages from X runs with standard deviation Y, coefficient of variation Z%, 99% confidence interval margin of error ±W% of the mean, and 99% prediction interval margin of error ±V% of the mean. CI bounds [A, B] and PI bounds [C, D] show the actual interval ranges._ For comparing between versions, it adds a workflow which runs on both **ARM64** and **X86** machine. It will also post the comparison between the versions like this: https://github.com/valkey-io/valkey/issues/2580#issuecomment-3399539615 --------- Signed-off-by: Roshan Khatri <rvkhatri@amazon.com> Signed-off-by: Roshan Khatri <117414976+roshkhatri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| benchmark-config-arm.json | ||
| benchmark-config-x86.json | ||