Benchmarking Graphite master on AWS

2016-09-12 12:15:25 by jdixon

Hello, friends. Just wanted to follow up the previous post with a quick update. As I've mentioned publicly, developing and cutting a new release from Graphite's master branch has become a personal and professional priority for me. And while I've become very familiar with much of the code base over the course of writing The Graphite Book and have thrown a lot of traffic at it over the last few months, I hadn't run any significant tests for performance regressions at scale (compared to the 0.9.15 release).

This round of tests used the same configuration and benchmarking processes as before. I neglected to mention this before, but all series of benchmarks started with installing Graphite using the Synthesize setup script. For the previous test I used Synthesize v.2.4.1 to install Graphite 0.9.15 on a 64-bit Ubuntu 14.04 LTS instance in Amazon's EC2 cloud. For this round I went with Synthesize v3.0.0RC2, which targets Graphite's master branch.

Editor's Note: This benchmark and blog post were completed a couple weeks ago but I'm only just now getting around to publishing it publicly. The results in this post, as well as the one preceding it, were never intended to represent the theoretical maximum throughput of a single Graphite node. They were intended to serve as a shared exercise (and catalyst for discussion) in benchmarking for the Graphite community.

The results of this round are fairly boring. Aside from minor improvements in CPU and Load consumption, there are no obvious regressions or significant improvements. Write operations, write times, and points per update are consistent with the previous benchmark. Given that the focus of the master branch has been in adding new features (with no intentional focus on performance improvements), I'm very pleased with the results. The only cause for possible concern was a slight increase in data retrieval times. I'm going to look into this further to see if there are any obvious culprits and if it's justification for pushing back the next release. Note that I don't have any useful rendering statistics for these rounds since I've been using Grafana (which performs client-side rendering).

To recap, the test host is a single EBS-optimized EC2 i2.4xlarge instance with a 400 GiB EBS Provisioned IOPS SSD (io1) with a requested 20k Max IOPS. All configuration details and benchmarking processes can be found in the previous blog post.

Fig 1: Grafana dashboard for Carbon & Whisper
Fig 2: Grafana dashboard for collectd
Fig 3: top output for Carbon

Add a comment:




max length 4000 chars