Unhelpful Graphite Tip #10 - Time Shifting

2012-04-25 08:44:44 by jdixon

Let's say you want to compare how a particular metric compares to some point in the past. This is a common practice in troubleshooting and capacity planning. What's the best way to achieve this in Graphite?

I might start off by selecting the past four weeks and visually discern the trends from week to week. Here's a graph showing the last month of AMQP activity. We can see that traffic was oscillating quite a bit over the first week and a half before smoothing out and gradually trending downward.

That's useful, but what if we want to directly correlate the activity between now and the same time two weeks ago? This is where the timeShift() function comes in. Let's take a look at the same 4-week period, but this time we'll review two weeks of current data and overlay it with a time-shifted span of the two weeks prior.

Note: If you weren't already aware, you can add the same metric to a graph multiple times by manually adding it via the Graph Data panel or by applying a function to the original metric before you add it again through the Tree or Auto-Completer.

Now we're talking. I don't know about you, but my brain has a much easier time processing this view. We can easily correlate the changes over these two 2-week cycles. And it's almost as if the composer window is wrapping the data for us from one interval to the next.

But what if we want to view each week as a discrete interval over the last month? Let's crank it up a notch.

There are definitely some interesting patterns in there to dive into later. In particular, I'm curious how much we're actually changing from week to week.

Let's go back to our previous example. Except this time we'll add a composite representing the percentage of our current data against its respective 2-week-prior value.

(Click on image for a full-size version)

Go forth and timeShift, my young Padawans.

Comments

at 2012-11-13 11:56:52, Olivier wrote in to say...

For my information, how do you send RabbitMQ monitoring statistics to Graphite ?

Thanks fro sharing Graphite tips.

Olivier.

at 2012-11-13 13:53:19, Jason Dixon wrote in to say...

Oliver - I don't personally use RabbitMQ, but a quick Google revealed a lot of results.

at 2012-12-11 13:36:29, Iancu wrote in to say...

Hi again Jason. Well after some effort I managed to get graphite and statsd running and sent in some data with a script I made in bash. The data I am sending is collected from snmp and my question is - how does one, except by using the url api, modify raw data in a fairly complex but free manner? I want to do some divisions / multiplications on the values of data and saw no functions that would allow it. Am I missing something? Thanks again, Iancu

at 2012-12-11 13:41:56, Jason Dixon wrote in to say...

@Iancu - use the scale() function to perform multiplication or division.

at 2012-12-11 16:41:54, Iancu wrote in to say...

Thanks again. With your advice and the documentation I found for the previous version of graphite about functions - it's missing from the latest version, don't know why, I managed to do what I want. Now I have another problem (hopefully I am not too annoying) - on my graphs the time is a bit off and I can't understand why or how - the system where I am polling from and where I am sending the data to (one and the same) and the system that's being polled both show one time and graphite is showing me something completely different, a difference of about 9 hours... Any idea what could generate this? (It was a common problem for all the graphs I tested, with or without statsd). Best regards, Iancu

at 2012-12-11 17:52:35, Jason Dixon wrote in to say...

@Iancu - I would strongly suggest joining the #graphite channel in freenode IRC. This blog is not very well suited for back-and-forth conversation or assistance. Thanks!

at 2012-12-11 17:56:11, Iancu wrote in to say...

Hi Jason - will do, thanks for the patience and the advice.

at 2013-01-31 14:21:20, Prachi Deodhar wrote in to say...

Can we compare a graph versus itself per day?

at 2013-01-31 14:27:24, Jason Dixon wrote in to say...

@Prachi - Sort of. You can always use the methods I explained above to add the same metric to the graph with timeShift. But there's nothing that just "overlays" the graph on itself with a timeShift.

at 2014-03-03 13:38:21, Raj wrote in to say...

Jason, Thanks for the article.

In my project I have a graph for whole day but I would like display it in 3 shifts (of 8hrs.) on same graph. I see that we can do this manually with timeshift(), but is there anyway to automatically render graph with 3 shifts. (Sorry if this is silly question...I am new to Graphite)

Add a comment:

  name

  email

  url

max length 4000 chars