Trending your GitHub Commits in Graphite

2012-07-22 22:45:55 by jdixon

Today I was browsing the list of service hooks that GitHub provides. I almost forgot that there's a simple WebHook service that POSTs commit information during the git post-receive hook to any external URL. This got me thinking that it would be nice to trend commit activity inside my Graphite server. Don't get me wrong... GitHub already provides some really nice visualization for project and committer activity on their site. However, as a data junkie, I'd love to be able to correlate this activity with my own application metrics.

This was a perfect fit for Backstop, the HTTP/JSON-to-Graphite bridge. After a couple hours of futzing around I had a working version. If you haven't used Backstop before, rest assured that getting started is pretty darn easy. In fact, if you're a Heroku customer, it's easy and free. There are just a few commands to get your own Backstop server running on Heroku.

$ git clone https://github.com/obfuscurity/backstop.git
$ heroku create
$ git push heroku master
$ heroku config:add CARBON_URLS=graphite.example.com:2003

Next you'll want to visit the Admin page for your GitHub project and click the Service Hooks tab to view all of the available hooks. Go to the WebHook URLs service and add your URL (e.g. http://frilly-lace-2030.herokuapp.com/github. Click the Test Hook button if you want to fire off a test metric, but there are two things to take note of:

  1. The first time you try to connect to Backstop after launching it, the backend socket connection to Carbon might fail and log an exception, and
  2. Test metrics are generated using older commit data; as such, you'll see the metric written to your Graphite server, but it will reflect the actual timestamp of the original commit.

That's basically all there is to it. Backstop will parse the JSON from GitHub and submit it as an annotation-style metric with a value of 1. These can be visualized as vertical lines with Graphite's drawAsInfinite() function as demonstrated in my Unhelpful Graphite Tip #1. If you want to trend all of the commits in a branch at once, just use a wildcard to aggregate the author field.

Add a comment:

  name

  email

  url

max length 4000 chars