Trending your GitHub Commits in Graphite

2012-07-22 22:45:55 by jdixon

Today I was browsing the list of service hooks that GitHub provides. I almost forgot that there's a simple WebHook service that POSTs commit information during the git post-receive hook to any external URL. This got me thinking that it would be nice to trend commit activity inside my Graphite server. Don't get me wrong... GitHub already provides some really nice visualization for project and committer activity on their site. However, as a data junkie, I'd love to be able to correlate this activity with my own application metrics.

This was a perfect fit for Backstop, the HTTP/JSON-to-Graphite bridge. After a couple hours of futzing around I had a working version. If you haven't used Backstop before, rest assured that getting started is pretty darn easy. In fact, if you're a Heroku customer, it's easy and free. There are just a few commands to get your own Backstop server running on Heroku.

Read the rest of this story...

Introducing Descartes

2012-07-10 17:51:45 by jdixon

Graphite is renowned for its usefulness and ease for prototyping new charts. It's also known for having a dashboard component that leaves much to be desired. In response the community has seen a rising tide of new dashboard projects aimed at filling this gap. The growing list of third-party Graphite dashboard projects is extensive, but continues to fall short in areas such as self-service, configuration, and collaboration.

Most of this software require users to generate dashboards from JSON or other command-line gymnastics. While this is reasonable for many operations folk, it's an impedance for the engineers and business-oriented users; the same users that we want using this software for making sound decisions. Graph views are static and inflexible for collaboration and historical dialogues. In response to these shortcomings I've started the Descartes project.

Read the rest of this story...

Collection of D3 Tutorials

2012-07-09 13:24:05 by jdixon

A friend of mine recently asked for some good D3 tutorials and sites. At second glance these are an awesome collection of examples for using D3 and general visualization work.

Pro: You don't have to scour the web for these yourself.

Con: It's unlikely you'll ever fully consume all the awesome.

The State of Employment

2012-07-08 17:39:48 by jdixon

Seems that it's common for folks to blog about changes in employment. I hate to be left out on the fun, so I'll take a brief moment to officially announce my pending "new-hire" status with GitHub, effective tomorrow.

Friends who've already heard the news pepper their congratulations with a sense of confusion as to why I'd leave a good thing at Heroku. Indeed, I think most people in our industry would rank Heroku and GitHub at the top of their list of prospective employers. Unsurprisingly, I loved my job. I've never worked with a team of engineers as highly skilled or dedicated to their mission as the men and women at Heroku. So why would I leave?

Read the rest of this story...

El Cheapo Network Graph

2012-07-08 12:45:58 by jdixon

Here's an embarrassingly simple script I threw together this morning to track network latency to a handful of remote websites/networks from my home internet. Yes, I understand that these numbers are highly influenced by my proximity to various CDN networks and bear no resemblance to how actual web browsing would perform concurrently. That isn't the point. This is merely to demonstrate a cheap and easy way to get more metrics into Graphite; and at the same time, providing me with some useful reference for when my home internet provider will inevitably have hiccups.

Read the rest of this story...

Graph Porn and Sharing

2012-07-01 13:43:14 by jdixon

Part of what I see myself doing (by writing blog posts, creating software like Tasseo, etc) is to try and help others learn better ways of communicating our operational knowledge through visualization tools and methodologies. While I've gotten a lot of positive feedback from my Graphite articles, what I haven't seen as much is a two-way sharing of the harvested data made possible through these experiences.

I think there are a couple possible reasons for this: first, we work with "propietary" data that our employers might not want divulged; second, we assume our data is immaterial and not worth sharing. For the former, I think this is a very similar argument that many of us had with employers during the push to open source software. There is much to be gained by sharing our raw data (perhaps without all of the proprietary metadata and labels that make it relevant to our business) and seeing those examples improved upon and returned by our peers.

Read the rest of this story...