obfuscurity.
hidden in plain sight

Update / TL;DR: Thanks to Bernd Ahlers (@berndahlers) for clueing me into the fact that you can call rufus-scheduler directly rather than indirectly through resque-scheduler. Because it uses Event-Machine, there's no need to run separate worker processes or queue up the jobs. Consider me sold. The changes have already been committed.

If you still want to read the original post, continue on.

Today I merged in a refactor of the Descartes bits that deal with metrics. Specifically, the live Metrics tab and sparklines view. This will have a profound effect on performance, but can also have a surprising effect on your wallet if you're not paying attention.

So, a little background on how Descartes used to operate and why this change was necessary. Not too long ago I added a new Metrics page that displays sparklines for every metric in your Graphite server and lets you click on them to create a composite graph. Although the page is still rather immature, it's useful for basic visualization and graph creation. Personally I think its major selling point right now is in the sparklines I mentioned. This is one thing that you don't really get with native Graphite -- being able to quickly see activity patterns on any metrics without going through the hassle of actually creating a graph. This is made that much more awesome by the presence of live filtering. Click on the Add to Graph button and you're presented with an additional input field that, as you type a string, will filter down the list of metric sparklines you're viewing in realtime.

Read the rest of this story...

In the process of hacking on a plugin for the Uptime project I realized that I needed a simple HTTP server capable of receiving and dumping JSON data via POST. I'm well aware of the awesome Python SimpleHTTPServer module, but alas it doesn't support POST requests.

Fortunately I was able to throw together a quick little server using the sample HTTP server on the Node.js project website along with their API docs. I know this is a ridiculously simple daemon, but someone else might find it as useful as I did.

Read the rest of this story...

Just this morning I discovered the Uptime project over on GitHub. The author bills it as "A simple HTTP remote monitoring utility using Node.js and MongoDB". I'm already in love with this tool thanks to its composability and ease of use.

The documentation over at the Uptime project is quite good, so I won't bore you with the details. The basic gist is that you'll want to have a MongoDB server available (OS X users can just brew install mongodb) and Node.js (at least version 0.8). Clone the repo locally and then run node app.js to start the monitor (web UI) and analyzer (check engine).

Read the rest of this story...

Registration for Monitorama opens up one week from today. We're in the highly unusual situation of being a first-year conference that will very likely sell out in almost as short a time as it took to plan it. However, while I'm thrilled that so many incredible people want to attend, I want to take a moment to make sure everyone fully understands what this event is truly about, and what I personally expect to come out of it.

Monitorama will be an inclusive conference. There will be no discrimination according to race, gender, sexual preference, programming language, operating system or editor. You will not be judged on your experience, your abilities as a programmer or the number of followers on GitHub. The only tacit requirement will be a passion for our shared open-source monitoring toolset and the tenacity to dig in, have fun and help advance the state of our craft over the course of this two-day event.

Many of you will write code. Some of you will work on documentation. Others will speak or present workshops to inspire the other participants and help bring focus to our mission.

Everyone who registers should do so with the understanding that they are expected to participate. Attendees are for other, lesser, conferences. Monitorama is all about getting shit done and having fun doing it. Do not let this scare you. We will all walk away from Boston knowing that there is great work yet to be done, but with the collective wisdom and progress gained from an intense program of collaboration and learning.

There are only 200 seats available for Monitorama 2013. I hope to see your name on the ledger, one week from today.

A few stray bits of information following my presentation at DevOpsDays Roma.

The talk has received a ton of positive feedback from everyone. The slides in particular have been getting a ton of redistribution on Twitter. I'm not sure if this is a sign that my deck is that much better than the actual talk, but whatever. I'm glad that people are finding it useful and/or informative.

Read the rest of this story...

I've just concluded a week in Italy as part of my visit to speak at DevOpsDays Roma. Most people don't know this, but I was an Architecture student at Georgia Tech many years ago. As such, I was exposed to a lot of Greek and Roman history. This made a lasting impression on me; I've always dreamed of visiting Rome and it was a stroke of luck when I heard about the conference and was eventually accepted to speak.

Read the rest of this story...

The curtain has lowered on another couple days of scalability lessons and "disaster porn" at this year's Surge conference. Despite my initial misgivings that the registration fees were too high, the conference organizers have once again put together an experience that is quite possibly the best among all technically-oriented events.

Read the rest of this story...

Kicking off this year's Surge conference was a pair of BoF sessions. The #monitoringsucks one was packed, to the extent that a number of us had to steal chairs from the Chef BoF across the hall. I remembered to write down some of the highlights from the session. Note that I'm not quoting anyone directly and am summarizing each speaker to the best of my recollection. If you were at the event and remember things differently, please notify me in the comments section below.

Read the rest of this story...

A couple weeks ago I uploaded a new screencast and tweeted about it, but I completely forgot to mention it here. This is a fairly thorough demonstration for installing Graphite from git checkouts on an Ubuntu 10.04 server. Please let me know if you have any questions about the content or ideas for future screencasts.

I would recommend watching it in fullscreen in at least 720p resolution. All commands and configurations referenced in the video can be found here.

We've noticed an increase in alerts recently at $DAYJOB. So naturally we thought it would be helpful to begin tracking Nagios alerts in Graphite. Alas, this will only help us going forward, so I wondered how difficult it would be to retrieve historic data from PagerDuty and import it into Graphite. Turns out it isn't too hard, although we have to work around some of the limitations in PagerDuty's Incidents API.

Read the rest of this story...