obfuscurity.
hidden in plain sight

Within the last hour I stumbled across a tweet from Dan Ryan mentioning a new hosted Graphite + StatsD service called Playfair. As you might expect, this piqued my interest.

Immediately, I thought of Hosted Graphite and wondered how this compares with their offering. Would it have its own dashboard? Was it a DigitalOcean-backed Graphite instance (admittedly, something I've considered trying to package up myself)? I hopped over to their website and looked around.

My initial impressions were positive. It's a very nice, professional looking website. The product is a managed Graphite and StatsD instance in the "Cloud". For a very reasonable $9 per month, you get the aforementioned services on 20GB of fast SSD on a "Dedicated virtual instance" with "Unlimited data retention". I already have a lot of questions bouncing around in my head, so I go digging for more information.

Sadly, there is no documentation on the public site. I assume the important bits are exposed once you login, so I decide to sign up for a quick peek. They offer a 30-day free trial but you have to give them your credit card. Seems like a no-brainer, and it basically is.

After signing up and confirming my email, I launch a new instance. A brief 90 seconds later, the instances page refreshes to display my new Graphite URL. Yes, I've exposed my host information in this screenshot; I don't much care who uses this test box, and as you'll find out soon, it wouldn't really matter if I did anyways.

As I wait for the page to load, I'm anxiously curious to see what the UI will look like. If you've used the newest Graphite (0.9.12), you might have noticed the new banner at the top. Admittedly, it's not a terribly sexy change, but getting rid of the old Graphite logo was something that was on my mind for a long time. Sadly, I was greeted by our old friend, which tells me this is very likely running the older 0.9.10 (possibly along with a security issue that was fixed in 0.9.11) .

Despite that, it was nice to see a number of metrics already populated in the metrics navigation tree. And yet, there's a strange collection of StatsD internal statistics that seem haphazardly organized. But the most striking was the appearance of someone else's Carbon agent metrics (instance-3-9-a) in my system? Obviously these aren't terribly useful or secretive, but it concerns me nonetheless. Perhaps this is an archive of their build process.

At this point I started to scrutinize harder and began looking for details that the average customer might overlook. I checked to see if I could login to the Django admin UI with my Playfair credentials. Nope. Wait, are they providing any sort of basic authentication to the Graphite UI itself? Nope. I was able to open up Firefox and browse the instance without any authentication (I was logged into Playfair with my Chrome session). So yeah, even if I wanted to keep you out of my Graphite instance, my only defense would be obfuscation.

I think it's fair to say this service should be considered alpha software, at best. This is the first I've heard of their company; which isn't to say that they haven't reached out to other members of the Graphite community, but you'd think that it would've been news among the core Graphite development team. And yet, I want to like this product. I think given enough time and resources, this could be a really nice offering for bootstrapping Graphite within many businesses.

Still, I have a number of other questions that need to be addressed. Unfortunately, that mystical documentation I hoped to find after signing up? Nada. This is a huge oversight, imho. Graphite is a project that tends to lack quality documentation in areas; if you want to succeed as a commercial offering, this is one of the first items you should address.

Last but not least, there is no mention of how they intend to provide "Unlimited data retention". What does this mean? Some sort of NoSQL backend? A modified Whisper format (unlikely)? I think the most likely scenario is that they're using Ceres and avoiding any rollups. And what about granularity? Based on my limited tests, they're only offering 1-minute granularity. This is acceptable for development environments or rollups, but is unsatisfactory for near-realtime monitoring.

I hope that Playfair is able to take this feedback and roll it back into the product. They have a very nice signup and deployment experience, but their actual deliverable leaves something to be desired. But I suspect that it will be "good enough" for many users wanting to dive into Graphite for the first time and see what the fuss is all about.

Update: Oh, and how to delete unwanted metrics without access to the underlying host? Someone was kind enough to send me a message via metric since writing this review. I'm not anxious to remove it, but it occurred to me that I wouldn't know how, even if I wanted to.