I’ve been hearing good things about OpenTSDB for some time and I’ve been meaning to take it for a spin.
Well I’ve finally gotten around to it.
- Pretty damn easy to set up. Considering the technology behind this.
The instructions are easy to follow and the build.sh script does all the hard work. I’m running a single HBase server. Not a cluster. HBase is the Hadoop database. Wooo Big Data. I’m running it on an AWS t1.micro (small Big Data). Just evaluating it at the moment.
- UI probably designed by an Engineer.
Functional but not so pretty. Hey I’m not saying I can do better! Frontend is not something I’m good at. You should see my horrible HTML. It would make your eyes bleed. Blink tag is still cool yeah? The thing that is kinda annoying is that it seems to use the time zone from my browser, which is not the time zone of my servers (UTC). Graphite’s UI took a long time for me to warm to. Now I’d like to see something like that for OpenTSDB. I believe there are some projects along these lines but I haven’t investigated.
- NoSQL to be Web Scale. Ok I can push billions of data points into HBase and never lose data like I do in Graphite (whisper) or Ganglia (RRD). Cool.
As I understand it, there’s no real difference between the concept of a Metric, and a key/value pair Tag. It’s all just data in NoSQL magic land.
This flexibility is proving difficult for me to work out how to get data in. I’m working on a project in my hack days at 99designs to suck metrics in from a variety of services and spit them out into a variety of backend systems. Ganglia and Graphite so far and now OpenTSDB. Just to see where it leads. It’s open source and on github here: https://github.com/99designs/vacuumetrix But I’m not sure how to use tags yet. There’s some talk on the mailing list about changing how this works, so I’m not going to fret too much. https://groups.google.com/d/msg/opentsdb/llAVqkKqFPw/_vQbf1MSX6sJ
- Downsampling. Ok this is really useful, and is something the likes of Ganglia and Graphite don’t do so well. Sometimes to see what is going on in the data you want to downsample.
- Aggregation is the default. OpenTSDB is designed to handle huge abouts of data. So aggregating metrics is naturally on.
- You have to manually register each new metric.
e.g. #tsdb mkmetric newrelic This is a bit sucky. Is there a way to do it automatically?
- tcollector I’ve no idea if this is working or not. Admittedly I’ve only been looking at OpenTSDB for a couple of hours. It’s not immediately clear how this works or what the metrics it may or may not be injecting are called. I have not yet looked at the source. I just tried to run something called startstop?!? If I have a few moments I’ll investigate further.
In conclusion. The prospect of not throwing away data is attractive. I will keep playing with this tool, get some real data in and see if it’s useful.
I recommend watching this video about OpenTSDB.