High Throughput Time Series Data

Greetings from the Canberra Big Data Meetup, on High throughput ingestion of time series data. This is at the Canberra Innovation Network/E29 offices. The first talk was by, Jordan Braiuka, from the host for the event, Canberra company Instaclustr, on using Cassandra to store 1 million metrics a minute. Not to be outdone, Mike Leonard, from Reposit Power (also a Canberra start-up), talked about ingesting 12 Million Data Points per Hour with OpenTSDB.





Instaclustr monitor servers, and Reposit Power the batteries in people's homes, but from a data storage point of view these are similar. The emphasis here is on being able to store large amounts of data quickly, and be able to query it for standard reports. For today's IT professionals, brought up with relational databases, the concepts used to do this efficiently will be novel. However, for someone like myself, trained in the mainframe pre-relational era, it is all very familiar. It is good to see there are modern tools to implement these decades old techniques, and proponents using them.