The Stack Archive

Why NoSQL is the future of IoT

Mon 15 Feb 2016

Smart watch

emanuel-marchalEmmanuel “Manu” Marchal, head of operations for Basho Europe, here explains the necessary preparations required to avoid potential IoT data challenges…

Experts predicted more wearable technology wrapped up under the Christmas tree in 2015, and they weren’t wrong. We were gifted with everything from simple fitness trackers to smart watches, glasses, jewellery, headgear, belts and e-textiles. After the “toys but no batteries” disaster of years gone by, I’d argue it’s a data flake-out that will worry parents next year as their children run around excitedly with their Christmas loot.

Sales of wearable technology have already quadrupled over the last five years, and the global wearables market is expected to reach a value of 19 billion US dollars in 2018 – more than ten times its value in 2013. And it’s not just wearables. According to Gizmag “nowadays kids expect their toys to connect to the internet, pair with smart devices, and let them join in the latest tech trends.”

But now as we move beyond the holiday season, last Christmas will likely prove a drop in the ocean compared to what is to come in 2016. The market for wearable technology is exploding. By 2019, research firm IDC predicts worldwide shipments will reach 173.4 million units – that’s a growth rate of nearly 23% over the next five years. Most, if not all of these new and exciting devices, are being either powered by and or leveraging data to make the experience complete.

It’s not just the amount of devices that are on the rise either. The sophistication of these wearable devices is also increasing. The average wearable device in 2013 had 1.3 sensors, but by 2019 they are expected to carry 4.1. The data these devices now need to capture, keep and distribute is growing in multiple directions – and with each device also leveraging at least one application the resulting data tsunami could be lethal for service providers without the right technology in place to ride it.

Weathering the data storm

This raises the question as to whether today’s brave wearable technology pioneers are fully prepared for the potential data challenge that is to come. Once turned on, if the promise of personalised engagement, instant connection or reliable multiplayer gaming is not delivered, excitement can quickly dwindle and brand promises are broken. In the future, meeting peak demand and availability while allowing for scalability will become business imperatives for many companies, to ensure this doesn’t happen.

A NoSQL database that uses a SQL-like query language will provide familiar semantics that users can quickly leverage to make it easy to write queries for data analysis.

There is growing awareness of the challenges that IoT-generated data presents. Not only has it become clear that traditional relational databases cannot manage the scale of IoT data, but many databases are also ill-equipped to handle the specialised nature of IoT data sets, including time series data – by which I simply mean data that is created with a timestamp. Yet the ability to effectively collect, store, and analyse time series data is critical to harnessing the IoT’s power to help businesses gain those valuable insights, power digital transformations, and drive more effective customer engagements.

Resolving the data storm with NoSQL

This year, the companies who are first to address the issue of managing time series data will possess a strong competitive advantage over those lingering behind. The starting point is to consider the data strategy “from the ground up”, and its beating heart is the database. Without the right solution in place, the ability of an IoT business to scale cost effectively with demand, while ensuring data is always correct and available – when and where it is needed – is severely limited.

Solving the unique challenges that come with time series data requires a database solution that is specifically optimised for the task. Modern NoSQL databases inherently solve many of the data volume and diversity issues that come with IoT, but these too are having to evolve to provide optimum storage and retrieval for this new time series data.

There are five things businesses need to look for in a database for Time Series IoT data:

The right structure and format

Is it designed to achieve maximum performance and availability for this specific data type? A distributed NoSQL database designed for Time Stamp will provide the read and write performance as well as the scalability and availability that IoT applications require while running on commodity hardware to reduce the overall costs of operations. Distributed design also allows businesses to keep the data close to the customer empowering those personalised experiences using local data centres.

Data Co-location

Time Series data is typically queried by time ranges, sources and location. A NoSQL database must ensure fast response time no matter the total volume of data and this can be achieved with the optimal co-location of time series data by quantum of time, source and geohash.

Fast, easy range queries

A NoSQL database that uses a SQL-like query language will provide familiar semantics that users can quickly leverage to make it easy to write queries for data analysis.

High write performance and availability

A distributed NoSQL database with a masterless architecture can ensure high availability and high performance for both read and write operations during peak loads, even in the event of a node failure.

Data compaction

As Time Series data ages, the granularity of the data often becomes less relevant to the purpose of useful analysis. To be able to store and retrieve the data efficiently, organisations must have the ability to roll up and compress the data according to the needs of the business.

Those who are already ahead of the wave will ultimately win in the IoT race with an ability to deliver on their brand promise by providing a personalised high quality customer experience and a simple, quick and straightforward engagement. In today’s highly competitive environment, it is this ground-up approach of optimising a company’s data strategy and architecture that will be the secret to a Merry Christmas in 2016.

Emmanuel “Manu” Marchal is the head of Basho’s Operations in Europe, the Middle East and Africa (EMEA). He is founder of one of the largest big data communities in Europe, Big Data London, which currently has more than 4,500 members. He is active on Twitter @manumarchal


feature IoT security wearables
Send us a correction about this article Send us a news tip