What to consider when defining your new data strategy
Mon 11 Dec 2017 | Mathias Golombek
Mathias Golombek, CTO at Exasol, writes on how businesses are adapting to incorporate new data strategies
Big data is no longer just hype and a buzzword. However, focussing on the term big data is missing the point. We have moved on to a place where business-oriented ‘data strategies’ should be the focus.
By changing the focus from big data to data strategies, businesses and teams are starting to see real value being delivered from data insights. The definition of Big Data used to be predicated on technological details. The driver of Map Reduce and the underlying technologies were software developers, who loved to work on parallel clusters and use open source stuff such as Hadoop.
But the projects were (too) rarely about business values or new insights, and especially not about maintainable and scalable analytical applications. That’s why so many Big Data projects failed.
Currently, the term Big Data is mostly used to describe the storing and processing of large data sets using technologies such as Hadoop and Spark. Too often ‘data lakes’ turn into passive ‘data reservoirs’ with the intrinsic value remaining untapped. This is why we must change the vernacular to ‘data strategies’. Businesses who cannot answer how that stored data can be analysed appropriately and leveraged to optimize their business, are wasting resources and will be left behind.
Solving business challenges
When it comes to data, the e-commerce and retail sectors have always been front-runners in adoption
Companies have to incorporate new data strategies to remain ahead of the competition. Only companies who work their data hard for insights will be able to optimise their business or create innovative new services and revenue streams. The importance of data is increasing in this digital world. As a consequence, companies are building new data competence centres outside of the IT departments, reporting directly to the C-levels (CFO, COO or directly under the CEO).
With the rise of the ‘Data Scientist’, the discussion surrounding Big Data has moved from purely technological questions to business insights. In the Competence Centres (Analytics, Business Intelligence, Data Science), multiple stakeholders are part of data projects and ensure that the right questions are asked to solve the actual challenges of disparate business departments.
It’s beneficial that new exciting technologies are constantly popping up to address specific needs, and they are replacing the unsuitable big software suites from the large vendors such as Oracle or IBM. Today, a heterogeneous, agile data ecosystem allows companies to do unprecedented things with data, opening up a whole new space to become better in business by utilising automated, predictive and prescriptive processes rather than just creating reports about the history.
Leading use cases
When it comes to data, the e-commerce and retail sectors have always been front-runners in adoption. They successfully applied data mining in the late 1990s, back when it was just called data science. Large retailers have millions of customers and are used to processing huge data volumes with the least possible delay.
With the rise of online shops, the e-commerce companies had to be quick to keep up with each other. As an example, the German unicorn company, Zalando has more than 70 data analysts who make sure that every single company decision is data-driven, and that their online shop is automatically optimized by crunching all the existing data they can access.
Companies can increase the success rate of data projects if they define the objectives early on
Besides the customer-centric markets, a completely new industry has risen in the form of the so-called data-driven service companies. Their business model relies completely on data. One example here would be the web tracking companies whose service is to analyse and optimize your website in real-time, such Webtrekk.
Setting your objectives
Objectives of projects haven’t been clearly set out from the beginning in too many instances. The initial impetus is often more about storing all the data and then looking what you could do with it later. Furthermore, the teams responsible are often too technology-driven and concentrated on questions whether Hadoop or Spark will be the solution for everything.
Nowadays, common sense has evolved so that technologies are implemented in such a way that they can be easily replaced at a later stage, and so that the key focus is on maximising the necessary capabilities for your specific application.
There are Big Data projects that specialise in analysing complex graph structures, text sentiment analysis tools for analysing your customer’s support emails appropriately, and in-memory databases that give you fast access to your data analysis.
Companies can increase the success rate of data projects if they define the objectives early on, and pick the appropriate data management technologies afterwards. To be able to do that, you have to create smart competence centres with people that have deep know-how about data management and the quickly evolving technology market.
Future business models
While data has long been a historic window for businesses (e.g. sales items, financial numbers), it is moving to become a source for predicting the future. The prevailing definition which encompasses Big Data’s evolution covers descriptive and diagnostic to predictive and finally prescriptive analytics.
New technologies such as artificial intelligence, in-memory databases, key/value stores, graph databases, stream processing tools and many more, move the goalposts. The question changes from technical limitations to the smart application of such technologies, in order to take innovative insights from all kinds of data sources.
The speed of adoption will increase more and more as a consequence, and data will more often form the bedrock of companies’ strategies. The exploding number of devices that collect data, e.g. smartphones or IoT sensors, will lead to exciting new applications that will make our daily life easier and easier while opening the space for completely new business models.
