Most companies, even if they don’t know it, already have way much more data than they can efficiently handle; they are literally drowning under data! So, do you believe the Big Data promise? That it’ll make a difference in your business performance and profits? Not so evident!
But first, let’s see what we are talking about? What is Big Data?
And when did it turn out to be BIG?
The true nature of Data and the Big Data promise
Data comes in two forms: structured and unstructured. When you join the two, it becomes Big, really BIG.
Structured Data is any kind of data a company has within its walls, that it has formatted to better handle it, mostly it’s proprietary information; meaning, the data has been extracted (gathered), categorised and linked under some kind of hierarchy in either Excel sheets or Databases. It is then considered useful (available); it can be easily manipulated, crushed, compared, etc… displayed as BI on dashboards and accessed anytime, anywhere (ideally) for analysis, interpretation, decision-making, etc.
Business Intelligence experts estimated a few years ago that only 4% of all data amongst enterprises’ walls was structured.
Unstructured Data has to do with the rest of it, all data related to your company; that is, conversations in your company’s emails, chats and blogs, technical information and specifications, all your social media conversations and all information published in media channels and experts about your company and your field of activities.
They say: “You access the Big Data fountain by simply overlaying (how do you do that?) unstructured data on top of structured data and use analytics software to visualize it”. That is the essence of the Big Data promise; it’ll be marvelous and easy !
In theory, you should get never-before-seen insights about almost anything; that’s the promise. In theory, you will be able to better predict product sales, target customers better, discover new emerging markets, reduce transportation cost, increase productivity, etc.
Big Data has evolved rapidly since the days of open source tools. It is now more mature with offerings to help you visualize Big Data from specialists like Tableau, Qlik, etc., and major players like Microsoft, SAP and Oracle.
But the reality is that companies are having major issues with feeding data to these analytics platforms.
Now, here are two highlights that should rise suspicions regarding the Big Data promise:
- according to a study by Xplenty, a third of BI professionals spend 50% to 90% of their time cleaning up raw data and preparing it to input in data platforms
- according to Cisco, only 28% of companies think they are generating strategic value from their data
Big Data as is is still not very much automated today, the data cleaning problem proves that companies still have to spend a lot of time sorting out and organizing data before it gets analyzed.
Hence, there is still a major problem at the source with the Big Data promise; feeding the right data, that is: identifying, gathering and curating data.
And the Internet of Things (IoT) will not help at all, on the contrary; as we get better at collecting data and increase the amount of available data, the problem will only get… BIGGER. Some think that AI will magically solve the Big Data issue. Well, that calls upon other issues…
We are all in for the virtue of progress but sometimes the wait-and-see attitude may turn out to be a big payoff. And even more so, considering that most companies are already doing the Big Data thing, they just don’t know it or they don’t call it that; instead they call it Business Intelligence, Sales and Predictability Reports, Performance Metrics (KPIs), Decision-making Tools, Dashboards, etc. Not to forget that many if not most enterprises now have multiple Social Media Accounts they use for Marketing, Clientele Service and Technical Support from which they throroughly analyze both conversations and engagement levels.
So, in the meantime, here’s a very simple strategy to help your company start on the right foot with Big Data:
- – Analyze what data you really need or you think you need to know to make better decisions, enhance management, increase productivity, etc.
- – Look at what you’ve got within your walls and identify what’s missing
- – Connect to or build bridges to connect to the source of valuable data you already have with the help of either your IT department or your IT consulting firm
- – Start curating and illustrating / displaying your Data in a meaningful way to your company
- – Provide access to this data anytime, anywhere, on any devices!
Trust me, your Data will start to feel really Big all of a sudden, it’ll mean a lot to your company and your decision-makers.
And if need be, then and only then:
- – Find out where you will get the data you are missing (feeding source)
- – Plan to restart from point 3 above
Before you start collecting, gathering and feeding your company with Big Data, think about the investment and the load of time and efforts it’ll take to extract that Intelligence needle from this haystack of Data… because somebody somewhere will have to “prepare the data” anyway! And ask yourself, is that Big Data promise really worth it?
Maybe you can start by just cleaning up your garden first and see what you’ve got! You might be surprised with what you find. And it may be quite BIG!
Denis Paul & Michel