Big Data’s Big Problem

There’s a common myth inside the discussion about the earth’s natural resources, namely that we run the risk of exhausting them. But a simple research of the matter quickly dispels this. We do not run the risk of “running out,” of our resources, but instead face the dilemma of not being able to extract them. As easier deposits of gold, phosphorous or oil are exhausted, the cost of mining these elements from more difficult source locations means they would be priced beyond consumption on the world market. It quite literally will become the “water water everywhere but not a drop to drink” scenario.

The world of Big Data faces a similar problem. In fact, so close is the parallel that many over the years have simply called Big Data the “next natural resource,” and noted the problem with extracting it as being the growing skills gaps in organizations. This singular issue, the absence of data literate professionals inside organizations that can adequately understand insights and data relevance, continually grows as the single biggest problem facing the growth of the industry. In many of our market research reports for companies in this space, we have seen an increase in the amount of interaction and sharing of articles from people such as Jordan Morrow, Global Head of Data Literacy at Qlik. As the foundation to a recent survey he recently conducted, Morrow cited the accuracy and danger of a 2017 Forbes article noting, “Increasingly, this data literacy divide will impeded organizations of all shapes and sizes from reaping higher rewards from their data investments.”

“Increasingly, this data literacy divide will impeded organizations of all shapes and sizes from reaping higher rewards from their data investments.”

As case in point, note the volume around a single blog posted by Snowflake earlier this year. In it, the author makes the case that one of the biggest hurdles to an enterprise organization ever achieving a maturity in the data and cloud space is simply the absence of resources outside the IT team to help plan and develop a data architecture.


So what can sellers of big data solutions and services do to help potential buyers enter this space confidently and successfully? We would offer the following three recommendations based on the current trends we are seeing in the industry as we constantly monitor the top competitors:

1. Focus content creation efforts strongly around DIY and free training content

Earlier this year, we noted multiple providers in the data space offering content intended to help companies trying to navigate the internal skills gap. In January, there was strong engagement around Amazon’s release of free digital training courses for AWS Big Data Training & Certification. Courses included introduction to Amazon EMR, and then allowed for continued online education for paid classroom training on Data Warehousing on AWS and Security Operation. Additionally, in the same period, there was strong engagement around content from Teradata. Using YouTube as the core venue, they posted educational content (promoted via Twitter) as a ten-part series of how-to videos illustrating tasks such as spinning up Teradata on AWS, Loading and Querying Data and Deleting a Cluster.

2. Develop certifications and training acknowledgments

In addition to being a provider of a data and analytics platform, institute courses that yield an official certification from your brand designating an individual as a professional. As an example, consider how much energy Tableau puts into singular events like “Certifiably Tableau Day” on March 15 promoted using #certifiablyTableau. All of their social media content on this day are testimonials from people certified by Tableau sharing how the certification both helped to chart and validate their career path and differentiate themselves in job searches. The intended goal is to get more people to understand the nature of the “professional” data scientist and help close the skills gap inside organizations by training more people on their platform. In addition, Informatica saw strong engagement around a blog post suggesting that even current data scientists need additional learning and that by seeking additional skills development and certifications simply makes them an in-demand asset inside the enterprise space today.

3. Guide prospects and clients through the skills gap challenge

Finally, providers of big data services should realize that part of the reason for the skills gap is that companies are not entirely sure who to hire or what skill sets to seek. The role of the data scientists is still somewhat fluid in definition, and understanding who to hire is something companies need help with. Note the graph below showing engagement for original content published by MicroStrategy. The twin peaks in the middle of the measuring period were due to the MicroStrategy World conference where they published a map for #IntelligentEnterprise. The map offers a collection of technologies and techniques to help customers chart the rough waters of building a successful data-driven organization. The engagement around this content shows the incredible interest and desire by companies to be educated and led in how to adequately stand up in the big data world.



There is no question that the world of big data and the advances it offers companies is the future. But the speed with which a company will realize these benefits will be directly dependent on whether or not they can staff appropriately to actually mine the data for relevance. Sellers of Big Data solutions need to realize that helping enterprise companies solve for the skills and data literacy gap needs to be a priority and that small steps (as noted above) can be taken immediately to help towards this goal.