Not wanting to be left behind in Small Data Land, businesses are quick to jump on the Big Data bandwagon. A report from Gartner found that more than 75% of companies have invested or are planning to invest in Big Data strategies. But what is it? And should you be considering the impact Big Data has on the future on your business?
What is Big Data?
In essence, Big Data is the accumulation of massive amounts of tiny pieces of data. But that definition is too simple, so I asked big data specialist David Karpook vice chairman of the Open Standards Consortium for Real Estate (OSCRE) and industry expert on Big Data Solutions, what his definition is. He told me that he defines Big Data by the three V’s “Volume - there has to be a lot of it, Velocity - so it’s generated quickly at a rapid pace in rapid cycles and Variety - data may take many different forms, from traditional numbers and words to photographs, videos, and sound recordings.”
Image Credit: KPCB
As you can see Big Data encapsulates information that fits nicely in an Excel spreadsheet, like customer data, and sales figures. But it also includes “unstructured data” like video and sound, which is much harder to fit into the confines of a spreadsheet. This brings up issues of Data Storage and also Data Aggregation - topics I will focus on in later blog posts.
When I say Big Data, I mean really Big. By 2020 it’s estimated that the digital universe will have produced 44 trillion gigabytes of data. To give you an idea of just how huge that is, that data would fill six stacks of iPads reaching from the Earth to the Moon. Take a look at this visualization tool to get an idea of data sizes:
Brought to you by Redcentric
Unusable Big Data
Data generation is increasing rapidly and its volume is astounding, it is also coming from a wide variety of sources. A report found that while consumers generate 80% of the data, while businesses have become custodians of this data.
Consumers are producing data through devices such as phones, tablets, TVs, drones, cars, GPS systems, social media, messaging apps, games consoles, and wearables. But most of this data isn’t usable. Bummer.
That’s because massive amounts of data are collated, but not necessarily processed. Think of Facebook, Mark Zuckerberg has spent the past decade amassing crazy amounts of personal information on all facebook users - but until recently it wasn’t clear what he going to do with this data.
For most businesses the data comes in and it sits there, like a big giant hairball, that is very difficult to unravel. It’s the job of Data Engineers to look at this hairball and find the useable threads in it.
Most of their work entails cleaning up databases, so getting rid of unusable data, and finding information that can be translated into something useful.
“In corporate settings data engineering is a bit like janitorial work, you’re cleaning up databases,” says Maksim Tsvetovat, IntellectsoftBig Data Scientist, and author of the book [Social Network Analysis for Startups](https://www.amazon.com/Social-Network-Analysis-Startups-connections-ebook/dp/B006WACLQS?ie=UTF8&ref=asapbc). “For Big Data to work, there has to be a discernible signal in the noise.”
Find a Signal in the Noise
Once you find a signal, says Tsvetovat, you build a system around it, and to the outside world, the system looks like magic.
Think of how Netflix knows which movies and TV shows you’d like to see before you’ve even heard of them. It’s like Netflix can read your mind! But it can’t. Like all Big Data systems, it works on an algorithm that bases its suggestions on your previous selections and browsing history. This is pure data science.
Maybe that sounds like we’re entering the realms of Massive Budgets, Dedicated IT Departments and BigCorporation Inc, but be assured that SMEs are in the best position to use Big Data.
Data Consultant Bernard Marr wrote in the Business Standard that “In many ways, Big Data is suited to small business in ways that it never was for big business - even the most potent insights are valueless if your business is not agile enough to act on them in a timely fashion. Small businesses have the advantage of agility, making it perfectly suited to act on data-derived insights with speed and efficiency.”
You’re probably using some semblances of Big Data without realizing it. Like most business, you might use Google Analytics to track website traffic - if so, you’re already using Big Data. Perhaps you use an email marketing platform like MailChimp to send you statistics about open and click-through rates to determine if your email marketing campaigns are effective. This is again Big Data.
Properly analyzed Big Data can help you determine customer behaviour, which extends to being able to make predictions about customer buying behaviour, and also help to reduce churn. It can also be used for analyzing and improving your business processes.
Our next blog post in the series will explore how you can effectively manage Big Data to help grow your SME.