Everybody’s talking about … Big Data

8 May 2014 by Rebecca in Business and finance

Unless you’ve been living under a rock for a few years you will have heard of Big Data. But is it all hype and hyperbole or will it really change the world?

Where did it come from?

The term Big Data was probably coined by John Mashey, chief scientist at Silicon Graphics. In the 90s if you were a Hollywood producer looking for cutting edge special effects or an intelligence agency looking for state of the art video surveillance you went to Silicon Graphics. The company was dealing with a huge amount of a new type of data and Mashey frequently gave talks on how this data would change the future, including one with the snappy title ‘Big Data and the Next Wave of Infrastress’.

Big Data gathered traction, especially in tech circles (i.e. people who actually knew what ‘infrastress’ meant). For the rest of us – it took a little longer. In 2012 Big Data burst out into the mainstream – it was a featured topic at the 2012 World Economic Forum in Davos, Switzerland. The US federal Government announced $200 million in funding for Big Data research programs and even the satirical cartoon Dilbert had something to say about Big Data.

But what does it mean?

Basically Big Data is the term used for the collection of very large, often very complex data sets that can now be analysed in non-traditional ways to provide insights that were unheard of a decade ago.

To give you an idea of just how big, Big Data is … if you take all the data that was created in the world from the dawn of civilization until the year 2010, the same amount of data will soon be generated every minute.

This explosion of data has come from advances in technology. The smart phone in your pocket is now more powerful than your desktop computer was just a few years ago and is capturing data all the time, including where you are (via GPS sensors). Whenever you are connected to the internet, via your phone, tablet or computer, data is being gathered about what are you doing – playing an app, listening to music, using the internet or social media. Every time you share a photo, tweet, update your Facebook page, shop online, walk past CCTV, call a recorded customer service line, send an email, use an e-reader, watch Netflix or upload video you’re adding to Big Data. There are 30 billion pieces of content uploaded to Facebook and eight years of video uploaded to YouTube every day! In fact there are now nearly as many bits of information in the digital universe as there are stars in the actual universe.

But it’s not the volume of data that is the potential game changer – it’s our ability to store, combine and analyze it.

Where will I find Big Data being used?

Big Data is everywhere and its impact can be witnessed in everything from sport to health and fitness to medicine to entertainment to crime prevention and business.

There are now smart TVs that remember your viewing preferences and use face recognition to ensure children can’t access age inappropriate material. Book e-readers are gathering data about your reading habits. There are wearable devices that track your activity, calorie intake and heart rate to help keep you healthy. You can get smart carpet which can raise the alarm if your elderly mother doesn’t make her usual morning cuppa. You can even get a smart nappy which will send a tweet when the nappy needs changing! Obviously a swift sniff of the offending backside would do just as well but there are sensors in the nappy which can check the contents for signs of infection. Sensors in a baby’s mattress which measure breathing and heart rate could herald the end of cot death.

Not only are we gathering information and data that we simply didn’t have access to before but vastly improved computer processing power and complex analytics is allowing us to combine large, messy and previously unconnected data sets to find out everything from the bizarre to the life saving. For example the US retailer Wal-Mart combined in-house sales data with external weather data and discovered that the sale of Pop-Tarts jumped as soon as there was a hurricane warning! No one knows why but the beauty of Big Data is that you don’t need to know why because the data does the talking. Now Wal-Mart move the Pop-Tart display to the front of the shop when a storm is approaching and Pop-Tart sales skyrocket. The FBI has also combined data from social media, CCTV cameras, phone calls and texts to track down criminals and predict the next terrorist attack. There are apps that alert police to gunfire in real time. It’s also now possible, based on their twitter posts, to predict which mothers will suffer from postnatal depression. Professional sport is using big data analytics to find talent and improve performance and politicians are using it to figure out where they need to campaign to win elections.

Can I use Big Data?

It is easy to be overwhelmed by Big Data especially when you consider that most businesses don’t even use the small data they currently have access to never mind harnessing the potential of Big Data.

There is already a backlash as people increasingly question just how useful it’s really going to be. It’s certainly true that most businesses will never be able to compete with the Big Data giants like Amazon, Facebook and Google. Luckily they don’t need to.

The trick for turning the hype into practical business savvy insights is to be really clear on your strategy, work out what questions you need to answer in order to deliver that strategy and focus on collecting and analysing only that data. Unless you’re an analytic powerhouse, collecting too much data is as useless as collecting too little.

Plus you don’t need to have Big Data to benefit from Big Data. It’s now possible to source external data to cross reference with your own internal data. For example many retailers are combining their own sales or loyalty card information with weather data or GPS data to send out targeted promotions.

Like most innovations Big Data has potential but only if you don’t allow yourself to get bamboozled by the media furore and wacky stories of futuristic ‘smart living’. Find out what you need to know in order to be better and establish what combination of traditional data and Big Data will help you to deliver that outcome.