Archive for April, 2016

Smart and Fast Data, not just Big Data…

Friday, April 22nd, 2016

I’ve just finished reading the book “Big Data: Using SMART Big Data, Analytics and Metrics To Make Better Decisions and Improve Performance” written by Bernard Marr. This book is a must read for Data Scientists and general people interested in data analytics. It is now clear that at the end of the day, it’s not about how much data you generate (in a scientific laboratory, in healthcare, in logistics, in the financial world, etc), it’s about how well you use it. Though Big Data has been recently deemed an over-hyped term, it’s not going to go away any time soon. I believe it will be the opposite; data science will be applied to all sectors in society. Information overload is a phenomenon and challenge we face all the time now. In fact, large-scale data analytics, predictive modeling, and data visualization are increasingly crucial for companies in both high-tech and mainstream fields to survive nowadays. Big data capabilities are a need, not a want today or tomorrow. Big Data is a broad term that encompasses a variety of angles. There are complex challenges within Big Data that must be prioritized and addressed; such as “Fast Data” and “Smart Data.” “Smart Data” means information that actually makes sense. It is the difference between seeing a long list of numbers referring to weekly sales vs. identifying the peaks and troughs in sales volume over time. Algorithms turn meaningless numbers into actionable insights. Smart data is data from which signals and patterns have been extracted by intelligent algorithms. Collecting large amounts of statistics and numbers bring little benefit if there is no layer of added intelligence and expertise. By “Fast Data” I am talking about as-it-happens information enabling real-time decision-making. An advertising firm, for example, needs to know how people are talking about their clients’ brands in real-time in order to mitigate bad messages. A few minutes too late and viral messages might be uncontainable. A retail company needs to know how their latest collection is selling as soon as it is released to continue or stop selling it. Public health workers need to understand disease outbreaks in the moment so they can take action to curb the spread. One example is Google Flu Trends detecting spikes in flu searches from Google. Twitter uses the same strategy to evaluate outbreaks of infectious diseases. A bank needs to stay abreast of geo-political and socio-economic situations to make the best investment decisions using a global-macro strategy. A logistics company needs to know how a public disaster or road diversion is affecting transport infrastructure so that they can react accordingly. One of the biggest evolutions of integrating smarter data into content experiences is that it levels the playing field with larger competitors who may have more resources to burn on advertising media. Using the totality of visitors as a whole – and deriving meaning from all of their content experiences, we can deliver more relevant and contextual experiences than our competitors. And today, we can deliver those solutions in much less expensive ways than the multi-million dollar solutions that may have historically been out of our reach. Big Data is just a problem. Smart Data is a solution that changes the game of marketing, and how we deliver better solutions for customers from this point forward. According to John Bollen, the four keys to converting big data into smart data are: 1) Organize and manage resources; 2) Identify your customers and/or targets; 3) Target this specific group of people and evaluate the outcome in real-time and 4) Use data analytics to look forward and to do forecasts. In addition, we have to remember that technology doesn’t solve the problem of changing big data to smart data. It’s more about process than technology. While tools are getting better at aggregating and parsing data, it’s ultimately up to us as data scientists to connect past behavior to future wants, preferences, needs, etc. The technological advancements seen all the way back to the Industrial Revolution have been about automating a manual process, not making us smarter. The questions without answers are “what processes are in place to handle the data?”, “what governance is in place?” (in other words, who’ll make decisions that extend from the data?), and finally, “how do we operationalize the data”? Technology won’t answer these questions. First, we must have the right people in place and processes established, then we look at how technology fits in. Smart and fast data generate reliable answers; however we need the right decision makers in the end.

Image Source: Digitalroute