Big (Web) Data. How to start?

In today's rapidly changing business environment of high-tech companies, managers can't rely solely on “educated guess” or on analysis of historical data to drive critical business decisions on real-time without risk of misdirecting the company strategy and its opportunities. Yes, data are essential but too many data and complex statistical models that are not suited to C-suite executives had caused an understandably slow pacing in making substantial investments in data analytics. In an HBR article published in October 2012, titled Making advanced analytics work for you, a practical guide on how to start adopting data analytics to drive profitability is presented. Below is a brief summary of the lessons learned:

  1. Choose The Right Data. Big data has to do with manage unstructured data coming from many sources, especially from social networks and the web. Leader companies must enable capabilities to mine, curate and do advanced data segmentation that matters to a specific business problem or opportunity. This is a very important first step to avoid a well-known “data-rich-but-insight-poor” syndrome affecting so many enterprises. For instance, perhaps a decade old, traditional statistical methods based on complex probabilistic models and historical data alone did well by helping executives understanding what's going on with their operations and by solving complex business problems in a generally slow-moving industry (e.g., manufacturing). Those same models are prone to fail in today's more demanding and fast-paced environment where high-tech companies compete. In an era of Facebook and Twitter, the company's product insights are on “unstructured data in form of conversation, photos, and video” as quoted in the HBR article cited above. More powerful APIs (Application Programming Interface) is transforming the way engineers and scientists are parsing data from multiple sources to build a real-time web app using modern technologies such as Node.js, Heroku, and Express. These insights are surprisingly relevant and actionable.
  2. Building Models that Predict and Optimize
  3. It is not about the data but the way you correlate them (e.g., apply advanced segmentation) to reveal useful information about key performance indicators (KPI). A model should always start from a business objective and a set of well-designed metrics that answer this specific business problem. This is also known as hypothesis-driven problem solving that are used by scientists.
  4. Organizational Change. Here's a quote from the above HBR article that says: “... managers don't understand or trust big data-based model.” This is a result of a mismatch between company culture and its capabilities to use big data and analytics – the lessons learned here are to develop analytics that is actionable and embedded into a simple user interfaces for front lines and C-suites executives.

The era of data analytics is now. Be nimble and bold as a startup are essential for big companies aiming to leverage the benefits of big data and analytics. Get ready with a myriad of innovative algorithms by using machine learning running on the cloud, and soon they will become a decisive competitive asset.