July 6, 2014

Big Data Crystal Ball – Looking Forward

A guest post by Clif Triplett, ex-CIO, Baker-Hughes and member of Flutura advisory board.

Big Data”; another big idea like “Cloud”; is it something that we should be embracing right now or be lost forever?   Is this another vague term that can be applied to countless projects?  Everyone is doing it our vendors will tell us.  You may have the question, “Am I actually running a “Big Data” project or do I just have to deal with a lot of data?”  What is “Big”?   

As I look forward, I think Big Data will begin to take different distinct forms as this other concept of “Cloud” has matured over recent years.  Business cases will become more obvious and solutions better defined.   The enablers to address  “Big Data” opportunities will also become more obvious as we see the dynamics of tomorrow dramatically evolve compared to just a few years ago. 

The most influential contributors emerging on the horizon that will change the game are the following:

  •         Explosion of sensors and digital inputs
  •       Very large memory machines (Terabytes of memory and more)
  •       Easy access to thousands of processors from Cloud providers like Azure and Amazon
  •       HPC environments and Cloud operating systems that can organize computing across many independent machines
  •       The continued explosion of readily available data on the internet (e.g. weather, stock market, government records, news, social media)
  •       Web crawlers that can locate and index information on the web      Predictive Analytic engines
  •       Open Source software (e.g. Predictive Analytic engines, search engines, cloud operating systems, and I anticipate the emergence of applications like those that might be able to judge and predict sentiment about topics)
  •       Proven use cases by industry
  • .     Commercial-Off-The-Shelf (CoTS) package solutions connecting multiple data sources and answering specific business challenges e.g. sales forecasts, predictive maintenance, process control, inventory management.
This convergence of technology and information now make it possible to define sub-classes of Big Data projects, such as Demand Forecasting, Failure or Defect Management, Market Sentiment, Situational Alternative Simulation, and many others I am sure.

Big Data product segments will also be clearer and strong leaders will emerge.  One of the product segments where we will see specialization will be hardware platforms.  We will see specialized machines such as HANA platforms and Solid State storage optimized for very large data sets.   We will also see Big Data Cloud platforms leveraging the momentum of two major technology trends; Cloud and Big Data.   There will be many analytic platforms; both open source and proprietary.  We will see boutique solutions to specific problems like product launch predictive sales analytics and advanced maintenance management programs. 

The other big player to emerge will be taxonomy platforms that assist in connecting disparate data sources and applying context to support integration.   Today most major applications provide APIs to their applications.  I foresee them in the future also providing taxonomy context to allow more dynamic integration of systems.

So, what are some of the issues that this future of Big Data will bring upon us in the near term?  Organizational change management will be top of the list.   This emerging trend surrounding Big Data will cause many new roles, new skill demands, new systems, and new approaches to business. In short; change.

First of all, we will face the issue of where is our Master Data and who is the owner?  This is a problem that has been with us for some time, but now it will emerge as a very strategic decision and not just an IT annoyance.  Second will be the issue of data quality and how do we clean up all this conflicting or missing data we have uncovered.  Our list will continue to grow and here are a few more to consider:

  •       Development of Business Cases to clean up data
  •       Engaging the business in data governance
  •       New Technologies for solving this new class of Big Data business solutions
  •       New skills required and a shortage of top talent
  •       Technology and Business roadmaps – what does it look like?
  •       Many new vendors emerging and the old favorites trying new things they have never done before – how do we know who knows what they are doing?
  •       Project estimating and run away projects

We have seen many of these challenges before and many of us will see them again.  It will be exciting times and we better start preparing now.  The planets are aligning and Big Data is arriving.  It still may be a bit still too far away for some to see it clearly, but keep your eyes open for it will be upon you soon.


2 comments:

  1. What is the reason for such a rapid growth in the demand for cloud technologies? Activewizards.com gives simple answer - companies, firms, organizations and organizations understand their importance. They are interested in the clouds, such services are in demand, which means that suppliers feel good. A study conducted by Oracle experts shows that more than 90% of organizations that use cloud technologies are faster innovating. 73% of respondents said that the cloud also helps to keep customers, and 76% noted that they attracted new ones with their help. Already, almost 20% of companies have reached "cloud maturity" in the development of private clouds. And more than half of the respondents said that they will come to this within two years. As for the obstacles to deploying hybrid cloud solutions in companies, the study identified three key positions. For 60% it is the ability to manage a variety of IT architectures, 57% of respondents noted the bandwidth of networks, and 52% - the relationship with the IT suite.

    ReplyDelete