July 6, 2014

Big Data Crystal Ball – Looking Forward

A guest post by Clif Triplett, ex-CIO, Baker-Hughes and member of Flutura advisory board.

Big Data”; another big idea like “Cloud”; is it something that we should be embracing right now or be lost forever?   Is this another vague term that can be applied to countless projects?  Everyone is doing it our vendors will tell us.  You may have the question, “Am I actually running a “Big Data” project or do I just have to deal with a lot of data?”  What is “Big”?   

As I look forward, I think Big Data will begin to take different distinct forms as this other concept of “Cloud” has matured over recent years.  Business cases will become more obvious and solutions better defined.   The enablers to address  “Big Data” opportunities will also become more obvious as we see the dynamics of tomorrow dramatically evolve compared to just a few years ago. 

The most influential contributors emerging on the horizon that will change the game are the following:

  •         Explosion of sensors and digital inputs
  •       Very large memory machines (Terabytes of memory and more)
  •       Easy access to thousands of processors from Cloud providers like Azure and Amazon
  •       HPC environments and Cloud operating systems that can organize computing across many independent machines
  •       The continued explosion of readily available data on the internet (e.g. weather, stock market, government records, news, social media)
  •       Web crawlers that can locate and index information on the web      Predictive Analytic engines
  •       Open Source software (e.g. Predictive Analytic engines, search engines, cloud operating systems, and I anticipate the emergence of applications like those that might be able to judge and predict sentiment about topics)
  •       Proven use cases by industry
  • .     Commercial-Off-The-Shelf (CoTS) package solutions connecting multiple data sources and answering specific business challenges e.g. sales forecasts, predictive maintenance, process control, inventory management.
This convergence of technology and information now make it possible to define sub-classes of Big Data projects, such as Demand Forecasting, Failure or Defect Management, Market Sentiment, Situational Alternative Simulation, and many others I am sure.

Big Data product segments will also be clearer and strong leaders will emerge.  One of the product segments where we will see specialization will be hardware platforms.  We will see specialized machines such as HANA platforms and Solid State storage optimized for very large data sets.   We will also see Big Data Cloud platforms leveraging the momentum of two major technology trends; Cloud and Big Data.   There will be many analytic platforms; both open source and proprietary.  We will see boutique solutions to specific problems like product launch predictive sales analytics and advanced maintenance management programs. 

The other big player to emerge will be taxonomy platforms that assist in connecting disparate data sources and applying context to support integration.   Today most major applications provide APIs to their applications.  I foresee them in the future also providing taxonomy context to allow more dynamic integration of systems.

So, what are some of the issues that this future of Big Data will bring upon us in the near term?  Organizational change management will be top of the list.   This emerging trend surrounding Big Data will cause many new roles, new skill demands, new systems, and new approaches to business. In short; change.

First of all, we will face the issue of where is our Master Data and who is the owner?  This is a problem that has been with us for some time, but now it will emerge as a very strategic decision and not just an IT annoyance.  Second will be the issue of data quality and how do we clean up all this conflicting or missing data we have uncovered.  Our list will continue to grow and here are a few more to consider:

  •       Development of Business Cases to clean up data
  •       Engaging the business in data governance
  •       New Technologies for solving this new class of Big Data business solutions
  •       New skills required and a shortage of top talent
  •       Technology and Business roadmaps – what does it look like?
  •       Many new vendors emerging and the old favorites trying new things they have never done before – how do we know who knows what they are doing?
  •       Project estimating and run away projects

We have seen many of these challenges before and many of us will see them again.  It will be exciting times and we better start preparing now.  The planets are aligning and Big Data is arriving.  It still may be a bit still too far away for some to see it clearly, but keep your eyes open for it will be upon you soon.


  1. What is the reason for such a rapid growth in the demand for cloud technologies? Activewizards.com gives simple answer - companies, firms, organizations and organizations understand their importance. They are interested in the clouds, such services are in demand, which means that suppliers feel good. A study conducted by Oracle experts shows that more than 90% of organizations that use cloud technologies are faster innovating. 73% of respondents said that the cloud also helps to keep customers, and 76% noted that they attracted new ones with their help. Already, almost 20% of companies have reached "cloud maturity" in the development of private clouds. And more than half of the respondents said that they will come to this within two years. As for the obstacles to deploying hybrid cloud solutions in companies, the study identified three key positions. For 60% it is the ability to manage a variety of IT architectures, 57% of respondents noted the bandwidth of networks, and 52% - the relationship with the IT suite.

  2. The wonderful thing about tape media is its ability to store so much information in such a compact amount of space.Self Storage

    1. Big data is a term that describes the large volume of data – both structured and unstructured – that inundates a business on a day-to-day basis. big data projects for students But it’s not the amount of data that’s important.Project Center in Chennai

      Spring Framework has already made serious inroads as an integrated technology stack for building user-facing applications. Corporate TRaining Spring Framework the authors explore the idea of using Java in Big Data platforms.

      Spring Training in Chennai

      The new Angular TRaining will lay the foundation you need to specialise in Single Page Application developer. Angular Training

  3. Excellent website you have here, so much cool information!.. click here

  4. The nearer you live to huge urban communities, at that point the less these progress toward becoming issues with which you'd have to concern yourself. Verizon Fios Double Play

  5. This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post. spell to end a relationship

  6. Consistently, the speed increments generally around 10% which just demonstrates the strive after speed of the worldwide clients. With the quick paced way of life we have now, the internet would likewise need to keep up.bing search history delete

  7. Pretty blog, so many ideas in a single site, thanks for the informative article, keep updating more article.
    Digital marketing course in chennai

  8. Hi, I find reading this article a joy. It is extremely helpful and interesting and very much looking forward to reading more of your work.. google ranking

  9. This article gives the light in which we can observe the reality. This is very nice one and gives indepth information. Thanks for this nice article. website build

  10. The crazy expense of a decent school instruction verges on the domain of criminal theft. bobsweep

  11. At that point there is the little matter of the considerable number of negatives that radiate from the web. Bobsweep reviews

  12. I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much. BD Phone Price

  13. Do you wish to take in more about ways to deal with shield your electronics from the effects of warmth, electrical flood occasions and electrical clamor? bobsweep pethair plus robotic vacuum and mop reviews

  14. They additionally appreciate discourse sheets. https://prywatnoscwsieci.pl

  15. For example, the consistent spread of information stream guarantees that the stockroom staff gets exact expectations about anticipated requests. office activator

  16. You will regularly find that the organization you work with may really furnish you with a suitable arrangement with regards to managing vitality utilization and solar panel numbers. Zonnepanelen Installateur

  17. beautifully written article, if most effective all bloggers presented the equal content as you, the internet would be a miles better region.. free tarot

  18. It is critical to buy extra cells to fill in as reinforcement in the event that you flop on a portion of the cells for example awful patch, broken cell, scratched, and so on. Zonnepanelen kopen