April 28, 2015

How we survived a near death experience in a Big Data Project ? A Story from the trenches

One month back Flutura was in  a meeting with a customer in Houston where we were involved in an IOT big data project . It was a very tense meeting ... Budgets were being slashed, heads were rolling because of the steep drop in Oil prices. The CEOs office was forensically examining all projects thru a massive magnifying glass. The tension in the room was palpable. The cross functional team had spent a lot of time building a platform and the CEO wanted to see tangible business outcomes and how it made money for them. Given the intense market pressure, if the team did not articulate an ultra specific path to monetization, the project was definitely going to be axed. After about 3 hours of intense conversation the cross functional team of Engineering and IT folks were able to hammer out a 1 slider to make the case to the CEOs office .The meeting happened a week later.The project survived. Everyone breathed a sigh of relief ....

So what did we learn from the near death experience ?

The beauty about this near death experience in executing the IOT big data project ( We must admit ...It was a quite nerve wracking experience for us at Flutura as we went thru the turbulence ) was that it forced us to focus on the core . We have synthesized our learnings and we would be the first ones to admit that many of them are obvious but we felt the strong need to reinforce and internalise them.
Lesson-1 :  Think Outcomes ( for example : The project would reduce fuel consumed by 3 % resulting in $ 34,000,000 savings per year in first 2 years )
Lesson-2 : Are we  solving a top 5 problem which keeps business awake at night ( for example one should not be solving problem no 18  in the problem list from a business perspective ... forcibly ranking problems helps )
Lesson-3 : Is Status Quo an option ? ( One would be surprised as to how status quo is an option for many industries, businesses and processes. You may be excited about the possibilities but )
Lesson-4 : Don't use the following words while speaking to CXO team
  • "Big data"
  • "Analytics", "Data Science", "Algorithms", "Statistics" etc etc
  • "Hadoop", "Spark" , "Data Lake", "Lamba Architecture" etc etc
These are sure ways to short circuit a project as it activates the wrong  neural pathways for attention starved executives
Lesson-5 : Spend 70 % of conversation in first 4-6 weeks of a big data project ontangible Business case and not solution or architecture.

To conclude

These are tough times ...Markets are undergoing tectonic transition ... Industries which were immune to recession are tightening the belt.Big data and analytics has a huge role to play in driving efficiencies in many industries. The simple but obvious things evade us. We are reminded of a quote by T S Eliot
What we call the beginning is often the end. And to make an end is to make a beginning. The end is where we start from.
We hope the above learnings reinforce your experience and happy to hear your stories :)


  1. There are lots of information about hadoop have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get to the next level in big data. Thanks for sharing this.
    Hadoop Training in Chennai | Big Data Training in Chennai

  2. • Great information, I like this kind of blog information really very nice and more I can easily new skills are develop after reading that post.
    ios training in chennai

  3. • Very good effort in collecting information.........
    sas training in chennai

  4. I am really enjoying reading your well written articles.
    It looks like you spend a lot of effort and time on your blog.
    I have bookmarked it and I am looking forward to reading new articles. Keep up the good work..
    Java Training in Bangalore
    Advanced Java Course in Bangalore
    Java Coaching Institutes in Bangalore
    Advanced Java Training Institute in Bangalore
    best hadoop training institutes in bangalore
    best big data training in bangalore

  5. Thanks for giving great kind of information. So useful and practical for me. Thanks for your excellent blog, nice work keep it up thanks for sharing the knowledge.
    interior designer company in noida

  6. nice work keep it up thanks for sharing the knowledge.Thanks for sharing this type of information, it is so useful.
    Epoxy Grout manufacturer

  7. I feel happy about and learning more about this topic. keep sharing your information regularly for my future reference. This content creates a new hope and inspiration with in me. Thanks for sharing article like this. the information which you have provided is better then other blog.
    Best IELTS Coaching centre in Dwarka

  8. Wonderful article… Thanks for sharing such useful article E Cigarette Australia

  9. Usually I never comment on blogs but your article is so convincing that I never stop myself to say something about it. You’re doing a great job Man,Keep it up.
    New Zealand education Consultants in Delhi

  10. Your Blog is appriciatable and nice. Bharat CSP is nothing but a business contributor and technology services provider to different kind of banks.

    Apply CSP
    CSP registration
    CSP provider
    bank CSP
    CSP kisok
    kisok banking

  11. Awesome and Valuable information in this blog. Pure Digital covers a wide range of services from hosting, building your first website to Website integrations and backend systems development.

    Website Design Company in New Zealand
    Website Development Company in New Zealand
    Hosting Company New Zealand
    Website Revamp Company in New Zealand
    Web Hosting New Zealand

  12. Thanks for giving great kind of information. Get Kiosk, CSP Mitra Banking Registration & Online Csp Application with Bank Mitra Website.

    CSP Apply
    CSP Online Application
    Apply for CSP
    Top CSP Provider in India
    Apply Online For Bank CSP
    Online Money Transfer