October 16, 2013

Upstream Sensor Data + Big Data Analytics = Game Changer in Oil n Gas industry


Each blow costs the Oil and Gas industry a Billion dollars.

Can we avoid it? Can we see it coming and take action? 

We also know that each operating rig consists of thousands of sensors. This sensor data is used to analyse and reduce HSE (Health, Safety and Environment) risk considerably and dollars.


To show in detail how this is done, Flutura will explore breakthroughs achieved with upstream oil and gas data pools and Big Data analytics. This would transform the way HSE (Health, Safety and Environment) risk is detected by unlocking patterns previously not seen. Let’s begin, by asking ourselves some key leading questions that may help us diagnose the problem better and help transform the way we have been dealing with this problem so far:

1.       How can disruptive innovations in big data processing help mitigate blow out risks in the Oil and Gas industry to save lives and secure our environment?

2.       What are the key data blind spots in data upstream in the rig?

3.       What are some of the powerful unanswered questions from MWD (Measurement while drilling) / LWD (Logging while drilling) data?

4.       How can we triangulate across data streams to see key early warning signs upfront?

5.       What if we can wear new lenses to see risk patterns previously undetected?

The Oil n Gas industry is ripe for innovative solutions as two factors are converging to create the perfect storm.

·         Disruption enabler-1 : An audacious problem related to Health, Safety and Environment ( HSE) issues

·         Disruption enabler-2 : Large headroom to juice untapped data assets in upstream part of Oil n Gas value chain


Flutura calls this next generation approach - Oil n Gas 'Last Mile' Safety Risk Analytics 2.0. This approach hits at the intersection of Sensor data + Big Data Platform Engineering + Deep Machine Learning. This is a definite turning point from the traditional ways of looking at data using Historian and/or traditional database systems for assessing risk.

Most risks which happen in the Oil n Gas industry are emanating from events which happen in the last mile. If one is able to triangulate on sensor and human generated event streams to detect early warning signs which have not been seen, then one can detect signals early on. Flutura believes that the best way to mitigate risk is by reacting strongly to weak signals in the last mile and this involves continuous triangulation  of  streaming MWD/LWD and SCADA data streams over a long period of time using a combination of platforms + people
So what does the last mile in the Oil industry look like? The last mile in the oil industry is a sensor jungle which consists of a variety of MWD (Measurement while drilling) and LWD (Logging while drilling) sensors and SCADA devices. Thousands of these sensors are measuring multiple things from drill rpm, mud flow rates, CO2 gas emissions, valve positions, pump states and in the process emit billions of events. By using innovations in Big data like Map reduce and horizontal scaling, the Oil and Gas  industry has enormous potential for  unlocking previously unseen patterns buried in the data which help reduce risk in the last mile.

1.       Is there a co-relation between the out of bound of drill rpm rhythms from MWD Sensor data and operators experience?

2.       What are the frequent sequence of events (mud flow, Drill pressure, temperature, CO2) exhibited in rhythm disturbances prior to a near miss event which can trigger shutdown actions before an adverse event occurs?

3.       Which parameters where out of preset bounds (2 sigma events) before an incident happened - pressure / temperature or rpm so that the checklists on recalibration can be fine tuned?

4.       Are there keyword frequencies in maintenance inspection notes like "vibration", "leakage" which can be mined from text mining process which gives early warning signals to an impending incident?

5.       Where their signals in identity management logs which provide a clue to forensically investigate safety incidents?

6.       What is the NBI- "Next Best Intervention" for a Drill, valve, pump etc based on its historical behaviour?

The data to answer the above questions is available in the last mile sensor data captured as illustrated below:


Throwing data-blind-spots into the spotlight

There are devices like pumps, valves and sensors attached to a drill which emit either change of value events or alarm events. These events can be stored in a central event repository instantiated on a Hadoop cluster with massively parallel processing capabilities which can scale in an economic manner. Storm can be used for event stream processing, PIG to do computations in data pipeline and Hadoop /HIVE to do batch based computation



3 key best practices to mitigate risk in Oil n Gas (HSE) using big data

So what are additional best practices which can be leveraged to mitigate HSE risks?

1.       TRIANGULATE :
Look for patterns which reinforce each other across SCADA/MWD/LWD/Surveillance video event streams - structured, unstructured, internal and external.

- What are the top 3 cross event stream triangulations which are important signals to watch out for?

2.       REACT STRONG TO WEAK SIGNALS :
Consider having organisational processes which can pick the weakest link and amplification processes for field intervention. An alert prioritisation framework which rank orders signals would help risk specialists respond in a surgical fashion

- How can a weak signal be picked up and transmitted in real time to a point of action?

3.       LAST MILE DATA BLIND SPOTS :
Be conscious of last mile data blind spots be it contractor data, incident notes or device logs.

- How can we build a last mile integrated 360 degree integrated data picture?
We at Flutura believe that answering powerful unanswered questions and executing the 3 key best practices would dramatically heighten situational awareness and mitigate operational risks.

New machine learning and big data innovations in processing billions of events and a variety of messy data sources using Map reduce paradigms has helped a lot of human generated companies to disrupt their markets. The question is how can Oil and Gas companies use the same data innovations levers on machine data to disrupt the way of managing last minute risks?

As Marcel Proust said “The real voyage of discovery consists not in seeking new landscapes, but in having new eyes”, we at Flutura too believe that discovering HSE (Health, Safety and Environment) risk consists not in looking for more systems but in seeing with ‘new analytical eyes’ to look at previously collected data.

Srikanth Muralidhara
Co-Founder
Flutura Decision Sciences & Analytics