December 8, 2012

8 TESTS TO DECODE BUSINESS ACCUMEN OF A DATA SCIENTIST



A data scientist at Flutura has to wear multiple hats in order to deliver next generation analytical solutions in the sectors we operate in namely energy, telecom, digital and health care industry. In order to do that he/she has to wear 3 hats

-         The BUSINESS  hat
-         The MATH hat
-         The DATA hat

Most of the time it’s easy to fathom the depth of the data scientists math / algorithmic knowledge and the depth of his/her understanding on handling high velocity data and unstructured data points. But one area of weakness is the business dimension. So how do you decide whether a data scientist can be put in front of the business? This blog talks about 8 different tests Flutura executes to decode the business acumen of a data scientist

Test-1: “RESONANT STORY TELLING” TEST


Human Beings are wired more to listen to stories than to read numbers. Flutura data scientists were doing data forensics on mobile app funnel drop analysis for an online travel agency was able distil the quintessential essence of all essences - That the mobile user who was getting dropped was a 20 something, last minute booker travelling between metros and trying to complete the transaction from a Samsung mobile using Android os and the friction point was the payment gateway
Therefore
-         Can the data scientist translate numbers into stories? This is a very important tool to build bridges with business. Else a data scientist has the risk of getting struck in the world of math and unable to make the connect.

Test-2: THE “STRING OF PEARLS” TEST


It’s very important for a data scientist to triangulate from key insights. A Flutura data scientist working on Telecom security use case was able to connect the dots when he was able to see a co-relation between multiple failed login attempts + successful patch download event and a surge in network traffic which was a result of the security hole in the patch which was downloaded.
Therefore
-         Can the data scientist connect the dots and form a “necklace” from the pearls of insights discovered from cryptic log file data points?

Test-3: “NEEDLE MOVEMENT” TEST


One of the biggest risks in a big data project is using data to solve the right problem. There are many use cases a data scientist can curate … How do we identify the use cases which are $ denting from the use cases which have marginal impact?.Big data use cases can be segmented into 2 categories … those which move the needle incrementally vs those which disrupt. Its very important to keep this distinction in mind. Flutura was able to shepherd an ecommerce company into introducing new payment products after most of the transactions were dropped at payment gateway. This minor tweak resulted in the friction point being removed and a huge upswing in revenues
Therefore
-         Can the data scientist tease out business themes where a use case can unlock disproportionate revenue making potential for the organisation?
-         How would a data scientist go about teasing out the business themes to move the needle?
-         Which are the best “impact zones” in a business process which are “ripe” for big data?

Test-4: “SNIFF THE DOMAIN OUT” TEST


Let’s face it – data driven domain knowledge can reduce the learning curve required to understand domain and is deeper than armchair based experiential knowledge. Multiple engagements Flutura has executed has proven to us that a data scientist can glean far more knowledge about the nuances of a business by doing getting his/her hands dirty on exploratory data analysis(EDA), and eyeballing univariate and bi-variate results.
Therefore
-         Can the data scientist “sniff the domain out” by examining EDA outputs and getting the business to put the numbers in context?

Test-5 : “ACTIONABILITY” TEST


Most of engagements , the end result is a  suave looking ppt with lots of eye candy graphs which result in a feel good effect but business is left wondering on the actions that can be driven out of the exercise. In Flutura our mantra has been “Actions not insights”. One of the use cases we executed resulted in high value customers who are vulnerable to churn away being redirected in real time to high touch contact centre agents who would call them instantly and offer an instant rebate to woo them back
Therefore
-         What was the data scientist’s role in operationalizing actions or did his prior engagements end with recommendations? There is a big difference between the two

Test-6 :  “USE CASE CURATION” TEST


Carving out new use cases and possibilities from new data pool is both an art and a science. A Flutura data scientist was able to use search logs which were typically discarded to decode the travel intent of an online booker – is it a price sensitive traveller or  a value conscious traveller ? is the traveller an early bird or a last minute booker. This use case to create behavorial tags from search logs resulted in more intelligent outbound actions
Therefore
-         Can we give a raw data set and can the data scientist take 3-5 minutes to curate an interesting possibility from the raw data set ?
-         Where would he or she start in the big data ocean and zero in on the right ‘catchment’ of use cases

Test-7: THE “NORTH POLE” TEST


Every big data voyage requires a north pole in terms of measuring success for the engagement. A data scientist must be extremely clear or what constitutes success for the business stakeholders be it a sandbox setup or a full fledged production setup of a Hadoop cluster.
Therefore
-         Can the data scientist work with business to articulate the ‘as is’ state and the expected ‘to be’ state of the decision making process after the analytical solution is implemented?

Test-8 : THE “WHAT DO YOU SEE” test


The ability to take an analytical output and translate them into a series of English statements – this constitutes Flutura’s “What do you see” test. The sample analytical outputs can be
-         Key word frequencies from text mining
-         Scatter plots
-         Box plots measuring behavioural volatility of customer balances
-         Bi-variate cross tab outputs
-         Clusters from a segmentation output etc
So
-         Can the data scientist construct 3-4 meaningful English statements from the above sample analytical outputs?
 If so he/she would have crossed the big chasm from math to a business pattern which can be perceived by business



So in a nutshell here are 8 questions to ask
-         “RESONANT STORY TELLING” TEST
o   Can the data scientist narrate a compelling and resonant story from the data patterns?
-         “STRING OF PEARLS” TEST
o   Can the data scientist connect the dots and form a “necklace” from the pearls of insights discovered from cryptic log file data points?
-         “NEEDLE MOVEMENT” TEST
o   Which are the best “impact zones” for use cases which are “ripe” for big data?
-         “SNIFF THE DOMAIN OUT” TEST
o   Can the data scientist “sniff the domain out” by examining analytical outputs and getting the business to put the numbers in context?
-         “ACTIONABILITY” TEST
o   What was the data scientist’s role in operationalizing actions or did his prior engagements end with recommendations?
-         “USE CASE CURATION” TEST
o   Can we give a raw data set and can the data scientist take 3-5 minutes to curate an interesting possibility from the raw data set ?
-         THE “NORTH POLE” TEST
o   Can the data scientist work with business to articulate the ‘as is’ state and the expected ‘to be’ state of the decision making process after the analytical solution is implemented?
-         THE “WHAT DO YOU SEE” test
o   Can the data scientist construct 3-4 meaningful English statements from clustering outputs, keyword frequencies, Box plots and other analytical outputs?


These tests are by no way collectively exhaustive or perfect. But it serves as a reasonable starting point to get the right DNA of Data Scientists into the organisation. Else we run the risk of having people who just knows how to create a Hadoop cluster :) as being labelled a data scientist.
As the saying goes “The real voyage of discovery consists not in seeking new landscapes but in having new eyes.”- Marcel Proust
Good luck with your efforts to recruit the rare species – the holistic data scientist :) !!!

1 comment: