The big data conundrum
Most system integrator organisations and product company IT departments now have a big data practice or at least a competency where they are busy ramping up engineers and data scientists who would be geared to respond to the next generation of data analytics challenges. Most buzz words like Hadoop , Flume, Scoop , Open Source are in the air but when it comes to applying these wonderful big data technologies to an application scenario most of these highly intelligent people draw a blank. Most do not have a clue of how to make a new relevant big data business use case for the industries they serve.
The intersection of big data technology and statistical principles are things which can be learnt directly. Business domain knowledge can typically be acquired by having worked or consulted in the industry. But are these three enough to convince a business leader to take the plunge into investing a huge amount of money, people and time into Big data? Not quite.
In many instances system integrators and IT departments resort to selling Big data as a fantastic technical breakthrough as opposed to showcasing how it can solve business problems better and help an organisation realize its long term vision. As a result of their insular nature system integrators and IT departments they are sometimes unable to relate to the challenges posed by the market dynamics on their customers. As a result of the above flawed approach many big data competencies are unable to gain a lot of mind share with the business leaders of their customer's organisations and face a lot of difficulty in selling any real big data transformations always tending to remain in technology proof of concept mode.
Business leaders are only interested in learning how their big data investment would relate back to real needs of the businesses they run .They want to know, in what way would big data help in reducing operational cost, gather market intelligence, provide for predictive analysis, make their customers come back for repeat business all related to the specific business organisations they lead. Also they would almost always challenge the need to migrate to big data technologies and methodologies if the needs of the business can be fulfilled with their current analytics investments.
1. Sell big data using an outside-in approach. That would entail understanding the business challenges and vision of the specific business first and then and making a determination if big data can indeed meet their challenges. Every business challenge might not need the application of big data solutions and this determination is very important before starting anything further.
2. Decompose the business challenges and vision statements into relevant business use cases and vet these use cases with business leaders and personnel to determine if they generate any interest in the community. Also determine if the existing business cases can be partially or fully met using existing analytics investments.
3. Once the business use cases have been determined and prioritized, bring in industry domain knowledge and mathematical experts to determine if these use cases can be addressed using business knowledge coupled with the application of known or custom modelling techniques. The outcome of this step should be a business solution with machine learning and statistical modelling techniques embedded into it.
3. After completion of the step 4 , feed the business solution to the technical team. Employ big data technical architects to conceptualize the customer landscape ,select big data tools and design the technical integration to technically support the business solutions. I would recommend the use of open source tools to keep costs in control and design a scalable architecture.
4. The final outcome of the above steps must be data or artifacts that directly address the challenges of the business use cases and the proposed solutions must collectively drive towards meeting the business vision of the organisation.
5. As a last step sell the business solution of the business leaders first, and then drill down into the how the business solution was arrived at showcasing the domain and statistical modelling techniques that went into it, finally concluding with a detailed description of the architectural landscape that would host these business solutions. To further justify the need for the bid data investment , as part of the technical piece also elaborate why the proposed solutions cannot be met with the business's existing analytics investments
6. For repeat business and expanding your big data footprint by adding additional use cases go back to step one.