Big Data Needs Big Connectivity

The discussion around #bigdata continues to gather momentum. Government departments, notably the NHS, and other organisations are making their databases available for open source projects to take the initiative, and crunch these vast data sets into consumable content.

Like many new digitally-themed trends, this brave new world isn’t as new as some would have you believe. Data warehousing, the process of extracting online, real-time data, has been around for many years. Here, the data is cleaned up, reorganised and aggregated for better reporting, and then delivered to a separate system for user analysis. Enterprise level tools to manage this remain big business; the only difference now is that there is vastly more data generated than before.

 Yet once aspect of Big Data that isn’t discussed enough is its potential connectivity. To really get the insight you need, your data needs to be combined with other sources to enrich its relevance and quality. Take closed-loop marketing. A discussion earlier this year amongst some colleagues drew some broad conclusions that in Europe, hardly any of the largest pharmaceutical companies have got this fully implemented. Certainly, there is some quality research to support this theory, with, as expected, technology being the major obstacle.

 It’s easy to understand why. With the electronic sales aid, or eDetail, mostly developed under the Sales and Marketing remit, and the underlying company sales and invoicing systems sitting with Operations, ensuring the right people are connected will be difficult. It could create the large, expensive IT projects that so many large companies, plus the NHS itself, are trying to avoid.

 Closed-loop marketing works if the eDetail usage stats are linked to sales figures. If, for example, you’d detailed a product to a physician six times in a year, you’d want to see the number of sales expressed as units and revenue for that customer. Ideally, there would be some growth of course. Yet the challenge is connecting these data sets together. It’s much more involved than simply extending the eDetail analytics to record the customer’s details ahead of the call. That data has to be transmitted and connected to the sales/invoicing system.

These challenges are not insurmountable of course. They just need careful planning. The exciting opportunity with big data is that further external sources could be applied to your internal data. Imagine applying Hospital Episode Statistics data. You could potentially see trends, at either national or local levels, showing an increase of patient admissions or episodes in a therapy area linked to sales. Then an economic data set could be added, maybe showing a link between austerity, well-being and prescriptions. Big data projects will succeed with connectivity.