pharmaphorumTowards the end of 2015, I was very flattered to be invited to provide some commentary for an upcoming publication on big data in the pharmaceutical industry, being developed by (not surprisingly) a company specialising in commercial data analytics.

I provided my thoughts on some simple questions – key highlights for the pharmaceutical industry in 2014, plus predictions and hopes for 2015, then promptly forgot about it in the rush to deliver projects before the end of the year. Last week, it was published and I was rather pleasantly surprised at how it turned out. Take a look for yourself (you will need to provide some details but it’s otherwise free).

Firstly, it’s nice to find myself in such esteemed company – there are some pretty well recognised industry experts in there. But secondly, and most importantly, it’s full of personal insights from people on their hopes for the future of pharma and hints at where big data might fit into this, rather than simply featuring some big data analysis (which I am sure would have been easy to do).

The relevance of this to healthcare is enormous.

Every day I hear commentary about how exciting big data is in all its glory and the plentiful applications it has in helping improve people’s health. And it does. For example, analysis of social media can reveal fascinating insights about perceptions of different diseases, the main challenges patients (and their support networks) face in managing them, how people are responding to using (or not) medicines in the real world and, of course, who the key influencers are in any given area.

Likewise, burgeoning sources of real-world data have the capability to revolutionise the way we approach prevention and management of particular conditions, in addition to the opportunity to radically reshape the drug development process in a more efficient direction. Think about the combination of personal genetic data, such as that collected by 23andMe, combined with real-world outcomes for patients with defined demographics taking a particular drug treatment and you can start to see the end of the current (and often inefficient) approach based on large controlled clinical trials.

I firmly believe that over the next ten years we will see research investment shift more and more from the clinic to the real world as devices and big data analytical technologies allow for a more informed view of the impact new interventions are having in more realistic patient populations.

But that’s not the point.

Amidst all this big data and excitement over the depths of analysis it can provide, it’s important to remember the value of smaller scale, more qualitative research – speaking to individuals to hear their thoughts directly. Analysis of social media data will identify some high level insights into how patients are managing (or not) a particular condition, but speak to a real patient and they will tell you what it is really like to live with that condition every day, the highs and lows of having to take medicines, the impact on their families and what’s really missing, whether it’s medical, informational or emotional, from the mix.

It is these small insights that can be missed in the mass of big data.

The combination of big data sources and modern analytical technologies is incredibly useful. It will tell us things about health that will deliver real benefit in the future in maintaining wellness or treating disease. But don’t hide behind those dashboards all the time – keep speaking to doctors, nurses, patients, caregivers and everyone else living it every day, because the benefit of their personal experience is incalculable.