By Ashoke Agarrwal
I have been a purveyor and user of market research for a little more than four decades now. At 26, I co-founded Francis Kanoi (Kanoi is my family name). I worked with my partner, the legendary Francis Xavier, in designing and executing what was then the most extensive market research study in the country. An all-India, 70,000 sample size study using the Bass Epidemiological Model to forecast the demand for household durables like refrigerators and TVs. The study found 33 subscribers in the first year, and four decades down the line is still going strong.
In the 90s, I directed research and planning at Ulka, with perhaps the strongest strategy team in the advertising world. Anil Kapoor, the MD and his handpicked team, genuinely believed in accessing and acting upon consumer research and insights. That was one of the reasons why Ulka, those days, sat at the high table in strategy sessions with clients’ marketing teams that went beyond advertising and creative strategy.
In the new millennium, I run, in partnership with other veterans of the marketing and marketing services industry, a marketing advisory service with market research as a core input.
Over the decades, market research has been slow to change. However, I have noticed an incipient change in this otherwise conservative industry over the past few years that will snowball into a paradigm shift over the coming decade.
Shifts in socio-economic and technological forces are driving this change.
The following key trends are powering the shift:
Hyper-individualisation: Sampling lies at the heart of traditional market research. At its most fundamental, sampling methodology is based on an assumption. Sampling assumes that a market divides into a finite number of segments usually defined by simple demographic parameters like age, gender, education, income and pop strata. Further, consumers in each segment behave homogeneously concerning their usage and attitudes towards the focus product or service category. On the strength of this assumption, traditional market research makes do with samples in the hundred and the thousands to assess and forecast markets with tens and hundreds of millions of entities. In essence, this ability is at the heart of the business model and viability of the market research industry. Unfortunately, this assumption has started failing, and doubts have started creeping into traditional market research methods.
The spectacular failings of one aspect of market research that gets frenzied media attention – election forecast polling – have further fuelled these doubts. The core assumption of homogenous demographic segments no longer holds because of the hyper-individualisation of opinions, attitudes, behaviour and lifestyles that the emergence of the digital world has wrought. In the old world, socio-economic forces, including mass media, lead to conformity among people of the same age, gender and social class groups. Today, with a proliferation of career and lifestyle choices, ever-increasing choices in products and services, and the ability to create a curated stream of media-based, every individual is a segment of one.
Big Data & Machine Learning: If market research is to survive, it will need to replace the traditional demographic segments-based sampling (this applies to both quantitative and qualitative market research). To my mind, the answer lies in the realm of Big Data coupled with machine learning. All commercial organisations will need to invest in the collection, warehousing, and analysis of Big Data covering all aspects of their business, including consumers. Big Data and its informed use are becoming critical to any organisation’s competitive strength.
In the age of hyper-individualisation, factors other than demographics drive attitudes, behaviour and lifestyles. For example, a twenty-five-year-old today might have more commonalities in attitudes and lifestyles with a forty-year-old than with any of the individual’s age cohorts. When applied to Big Data on consumers, machine learning can unearth such patterns and build a market segmentation map beyond demographics to actual behaviour. This shift gives back market research its predictive power. For example, in micro-lending, a growing fintech area, such analysis has led to factors like the day of the week an individual applies for a loan being an indicator of his creditworthiness.
There are many sources of Big Data. Governments and social organisations put out public sources. In addition, companies and organizations collect private first-party data from their operations. In fact, for an organization like Facebook and Google, this data is the lifeblood of their business. Finally, a fast-emerging source of Big Data is voluntary disclosures by individuals. In industry parlance, such Big Data collected through voluntary disclosures by individuals is called Zero-Party Data. Heightened privacy concerns and more stringent regulations will herald a new age of Zero-Party Data. I believe this will become the backbone of a new paradigm in market research.
Zero-Party Data, Data Vaults and Active Research: The market research agency of tomorrow will be a collector, storer and analyser of Zero-Party Big Data. The agency will partner with every individual whose data it collects and stores. The collection will be permission-based. The usage will be transparent, and, using blockchain technology, the individual will get a share of all revenues generated from using their data.
The agency will invest in machine learning to unearth patterns and, based on these patterns and directed probes will run market research projects and forecasts for its clients. In addition, the agency will anonymise all data used in such research projects.
Beyond market research, Zero-Party Data and Data Vaults opens up a whole new arena to marketing. I call this arena Active Research. With the individual’s permission, the agency opens up a one-on-one conversation between the individual and brands to unearth needs and desires, customize and test products and services and make relevant offers. The agency mediates and adds value by using state-of-the-art probes and is informed by the ongoing learnings across the database. Since the agency’s Zero-Party Big data covers a substantial part of the market, Active Research becomes a real-world test marketing option and even a viable Route-to-Market (RTM).
Beyond Zero-Party Big Data, Data Vaults and Active Research, other tech-driven significant shifts are likely to drive change in market research over the coming decades. The two technologies that drive this change will be
:: The emergence of NLP-driven custom-framework-based fourth-generation search engines to generate curated reports will transform secondary research.
:: Research in the metaverse will take ethnographic research to the next level.
I hope to discuss the above two horizons in market research in later columns.
Ashoke Agarrwal is a veteran advertising professional with around four decades in advertising and marketing services. Agarrwal, a chemical engineer from IIT Mumbai and a postgraduate from IIM Bangalore, is a pro-entrepreneur with past and current ventures in market research, advertising, CGI, e-learning and brand consultancy. He writes on MxMIndia every other Thursday. His views here are personal.
Last week I underwent some major dental work. Dealing with excruciating pain and dizziness induced by a bouquet of medicines, I took to Netflix for comfort. Partly my objective was to try and doze off, hence I deliberately chose not so addictive content. And I decided to watch the much acclaimed (read slammed) Ranbir Kapoor starrer, Animal.
A certain TVC by a service brand called Cred initialied random thoughts. And you can squarely blame it for this piece.