ML, LLM, Graphs & Market Modelling

 

 

By Ashoke Agarrwal

 

Ashoke AgarrwalNine months after the launch of ChatGPT, the hype has died down and the real work of building upon the burgeoning availability of Large Language Models (LLMs), of which ChatGPt is but one example, has begun.

 

The interest of businesses in the concept of Big Data rose exponentially in 2012. Many expected a paradigm shift in consumer marketing based on nifty analytical systems driven by Big Data. Much was expected in data-driven decision-making, personalisation and customisation, targeted advertising, sharper segmentation, predictive analytics and real-time insights. In 2015, in its report titled ‘Big Data, Analytics and the Future of Marketing and Sales’,  McKinsey laid out the expectations.

 

In 2023, the future is different from what McKinsey expected. The paradigm certainly shifted for Google and Meta, who cornered the advertising market based on a humongous amount of real-time Big data and state-of-the-art analytical engines.

 

Change also happened for brands that marketed and sold products and services to B2C and B2B markets. However, the difference had little to do with their use of Big Data and advanced analytical systems. It was in the emergence of the digital universe as a product, go-to-market and communication platform.

 

Dig a little, and you will find that the thesis that brands under-exploited the opportunity that Big Data presented to them mainly because they used legacy databases and ERP systems from the likes of SAP, Oracle and Salesforces to collect and store their Big Data. As a result, while many created special teams to mine, warehouse and analyse Big Data, most crucial business, marketing and sales decisions continued to be based on traditional business analysis and market research.

 

Meanwhile, Google and Meta (then Facebook) invested in an analytical system that eschewed the rigidity of the traditional IT-age relational databases – essentially tables with rigid rows and columns. In a seminal decision that presaged the age of AI, they decided to populate their databases in graphs – a network of nodes with many properties with multiple links to other nodes, with each link specifying a kind of relationship with a property of the originating node and a property of the linking node.

 

Such a database structure allowed for:

:: Fast and more varied analysis

:: More immediate additions and reconfiguration of the database as conditions change

:: Better situational analysis and insight discovery through “what-if” analysis that changed node and link configurations

 

Based on their ever-expanding, ever-enriched graphs and the use of Machine Learning (ML) driven near-real-time analysis, Google and Meta created a mighty advertising service with the confidence to offer brands a pay-per-click service. As a result, brands willy-nilly outsourced the harnessing of the real opportunity of the digital and Big Data age to Google, Meta and other digital ad exchanges that sprang up.

 

It is no surprise that Google and Facebook are among the most advanced AI players today, including in the field of LLMs with Google’s Bard and Facebook’s Llama. The essential process that powers LLMs is that they parse large storehouses of text into triples of subjects, objects and predicates and then model them into graphs with the nodes consisting of subjects and objects and the links signifying relationships in the form of predicates.

 

As Google’s and Meta’s LLMs improved power, it turbocharged their graph databases, allowing them to automate the process of incorporating unstructured into their graphs. Perhaps left to themselves, Google and Meta would not have exposed their LLMs to the public as they have done so now but kept it themselves as a background technology powering customer-facing applications. Instead, OpenAI and ChatGPT forced their hand.

 

The resultant hype around ChatGPT has kickstarted the age of AI, with the world at large now seriously scrambling to incorporate AI into businesses, Governments, schools, hospitals and wars. Publications like The Economist and others have started ranking Fortune 500 companies based on the potential competitive advantages/disadvantages that AI can deliver to them!

 

How will businesses in general and marketers in particular respond to the new horizons of AI?

 

The powers of MI, Graph Theory and Advanced Modelling will create a new platform that will change how businesses, if not entire societies, are run. Just as the arrival of digital media changed economies at the core, Graphs with Big Data inputs from structured and unstructured sources (processed through LLMs) will create dynamic market models that will change how businesses are structured, with business and marketing strategy being almost wholly automated with human inputs needed only at the highest meta-strategy level. The shift will be paradigmatic enough for the world to label the resultant business order as the Nth Industrial Revolution, a sub-set of the First AI Revolution that will redefine human society.

 

The question is whether individual businesses will seize this opportunity, invest and build proprietary dynamic models that will run their companies, or will they, once again, as they did with the digital revolution, outsource it to the next Google or Meta?

 

The stakes this time are higher. Not only will the businesses themselves be more beholden to those who own and run the models, but in the process, society will create AI-driven behemoths that will be the first step to the dysfunctional system that those who fear AI imagine.

 

Therefore, it is incumbent on business leaders and thinkers to pay close attention to this evolving opportunity and invest all that is needed to harness it before it becomes an insurmountable challenge.