5 Ways To Hybridize Predictive AI And Generative AI

The solution? Generative AI (genAI) and predictive AI can mutually enhance each other's capabilities.

GenAI struggles with reliability. For instance, although nearly three-quarters of lawyers intend to utilize genAI in their work, their AI tools experience hallucinations at least one-sixth of the time.

Predictive AI, on the other hand, faces challenges in usability. Despite its long-standing success in enhancing large-scale business operations, it only taps into a small portion of its potential because deploying it requires stakeholders to have a semi-technical understanding.

These two types of AI – essentially two categories of machine learning use cases – are poised to address each other's shortcomings. Here are five ways they can collaborate effectively.

1. Predictive Intervention For GenAI

Predictive AI can achieve what might otherwise be unattainable: fulfilling genAI's ambitious promise of autonomy – or at least a significant portion of it. By identifying cases that necessitate human intervention, an otherwise unreliable genAI system can gain the necessary trust for widespread deployment.

Consider a genAI-based question-answering system. Such systems can be reliable when limited to answering questions about a few pages of information, but their performance is questioned in more expansive systems. Suppose the system is 95% reliable, meaning users receive incorrect or problematic information 5% of the time. This often makes it unviable for deployment.

The solution lies in predictive intervention. If predictive AI identifies the 15% of cases most likely to be problematic for human review, this could reduce the rate of problematic content reaching customers to an acceptable 1%.

For more information, see this Forbes article, where I delve deeper into this approach.

The remaining four methods of integrating predictive and generative AI each assist in the reverse direction: genAI making predictive AI more user-friendly and accessible.

2. Chatbot Assistant For Predictive AI

GenAI is accessible to everyone, as it responds to human-language prompts, but predictive AI is not easily accessible to the average business user. To leverage it, a business professional requires the support of data scientists and a semi-technical understanding of how machine learning models enhance operations. Since this understanding is often lacking, most predictive AI projects fail to be deployed – even with data scientists available.

An AI chatbot can bridge this gap. Properly configured, it provides business users with a virtual, straightforward data scientist that guides the project and answers any questions about predictive AI. It acts as an assistant and thought partner, clarifying, elucidating, and suggesting, and answering endless questions without the user feeling they are bothering or asking "silly questions."

For instance, for a marketing project using predictive AI, I asked a well-prompted chatbot (powered by Anthropic’s Claude Sonnet 3 large language model) to explain the profit curve "for a 10-year-old using a story." It responded with a charming and easily-understood description of the diminishing returns faced when marketing a lemonade stand.

For more information, see this Forbes article, where I explore this use of a chatbot in greater detail.

3. Coding For Predictive AI

An interesting story. Despite being a data scientist for over 30 years, my focus on thought leadership had kept me away from hands-on practice for so long that, until recently, I had never used scikit-learn, which has become the leading open-source solution for machine learning.

However, in the genAI era, getting started was incredibly easy. I simply asked an LLM, "Write Python code to use scikit-learn to split the data into a training set and test set, train a random forest model, and then evaluate the model on the test set. For the training data, load a (local) file 'XYZ.csv'. The dependent variable is called 'isFraud'. Include clear comments on every line. Ensure your code can be used within Jupyter notebooks and include any necessary 'import' lines."

It worked. Moreover, the generated code served as a tutorial for various uses, eliminating the need for me to sift through scikit-learn documentation (which can be tedious!).

For more information, this approach will be covered in a Machine Learning Week training workshop, "Automating Building of Predictive Models: Predictive AI Generative AI," scheduled for June 5, 2025.

4. Generating Predictive Features

Since LLMs excel at processing human language – the realm of natural language processing and also known as processing unstructured data – they may outperform standard machine learning methods for certain language-intensive tasks, such as detecting misinformation or analyzing the sentiment of online reviews.

To create a proof-of-concept, we utilized a Stanford project that tested various LLMs on different benchmarks, including one that measures how often a model can determine whether a given statement is true or false. Under certain business assumptions, the resulting detection capabilities proved valuable, as detailed in this Forbes article.

More broadly, rather than serving as a complete predictive model, an LLM may be better suited for feature engineering – converting unstructured data fields into features that can be used as input for a predictive model. For example, Dataiku enables users (typically data scientists) to choose which LLM to use and what task to perform, such as sentiment analysis. Another example is Clay, which derives new model inputs from across the web using an LLM. For decades, NLP has been used to transform unstructured data into structured data that can then be utilized by standard machine learning methods. LLMs represent a more advanced form of NLP for this purpose.

5. Large Database Models

While LLMs have been making waves, another emerging AI trend is quietly gaining traction: large database models.

LDMs complement LLMs by leveraging the world's other primary data source: enterprise databases. Instead of tapping into the vast wealth of human writing such as books, documents, and the web itself—as LLMs do—LDMs utilize a company's tabular data.

Swiss Mobiliar, Switzerland's oldest private insurance company, employed LDMs to drive a predictive AI project. Their system informs sales staff of the likelihood of closing a new client, allowing them to adjust their proposed insurance quotes accordingly. The deployed system resulted in a significant increase in sales.

Swiss Mobiliar will present these results at Machine Learning Week 2025. For further details, see also my Forbes article on large database models.

Hybrid AI As An Antidote To AI Hype

Predictive AI and genAI are interdependent. Integrating the two will resolve their respective issues, expand the ecosystem of tools and approaches available to AI practitioners, and bring together what is currently a fragmented field into a more cohesive one.

Most importantly, these hybrid approaches will prioritize AI value over AI hype by focusing on project outcomes rather than treating any single technical approach as a cure-all.

In a few weeks, I will deliver a keynote address on this topic, "Five Ways to Hybridize Predictive and Generative AI," at Machine Learning Week, June 2-5, 2025, in Phoenix, AZ. In addition to my keynote, the conference will feature an entire track of sessions on how organizations are applying such hybrid approaches. You can also view the archive of a presentation on this topic that I gave at this online event.

The above is the detailed content of 5 Ways To Hybridize Predictive AI And Generative AI. For more information, please follow other related articles on the PHP Chinese website!