In spite of a decade’s long investment in its cognitive capabilities, IBM Watson seldom comes up in the same sentence as ChatGPT, Bard, Claude, LaMDA or a dozen or so other efforts to leverage large language models (LLMs) for automating customer service. The rollout of Watsonx.ai may bring IBM back into the discussion.

Given the amount of data companies have about their customers, products, and workflows, we wrote in a previous article that companies should consider taking the stance: “You Are Your Own LLM.” But taking a DIY approach to implementing LLMs can be difficult.

Three Methods to “Be Your Own LLM”

As described in a recent Harvard Business Review article, there are 3 basic approaches to ensuring generative AI technologies understand your business:

1. Training a custom LLM from scratch – expensive and requires specialized ML skills
2. Fine-tuning a foundation model with enterprise data – complex but more feasible
3. Using retrieval-augmented generation (RAG) to query vector databases that contain your corporate data – limits customization

Each approach has its own level of complexity, expense and risk. IBM’s new service offering militates toward method number 2.

Fine-Tuning a Foundational Model on Your Company’s Data

With Watsonx.ai, method number 2 (described above) becomes more attainable for organizations wishing to ensure an AI-powered customer assistant is specifically trained on company data. IBM’s offering includes Watsonx.ai, an automated platform designed to simplify fine-tuning of foundation large language models (LLMs) for business applications, including customer service. It provides a library of pretrained open-source models which IBM selected from Hugging Face. It also offers datasets and annotation tools to prepare enterprise data for fine-tuning these models. The platform handles data preprocessing and cleaning to prepare enterprise datasets. It also automates hyperparameter optimization to fine-tune the model for optimal performance.

The Prompt Lab interface is conversational in nature. It enables guiding model training through demonstrative examples instead of hard-coded supervision. Users can experiment with prompting techniques like in-context learning.

For customer service, Watsonx.ai could be leveraged to fine-tune and adapt models to company-specific terminology, product knowledge, conversation workflows, and language. This training would result in highly customized and effective conversational agents.

Watsonx.ai manages the computational resources required for data-intensive fine-tuning on many GPUs. It monitors for training convergence and performance. Fine-tuned models can be deployed through the platform for inference via APIs. It enables continuous retraining on new data for keeping models up-to-date.

Watsonx.ai is a Significant Improvement to Pre-trained Models

Being your own LLM can require significant effort, depending on the strategy you select. By providing pre-built models, automated fine-tuning tools, and interfaces for prompting, Watsonx.ai aims to reduce the ML expertise required for customizing foundation models using the “fine-tuning” approach. Watsonx.ai is a significant re-tooling and upgrade to the Watson Assistant services Derek Top described in this post. Rather than relying on an existing library of pre-trained conversation models for specific verticals, Watsonx.ai is tailored to fine-tune existing LLMs to suit specific business needs like customer service conversations. Efficacy still depends on data quantity and relevance. But this will no longer be the exclusive domain of software engineers or user interface developers. Watsonx.ai aims to make this approach for customizing LLM’s more accessible to enterprises.

Categories: Intelligent Assistants