Overcoming hurdles to AI chatbots in customer service

We all face the choice in digital customer service: Talk to a live agent? Or try the company’s AI-chatbot, a computer program that uses artificial intelligence and natural language processing to troubleshoot and resolve issues?

For businesses, AI-chatbot technology can mean substantial savings, notes Evgeny Kagan, an assistant professor at Johns Hopkins Carey Business School whose work focuses on operations management and business analytics.

“Chatbots are essentially free once you have them up and running. They can handle an almost limitless number of customers,” says Kagan. He points to market research showing that the retail banking industry alone could save $8 billion a year through widespread implementation of the technology.

But even though it’s cost-effective and the technology has advanced, there are still hurdles to adoption. So, Kagan and two Carey colleagues recently embarked on a study to find out why.

“Most research to date has focused on questions related to chatbot design, such as whether anthropomorphism [human likeness] helps or hurts adoption,” Kagan notes. “However, as ChatGPT and other AI tool performance continues to advance, appearance may soon become irrelevant. We wanted to study chatbot adoption from a broader service design perspective: What drives customer willingness to engage with a chatbot? And what are some simple remedies businesses can take to increase chatbot uptake?”

A nudge to save time

Kagan, who will teach at Johns Hopkins' new 555 Penn location in Washington, D.C., this fall, notes that his three-person research team was uniquely equipped to tackle these questions. Carey School Professor Maqbool Dada brings a wealth of experience in mathematical modeling aimed at optimizing systems for performance. “His guidance throughout the project has been crucial,” Kagan says. Assistant Professor Brett Hathaway, who has since left Carey to join the faculty of Brigham Young University’s Marriott School of Business, spent the first part of his career working in — and later managing — a banking call center.

“That was unique experience we wanted to leverage. Not many people know what happens in a call center from the inside,” says Kagan, whose own focus on behavioral economics and decision-making nicely rounded out the team.

Through an exploratory survey of online users of customer service chatbot technology, followed up by three experiments, the researchers arrived at some valuable conclusions, which they share in a working paper, “AI Chatbots in Customer Service: Adoption Hurdles and Simple Remedies.”

The trio found one common hurdle: algorithm aversion. It’s the term for consumers’ innate preference to interact with other humans rather than an algorithm. But Kagan and team found it could be fairly easy for businesses to overcome.

“Some people are distrustful of robots in general and of the fragmented ‘conversations’ that occur during AI encounters. They prefer the more continuous interaction that comes with a human conversation,” says Kagan. “We found that to get around algorithm aversion, chatbot operators can point to the time-saving advantages of using the chatbot.”

For example, he says, the company could note that the average wait time to talk to a customer service representative is 25 minutes, while the chatbot is available immediately — and can solve 60 percent of all queries.

“Chatbot performance information is something that companies can easily pull from their data,” Kagan says. “If the performance data look favorable, companies can use it as a ‘nudge’”— communicating a clear, tangible time-saving advantage to customers, who can then make informed decisions about whether to use chatbots or not.

With such a nudge in place, the researchers found in their experimental models that the impact of algorithm aversion almost disappears. “That’s good news for companies,” he says.

Jumping the queue

More difficult to overcome, the team found, is what’s known as “gatekeeper aversion.”

Gatekeepers, who are the least expensive first-line responders to customer queries, protect higher-paid employees’ time. Gatekeepers are crucial to the customer service industry — and they can be the source of endless frustration for consumers, who get caught in a seemingly continual loop of time-consuming phone transfers that don’t address or resolve their issue.

“People react very negatively to that,” says Kagan. “We hate being transferred, whether we’re dealing with a human or a robot.”

In their chatbot experiments, Kagan and his colleagues found that gatekeeper aversion is very persistent and hard to overcome. “From the outset, users perceive the risk of chatbot failure to be high, and they don’t want to engage.”

While this hurdle is more difficult to handle, the team did find one potentially promising solution: “priority queues.” The idea here is that the user is promised at the outset that if the chatbot fails to resolve their issue, they will be expedited to a live agent. “Essentially you’re telling them they can jump the line and get priority access to a human if they try the chatbot first,” Kagan says.

Using experimental data, he and his colleagues found that by implementing both an expected waiting time “nudge” and “queue priority rules,” companies using chatbot technology could reduce staffing costs by up to 22 percent.

While some companies have begun to adopt these strategies, Kagan notes, implementation is still at a very basic stage. Nudging, for example, is typically used in a vague way. “A company’s website might note that customer service representatives are only available during weekdays from 9 to 5, while the chatbot is available to answer your questions 24/7,” Kagan says. “Adding information about chatbot performance and speed would make for a more impactful nudge.”

As technology advances and consumers become ever more comfortable using AI-chatbots, “customer service companies will experience a major transition in their business model,” Kagan predicts, making insights like those gained by his team ever more valuable.