As Cora, the chatbot developed by RBS, evolves from its text-based origins into a life-like “digital human”, the insight team at the UK bank is on a parallel track searching for appropriate metrics to measure “her” effectiveness.
Ian Goulding, insight lead at the bank, addressed this topic at the recent MRS Financial Services Research conference in London, where he asked: “How do we measure our new channels and new touchpoints against things that are relevant to us internally, but also to customers?”
That task has to be considered in a wider context than just the banking category, he added, and is further complicated by the fact that chatbots are still relatively new in financial services.
“Customers don’t have a massive battery of experience or expectations around them, so that’s something we need to consider when we’re creating metrics,” he said. (For more, read WARC’s report: How do you measure a chatbot?)
The bank’s research in this area highlighted how customers are currently more familiar with chatbots in the retail space and how expectations are set by the category.
“In financial services, it’s a much wider set of parameters,” Goulding noted – not only are there many more things a customer might want to talk about but these will also frequently involve personal financial details, so expectations are going to be higher and financial services bots will be assessed far more critically than those in the retail space.
But there are some common areas to consider, including understanding and the need to make what a bot says simple and concise. Tone needs to be nuanced with “some engaging aspects, so people feel like ‘yes, I do actually want to engage with it’”.
And “humanity” needs to be pitched right. Many users anthropomorphise the experience but where they might accept an error in a retail context, they won’t when it’s financial services.
Some of these three areas are more easily measured than others – understanding, for example. Tone is more difficult, humanity even more so. At the same time, customer expectations are constantly changing as more people come in contact with bots and as machine learning makes bots better at what they do.
“As a research team we need to work closely with our tech teams, with our digital teams to get a sense of what Cora can and can’t do on almost a six-monthly basis,” said Goulding.
”If we don’t do that, we can’t understand the customer expectations, and we can’t measure the chatbot properly.”
Ian Goulding, insight lead at the bank, addressed this topic at the recent MRS Financial Services Research conference in London, where he asked: “How do we measure our new channels and new touchpoints against things that are relevant to us internally, but also to customers?”
That task has to be considered in a wider context than just the banking category, he added, and is further complicated by the fact that chatbots are still relatively new in financial services.
“Customers don’t have a massive battery of experience or expectations around them, so that’s something we need to consider when we’re creating metrics,” he said. (For more, read WARC’s report: How do you measure a chatbot?)
The bank’s research in this area highlighted how customers are currently more familiar with chatbots in the retail space and how expectations are set by the category.
“In financial services, it’s a much wider set of parameters,” Goulding noted – not only are there many more things a customer might want to talk about but these will also frequently involve personal financial details, so expectations are going to be higher and financial services bots will be assessed far more critically than those in the retail space.
But there are some common areas to consider, including understanding and the need to make what a bot says simple and concise. Tone needs to be nuanced with “some engaging aspects, so people feel like ‘yes, I do actually want to engage with it’”.
And “humanity” needs to be pitched right. Many users anthropomorphise the experience but where they might accept an error in a retail context, they won’t when it’s financial services.
Some of these three areas are more easily measured than others – understanding, for example. Tone is more difficult, humanity even more so. At the same time, customer expectations are constantly changing as more people come in contact with bots and as machine learning makes bots better at what they do.
“As a research team we need to work closely with our tech teams, with our digital teams to get a sense of what Cora can and can’t do on almost a six-monthly basis,” said Goulding.
”If we don’t do that, we can’t understand the customer expectations, and we can’t measure the chatbot properly.”
Sourced from WARC
