The regulator said this could include “providing financial guidance, tailored explanations of financial concepts, or agent-based chatbots for resolving complex consumer queries”.
It tested two large language models (LLMs), similar to ChatGPT, to explore how they could be used and any limitations.
The research paper said AI had the potential to “drive inclusive innovation and growth in financial services through improved consumer support”, but needed “adequate testing” to ensure mitigations were considered.
The regulator said: “The FCA’s approach to the use of AI by the firms we regulate is outcomes-focused, and therefore, testing – utilising a variety of methods – can be an important way for firms to understand the outcomes their consumers are receiving.
“This can help them to identify potential issues and provide an inclusive approach, which can support consumer trust. Trust contributes to positive interactions with consumers, and can lead to better consumer outcomes where it promotes, for example, saving.”
What mortgage and protection advisers should take from the FCA’s AI stance
Sponsored by Sesame Bankhall Group
It added that the research was to spark debate and contribute to academic discussion, and was not a form of guidance or direction, nor does it set out expectations of how firms should manage AI risks.
Engaging with AI at the right time
The FCA found that LLMs did seem able to simplify concepts, but testing their outputs still required human and automated evaluation.
It also found that how and where an LLM was used in a customer journey could impact outcomes such as comprehension and engagement. This was tested with a simulation of the customer journey when choosing a savings account.
The findings from the test group suggested that adding a chatbot onto the Q&A guidance page did not help participants choose the right account, suggesting how information was presented to a consumer might impact how effective it is.
The engagement with the chatbot was found to be so low that even if the information was useful, consumers did not use it. However, those who did engage with the chatbot on the Q&A page were still less likely to choose the right product, again suggesting this was down to how the information was presented.
Consumers want more automated processes
Lastly, the research suggested many consumers wanted some form of automated support.
The research participants said they were interested in using AI to compare products, understand financial guidance, and seek personalised advice.
The FCA concluded that exposing people to an LLM chatbot meant they were more likely to want to use AI in the future when making financial decisions.
Going forward, the regulator will conduct pilots with firms of AI models and said it wanted to better understand how LLMs can be designed so the full benefits are realised.
