Big banks warn that AI agents could lead to uptick in scams

While AI true believers worry about doom-and-gloom scenarios of AI destroying the world, some major financial institutions are sounding the alarm over a more grounded AI-related concern: scams.

On Tuesday, a consortium of big global banks put out a “principles paper” detailing their apprehensions regarding AI assistants as it relates to commerce. It also details principles that they would like to see followed as AI companies develop and continue to work on the technology.

Namely, the banks are concerned that AI agents empowered to act on behalf of customers will lose money to scams and fraud and, in turn, hurt merchants.

“As highly regulated financial entities, we are focused on managing risk effectively as emerging technologies arise,” reads the opening of the paper. “Consumers are unclear if AI agents will act in their interests.”

“They are concerned that AI agents may buy the wrong thing or spend too much – or even worse, lose their money to scams and fraud,”  the paper continues. “They are not sure whether they will be protected or who they ​will need to go ​to if things ⁠go wrong.”

The paper, called Building Trust in Agentic Commerce, was published by Bank of America, Capital One, ASB Bank, Commonwealth Bank of Australia, ING Group, and NatWest Group.

Agentic commerce introduces new safety risks, with potential for higher rates of scams, fraud and disputes. There may be mismatched expectations between consumers, AI agents and merchants on what product or service should be provided, or when and how, or who would be liable if an AI agent exceeds its authority. Some providers may engage in unsafe practices, including requesting consumer card details and entering them directly into websites, prioritizing payment methods with lower protections, and not complying with payment processing standards and payment scheme rules. Malicious actors may attempt new attack vectors for scams and fraud, including compromising or impersonating AI agents and merchants, and engaging in new forms of social engineering.

The groups also voiced their concern for merchants and business owners, as the rise of AI shopping chatbots can likely lead to an uptick in credit card disputes and chargebacks due to the actions of the AI agent.

The paper outlines five principles — Transparency, Safety, Privacy & Data, Choice, and Interoperability — which the banks would like to see the AI industry follow.

For example, the financial institutions believe that the AI companies should prioritize consumer and merchant safety and consent when it comes to users’ data and privacy. The banks would also like the AI companies to respect consumers’ choices in the market without locking them in with restrictions.

The paper’s release date certainly had interesting timing. It was released within 24 hours of the reported discovery of a serious zero-day vulnerability within Meta’s Muse agentic AI assistant. The paper was also dropped within hours of Amazon’s announcement that the e-commerce giant would be blocking Muse from making purchases on its platform.