Wall Street firms, including Goldman Sachs Group Inc., Citigroup Inc., and JPMorgan Chase & Co., are raising alarms over the emerging risks associated with the use of artificial intelligence in finance. These concerns, highlighted in their latest annual reports, encompass potential software hallucinations, the impact on employee morale, and the increasing sophistication of cybercriminals.

These firms have noted several risks tied to AI, including the potential for flawed AI models and the challenge of keeping up with rapid technological advancements. JPMorgan has expressed concerns about AI-driven "workforce displacement" potentially affecting staff morale and retention, while also increasing competition for skilled tech employees, as per their 2024 10-K report.

Increased AI adoption poses further risks of cyberattacks and misuse. "Having those right governing mechanisms in place to ensure that AI is being deployed in a way that's safe, fair and secure — that simply cannot be overlooked," Ben Shorten, Accenture Plc's lead for finance, risk and compliance for banking and capital markets in North America, mentioned in a recent interview.

Citigroup's report outlines the dangers of rolling out generative AI, which could produce "ineffective, inadequate or faulty" results that may impact its reputation and operations negatively. This reflects a broader industry concern over maintaining high data quality to avoid the pitfalls of outdated or biased data sets.

With the EU's Artificial Intelligence Act now in effect, introducing strict rules for AI usage, banks face a new regulatory landscape. This act significantly shapes how AI systems are marketed and used within the EU, impacting many U.S. banks with operations in the region. The outlook in the U.S. remains uncertain, adding another layer of complexity to the banks' operational environment.

As part of their strategic response, banks are not only enhancing their internal AI capabilities but also integrating third-party technologies. Morgan Stanley, for example, is employing AI tools like a ChatGPT-like interface to automate mundane tasks, thereby improving efficiency and data privacy, especially in remote work settings, according to their latest annual report.

AI's potential and pitfalls are starkly outlined by JPMorgan's CEO, Jamie Dimon, who compares AI's impact to the steam engine, suggesting it could "augment virtually every job."