CFOTO/Future Publishing via Getty Images
- ChatGPT's market and investing knowledge is spotty, and the bot isn't a good tool for investors yet, DataTrek said.
- The firm asked the chatbot seven questions related to markets, and some of its answers were vague or incorrect.
- Still, observers point to ChatGPT's rapid development, with the latest version released last month.
Open AI's ChatGPT still has major holes in its knowledge of markets and investing, and it's going to be awhile before the hyped artificial intelligence bot can be of use to investors, according to DataTrek.
In a note on Friday, the research firm tested the capabilities of the trendy AI tool, asking it seven questions related to markets and investing. The chatbot refused to answer four questions, and it incorrectly or vaguely answered two other questions, the firm said.
For some questions – such as when it was asked why small cap stocks have suffered in 2023 – its answer was vague and could be applied to other areas of the market.
At times, its answers were completely wrong. When asked for the performance of the S&P 500 last year, ChatGPT said the benchmark stock index had a net return of 26.9%, though the index actually plunged 20% last year amid rising inflation and the Fed's aggressive rate hikes.
The bot also refused to give its opinion on specific stocks, claiming it couldn't predict the future with certainty. That's the same response ChatGPT gave when Insider asked for its 2023 market outlook, where it declined to give an opinion and merely said stock performance would be affected by economic conditions.
"Generative AI has a long way to go before proving useful in the financial services industry or as a tool for individual investors. ChatGPT does not make recommendations, but rather offers generic responses you might also find in an economic textbook and that won't get OpenAI in hot water with regulators," DataTrek co-founder Jessica Rabe said.
ChatGPT sparked a viral craze in markets earlier this year, with a myriad of names soaring on even vague associations with the emerging technology.
The chatbot could potentially serve as an assistant in portfolio management, one academic paper said, though researchers noted ChatGPT was unlikely to "prophet" anything in the financial realm. Investment managers also don't seem to be worried about the bot taking over their jobs, as it only seems to be capable of surface level thinking for now.
However, observers are quick to point to the rapid development of AI. In March, Open AI released the newest version of the bot, GPT-4, which the company claims is capable of "advanced reasoning" and "creativity."
Outside of finance, the technology is already making headway in highly complex fields like medicine, passing a licensing exam with flying colors and easily diagnosing a rare illness.