Chip Somodevilla/Getty Images
- We asked ChatGPT and Claude to judge President Joe Biden's cognition based on talks with Robert Hur.
- Special counsel Hur interviewed Biden over his handling of classified documents.
- After analyzing Hur's interview transcript, the AI chatbots said the US president sounded fine.
President Joe Biden's cognitive abilities may be better than some say they are — at least according to two AI chatbots.
The 81-year-old's mental acuity has returned to the spotlight after special counsel Robert Hur investigated Biden's handling of classified documents. Hur ultimately recommended in February that no charges be made, in part because the president is an "elderly man with a poor memory" who would elicit sympathy from the jury. One factor that played into the report was Biden forgetting when he was vice president and when his son Beau died.
However, a full transcript of the president's five-hour interview with Hur, which dropped on Tuesday, has raised further questions. Some who analyzed it said Hur's claims about Biden's memory issues were overstated. He seemed to be able to recall the day and month his son Beau died, and recognized his errors when he misstated which years he served as vice president.
To get some additional opinions, Business Insider uploaded the transcript into OpenAI's ChatGPT and Anthropic's Claude Pro. Using the same prompts, we asked the AI chatbots to assess Biden's cognitive abilities and age-related issues based on factors like clarity of thought, coherence, language use, memory recall, and "any other relevant indicators present in the text."
These chatbots are run on large language models, or LLMs, computer programs trained on troves of web data that generate human-like responses.
And both chatbots' verdicts were that Biden's cognitive abilities are just fine.
"The interview transcripts reflect that President Biden was able to communicate effectively, recall details from his tenure, and engage in logical reasoning during the interview," ChatGPT wrote. "These aspects are essential indicators of cognitive abilities and suggest that he could perform cognitively demanding tasks during the interview."
Claude responded similarly, though noted that age may have something to do with Biden's responses.
"President Biden demonstrates mostly intact cognitive abilities, with clear and coherent thinking, appropriate language use, and the ability to recall relevant information," Claude wrote. "While there are some age-related changes noted, they do not appear to significantly impact his overall cognitive functioning in the context of this interview."
It's important to note that ChatGPT's and Claude's claims should be taken with a grain of salt.
Launched in November 2022, ChatGPT has been found to produce bias and generate factual errors, as do other AI bots. An August study found that ChatGPT generated false information when asked to design cancer treatment plans. In January 2023, tech news outlet CNET was forced to issue corrections after an AI-generated article it published contained multiple inaccuracies. Anthropic launched Claude in March 2023; in a recent article, it said Claude can "occasionally produce responses that are incorrect or misleading."
The AI chatbots broke down the reasoning behind their conclusions on the Hur talks.
When it came to Biden's memory, ChatGPT found that he displayed memory recall by describing interactions and processes related to handling classified documents with context. Claude said that was reflected by his ability to provide specific details and anecdotes from his time as vice president.
The chatbots also pointed out Biden's seemingly good verbal reasoning skills. ChatGPT said the president demonstrated his ability to "process information" and "engage in complex discussions" by answering clarifications and follow-up questions. Claude wrote he "generally demonstrates clear and coherent thinking" and can "largely stay on topic" despite occasional tangents.
To summarise, "Based on the transcripts, President Biden shows no significant difficulty in articulating his thoughts, recalling past events, and engaging in detailed discussions on various topics, all of which are aspects of cognitive functioning," ChatGPT wrote. Claude came to a similar conclusion.
The chatbots didn't agree on everything, though. When asked to guess the President's age, ChatGPT said it's "not feasible" to guess how old he is based on the transcript alone. Claude, however, correctly answered the question based on previous leaders' careers.
"Considering the average age of US Presidents and the timeline of events referenced in the transcript, it is reasonable to estimate that President Biden is likely in his late 70s or early 80s," Claude concluded.
The President's office didn't immediately respond to a request for comment from BI before publication.
Axel Springer, Business Insider's parent company, has a global deal to allow OpenAI to train its models on its media brands' reporting.