Google/BI
- Google's AI Overviews have dropped off significantly in recent months, per a new study.
- The company said in May it would scale back the feature, which was generating some odd results.
- Google's AI Overviews have gotten shorter in length, too.
Google's AI Overviews in search results have dropped off significantly in recent months, according to a new study. The search giant is rethinking how it uses artificial intelligence in its biggest product.
Researchers at SE Ranking have been closely tracking how often Google's AI-generated responses appear in results and the sources the tool cites.
In a test of around 100,000 searches on Google conducted in July, just 7.47% produced an AI Overview, which was a 1.24% drop from the previous month, the researchers found. That's also a huge fall-off from February when Google was beta testing AI Overviews ahead of a full public release, where SE Ranking found that 64% of queries returned an AI response.
The research concluded that the average length of each AI Overview had dropped to 2,633 characters, almost a 40% decrease from the month prior. In terms of who's getting cited the most, Forbes, Business Insider, and Entrepreneur came out as the top-linked media outlets in the July study.
"We designed AI Overviews to appear for queries where they're helpful and provide value beyond existing features on the results page, and they continue to show for a large number of searches," a Google spokesperson said in a statement. "As we've shared, we're continuing to refine when and how we show AI Overviews so they're as useful as possible, including a number of recent technical updates to improve response quality."
The spokesperson said the SE Ranking research results did not reflect the company's own findings.
Google announced at the end of May that it would scale back AI Overviews after users complained they were getting incorrect and sometimes strange answers, such as telling people to eat glue. The company had only made the feature available to the public two weeks prior.
Liz Reid, Google's head of search, said at the time that Google "didn't get it right" and that it would refine and improve the feature, including more restrictions on user-generated content that could give "misleading" advice.
It was the second blunder in a double-whammy for the search giant, which had also scaled back its Gemini image generator following fierce backlash over how the tool was generating historically inaccurate images of people of color.
The authors of the SE Rankings study acknowledged that AI Overviews are changing fast, but the data gives us a rare look into how Google is moving more cautiously with AI results.
SE Rankings also found that as of July 11, AI Overviews cited no Reddit or Quora Links. The two websites have rocketed in prominence in Google Search after the company said it would promote more online forums in results.
However, some niche topics saw an increase in search results featuring AI Overviews. Queries pertaining to relationships had the most, with 40.64% of related keywords triggering AI Overviews in results.
Are you a current or former Google employee with more insight to share? Got a tip? You can contact this reporter securely on Signal at hughlangley.01 or email at hlangley@businessinsider.com.