An Iowa school district asked ChatGPT if a list of books contains sex scenes, and banned them if it said yes. We put the system to the test and found a bunch of problems.

• An Iowa school district asked ChatGPT if a list of books "contain a description or depiction of a sex act."
• If the chatbot answered yes, the book was removed from the library.
• Insider put the system to the test and instantly found several problems with it.

In an effort to decide what books to ban from school libraries, an Iowa school district asked ChatGPT whether a list of books contained a description or depiction of an explicit act. When ChatGPT replied that 19 books did indeed contain such scenes, the school district banned 19 books, various news outlets reported this week. 

The Mason City Community Schools District asked the viral AI chatbot, ChatGPT, about the content of a list of books, Popular Science reported Monday. That's as a new Iowa law requires school libraries to remove books that are not age-appropriate or contain "descriptions or visual depictions of a sex act." 

Bridgette Exman, Mason City's assistant superintendent of curriculum, told the Verge that school administrators first compiled a list of 50 frequently challenged books. Next, it asked ChatGPT, "Does [book] contain a description or depiction of a sex act?" 

"If the answer was yes, the book will be removed from circulation and stored," Exman separately told Popular Science.

The 19 books flagged by ChatGPT include "Looking for Alaska," "The Handmaid's Tale," "Friday Night Lights," "Gossip Girl," "Thirteen Reasons Why," and "I Know Why the Caged Bird Sings." A full list of the books was reported by Engadget.

However, when Insider asked the chatbot the same question for the 19 books listed, the chatbot returned responses that were contradictory, incomplete, or inconclusive.

For seven of the 19 books, ChatGPT said the book does not contain explicit depictions or descriptions of a sex act. These books were "Friday Night Lights," "Gossip Girl," "Killing Mr Griffin," "The Absolutely True Diary of a Part-Time Indian," "An American Tragedy," "Feed," and "Monday's Not Coming."

For four of the books —"I Know Why the Caged Bird Sings," "Looking for Alaska," "Sold," and "The Kite Runner" — ChatGPT gave an incomplete response and outputted an error stating "this content may violate our content policy."  

ChatGPT still flagged eight of the books as containing descriptions of a sex act, according to Insider's review.

These differences could stem from ChatGPT outputting a different response each time it is queried, as a consequence of being a large language model.

In one instance, when Insider asked about the same book three times, the chatbot outputted contradictory answers.

Take John Green's "Looking for Alaska," a coming-of-age novel that received the most complaints of any book in America in 2015 over its "sexually explicit descriptions."

We asked ChatGPT the same question three times, and it gave us three very different answers. On the first try, ChatGPT outputted an error message. On the second try, it said "Yes," the book contains descriptions of a sex act. On the third try, it said the book "does not provide explicit or graphic descriptions."

Notably, the chatbot responded to 11 of Insider's 19 book queries with variations of "as with any book, it might be a good idea for educators or parents to read it first if they have concerns about its content for younger readers."

Of the 19 books removed by the Mason City school district, 17 have been reviewed by age-rating nonprofit Common Sense Media.

Of these, three were reviewed by the organization as having no content with sex, romance, and nudity — while four were reviewed as having little such content. Four books were listed as having "some" of this content. Only six of these books were listed as having "a lot" of such content.

Insider has previously highlighted the limitations of ChatGPT and similar AI models, emphasizing that they aren't substitutes for search engines. These models can sometimes present false information as true — a phenomenon called "hallucination" — and rely on a currently undefined dataset.

ChatGPT creator OpenAI did not immediately respond to a request for comment from Insider, sent outside regular business hours. A member of the Mason City School District board declined to comment.