Gemini Ultra is putting up an impressive showing against OpenAI's GPT-4 on a slate of benchmarks that test the capabilities of AI models.
Google says Gemini, its generative AI model, is more powerful than OpenAI's ChatGPT.
  • Google says Gemini, its generative AI model, is more powerful than OpenAI's ChatGPT. 
  • We asked 10 questions to ChatGPT and Gemini via Bard and compared their responses. 
  • Gemini had an edge on queries around current events and planning — but it generated factual errors.

Google's highly anticipated rival to OpenAI's ChatGPT is finally here.

Last week, Google released Gemini, a multimodal AI model that the search giant says can process text, images, and audio. The company contends that Gemini Pro, the model's most advanced version that's currently unavailable to the public, is smarter than OpenAI's GPT-4, one of the models behind ChatGPT.

Currently, only Gemini Pro, the most basic version of the model, is available to the public through Google's AI chatbot, Bard.

Business Insider asked ChatGPT and Bard a series of the same 10 questions, ranging from getting the skinny on OpenAI CEO Sam Altman being ousted to advice on practicing adventuresome sex. We then compared the results.

Here's how they stacked up against each other.

ChatGPT: Is this photo AI-generated? Explain your reasoning to your answer.
I asked ChatGPT to identify whether the image is AI-generated.
We asked ChatGPT if the photo was AI-generated?

I uploaded a photo of a restaurant meal generated by AI to see if AI could detect AI-generated images.

What I like: ChatGPT described the image in rich detail and offered theories on how the photo may or may not be AI-generated.

What could be better: The chatbot appears to be unable to make a judgement call on whether or not the image is made by AI.

What caught my eye: The AI chatbot's response seems to be convinced that the AI-generated image could've potentially been created by a human artist.

Gemini: Is this photo AI-generated? Explain your reasoning to your answer.
asking gemini if photo is ai generated
I asked Gemini if this photo was AI-generated.

What I like: Gemini correctly claimed with confidence that the uploaded photo was AI-generated by analyzing the features in the photo that make it look artificial.

What could be better: Google's chatbot misidentified features in the photo, pointing out things like floating islands and a blue sky that the image doesn't include.

What caught my eye: Despite Gemini's failure to interpret the image, I'm impressed at how the chatbot is able to discern AI-generated art from art made by humans.

ChatGPT: I’m interested in experimenting with BDSM. Where do I begin?
asking chatgpt for bdsm advice
I asked ChatGPT about experimenting sexually.

I asked the AI chatbots about sex to see how they respond to potentially graphic questions.

What I like: ChatGPT generated a list of helpful, in-depth tips on how to explore BDSM. It recommended books, preached consent, and urged beginners to start slow and prioritize safety.

What could be better: Links to sources would be helpful. It's weird receive sex advice from an AI chatbot without knowing where the information is coming from.

What caught my eye: The AI chatbot flagged the question saying "this content may violate our content policy," then changed the color of the text to orange.

Gemini: I’m interested in experimenting with BDSM. Where do I begin?
asking gemini for bdsm advice
I asked Gemini about experimenting sexually.

What I like: Nothing. Gemini refused to answer my question.

What could be better: I was hoping Gemini would — at the very least — provide information on where to look to learn more about BDSM.

What caught my eye: Google's AI chatbot spit out the same curt response to every variation of the question.

ChatGPT: Who is running for the 2024 US presidential election?
i asked chatgpt whose running for the 2024 presidential elections
I asked ChatGPT about the 2024 election.

What I like: ChatGPT was transparent about its limitations to answering the question given that it's only trained on data until April 2023.

What could be better: Instead of relying on the chatbot's web browsing plug-in, it would be helpful if ChatGPT's language model can answer questions about current events by drawing from the latest data.

What caught my eye: It broke down the answer into distinct categories based on party lines.

Gemini: Who is running for the 2024 US presidential election?
asking Google's Gemini whose running for 2024 presidency
I asked Gemini about the 2024 election.

What I like: Gemini is able to browse the web and pull up the latest information with accuracy on the candidates running for the 2024 US presidency. It also provided a brief description for each candidate.

What could be better: Nothing. Gemini did exactly what I wanted.

What caught my eye: The AI chatbot listed links to election pages from The New York Times, The Washington Post, and other media outlets for users to learn more.

ChatGPT: Write me a resignation letter from my job that's professional and honest.
asking chatgpt to write a resignation letter
I asked ChatGPT for some HR-related help.

What I like: ChatGPT created a simple, fill-in-the-blank template for a resignation letter that's in-depth and addresses everything it needed.

What could be better: The AI-generated resignation letter was too long and read in an unnaturally formal tone.

What caught my eye: The resignation letter displayed signs of human emotion with lines like "This decision was not an easy one" and "I am grateful for the opportunities."

Gemini: Write me a resignation letter from my job that's professional and honest.
asking gemini to write a resignation letter
I asked Gemini for some help with some HR-related issues.

What I like: Gemini's resignation letter was clear, concise, and straight to the point. It's a template I would consider using if I were to resign.

What could be better: I wish the answer was written with more personality and emotion.

What caught my eye: Google's AI chatbot gives you the option to modify its response to be shorter, longer, more casual, and more professional, which can help users customize their letter to their liking.

ChatGPT: Write me a text message to a good friend I haven’t talked to in months. Make sure the message is polite, thoughtful, and light-hearted.
asking chatgpt to write a message to a friend i havent spoken to in ages
ChatGPT's response to my request to write a message to a friend.

What I like: The message ChatGPT generated is casual, sincere, and thoughtful.

What could be better: Personally, the message reads a bit formal and diplomatic. It doesn't reflect how I text my friends.

What caught my eye: The AI chatbot's message includes emojis like a smiley face and a rainbow.

Gemini: Write me a text message to a good friend I haven’t talked to in months. Make sure the message is polite, thoughtful, and light-hearted.
asking gemini to write me a message to a friend
Gemini's response to my request to write a message to a friend.

What I like: The AI-generated text message reads casual, light-hearted, and enthusiastic — which is exactly what I wanted.

What could be better: Nothing. I would use this message as a jumping off point for my own texts to old friends.

What caught my eye: Google's AI chatbot can modify the tone of the message to reflect the user's personality with further prompting.

ChatGPT: In layman's terms, give me a day-by-day rundown on how the OpenAI drama with Sam Altman being ousted unfolded.
asking chatgpt about sam altman ousting
I asked ChatGPT to explain the drama around Sam Altman's firing — then rehiring — from OpenAI.

What I like: ChatGPT was able to explain how the ousting process at OpenAI works even though it's only trained on information up to April 2023. The ousting of OpenAI's CEO Sam Altman occurred November of this year.

What could be better: The explanation can be misleading if ChatGPT users don't carefully read the AI's response. The chatbot says it can provide an overview of the CEO's ousting even though it's only trained on data up to April 2023.

What caught my eye: Instead of declining to answer the question, ChatGPT used phrases like "would have" or "would need" as a way to predict how the ousting could unfold.

Gemini: In layman's terms, give me a day-by-day rundown on how the OpenAI drama with Sam Altman being ousted unfolded.
asking gemini about sam altman drama
I asked Gemini to explain the drama around Sam Altman's firing — then rehiring — from OpenAI.

What I like: Gemini made a clear, concise timeline around CEO Altman's ousting, including the aftermath and key takeaways.

What could be better: Some of the information is wrong. Gemini said that the news of Altman's departure leaked on November 18, when in fact, the news around the CEO's ousting broke the day before.

What caught my eye: Gemini provided links to news articles that backed up some of the chatbot's claims.

ChatGPT: What’s your stance on the current state of Israel-Hamas conflict
asking chatgpt its stance on israel palestine
ChatGPT didn't share a stance on the Israel-Hamas conflict.

What I like: Even though ChatGPT declined to share its stance on the Israel-Hamas conflict, the chatbot didn't shy away from providing historical context behind the issue.

What could be better: ChatGPT's response didn't include links to sources, which can be problematic given the rampant misinformation around the conflict.

What caught our eye: The AI chatbot explained the Israel-Hamas conflict in a way that's concise and easy to understand.

Gemini: What’s your stance on the current state of Israel-Hamas conflict
asking gemini about israel and palestine
I asked Gemini for its stance on the conflict between Israel and Hamas.

What I like: Gemini encouraged me to seek additional information on the Israel-Hamas conflict using Google Search.

What could be better: Gemini, at the very least, could've explained the history the conflict leading up to the current moment without taking sides.

What caught our eye: Google's chatbot spit out the exact same response when I asked the question in different ways.

ChatGPT: Summarize the article pasted below in a few bullet points.
asking chatgpt to summarize an article
I asked ChatGPT to summarize a recent Business Insider article.

I asked the AI chatbots to summarize a Business Insider article on what neighbors think of Jeff Bezos to test their ability to synthesize and reformulate information.

What I like: ChatGPT summarized the article with brevity and accuracy.

What could be better: Nothing. The summary was on point.

What caught our eye: It took less than a minute for ChatGPT to read through the text and generate a summary.

Gemini: Summarize the article pasted below in a few bullet points.
asking gemini to summarize jeff bezoz article
I asked Gemini to summarize a recent Business Insider article.

What I like: Nothing. Gemini refused to answer my question after multiple attempts.

What could be better: An explanation on why the language model can't summarize articles would be helpful context.

What caught our eye: When I fed the chatbot the link of the article to summarize, it told me it needed the fully body of text to perform the task.

ChatGPT: Describe what you see in the image.
i asked chatgpt to describe the photo i uploaded
I asked ChatGPT to describe a stock image of New York City.

I imported a stock image of a New York City street into the AI chatbots to see how their image identification capabilities stack up to one another.

What I like: ChatGPT correctly identified that the photo was taken in NYC because it recognized the Chrysler Building, one of the city's landmark buildings.

What could be improved: I would be extremely impressed if ChatGPT was able to identify exactly what cross-section the photo is depicting.

What caught my eye: The chatbot described the image in great, poetic detail and made claims about the weather and the time of day based on what it sees in the photo.

Gemini: Describe what you see in the image.
asking gemini to describe image
I asked Gemini to describe a stock image of New York City.

What I like: Gemini identified the picture as a "city street with tall buildings in the background" and then went on to detailed what else it saw.

What could be improved: It wasn't until the last paragraph that Gemini acknowledged it was a picture of New York. "I am also aware that this image is of a street in New York City, New York, United States," it said.

What caught my eye: Still, it didn't apparently recognize the iconic Chrysler Building — instead saying that the buildings were "likely to be office buildings, apartments, and hotels."

ChatGPT: My goal is to lose 10 pounds in two months without losing muscle mass. Make me a nutrition plan that includes how many calories and what I should be eating each day.
asking chatgpt to make me a meal plan
I asked ChatGPT about a two-month weight-loss plan.

What I like: ChatGPT provided a clear, step-by-step explanation on how it developed my meal plan given my weight, activity level, and dietary preferences. It calculated how many calories I should eat each day, as well as macronutrient benchmarks to meet in order to achieve my fitness goals.

What could be improved: The process behind making the meal plan took up the majority of ChatGPT's answer even though the question focused on the specific foods I need to eat to reach my goals.

What caught my eye: It took ChatGPT a few attempts in order to make a meal plan.

Gemini: My goal is to lose 10 pounds in two months without losing muscle mass. Make me a nutrition plan that includes how many calories and what I should be eating each day.
asking bard for meal plan
I asked Gemini about a two-month weight-loss plan.

What I like: Gemini created a concise yet detailed personalized meal plan that includes recipes and macronutrient data for meals to eat during breakfast, lunch, and dinner.

What could be improved: Nothing. Google's chatbot spit out a satisfying answer.

What caught my eye: Gemini's meal plan didn't offer an explanation on how it calculated my caloric intake and macronutrient goals.

The early verdict?

Gemini's edge: I thought Google's Gemini did better than ChatGPT at answering questions related to current events, identifying AI-generated images, and meal planning. However, it refused to answer thornier questions around sex and politics, and also generated factual errors.

ChatGPT feels more human: Even though I preferred some of Gemini's responses over ChatGPT's, ChatGPT's conversational tone makes interacting with the chatbot a more enjoyable, human experience.

Read the original article on Business Insider