Google's Gemini video demonstration wowed plenty of people at launch, but a new report suggests that the company could have staged it by spoon-feeding the questions to its AI model.
What Happened: Although Alphabet Inc.'s (NASDAQ: GOOG) (NASDAQ: GOOGL) Google is claiming superiority over Microsoft Corp.-backed (NASDAQ: MSFT) OpenAI's GPT-4, Gemini's video demonstration could have been staged, according to a Bloomberg report.
The staged part of the Gemini demonstration hinges on a statement by a Google spokesperson and the YouTube video's description.
"For the purposes of this demo, latency has been reduced and Gemini outputs have been shortened for brevity," said a Google disclaimer in the Gemini video description.
What this means is that the time taken for Google to pose a question to Gemini and for the AI model to process it and respond is misleading, going by the video demonstration.
Latency being reduced manually by Google here means that the company made Gemini's responses appear faster than they were. These almost real-time-like responses were one of the things that Gemini was praised for after its launch on Dec. 6.
Moreover, adding to the skepticism about Gemini's accuracy, a Google spokesperson revealed that the AI model's conversational style of responses was staged to an extent. The company said that the video demonstration was made "using still image frames from the footage, and prompting via text."
If the smoothness of the conversation has you wowed, you might want to reconsider it for the time being.
Gemini's Win Over GPT-4 Could Be Short-Lived: If that is not enough, Gemini's win over GPT-4 could also be short-lived.
Buried in the Google Gemini report is a benchmark comparison of Ultra and Pro versions of the large language model with OpenAI's GPT-4 and GPT-3.5, apart from Meta Platforms Inc.'s (NASDAQ: META) Palm 2-L and Anthropic's Claude 2.
The version of Gemini that Google demonstrated in a pre-recorded video on Wednesday, Ultra, will arrive only in 2024.
To make matters worse for Google, OpenAI has taken progressively less time to launch a new version of GPT.
OpenAI took approximately 30 months to go from GPT-3 to GPT-3.5 - GPT-3 was first launched in beta in June 2020, while GPT-3.5 was quietly launched in Nov. 2022.
It took the AI startup only four months to launch GPT-4 in March 2023, while GPT-4 Turbo took a little over seven months.
Google has only benchmarked Gemini's performance against GPT-4, so OpenAI could beat it again in a few months - essentially, by the time Gemini Ultra is made available to the public, OpenAI could launch a new GPT version that is better.