Google has a lot to prove with its AI efforts — but it can't seem to stop tripping over its own feet.
Earlier this week, the tech giant announced Gemini, its most capable AI model to date, to much fanfare. In one of a series of videos, Google showed off the mid-level range of the model dubbed Gemini Pro by demonstrating how it could recognize a series of illustrations of a duck, describing the changes a drawing went through at a conversational pace.
But there's one big problem, as Bloomberg columnist Parmy Olson points out: Google appears to have faked the whole thing.
In its own description of the video, Google admitted that "for the purposes of this demo, latency has been reduced, and Gemini outputs have been shortened for brevity." The video footage itself is also appended with the phrase "sequences shortened throughout."
In other words, Google misrepresented the speed at which Gemini Pro can recognize a series of images, indicating that we still don't know what the model is actually capable of.
In the video, Gemini wowed observers by using its multimodal thinking chops to recognize illustrations at what appears to be a drop of a hat. The video, as Olson suggests, also offered us "glimmers of the reasoning abilities that Google’s DeepMind AI lab have cultivated over the years."
That's indeed impressive, considering any form of reasoning has quickly become the next holy grail in the AI industry, causing intense interest in models like OpenAI's rumored Q*.
In reality, the demo wasn't just significantly sped up to make it seem more impressive, but Gemini Pro is likely still stuck with the same old capabilities that we've already seen many times before.
"I think these capabilities are not as novel as people think," Wharton professor Ethan Mollick tweeted, showing how ChatGPT was effortlessly able to identify the simple drawings of a duck in a series of screenshots.
Did Google actively try to deceive the public by speeding up the footage? In a statement to Bloomberg Opinion, a Google spokesperson said it was made by "using still image frames from the footage, and prompting via text."
In other words, Gemini was likely given plenty of time to analyze the images. And its output may have then been overlaid over video footage, giving the impression that it was much more capable than it really was.
"The video illustrates what the multimode user experiences built with Gemini could look like," Oriol Vinyals, vice president of research and deep learning lead at Google’s DeepMind, wrote in a post on X.
Emphasis on "could." Perhaps Google should've opted to show the actual capabilities of its Gemini AI instead.
It's not even the first time Google has royally screwed up the launch of an AI model. Earlier this year, when the company announced its ChatGPT competitor, a demo infamously showed Bard making a blatantly false statement, claiming that NASA's James Webb Space Telescope took the first image of an exoplanet.
As such, Google's latest gaffe certainly doesn't bode well. The company came out swinging this week, claiming that an even more capable version of its latest model called Gemini Ultra was able to outsmart OpenAI's GPT-4 in a test of intelligence.
But from what we've seen so far, we're definitely going to wait and test it out for ourselves before we take the company's word.
More on Gemini: Google Shows Off "Gemini" AI, Says It Beats GPT-4
The post Google Admits Gemini AI Demo Was at Least Partially Faked appeared first on Futurism.
Continue reading here:
Google Admits Gemini AI Demo Was at Least Partially Faked
- Futurist Serata featuring artist Luca Buvoli at Brown (Nov. 20) - November 7th, 2009 [November 7th, 2009]
- FUTUR1SM00GGI - November 8th, 2009 [November 8th, 2009]
- ‘Futurism on Film’ Series this month in NYC - November 8th, 2009 [November 8th, 2009]
- Schedule of Futurist Events in NYC (PERFORMA 09: Nov 1-22) - November 8th, 2009 [November 8th, 2009]
- ‘Futurismo/Futurizm: The Futurist Avant-Garde in Italy and Russia’ (Nov. 13 + 14) - November 8th, 2009 [November 8th, 2009]
- ‘Beyond Futurism: F.T. Marinetti, Writer’ conference at Columbia (Nov. 12+13) - November 8th, 2009 [November 8th, 2009]
- Futurism and Cars at the Museo Nicolis - November 8th, 2009 [November 8th, 2009]
- MoMA Film Series Marks Centenary of Futurism with Films - November 8th, 2009 [November 8th, 2009]
- ‘Bergson+Futurism. Speed in thought’ - Madrid (Nov. 5) - November 8th, 2009 [November 8th, 2009]
- ‘The Future in Five Senses: Echoes of Italian Futurism in New York Architecture and Design’ Nov. 16th NYC - November 8th, 2009 [November 8th, 2009]
- New World-Wide Climate Treaty in 2010 More Likely - November 8th, 2009 [November 8th, 2009]
- Tar Sands CCS Myth Shattered - November 8th, 2009 [November 8th, 2009]
- Smart Grid and Smart Meters Get Big Grants - November 8th, 2009 [November 8th, 2009]
- Pollution Makes Methane Even More Dangerous - November 8th, 2009 [November 8th, 2009]
- Climate Change Bill Hearing Video - November 8th, 2009 [November 8th, 2009]
- New Satellite to Monitor Water and Plant Growth - November 8th, 2009 [November 8th, 2009]
- Spiritual Battle Awaits the Deniers and Skeptics - November 8th, 2009 [November 8th, 2009]
- Effects of Climate Change are Observed World-Wide - November 8th, 2009 [November 8th, 2009]
- Get Yer Global Warming Science Here - November 8th, 2009 [November 8th, 2009]
- TckTckTck Wake up Call — Delay Kills - November 8th, 2009 [November 8th, 2009]
- Canada’s Awful Gold Rush - November 8th, 2009 [November 8th, 2009]
- Climate Change Talks Spark Global Backlash by Businesses - November 8th, 2009 [November 8th, 2009]
- World May Need Extra Year for Climate Treaty - November 8th, 2009 [November 8th, 2009]
- Senator Boxer Moves Climate Bill Despite Republican Obstructionism - November 8th, 2009 [November 8th, 2009]
- Lights out for incandescent lights? - November 8th, 2009 [November 8th, 2009]
- Sutures from Bacteria - November 8th, 2009 [November 8th, 2009]
- Remote-Controlled Pigeons - November 8th, 2009 [November 8th, 2009]
- Apple Announces iPhone Release Date - November 8th, 2009 [November 8th, 2009]
- UK Government Envisions a Grim Future - November 8th, 2009 [November 8th, 2009]
- Top Ten Emerging Technologies for the Environment - November 8th, 2009 [November 8th, 2009]
- DIY Mobile Networks - November 8th, 2009 [November 8th, 2009]
- Stem-Cell Treatment Cures Type 1 Diabetes - November 8th, 2009 [November 8th, 2009]
- Is Tesla Getting the Electric Car Right? - November 8th, 2009 [November 8th, 2009]
- The Future of TV News - November 8th, 2009 [November 8th, 2009]
- Bruce Sterling on Earth-Friendly Pervasive Computing - November 8th, 2009 [November 8th, 2009]
- First Step Toward Organ Regeneration in Humans - November 8th, 2009 [November 8th, 2009]
- IBM's "Five in Five" - November 8th, 2009 [November 8th, 2009]
- Outsourced Journalism - November 8th, 2009 [November 8th, 2009]
- Is True Global Democracy the Next Great Political Movement? - November 8th, 2009 [November 8th, 2009]
- The Risks of Autonomous Robots - November 8th, 2009 [November 8th, 2009]
- Microsoft Introduces "Tabletop" PC - November 8th, 2009 [November 8th, 2009]
- Britain Piloting First Biofueled Train - November 8th, 2009 [November 8th, 2009]
- Self-Healing Plastic - November 8th, 2009 [November 8th, 2009]
- Bird Population Falls Over Past 40 Years - November 8th, 2009 [November 8th, 2009]
- The iPhone Revolution? - November 8th, 2009 [November 8th, 2009]
- The End of "Cheap Food"? - November 8th, 2009 [November 8th, 2009]
- How to Stop -- Or Live With -- Global Warming - November 8th, 2009 [November 8th, 2009]
- MIT Demonstrates "Wireless Electricity" - November 8th, 2009 [November 8th, 2009]
- Unintended Consequences of Biofuels - November 8th, 2009 [November 8th, 2009]
- Time to Focus on the Big Picture in Copenhagen - December 12th, 2009 [December 12th, 2009]
- Protests in Copenhagen - December 12th, 2009 [December 12th, 2009]
- Mario Guido Dal Monte exhibit - December 13th, 2009 [December 13th, 2009]
- Futurism News Bulletin, xvi - December 13th, 2009 [December 13th, 2009]
- Viva il Futurismo! (video trailer) - December 13th, 2009 [December 13th, 2009]
- 3 exhibits in Gorizia! - December 13th, 2009 [December 13th, 2009]
- Forthcoming: ‘Antidiets of the Avant-garde’ by Cecilia Novero - December 13th, 2009 [December 13th, 2009]
- Pubblicità e propaganda. Ceramica e grafica futuriste at the Wolfsoniana - December 13th, 2009 [December 13th, 2009]
- Balla’s home scheduled to open in 2010 - December 13th, 2009 [December 13th, 2009]
- Futurismo a Savona - December 13th, 2009 [December 13th, 2009]
- ‘Zang Sud Sud’, Cosenza - December 13th, 2009 [December 13th, 2009]
- Conference in Rome (Dec. 10) - December 13th, 2009 [December 13th, 2009]
- Climate Hackergate: A Well-Orchestrated Campaign of Harassment - December 13th, 2009 [December 13th, 2009]
- The Sad Story of Cap and Trade - December 13th, 2009 [December 13th, 2009]
- How to Waste Trillions on Capturing Carbon - December 13th, 2009 [December 13th, 2009]
- Smack the Email Hack Attack - December 13th, 2009 [December 13th, 2009]
- EPA About to Declare CO2 a Public Danger - December 13th, 2009 [December 13th, 2009]
- Copenhagen Summit Starts with Virtually There Media - December 13th, 2009 [December 13th, 2009]
- Climate Scientist Gets Blunt on Trading Scheme - December 13th, 2009 [December 13th, 2009]
- One Climate Change Editorial in 56 Newspapers, 45 Countries - December 13th, 2009 [December 13th, 2009]
- This Decade Will be Hottest Ever on Record - December 13th, 2009 [December 13th, 2009]
- Divide and Conquer - December 13th, 2009 [December 13th, 2009]
- Leave the Coal in the Hole! - December 13th, 2009 [December 13th, 2009]
- COP15: Two Agreements Coming - December 13th, 2009 [December 13th, 2009]
- Climate and Copenhagen News December 10 - December 13th, 2009 [December 13th, 2009]
- Sea Level Already Rising on Atlantic Coast - December 13th, 2009 [December 13th, 2009]
- ‘Umbria Veloce’ in Perugia - December 14th, 2009 [December 14th, 2009]
- An Instable CO2-Filled Ocean - December 14th, 2009 [December 14th, 2009]
- ‘Futurismi a Ravenna’ opens Dec. 19 - December 15th, 2009 [December 15th, 2009]
- ‘Futurism and the Technological Imagination’ – 30% discount until Jan. 15 - December 15th, 2009 [December 15th, 2009]
- Protecting Our Lungs at Copenhagen - December 15th, 2009 [December 15th, 2009]