Google Earth could be used to make fake evidence before capability rollback.
SOPA Images/LightRocket via Getty Images
Twice in one month, companies full of exceptionally smart people shipped AI features capable of producing convincing synthetic imagery that could be misrepresented as evidence, watched the public demonstrate the risk within hours or days and then withdrew, restricted or revised the feature. The question is no longer whether these tools can be misused. It is why foreseeable misuse keeps being discovered by the public instead of in the review that approved the launch.
The latest high-profile example was Google. The company added image generation to Google Earth on the web in late July, inviting users to zoom to a place, tap Create Image and type, in Google’s own words, “whatever you want to see.” The tool produced synthetic, photorealistic images grounded in Google’s authentic satellite, aerial and 3D imagery for the exact location selected.
Within hours, researcher Henk van Ess generated refugees at the Mexican border and a nuclear plant in Iran. Other researchers generated false disaster imagery, including a bomb crater at a hospital in Gaza. One day after launch, Google rolled the feature back “while we work on implementing stronger guardrails,” acknowledging that “people uniquely trust Google Earth for a reliable view of the world.”
The generated images did not alter Google Earth’s shared base imagery or appear in the main Google Earth experience for other users. Google said they were marked as AI-generated. But they could be screenshotted, downloaded and circulated outside the product, which is exactly what happened.
Meta Ran The Same Play Three Weeks Earlier
In early July, Meta shipped a feature alongside its Muse image model that let users mention public Instagram accounts as reference material for AI-generated images. The account owner was not notified when their account was referenced. Public adult accounts were included by default unless the owner changed their settings.
The backlash over consent, likeness use and the obvious path to non-consensual imagery took about three days to end the experiment. Meta’s statement said the feature “missed the mark.”
The cycle has run before. I wrote in January about xAI shipping an image tool that generated sexualized imagery of children before its safeguards caught up. xAI did not simply remove the capability. It restricted access and adjusted controls after public backlash.
Image models have also been producing plausible receipts and invoices on request for more than a year. Ship, public backlash, then withdrawal, restriction or newly announced safeguards. The pattern has become commonplace, and each round ends with a statement describing as a surprise something foreseeable from the feature description alone.
Martino Jerian sees it the same way. His company, Amped Software, builds the image and video analysis tools forensic examiners use to test photos and footage, which makes him one of the people who cleans up what these launches leave behind.
“How do features with such clear misuse potential make it through the design, review, and approval process?” he asked in a LinkedIn post after the Google Earth reversal.
His standard is not that every conceivable abuse should stop a launch. It is that deception, impersonation and fake evidence are foreseeable misuses, and foreseeable misuse should be weighed “before a public launch rather than after the backlash begins.”
Every Demo Prompt Is Also A Fraud Prompt
The marketing demo and the fraud scenario are the same feature with different nouns.
Google pitched the Earth tool for real-estate renderings: add a cabin to this empty lakefront lot. On a claims desk, that prompt reads: add storm damage to this roof. A tool that can reimagine an intersection for an urban planner can reimagine the same intersection for a party disputing what a driver could see.
That matters because the imagery these tools imitate carries courtroom weight. Aerial and satellite images can help establish property lines, sight lines in crash reconstructions, roof condition in storm claims and scene geography in criminal trials.
Courts have long recognized that machine-generated imagery can raise questions of malfunction, tampering and authentication. When a defendant challenged a Google Earth exhibit in 2015, the Ninth Circuit wrote that concerns about tampering “are addressed by the rules of authentication.” But the defendant in that case had not actually objected to the exhibit’s authentication, so the decision did not establish a simple rule for authenticating disputed Google Earth imagery. The case nonetheless illustrates the legal framework: the issue is not whether a digital image looks persuasive, but whether its origin, handling and claimed meaning can be established.
What changed is the threat model. A party no longer needs specialized geospatial tools, source-data access or sophisticated image-editing skill to create a plausible, location-grounded synthetic visual. The authentication question is not new. The speed, accessibility and visual credibility of the manipulation are.
For one day in late July, it took a sentence.
The Rollback Restores The Feature List, Not Trust
What a walk-back cannot do is un-demonstrate the capability. Everyone who watched the Google Earth cycle now knows that a screenshot of a trusted geospatial environment can be turned into a location-grounded synthetic image in seconds.
That does not make every authentic aerial exhibit suspect. It does mean that authentic exhibits now face a more practical and easily weaponized version of an old authentication question: What is the source? What happened between that source and the image now offered? Who can reproduce the process?
Google’s rollback note said the generated images were watermarked as AI-generated. The public tested the verification path too. Researcher Tal Hagin fed the fake Gaza hospital image back to Google’s Gemini chatbot and asked it to check for SynthID, the company’s invisible watermark. Gemini reported, “No reliable signals were detected indicating how the content was created.”
Futurism ran its own test. Gemini said it was “unsure if the image was made with Google AI.”
Those results do not prove that the watermark was absent or that SynthID never works. They show something more operationally significant: Google’s public verification path did not reliably identify a high-risk synthetic image made through Google’s own Earth workflow. A provenance signal that requires someone to check it, then returns an inconclusive answer, cannot carry the evidentiary weight users may assume it carries.
The capability did not leave either. The model behind the feature, Nano Banana 2, remains available in Gemini, where users can upload an image, including a map or Earth screenshot, and request edits in natural language. The Earth rollback removed an especially frictionless, location-grounded workflow. It did not remove the broader capacity to edit geospatial imagery. The rollback removed a doorway. The room is still there.
Companies will keep building image tools, and there are real uses for every one of these features. That is exactly why the review question Jerian is asking deserves an answer before the next launch instead of after it.
Somewhere right now, a feature with the same obvious dual use is moving through an approval process on the strength of its demo. It will ship, someone will type the fraud version of the marketing prompt within hours, and the company will discover in public what a digital forensic examiner could have told it in the design meeting.
The feature will be reversible. What it teaches the world about what can be faked, and what can be trusted, is not.