By: Thomas Stahura
Most folks think they can tell the difference between AI-generated and human-created content.
Me especially: An avocado chair, yea that's AI. Dripped out pope? AI again. Will Smith eating spaghetti? You already know what it is!
Even so, last month, a video of an emotional support kangaroo attempting to board a plane went viral on twitter. The original clip, posted on Instragram, was just weird enough to get attention, but not weird enough to be immediately flagged as AI. Once on X, the video gained 58 million views, with some comments claiming it to be real. Strange because once unmuted, the Australian gibberish gives it away as AI. That and the nonsensical text.
Still, since the launch of Veo 3, there's been an explosion of AI videos in my social feeds. And I don’t think it’s just me since stuff like bigfoot vlogs, alien street interviews, and AI ASMR are getting hundreds of thousands and sometimes even millions of views. John Oliver dedicated a segment on Last Week Tonight on the topic.
The video models have gotten so good that it feels like we are at a tipping point where, given enough creative ideas, it is now economically viable to run large scale AI content farms.
Especially if the social platforms in question are also building their own generative models. Therefore, the burden of authenticity, now more than ever, is on the individual. The thing is, AI watermarking and detection is still extremely faulty.
Tools like SynthID and IMATAG both modify pixel values in a structured and pseudo-random way during generation. This is usually done within the model itself so the watermark is spread across the image making it more robust to simple edits and cropping. However, converting the image to lossy formats (jpg) multiple times or downscaling then upscailing the image will compromise the watermarks integrity. Not to mention most open source models don't watermark at all.
Video is similar but the mark is embedded across multiple frames, sometimes with temporal consistency checks to make sure it survives basic video editing. Still, heavy video effects and excessive cuts will break the temporal consistency of the watermark.
Text is the trickiest. The most common watermarking method is to tweak the probability distribution of word/token selection during generation. For example, models are nudged to pick certain synonyms or sentence structures that, when analyzed statistically, reveal a hidden pattern (think gpt isms). Google’s SynthID Text and similar methods use error-correcting codes to make the watermark more robust, but if you paraphrase or summarize the text, the watermark gets wiped out.
AI watermarking, much like online bot detection, is stuck in a perpetual arms race wherein every advance in watermarking is quickly met by new evasion tactics.
This arms race is compounded by the open-source explosion. Anyone can fine-tune or fork a model, disabling watermarking entirely or even inserting their own. The barrier to entry for running a “clean” (i.e., unwatermarked) content farm is basically zero. And as the models get better, the uncanny valley shrinks — making it harder for even the most online, AI-savvy users to spot fakes. Hence, an emotional support kangaroo fooling the world.
This is where regulation steps in — or at least tries to. The EU AI Act, mandates transparency for synthetic content, requiring clear labeling of AI-generated media and watermarking for anything that could be mistaken for real. The idea is to force platforms and creators to disclose what’s real and what’s not, shifting some of the burden of authenticity off the individual and onto the companies building and distributing this tech.
Contrast that with the US, where regulation is still mostly vibes and voluntary commitments. There’s no federal law mandating watermarking or disclosure for AI-generated content. Instead, it’s a patchwork of executive orders, industry pledges, and state-level bills — none of which have real teeth. Except for the TAKE IT DOWN act, which was signed into law by President Trump in May after quickly making its way through congress.
This act, which gained rare bipartisan support, requires social media platforms to remove nonconsensual intimate imagery (NCII) within 48 hours of a victim’s request. It also imposes criminal penalties for those who create or distribute such content.
The FTC, which announced a 10% staff cut under pressure from DOGE, is responsible for enforcing the act leading to worry about that agency’s capacity to handle violations at scale. Critics also argue that the language is too broad since it doesn’t explicitly exempt other types of legal synthetic content. Some fear this could result in platforms over-censoring and stifling free speech. But it at least signals an effort in Washington to crack down on the most insidious uses of generative AI.
Though legislation alone can’t keep pace with the speed and scale of synthetic media abuse. A new crop of startups are emerging in deepfake detection. Clarity (Ascend Portco), backed by Bessemer and Walden Capital, uses video and audio AI to “score” media in real time to detect and prevent digital impersonations. More and more, it seems that trust is the product, and startups are racing to sell it before the next viral hoax hits.
It can feel like a whack-a-mole game with new AI-generated images and the counterpunch of watermarks, regulation, and detection tech. My main fear is getting so numb to the fakery that we stop believing anything at all.
Stay tuned!