Meta's Un-Stable Signature
https://hackerfactor.com/blog/index.php?/archives/1098-Metas-Un-Stable-Signature.html
ementally · 9 days ago
6 comments
https://hackerfactor.com/blog/index.php?/archives/1098-Metas-Un-Stable-Signature.html
ementally · 9 days ago
6 comments
flaxxer · 5 days ago
also, easily bypassed now: https://twotensors.ai/
tarpitt · 5 days ago
There is actually an older method for countering steganography and adversarial image generation attacks: https://en.wikipedia.org/wiki/Gaussian_blur
miohtama · 5 days ago
AFAIK these methods claim to be blur resistant
masfuerte · 5 days ago
Yes. These methods don't work reliably. Apply a blur and they still don't work reliably.
miohtama · 5 days ago
Related to this, the EU AI Act requires mandatory watermarking that is cannot be removed or is illegal to remove.
https://digital-strategy.ec.europa.eu/en/policies/eu-icons-l...
If Facebook already embeds user IDs in images (AI or no AI) I can only drool to think what kind tracking, advertising and mass surveillance opportunities are coming.
N19PEDL2 · 5 days ago
How can a watermark be unremovable?
embedding-shape · 5 days ago
The actual rules don't say that I think, it's more about the intention that the watermark is embedded with the image/multimedia itself, so it's persisting even if someone "right-click > save" the image or takes a screenshot, not literally regulated the watermark has to be unremovable.
> (Summary) The icon should be directly embedded into the deep fake or published text (except for creative works), unless equivalent alternatives are available such as a user interface overlay. The icon must be visible when content is reshared or downloaded.
charcircuit · 5 days ago
It says the icons are optional. So that icon must not be what the other person was talking about.
embedding-shape · 5 days ago
> It says the icons are optional. So that icon must not be what the other person was talking about.
What "watermark" are they talking about if not the label/icons? The label/icon in question are what the whole "EU Icons for labelling AI-generated content" thing is about, someone correct me if I'm having a big brain fart.
Ohentis · 5 days ago
I imagine the goal is for everything to use something like Google's synthid.
embedding-shape · 5 days ago
That sounds like one possible implementation, not the goal per se. The goal (the explicit/stated one at least) is to give people a heads up what's AI generated vs not, when that's unclear.
InsideOutSanta · 5 days ago
I don't see anything about watermarking in the linked article, it's about labelling requirements. It describes situations where you are required to disclose if an image was AI-generated.
richardfey · 5 days ago
This is a great statistical analysis and it was a pleasure to read, but I wasn't expecting the claims to be so poorly supported. There's also a reply from one of the Meta authors there, worth checking out.
hackerfactor1 · 3 days ago
Which claims do you think are poorly supported? I'm the author; I tried to include everything other people need to repeat the same experiments. I've even had two people write in directly to me, stating that they have been able to replicate my findings.
Or are you referring to the claims from Meta, Google, and Adobe -- which failed to hold up under independent evaluation.
You are also correct that one of the meta authors wrote in a comment. However, he demonstrated a clear lack of understanding regarding what makes the bits "independent" or how to resolve the independence problem.
richardfey · 3 days ago
> Or are you referring to the claims from Meta, Google, and Adobe -- which failed to hold up under independent evaluation.
This.
> However, he demonstrated a clear lack of understanding regarding what makes the bits "independent" or how to resolve the independence problem.
Yes, I didn’t want to call it out explicitly, but this is exactly the kind of thing that would have made my undergraduate statistics professor lose patience.
itake · 5 days ago
A watermark is not just “transparency.” It can reveal what tool someone used, how they work, or that an image came from a stigmatized platform. In sensitive contexts—politics, sexuality, medical issues, protest material, or private expression—that can become surveillance.
I am working on Saigon Watermarks: https://apps.apple.com/us/app/saigon-watermark/id6777061197 for detecting and removing provenence markers in AI.
The tool also removes c2pa markers, which google is now linking the device that took the photo with the photo.
scary stuff.
https://security.googleblog.com/2025/09/pixel-android-truste...
RobotToaster · 5 days ago
No android version?
itake · 5 days ago
Still working on it...
I'm waiting on Apple to approve the MacOS version. After I will either focus on removing SynthID (currently not supported) or releasing android.
hparadiz · 5 days ago
Another reason to drop both iOS and Android.
mort96 · 4 days ago
In favour of what, might I ask?
Keep in mind that the replacement must be a first class citizen in the society I live in. That means being able to easily transfer money using the app everyone uses where I live. It means public transport ticket apps. It means banking apps. It means iMessage, Facebook Messenger, Snapchat and Signal.
UltraSane · 5 days ago
When generative AI can create such good fake images a valid c2pa linked to the source camera will become mandatory for an image to be considered authentic.
nocoolnametom · 5 days ago
This is what REALLY pains me about this discussion: I am 100% about personal digital freedom, but I am also 100% opposed to promoting political violence and promoting theft and grift using generative AI. If C2PA is going to work towards one goal by being diametrically opposed to the other then it _cannot_ be a useful tool and we need an _actual_ solution. I was extremely excited by C2PA until today and now am only disappointed that there isn't already some better solution.
Edit: Thinking through this a bit more, I think the goal of _authenticating_ a photo using C2PA is still useful. If the goal is to remove them to get a "naked" image, that's fine, such an image is then inherently no more or less trustworthy than any other image. If the goal is to figure out how to reproduce a valid provenance chain on top of an altered image then I have problems with that.
kamranjon · 5 days ago
How common is it for peer reviewed papers like this to be so far off their claimed findings?
“According to Google's peer-reviewed and published paper, they claim to have a true positive rate (TPR) above 99.97% -- meaning that they will miss their own watermarks less than 1 in 10,000 times. However, my own empirical testing found that is it much closer to 1 in 20.”
a34729t · 5 days ago
If there were bounties for invalidating peer reviewed research, I suspect this would be a lot leas common.
f33d5173 · 4 days ago
The point is to embed a particular signature, which was generated randomly, in the image. The distance function discussed is the same as the popcount of the xor. It's well know that the xor of random data with correlated data is statistically random. Hence, however well correlated the signatures of unwatermarked images may be with each other, they would show no correlation with the signature of a watermarked image. That is, unless the watermark by extremely bad luck happened to be near one of these clusters the author discovered. This does represent a genuine flaw, but an extremely minor one, and one that can be easily mitigated with no changes to the underlying algorithm,