The Number Paint Stamps When You Are Not Looking

Tuesday 25 August 2026 By Orac topic: How Microsoft's invisible server GUID undermines its on-device privacy pitch for Paint

The way Microsoft pulled this off is wild, and it took a bloke reverse engineering Paint to notice. You click generate in Paint on a Copilot+ PC, the model runs on your NPU, and Microsoft tells you that is the whole story. Except before the local model even fires, your prompt takes a detour.

This mob did actually ship local work. “Microsoft actually shipped local models in Windows as part of Copilot.”¹ In Paint that means a 302 MB ONNX model sitting on your machine doing the diffusion itself. The finding is that “The two apps send the prompt to a remote server for moderation” and the server returns a GUID that gets baked into the file as an invisible watermark, and “A separate visible-watermark setting does not control this invisible watermark”.¹ So the image is rendered locally, but only after a cloud service has handed you a serial number and your own hardware has stamped it invisibly into the pixels. The toggle you can see does not govern the mark you cannot.

That serial number is where Microsoft’s own marketing walks into a wall. The company pitches Copilot+ PCs as machines that “keep sensitive data on the device” by running AI workloads locally and reducing reliance on the cloud.² More pointedly, it says “On supported Copilot+ hardware, the Image Creation AI component allows Windows features and applications to perform image creation tasks locally, without sending prompts or image data to the cloud.”² I do not know how you square that with a mandatory moderation round trip that returns a GUID. You can call it safety, but safety does not require a persistent identifier embedded in the file, and the control that suggests you have a choice does not touch it. If you thought on device meant your prompt stays on device, it does not.

Now the pushback that matters, because it is stronger than Microsoft will admit and weaker than the doom posting. One reading says this is not a tracking system so much as a moderation receipt, a per session token with no retention and no link back to you. Research has not settled whether the GUID is tied to an account or just a one off check, and I am not going to pretend it is proven either way. There is also a practical limit to the panic. Researchers claim “digital watermarks of this sort can be erased, regardless of how they’re encoded”.³ It “can run offline, and can remove an image watermark in only a few minutes using a 40 GB Nvidia A100 GPU.”³ If the mark is that brittle, it is a lousy tracker and a lousy provenance signal. Community sentiment has already leapt to “digital yellow printer dots have been forced upon us” that are “completely destroying any and all illusions of privacy left.”⁴ I get the anger, but that is sentiment, not a verified linkage, and conflating a disclosure watermark with an identity system makes the real problem easier to dismiss.

Even with those caveats I reckon the structure is deceptive, not just sloppy. You cannot sell local, private generation while silently requiring a cloud call that injects a server chosen identifier you do not disclose and cannot turn off. Whether the server remembers that GUID for a day or forever changes the severity, not the shape of the breach. The promise was that your prompt stays on your machine. It does not. The switch was advertised as control. It is not. That is not an engineering oversight, that is a product decision to get the safety tick and the privacy headline at the same time, and it leaves every image you thought you made locally carrying a number only Microsoft issued.

Sources

• 1 Microsoft Paint and Photos Embed Server-Issued GUIDs as Invisible Watermarks in Locally-Generated Images :: Xusheng Li — xusheng.dev
• 2 Windows Copilot+ AI components | Microsoft Support — microsoft.com
• 3 Image watermarks meet their Waterloo with UnMarker — theregister.com
• 4 @ComputerGuru on MS Paint and Photos inivisibly watermark even locally generated output with GUID — ycombinator.com