You bought a Copilot+ PC partly because the AI runs on your hardware. No cloud bills, no rate limits, no prompts leaving the machine. You open Paint, type a prompt into Cocreator, and the NPU generates the image locally. It feels offline.
It is not offline. And the image you save carries something you cannot see: a 16-byte globally unique identifier, issued by a Microsoft server, embedded into the pixels themselves.
That is the conclusion of a detailed reverse engineering writeup published last week by a security researcher, and it climbed to the top of Hacker News over the weekend with over 500 points and 200+ comments. I do not own a Copilot+ PC, so I have not reproduced this myself. Full disclosure: everything below comes from the researcher's published analysis, Microsoft's own support documentation, and discussion in the original thread. I verified each claim against those sources, and I will link all of them.
What makes this story worth 15 minutes of your time is not outrage. It is the engineering. This is one of the clearest public examples of how "local AI" and "private AI" have quietly become two different things, and the design decisions Microsoft made here teach you a lot about how provenance systems are being built across the industry.
What the researcher actually found
The investigation started, as many do, with an oddly large file. Paint ships a DLL called Watermarker.dll, roughly 1.67 MB. That is suspiciously heavy for what should be trivial functionality: drawing a small visible watermark into a corner of an image. The researcher asked an AI assistant to help analyze the DLL and found it does far more than the visible watermark.
Tracing the call tree inside Paint's PaintAIManager.dll, the researcher found that after the local Stable Diffusion model finishes generating an image, Paint calls a function called WmkWriteWatermark with a 16-byte payload. That payload is a GUID, a globally unique identifier.
The detail that caught my attention as an engineer: if WmkWriteWatermark fails, Paint does not quietly return the unmarked image. It converts the entire generation into an error. The watermark is not a nice-to-have. It is a hard requirement of the pipeline. An image without the embedded ID, by design, must not exist.
The same Watermarker.dll exists inside the Microsoft Photos app, backing its Image Creator and Restyle Image features. Both go through the same wrapper: run the local model, parse a GUID, convert pixel formats, call WmkWriteWatermark. There is one behavioral difference worth noting. If the watermark encoder fails in Photos, the code logs the error and appears to continue returning the image anyway. Paint is strict. Photos is lenient. Same DLL, two failure philosophies.
Where the GUID actually comes from
Here is the part that breaks the "local" mental model.
The GUID is not generated on your device. Before Paint runs the local image model, AIServices.dll sends your prompt and chosen style to a remote Microsoft endpoint for content moderation. The server responds with two ID strings: a promptGenerationId and a watermarkId. Paint parses both as GUIDs and rejects the response if either comes back as zeros.
The watermarkId is the 16-byte value that ends up embedded in your image's pixels. The promptGenerationId is used in the C2PA signing flow, which we will get to.
So the data flow on a Copilot+ PC looks like this:
You type a prompt
|
v
Prompt + style --> Microsoft moderation endpoint (network)
|
v
Server returns promptGenerationId + watermarkId
|
v
Local Stable Diffusion runs on the NPU (offline)
|
v
WmkWriteWatermark embeds watermarkId into pixels
|
v
C2PA Content Credentials attached (with another network call)
|
v
Saved PNG/JPEG/GIF/.paint file
The generation is local. The identity of the generation is not.
There is one more linking behavior the researcher documented. With each new moderation request, Paint sends the previous promptGenerationId as lastPromptGenerationId. Successive generations from the same session are explicitly chainable by the server. Your tenth image is quietly associated with your first.
Microsoft's own support page for Cocreator is actually candid about the architecture, if you read it carefully: to use these features you must be signed in with a Microsoft account and connected to the internet, because safety systems run in the cloud via Azure online services, and Microsoft "collects attributes such as device and user identifiers, along with the user prompts, to facilitate abuse prevention and monitoring." The page also states Microsoft does not store your input or generated images. What the support page does not spell out is that the moderation response includes a per-prompt GUID that gets written into the pixels of the file you save.
Two watermarks, one toggle
Confusion in the HN thread mostly came from one fact: Paint has a settings toggle for a visible watermark on AI images, and users assumed turning it off disabled watermarking. It does not. The visible watermark and the invisible GUID watermark are separate systems. The toggle controls the former only. Based on the researcher's analysis, there is no user-facing setting anywhere that disables the invisible watermark, and Paint's strict failure behavior ensures an unmarked image never reaches your disk through the normal path.
That distinction matters for how you talk about this with colleagues. "You can turn off the watermark" is true for the stamp you can see. It is false for the identifier you cannot.
The C2PA layer ties it together
Pixels are only half the story. Paint also attaches C2PA Content Credentials to saved AI images, implemented in ProvenanceHelper.dll and provenancesdk.dll. C2PA is the industry-standard provenance framework backed by the Coalition for Content Provenance and Authenticity, which Microsoft co-founded. Think of it as a signed manifest glued to the file that answers "was this made or edited by AI?"
The researcher's finding is that the C2PA manifest and the invisible watermark are not independent. The signed c2pa.soft-binding assertion in the manifest names "Microsoft InvisMark" and records the same per-generation identifier carried in the pixels. And the provenance signing step involves another network request that sends the PromptGenerationId, which the server originally issued alongside the watermarkId. The server can therefore associate the signed manifest with the invisible watermark sitting in the submitted pixels.
Practical consequence: stripping metadata does not strip the watermark. Delete the C2PA manifest, screenshot the image, re-encode it, and the invisible pixel-level mark is designed to survive. That is the entire point of a soft-binding watermark, and it is the same design philosophy behind Google's SynthID. If you want to check your own images, the researcher notes the generation flow limits saves to C2PA-preserving formats: PNG, JPEG, GIF, and Paint's .paint format.
Is this nefarious? The honest answer is complicated
It is tempting to read this as spyware. The reality deserves more nuance, and I want to give both sides fairly.
The case that this is reasonable: The EU AI Act's Article 50 transparency obligations took effect on August 2, 2026, and require AI-generated content to carry a detectable, machine-readable mark. Every major AI vendor is shipping provenance right now. Watermarks that survive screenshots and re-encoding are the only kind that work against actual misuse, and abuse monitoring genuinely needs per-generation identifiers to function. Microsoft publicly discloses the remote safety checks, the account requirement, and the C2PA credentials. The Register's coverage notes Microsoft has arguably gone beyond the EU's minimum requirements.
The case that this is a problem: What Microsoft discloses and what it does are different in degree, not kind. The disclosure says "content credentials" and "cloud safety services." The reality is a prompt-specific GUID, issued per request, chainable across a session, embedded invisibly in pixels, with no off switch, in an app called Paint that people have trusted for 40 years as a dumb canvas. As the researcher put it, "local" does not mean offline: the prompt still goes to Microsoft, and the finished image's identity comes back from Microsoft. One HN commenter drew the parallel to laser printer tracking dots, the machine-identifying yellow dots manufacturers have embedded in printouts for decades, which alarmed privacy advocates when exposed. If a subpoena or breach ever linked watermarkIds to Microsoft account records, every image becomes attributable. Microsoft says it does not store the images; the GUID linkage is what makes the question worth asking at all.
My own read: the engineering is competent and the motive is plausibly regulatory compliance plus abuse prevention. The communication is the failure. A user toggling off the visible watermark reasonably believes they opted out of watermarking. They did not.
What this teaches you if you build AI features
I write a lot about Spring Boot and AI agent systems, and this story is directly relevant to anyone shipping AI features in their own products. A few transferable lessons from Microsoft's design:
- Decide failure semantics deliberately. Paint fails the generation if watermarking fails; Photos logs and continues. Both shipped. That inconsistency between sibling apps in the same OS suggests the decision was made twice by different teams. When you add a compliance or safety step to a pipeline, decide explicitly: is it a hard gate or an observability event?
- "Local" is a claim about inference, not about the system. If your on-device feature phones home for moderation, entitlement checks, or telemetry, say so in plain language near the feature, not in a policy document three clicks away.
- Linking identifiers across requests is a privacy decision. Sending lastPromptGenerationId creates a session graph almost for free. Cheap to build, expensive in trust when discovered.
- Provenance is arriving whether you like it or not. C2PA-style manifests and survivable watermarks are becoming table stakes under the EU AI Act. If you generate or edit media server-side, budget for the provenance layer now rather than retrofitting it.
Your AI image provenance checklist
Save this one. Five questions to ask of any AI image feature you use or build in 2026:
- Does generation require network? If the model runs locally but the feature needs internet, find out what crosses the wire. Prompts, identifiers, and entitlement checks are the usual suspects.
- Is there a watermark, and can you actually disable it? Assume the visible toggle controls only the visible layer unless documentation says otherwise.
- What survives a screenshot? C2PA manifests die with metadata. Soft-binding pixel watermarks are built to survive. Plan your threat model accordingly.
- Are generations linkable to each other or to an account? Per-request GUIDs plus account requirements equal attributability.
- What does the vendor disclose versus what exists? The gap between the support page and the disassembly is where the real story lives.
The bottom line
Microsoft Paint, an app most developers dismissed years ago, now contains a locally executing Stable Diffusion pipeline, a remote prompt moderation service, a survivable invisible watermark carrying a server-issued GUID, and a signed C2PA provenance manifest, all in a 1.67 MB DLL most people never noticed. The researcher's work is a gift to the industry: read the writeup yourself, because the level of call-tree detail is genuinely educational regardless of how you feel about the conclusion.
The era where "it runs on my machine" implied "it is private" is over. Provenance infrastructure like this is spreading to every AI feature on every platform, driven by regulation and real misuse. That is probably necessary. But necessary systems deserve honest disclosure, and a hidden GUID in Paint is not that.
I write about Java, Spring Boot, and AI every week. Subscribe, it's free.
Have you checked what your AI tools send over the network? What was your experience, and did anything surprise you?
Sources: The reverse engineering writeup is at xusheng.dev. The Hacker News discussion has worthwhile pushback in the comments. The Register covered the findings, and Microsoft's Cocreator support page documents the account, connectivity, and data collection requirements in the vendor's own words.
