Independent AI safety checker

I investigate AI, and publish what I find.

Evidence-first, reproducible write-ups on how AI systems handle your code, your data, and your trust — captured, verified, and open to anyone who wants to check my work. No affiliation with any vendor.

Reports

02
2026-07-29 [BY DESIGN] Cursor · Codebase Indexing · v3.13.21 · file-content upload

Cursor's Codebase Indexing uploads your file contents — as plaintext.

Cursor's docs say it “scans and indexes your source files,” which reads as local. It isn't: when indexing runs, your files upload to api2.cursor.sh. I captured an upload, decoded the body, and recovered a canary source line verbatim — plaintext, gzip-compressed inside TLS. The word “upload” only appears in Cursor's engineering blog (“uploads every file when a codebase is new”), not the indexing help or security pages most people read. Cursor says it discards the plaintext after making embeddings and keeps only embeddings + obfuscated paths — a server-side claim I can't verify.

dest api2.cursor.sh rpc FastUpdateFileV2 body plaintext (gzip in transit) client v3.13.21
SettingStops your code uploading?
Codebase Indexing — offyes — nothing uploaded[STOPS IT]
Privacy / “Ghost” Modeno — upload still sent (x-ghost-mode: true)[NO]
Share data / training opt-outno — governs model training, not codebase indexing[NO]
the smoking gun
$ # decode the FastUpdateFileV2 upload body POST api2.cursor.sh/aiserver.v1.RepositoryService/FastUpdateFileV2 → 200 $ gunzip < body | grep NEEDLE const marker_29 = "ZZQCANARYNEEDLE7391PLAINTEXTPROBE-29"; ← my canary file, recovered verbatim from the upload

The investigation

  1. Route Cursor's traffic through me
    mitmproxy + a locally-trusted CA; Cursor launched through the proxy on a canary repo of clearly-marked fake files.
  2. Two calls to api2.cursor.sh
    SyncMerkleSubtreeV2 (a manifest of obfuscated-path + content-hash) then FastUpdateFileV2 (the bulk upload, ~700 KB batches).
  3. Decode the body
    the FastUpdateFileV2 payload decodes to plaintext source (some batches gzip-compressed inside TLS). My canary line came back verbatim — not just embeddings.
  4. Turn indexing off
    with Codebase Indexing disabled, the client uploaded nothing — I checked with Privacy Mode both on and off. The switch that matters is indexing.
  5. Privacy Mode doesn't stop it
    the upload requests carry x-ghost-mode: true while still uploading. Privacy/Ghost mode governs retention, not whether data is sent.
  6. The disclosure gap
    the help page says “indexes your source files” (sounds local); only Cursor's eng blog says “uploads every file when a codebase is new.”
The point

“Indexing” is not local. When it's on, your files leave your machine as plaintext to be embedded on Cursor's servers. Cursor says it discards the plaintext afterward and keeps only embeddings + obfuscated paths — I can't verify what happens server-side, only that the code is sent. The one control that reliably stops it is turning Codebase Indexing off; Privacy Mode does not. Users deserve that stated where they'll read it, not only in an engineering blog.

2026-07-13 [FIXED] Grok Build CLI · v0.2.93 → 0.2.106 · data exfiltration

Grok's coding CLI uploaded your whole repo — and your git history.

It packaged the entire repository — including files it was told not to open and secrets deleted from history — into a git bundle and shipped it to Google Cloud Storage. I captured it on the wire and cloned it back. As of 2026-07-13 xAI's server returns disable_codebase_upload: true and it no longer fires. Its new /privacy opt-out changes what xAI retains, not what leaves your machine — see the timeline below.

binary SHA-256 2a97ba67…d767c dest gs://grok-code-session-traces bundle clones clean reproduced ×2 codebases
ToolUploads your whole repo?
Claude Codeno — only files it opens[LOCAL]
Codexno — only files it opens[LOCAL]
Geminino — only files it opens[LOCAL]
Grokwhole repo + git history → cloud[FIXED]
the smoking gun
$ grok -p "reply OK, do not open any files." OK. ▸ POST cli-chat-proxy.grok.com/v1/storage → 200 (body = "# v2 git bundle") $ git clone captured.bundle recovered/ && cat recovered/never_read.txt CANARY-NEVERREAD :: the agent was told not to open this file. recovered: 47 files · 4 commits · full history

The investigation

  1. Route its traffic through me
    mitmproxy + a locally-trusted CA; HTTPS_PROXY → every request Grok makes is logged.
  2. Tell it to do nothing
    prompt: "reply OK, do not open any files." Nothing should leave the machine.
  3. It uploaded anyway
    POST /v1/storage → 200, body magic # v2 git bundle. A whole-repo bundle, on an idle prompt.
  4. Clone the captured bytes
    git clone of the wire body reconstructed the repo — including the file marked never-read, verbatim.
  5. Measure the blast radius
    full git history rode along — secrets deleted months ago were still in the bundle; read .env went to /v1/responses.
  6. Try to opt out
    turned off "Improve the model" — /v1/settings still returned trace_upload_enabled: true; the upload still fired.
  7. Prove it's not a fluke
    reproduced on a second, unrelated codebase; other researchers confirmed on their own private repos (269 blobs · 219 MB).
  8. The walk-back
    2026-07-13: same client, but /v1/settings now returns trace_upload_enabled: false + disable_codebase_upload: true. Upload no longer fires. I can't prove I caused it — only the timeline.
  9. The fix arrives
    xAI ships a /privacy opt-out command and defaults the codebase upload off. Retested on grok 0.2.99 (binary SHA 01bcacec…). Reconfirmed still off on 0.2.106 (2026-07-22).
  10. I tested the opt-out
    A/B on the wire: opt-in vs opt-out send the same requests — session traces and model turns leave either way. The only change is /v1/traces returning 200204.
  11. Retention, not transmission
    /privacy sends PUT {"codingDataRetentionOptOut": true} — a server-side "don't keep it," not a block on what's sent. Your data still leaves the machine; the upload code still ships in the binary.
The point

What actually stopped the upload was a silent global flagdisable_codebase_upload: true — that applies whether you opt in or out. /privacy is a per-session retention toggle, not the switch that fixed this, so it shouldn't be pointed to as the control. And no developer should have to run an opt-out after every session to keep their own code off someone else's servers. The right default is off.

New investigations land here. Something worth digging into? [email protected]

Coverage

press & X
Interview
“Like many developers, I have high respect for Elon and for the teams at SpaceXAI. But the way they’re handling this one is not what I expected.”
cereblab, interviewed by Julie Lee — Inc.
As featured in
The Register The Verge The Hacker News Cybernews TechTimes GIGAZINE IBTimes UK Crypto Briefing Meteora Web Landian · 蓝点网 International Cyber Digest GBHackers ByteIota GlitchWire Penligent Developers Digest explainX AI Weekly The Agent Report Hive Security
On X