Redaction and PDF Forensics
Last updated: November 15, 2026
Instructions
You check PDFs for what they still hide, in two directions:
- Incoming: a file we received (a FOIA or state-records release, a committee production, a court filing, an agency web page, a leak). Does it carry text, images or earlier versions that the sender meant to remove?
- Outgoing: a file Citizen Analyst is about to publish, attach or send. Is every redaction real, and does the file name anyone it should not?
The two directions use the same checks. They differ in what a hit means: incoming, it is a finding about someone else’s file and an editorial question (§5); outgoing, it is our failure and the file does not leave (§4).
Everything runs locally. Never upload a source document, an unpublished production or a draft to an online redaction checker or PDF service: the service then holds the document. Free Law Project’s x-ray runs on our own machine for exactly this reason.
Use a different skill when:
| Task | Use instead |
|---|---|
| Logging, numbering, coding and citing a production | Citizen Analyst’s production-review method (this skill’s checks run on its working copies) |
| Litigation discovery, privilege review, FOIA exemptions, Vaughn indexes | litigation-support-ediscovery-analyst |
| Source intake, encrypted channels, metadata stripping in general, source audio | secure-source-communication |
| Is this leaked file or image authentic? | media-verification-specialist, then source-credibility-scoring |
| Requesting records or appealing a withholding | foia-and-sunshine-laws; requests are drafted for a human to file |
Ground every factual or legal claim through scripts/ca_research.py and cite it. Never state from memory whether something may lawfully be published (sealed material, a protective order, a statute that bars naming a victim): that question goes to counsel (§5).
1. What a PDF can still hide, and what catches it
A PDF is not a picture of a page. It holds text objects, images, drawings, annotations, metadata and, after an incremental save, earlier versions of itself. A redaction is real only when the hidden content is gone from the file, not merely covered.
The “caught by” column is our test (2026-11-13, x-ray 0.3.6, PyMuPDF 1.24.14, poppler pdftotext/pdfimages, one synthetic page per case), not a vendor claim.
| # | Failure | What is left in the file | x-ray | Also caught by |
|---|---|---|---|---|
| F1 | Black rectangle drawn over live text | The text | Yes | pdftotext; select-all and copy |
| F2 | Black square annotation over live text | The text | Yes | pdftotext |
| F3 | Redaction marked but never applied (a redact annotation not yet applied) | The text | No | pdftotext |
| F4 | Black box drawn over a scanned image | The whole unredacted image, under the box | No | pdfimages extracts it intact (pixel-identical in our test) |
| F5 | Scan redacted in the image, but its OCR text layer kept | The text, invisible | No | pdftotext |
| F6 | Redaction applied, then saved incrementally | The earlier revision, appended before it | No | Revision count (§2): truncating our test file at its first %%EOF gave back the full unredacted text, while x-ray and pdftotext on the final file both came back clean |
| F7 | Author, organisation, software, dates, file paths in metadata (Info dictionary and XMP) | Who made it, on what, when | No | pdfinfo, exiftool |
| F8 | Comments, tracked changes carried over from Word, hidden layers, attachments, form-field values, bookmarks | Whatever they contain | No | Acrobat’s Sanitize, pdfinfo (form, JavaScript), a viewer’s attachments and layers panes |
| F9 | Text truly removed, but the gap width and the positions of the remaining characters stay | Enough to narrow or identify a short redacted name | No | Nothing automatic. See §4 step 6 |
x-ray is right about what it says it does: Free Law Project describes it as supporting “only the most basic (and most common) type of bad redaction, rectangles on top of text”. It also skips rectangles at the very top of a page (bottom edge at or above 43 points, treated as headers) and filters boxes whose recovered text is only dates or repeated characters (scripts/lib/redaction_check.py docstring, read from x-ray’s source). An x-ray result of “nothing found” means F1 and F2 were not found. It does not mean the file is clean.
F6 is the one that fools careful people: a correct redaction tool, then a “Save” that appends the change instead of rewriting the file. The original stays inside. PDFResurrect was written to pull every earlier version out of public PDFs, by finding the %%EOF marker each revision leaves (757 Labs, “Faith in the Format”). ExifTool behaves the same way on PDFs: its edits are appended, so “old information is never actually deleted from the file” until the file is rewritten (ExifTool PDF tag documentation).
F9 is research, not folklore: Bland, Iyer and Levchenko showed that glyph sizes and positions around a redaction leak information; redacting a surname from a Word-made PDF in 10-point Calibri left enough to identify the name uniquely in 14 percent of cases, which they call a lower bound (WIRED, 25 Nov 2022).
2. The checks
All local. Run them on a working copy, never on the original as received (§3).
| Check | Command | Read it as |
|---|---|---|
| Text under boxes (F1, F2) | make xray PDF= in the citizen-analyst repo (exit 0 clean, 1 findings, 2 could not check) |
A finding prints the page, the box and up to 40 characters of the recovered text. Exit 2 is not clean |
| All extractable text (F1–F3, F5) | pdftotext |
The machine’s select-all. Search it for every term that should be gone. On a scan, any text at all means an OCR layer exists |
| Images (F4) | pdfimages -list , then pdfimages -png into a scratch directory |
A full-page image on a redacted page: extract it and look at it without the box |
| Revisions (F6) | .venv/bin/python -I -c "import fitz,sys; print(fitz.open(sys.argv[1]).version_count)" |
1 is one revision. More than 1 means earlier versions are inside. Counting %%EOF is a rough proxy only: a linearized file (pdfinfo says Optimized: yes) carries two markers and one revision |
| Metadata (F7) | pdfinfo ; exiftool -a -G1 |
Author, Creator, Producer, dates, custom keys, XMP |
| Hidden extras (F8) | pdfinfo (Form, JavaScript lines); a viewer’s layers, attachments and comments panes |
Anything present is read before the file is used or published |
make xray and its library (scripts/lib/redaction_check.py) fail closed: when x-ray cannot run, they say CANNOT CHECK, never “clean”.
Read the findings, do not store them. A finding is a pointer to what was meant to be hidden. Do not paste recovered text into a ticket, a chat or a shared document; note the page and the box.
3. Incoming: a PDF we received
- Ingest first. Follow the production-review method’s ingest step: save the original read-only in
originals/, SHA-256 it, log it, number it. Every check below runs on a copy inworking/. Never re-save the original: even opening and saving it can rewrite it. - Run every §2 check, not only x-ray. A FOIA release is often a scan (F4, F5); a court filing is often a Word export (F1, F7, F8); an agency web PDF may have been edited and saved many times (F6).
- Record the result in the production log’s
custody_notefor that row: which checks ran, on which date, by whom, and “no findings” or “findings on pages N, M”. Do not record the recovered text there. - A hit is a finding about the file, not a fact about the world. Recovered text is what someone drafted and then tried to hide; it can be wrong, superseded or a placeholder. Code it as a lead in the production log and verify it against an independent record before it is reported.
- Withheld is not the same as failed. A FOIA exemption cited on a page (
litigation-support-ediscovery-analyst, the exemption table) is a legal withholding. Text recovered from under it raises §5, and the agency’s cited exemption goes in the log beside the finding. - Escalate before reading further when the recovered material is on its face classified, privileged, sealed by a court, or names a victim, a minor or a confidential informant (§5).
4. Outgoing: before Citizen Analyst publishes a PDF
The rule from secure-source-communication §4 is the rule here: burn redactions in; never draw a box over text. This section is the full procedure.
- Ask whether a facsimile is needed at all. Quoting the passage, or retyping the page with the redactions as plain
[redacted], publishes no file layers, no metadata and no glyph positions. Publish the PDF only when the document’s form is itself the evidence. - Redact by removal. Use a tool that deletes the content under the mark: Acrobat’s Redact tool with the marks applied, or PyMuPDF (
page.add_redact_annot(rect), thenpage.apply_redactions()). Never a drawing tool, a highlighter or a black font. An unapplied redact mark is F3: apply it. - Scans: redact the image and drop the OCR layer (F4, F5). Burn the mark into the image itself, and remove or regenerate the text layer after redaction; never keep the pre-redaction OCR.
- Sanitize and strip metadata (F7, F8). Acrobat: Sanitize document removes metadata, comments, hidden layers and similar hidden content (Adobe help). Elsewhere: MAT2, which may rasterize a PDF and leave its text unselectable (its man page says so; MAT2 also does not handle watermarks), or PyMuPDF’s
doc.scrub(). ExifTool alone is not enough on a PDF: its removal is appended and reversible untilqpdf --linearize in.pdf out.pdfrewrites the file (ExifTool PDF tag documentation). - Rewrite the file; never save incrementally (F6). In PyMuPDF,
doc.save(out, garbage=4, deflate=True)to a new path, neversaveIncr(). In other tools, Save As a new file. In our test, a full rewrite after redaction andscrub()left one revision, no findings and an empty Author field; the incremental save of the same redaction left two revisions and the original text recoverable. - Names: consider the gap (F9). For a short name whose exposure would harm someone, a removal-based redaction can still leak through the width of the gap. Prefer step 1 (retype or quote), or redact the whole line or passage rather than the name alone.
- Verify the final exported file with every §2 check:
make xrayexit 0,pdftotextcontains none of the redacted terms, no unredacted image under any mark, revision count 1, metadata empty or deliberate. Then open it and select-all on each redacted page. - Log it: file SHA-256, date, who checked, each check’s result. The file that was checked is the file that is published; any later edit means checking again.
The upload gate is a backstop, not the check. WPClient.upload_media() refuses any PDF (by .pdf suffix or %PDF- content) in which x-ray finds text under a box, and refuses every PDF when x-ray cannot run. It has no bypass; the fix is to redact properly and upload again. It runs x-ray only, so it cannot catch F3–F9. Passing the gate is not evidence that steps 2–7 were done.
5. Publishing what someone else failed to hide
Recovering text from a failed redaction is easy; deciding to publish it is an editorial and, sometimes, a legal decision. It is never made by the person who ran the check alone.
- It can be the story. In January 2019, reporters read the text under the black bars in Paul Manafort’s lawyers’ court filing by copying it out, and reported it, including that he was accused of sharing polling data with an associate (CJR; CNBC). Information about official conduct that an official or a party failed to hide can be newsworthy.
- It can be the harm. In February 2026, lawyers for Jeffrey Epstein’s victims asked federal judges to force the Justice Department to take down the files it had posted, citing “thousands of redaction failures” affecting nearly 100 survivors; DOJ said it was removing documents for further redaction on request (CNN, 2 Feb 2026). Names left visible in part in one document and in part in another made people identifiable (NPR, 6 Feb 2026). A newsroom that republishes such a page, or names someone from it, repeats the harm after the source has corrected it.
House rules:
- Escalate every recovered passage to the editor before it enters a draft. Write down what was recovered, from which file and how, and who decided.
- Never publish, from a failed redaction, the identity of a victim, a minor, a confidential informant, a witness who was promised confidentiality, or a private person who has not passed
patriot-private-citizen-inclusion-gate. - Never republish the unredacted file. If the story depends on the recovered text, report it in our words with our verification; link the producer’s own copy, not ours.
- Sealed, classified or privileged on its face, or under a protective order: stop and go to counsel before anyone reads further (the production-review method’s safety rule). Whether it may be published is a legal question this skill does not answer.
- Say how we know. If recovered text is reported, the article says it came from a redaction that did not hold, so a reader can judge it.
- Verify it like any other lead (§3 step 4). A failed redaction tells you what someone wrote, not that it is true.
6. Protecting our own sources (the outgoing half of secure-source-communication)
A document a source gave us can identify the source even when every redaction holds. Before any part of it is published or shared outside the trust group:
- Metadata names people. Author, organisation, software and edit dates (§1 F7);
secure-source-communication§3 has the stripping tools. The NSA warned in 2005 that converting a Word document to PDF “does not remove all [sensitive] metadata automatically” (FAS summary of Redacting with Confidence). A 2021 study of 39,664 PDFs published by 75 security agencies in 47 countries found only 7 agencies sanitizing any of them, and sensitive information still present in 65 percent of the sanitized files (Adhatarao and Lauradoux, arXiv 2103.02707). - Content identifies too. Stripping removes metadata, not content: a distribution list, a unique phrasing, a document version only a few people received, or a watermark. MAT2’s own documentation says it does not anonymise content and cannot handle watermarking. Weigh these before publishing a facsimile; retyping (§4 step 1) removes most of them.
- Keep the original encrypted, as received, for chain of custody (
secure-source-communication§3); publish only the checked derivative. - Never upload the source’s file to an online checker, converter or AI service (Instructions).
Sources
Read through scripts/ca_research.py on 2026-11-13 unless marked “our test”.
- Free Law Project, “X-Ray Bad Redaction Detector”, https://free.law/projects/x-ray; repository https://github.com/freelawproject/x-ray (BSD 2-Clause).
- Citizen Analyst
scripts/lib/redaction_check.pyandWPClient.upload_media()(JR P5.1, PR #445): x-ray’s limits as read from its source, the fail-closed behaviour and the upload gate. - Our test, 2026-11-13: x-ray 0.3.6, PyMuPDF 1.24.14, poppler
pdftotextandpdfimages, one synthetic page per failure F1–F6, F7 (Author field). Results as in §1 and §4 step 5. - ExifTool, PDF Tags (“All metadata edits are reversible … old information is never actually deleted from the file”;
qpdf --linearize), https://exiftool.org/TagNames/PDF.html. - MAT2 manual page (PDFs may be converted to images; lightweight mode), https://manpages.ubuntu.com/manpages/focal/man1/mat2.1.html; MAT2’s stated limits on content and watermarking, quoted at https://www.whonix.org/wiki/Metadata.
- Adobe, “Sanitize PDFs and remove hidden content in Acrobat Pro” (metadata, comments, hidden layers), https://helpx.adobe.com/acrobat/desktop/protect-documents/redact-pdfs/sanitize.html.
- 757 Labs, “Faith in the Format: Unintentional Data Hiding in PDFs” (pdfresurrect,
%%EOFper revision), https://757labs.org/projects/pdfresurrect/pdfresurrect_paper.pdf. - WIRED, Matt Burgess, “Redacted Documents Are Not as Secure as You Think”, 25 Nov 2022 (Bland, Iyer and Levchenko; the 14 percent surname result), https://www.wired.com/story/redact-pdf-online-privacy.
- Federation of American Scientists, “NSA: Redacting With Confidence”, 20 Jan 2006, https://fas.org/publication/nsa_redacting_with_confidence.
- Supriya Adhatarao and Cédric Lauradoux, “Exploitation and Sanitization of Hidden Data in PDF Files”, arXiv:2103.02707 (2021), https://arxiv.org/abs/2103.02707.
- Columbia Journalism Review, “Thank you to everyone who can’t redact documents properly” (Manafort filing, Jan 2019), https://www.cjr.org/analysis/manafort-mueller-redacted-document-ukraine.php; CNBC, 8 Jan 2019, https://www.cnbc.com/2019/01/08/mueller-accused-manafort-of-lying-about-sharing-polling-data-with-spy.html.
- CNN, “Epstein victims’ lawyers ask judges to force takedown of released files, citing ‘thousands of redaction failures'”, 2 Feb 2026, https://www.cnn.com/2026/02/02/politics/epstein-victims-demand-takedown-epstein-files; NPR, 6 Feb 2026 (search summary; the page did not extract), https://www.npr.org/2026/02/06/nx-s1-5702692/epstein-files-doj-trump-clinton-oversight.
Cross-references
- Citizen Analyst’s production-review method: ingest, hashes, control numbers and the production log this skill writes its results into.
litigation-support-ediscovery-analyst: FOIA exemptions, privilege, and production formats.secure-source-communication: §3 metadata stripping, §4 redaction (this skill is its full procedure), §5 source audio.patriot-private-citizen-inclusion-gate: before any private person recovered from a failed redaction is named.media-verification-specialist,source-credibility-scoring: authenticity of a leaked file.foia-and-sunshine-laws: getting the records (requests are drafted for a human to file), and appealing a withholding.
Safety
- Local only. No source document, production or draft goes to an online checker or converter.
- x-ray clean is not clean. It finds text under rectangles and nothing else (§1). Run every §2 check.
- Fail closed. “Could not check” is never “checked clean”, for
make xrayor for a person. - Never a box over text in anything we publish. Redact by removal, rewrite the file, verify the final file.
- Recovered text is a lead. Verify it before it is reported, and say how it was obtained.
- Escalate before publishing anything recovered from a failed redaction; never publish a victim’s, minor’s, informant’s or ungated private person’s identity from one.
- Sealed, classified or privileged material goes to counsel first. Not legal advice.
- Symmetry. The same checks and the same publishing test apply whoever produced the file and whoever it embarrasses.
