Redaction and PDF Forensics
Media & Document Verification

Redaction and PDF Forensics

Skip to main content
Table of Contents
< All Topics
Print

Redaction and PDF Forensics

Last updated: November 15, 2026

Instructions

You check PDFs for what they still hide, in two directions:

  • Incoming: a file we received (a FOIA or state-records release, a committee production, a court filing, an agency web page, a leak). Does it carry text, images or earlier versions that the sender meant to remove?
  • Outgoing: a file Citizen Analyst is about to publish, attach or send. Is every redaction real, and does the file name anyone it should not?

The two directions use the same checks. They differ in what a hit means: incoming, it is a finding about someone else’s file and an editorial question (§5); outgoing, it is our failure and the file does not leave (§4).

Everything runs locally. Never upload a source document, an unpublished production or a draft to an online redaction checker or PDF service: the service then holds the document. Free Law Project’s x-ray runs on our own machine for exactly this reason.

Use a different skill when:

Task Use instead
Logging, numbering, coding and citing a production Citizen Analyst’s production-review method (this skill’s checks run on its working copies)
Litigation discovery, privilege review, FOIA exemptions, Vaughn indexes litigation-support-ediscovery-analyst
Source intake, encrypted channels, metadata stripping in general, source audio secure-source-communication
Is this leaked file or image authentic? media-verification-specialist, then source-credibility-scoring
Requesting records or appealing a withholding foia-and-sunshine-laws; requests are drafted for a human to file

Ground every factual or legal claim through scripts/ca_research.py and cite it. Never state from memory whether something may lawfully be published (sealed material, a protective order, a statute that bars naming a victim): that question goes to counsel (§5).


1. What a PDF can still hide, and what catches it

A PDF is not a picture of a page. It holds text objects, images, drawings, annotations, metadata and, after an incremental save, earlier versions of itself. A redaction is real only when the hidden content is gone from the file, not merely covered.

The “caught by” column is our test (2026-11-13, x-ray 0.3.6, PyMuPDF 1.24.14, poppler pdftotext/pdfimages, one synthetic page per case), not a vendor claim.

# Failure What is left in the file x-ray Also caught by
F1 Black rectangle drawn over live text The text Yes pdftotext; select-all and copy
F2 Black square annotation over live text The text Yes pdftotext
F3 Redaction marked but never applied (a redact annotation not yet applied) The text No pdftotext
F4 Black box drawn over a scanned image The whole unredacted image, under the box No pdfimages extracts it intact (pixel-identical in our test)
F5 Scan redacted in the image, but its OCR text layer kept The text, invisible No pdftotext
F6 Redaction applied, then saved incrementally The earlier revision, appended before it No Revision count (§2): truncating our test file at its first %%EOF gave back the full unredacted text, while x-ray and pdftotext on the final file both came back clean
F7 Author, organisation, software, dates, file paths in metadata (Info dictionary and XMP) Who made it, on what, when No pdfinfo, exiftool
F8 Comments, tracked changes carried over from Word, hidden layers, attachments, form-field values, bookmarks Whatever they contain No Acrobat’s Sanitize, pdfinfo (form, JavaScript), a viewer’s attachments and layers panes
F9 Text truly removed, but the gap width and the positions of the remaining characters stay Enough to narrow or identify a short redacted name No Nothing automatic. See §4 step 6

x-ray is right about what it says it does: Free Law Project describes it as supporting “only the most basic (and most common) type of bad redaction, rectangles on top of text”. It also skips rectangles at the very top of a page (bottom edge at or above 43 points, treated as headers) and filters boxes whose recovered text is only dates or repeated characters (scripts/lib/redaction_check.py docstring, read from x-ray’s source). An x-ray result of “nothing found” means F1 and F2 were not found. It does not mean the file is clean.

F6 is the one that fools careful people: a correct redaction tool, then a “Save” that appends the change instead of rewriting the file. The original stays inside. PDFResurrect was written to pull every earlier version out of public PDFs, by finding the %%EOF marker each revision leaves (757 Labs, “Faith in the Format”). ExifTool behaves the same way on PDFs: its edits are appended, so “old information is never actually deleted from the file” until the file is rewritten (ExifTool PDF tag documentation).

F9 is research, not folklore: Bland, Iyer and Levchenko showed that glyph sizes and positions around a redaction leak information; redacting a surname from a Word-made PDF in 10-point Calibri left enough to identify the name uniquely in 14 percent of cases, which they call a lower bound (WIRED, 25 Nov 2022).


2. The checks

All local. Run them on a working copy, never on the original as received (§3).

Check Command Read it as
Text under boxes (F1, F2) make xray PDF= in the citizen-analyst repo (exit 0 clean, 1 findings, 2 could not check) A finding prints the page, the box and up to 40 characters of the recovered text. Exit 2 is not clean
All extractable text (F1–F3, F5) pdftotext - The machine’s select-all. Search it for every term that should be gone. On a scan, any text at all means an OCR layer exists
Images (F4) pdfimages -list , then pdfimages -png into a scratch directory A full-page image on a redacted page: extract it and look at it without the box
Revisions (F6) .venv/bin/python -I -c "import fitz,sys; print(fitz.open(sys.argv[1]).version_count)" 1 is one revision. More than 1 means earlier versions are inside. Counting %%EOF is a rough proxy only: a linearized file (pdfinfo says Optimized: yes) carries two markers and one revision
Metadata (F7) pdfinfo ; exiftool -a -G1 Author, Creator, Producer, dates, custom keys, XMP
Hidden extras (F8) pdfinfo (Form, JavaScript lines); a viewer’s layers, attachments and comments panes Anything present is read before the file is used or published

make xray and its library (scripts/lib/redaction_check.py) fail closed: when x-ray cannot run, they say CANNOT CHECK, never “clean”.

Read the findings, do not store them. A finding is a pointer to what was meant to be hidden. Do not paste recovered text into a ticket, a chat or a shared document; note the page and the box.


3. Incoming: a PDF we received

  1. Ingest first. Follow the production-review method’s ingest step: save the original read-only in originals/, SHA-256 it, log it, number it. Every check below runs on a copy in working/. Never re-save the original: even opening and saving it can rewrite it.
  2. Run every §2 check, not only x-ray. A FOIA release is often a scan (F4, F5); a court filing is often a Word export (F1, F7, F8); an agency web PDF may have been edited and saved many times (F6).
  3. Record the result in the production log’s custody_note for that row: which checks ran, on which date, by whom, and “no findings” or “findings on pages N, M”. Do not record the recovered text there.
  4. A hit is a finding about the file, not a fact about the world. Recovered text is what someone drafted and then tried to hide; it can be wrong, superseded or a placeholder. Code it as a lead in the production log and verify it against an independent record before it is reported.
  5. Withheld is not the same as failed. A FOIA exemption cited on a page (litigation-support-ediscovery-analyst, the exemption table) is a legal withholding. Text recovered from under it raises §5, and the agency’s cited exemption goes in the log beside the finding.
  6. Escalate before reading further when the recovered material is on its face classified, privileged, sealed by a court, or names a victim, a minor or a confidential informant (§5).

4. Outgoing: before Citizen Analyst publishes a PDF

The rule from secure-source-communication §4 is the rule here: burn redactions in; never draw a box over text. This section is the full procedure.

  1. Ask whether a facsimile is needed at all. Quoting the passage, or retyping the page with the redactions as plain [redacted], publishes no file layers, no metadata and no glyph positions. Publish the PDF only when the document’s form is itself the evidence.
  2. Redact by removal. Use a tool that deletes the content under the mark: Acrobat’s Redact tool with the marks applied, or PyMuPDF (page.add_redact_annot(rect), then page.apply_redactions()). Never a drawing tool, a highlighter or a black font. An unapplied redact mark is F3: apply it.
  3. Scans: redact the image and drop the OCR layer (F4, F5). Burn the mark into the image itself, and remove or regenerate the text layer after redaction; never keep the pre-redaction OCR.
  4. Sanitize and strip metadata (F7, F8). Acrobat: Sanitize document removes metadata, comments, hidden layers and similar hidden content (Adobe help). Elsewhere: MAT2, which may rasterize a PDF and leave its text unselectable (its man page says so; MAT2 also does not handle watermarks), or PyMuPDF’s doc.scrub(). ExifTool alone is not enough on a PDF: its removal is appended and reversible until qpdf --linearize in.pdf out.pdf rewrites the file (ExifTool PDF tag documentation).
  5. Rewrite the file; never save incrementally (F6). In PyMuPDF, doc.save(out, garbage=4, deflate=True) to a new path, never saveIncr(). In other tools, Save As a new file. In our test, a full rewrite after redaction and scrub() left one revision, no findings and an empty Author field; the incremental save of the same redaction left two revisions and the original text recoverable.
  6. Names: consider the gap (F9). For a short name whose exposure would harm someone, a removal-based redaction can still leak through the width of the gap. Prefer step 1 (retype or quote), or redact the whole line or passage rather than the name alone.
  7. Verify the final exported file with every §2 check: make xray exit 0, pdftotext contains none of the redacted terms, no unredacted image under any mark, revision count 1, metadata empty or deliberate. Then open it and select-all on each redacted page.
  8. Log it: file SHA-256, date, who checked, each check’s result. The file that was checked is the file that is published; any later edit means checking again.

The upload gate is a backstop, not the check. WPClient.upload_media() refuses any PDF (by .pdf suffix or %PDF- content) in which x-ray finds text under a box, and refuses every PDF when x-ray cannot run. It has no bypass; the fix is to redact properly and upload again. It runs x-ray only, so it cannot catch F3–F9. Passing the gate is not evidence that steps 2–7 were done.


5. Publishing what someone else failed to hide

Recovering text from a failed redaction is easy; deciding to publish it is an editorial and, sometimes, a legal decision. It is never made by the person who ran the check alone.

  • It can be the story. In January 2019, reporters read the text under the black bars in Paul Manafort’s lawyers’ court filing by copying it out, and reported it, including that he was accused of sharing polling data with an associate (CJR; CNBC). Information about official conduct that an official or a party failed to hide can be newsworthy.
  • It can be the harm. In February 2026, lawyers for Jeffrey Epstein’s victims asked federal judges to force the Justice Department to take down the files it had posted, citing “thousands of redaction failures” affecting nearly 100 survivors; DOJ said it was removing documents for further redaction on request (CNN, 2 Feb 2026). Names left visible in part in one document and in part in another made people identifiable (NPR, 6 Feb 2026). A newsroom that republishes such a page, or names someone from it, repeats the harm after the source has corrected it.

House rules:

  1. Escalate every recovered passage to the editor before it enters a draft. Write down what was recovered, from which file and how, and who decided.
  2. Never publish, from a failed redaction, the identity of a victim, a minor, a confidential informant, a witness who was promised confidentiality, or a private person who has not passed patriot-private-citizen-inclusion-gate.
  3. Never republish the unredacted file. If the story depends on the recovered text, report it in our words with our verification; link the producer’s own copy, not ours.
  4. Sealed, classified or privileged on its face, or under a protective order: stop and go to counsel before anyone reads further (the production-review method’s safety rule). Whether it may be published is a legal question this skill does not answer.
  5. Say how we know. If recovered text is reported, the article says it came from a redaction that did not hold, so a reader can judge it.
  6. Verify it like any other lead (§3 step 4). A failed redaction tells you what someone wrote, not that it is true.

6. Protecting our own sources (the outgoing half of secure-source-communication)

A document a source gave us can identify the source even when every redaction holds. Before any part of it is published or shared outside the trust group:

  • Metadata names people. Author, organisation, software and edit dates (§1 F7); secure-source-communication §3 has the stripping tools. The NSA warned in 2005 that converting a Word document to PDF “does not remove all [sensitive] metadata automatically” (FAS summary of Redacting with Confidence). A 2021 study of 39,664 PDFs published by 75 security agencies in 47 countries found only 7 agencies sanitizing any of them, and sensitive information still present in 65 percent of the sanitized files (Adhatarao and Lauradoux, arXiv 2103.02707).
  • Content identifies too. Stripping removes metadata, not content: a distribution list, a unique phrasing, a document version only a few people received, or a watermark. MAT2’s own documentation says it does not anonymise content and cannot handle watermarking. Weigh these before publishing a facsimile; retyping (§4 step 1) removes most of them.
  • Keep the original encrypted, as received, for chain of custody (secure-source-communication §3); publish only the checked derivative.
  • Never upload the source’s file to an online checker, converter or AI service (Instructions).

Sources

Read through scripts/ca_research.py on 2026-11-13 unless marked “our test”.

  • Free Law Project, “X-Ray Bad Redaction Detector”, https://free.law/projects/x-ray; repository https://github.com/freelawproject/x-ray (BSD 2-Clause).
  • Citizen Analyst scripts/lib/redaction_check.py and WPClient.upload_media() (JR P5.1, PR #445): x-ray’s limits as read from its source, the fail-closed behaviour and the upload gate.
  • Our test, 2026-11-13: x-ray 0.3.6, PyMuPDF 1.24.14, poppler pdftotext and pdfimages, one synthetic page per failure F1–F6, F7 (Author field). Results as in §1 and §4 step 5.
  • ExifTool, PDF Tags (“All metadata edits are reversible … old information is never actually deleted from the file”; qpdf --linearize), https://exiftool.org/TagNames/PDF.html.
  • MAT2 manual page (PDFs may be converted to images; lightweight mode), https://manpages.ubuntu.com/manpages/focal/man1/mat2.1.html; MAT2’s stated limits on content and watermarking, quoted at https://www.whonix.org/wiki/Metadata.
  • Adobe, “Sanitize PDFs and remove hidden content in Acrobat Pro” (metadata, comments, hidden layers), https://helpx.adobe.com/acrobat/desktop/protect-documents/redact-pdfs/sanitize.html.
  • 757 Labs, “Faith in the Format: Unintentional Data Hiding in PDFs” (pdfresurrect, %%EOF per revision), https://757labs.org/projects/pdfresurrect/pdfresurrect_paper.pdf.
  • WIRED, Matt Burgess, “Redacted Documents Are Not as Secure as You Think”, 25 Nov 2022 (Bland, Iyer and Levchenko; the 14 percent surname result), https://www.wired.com/story/redact-pdf-online-privacy.
  • Federation of American Scientists, “NSA: Redacting With Confidence”, 20 Jan 2006, https://fas.org/publication/nsa_redacting_with_confidence.
  • Supriya Adhatarao and Cédric Lauradoux, “Exploitation and Sanitization of Hidden Data in PDF Files”, arXiv:2103.02707 (2021), https://arxiv.org/abs/2103.02707.
  • Columbia Journalism Review, “Thank you to everyone who can’t redact documents properly” (Manafort filing, Jan 2019), https://www.cjr.org/analysis/manafort-mueller-redacted-document-ukraine.php; CNBC, 8 Jan 2019, https://www.cnbc.com/2019/01/08/mueller-accused-manafort-of-lying-about-sharing-polling-data-with-spy.html.
  • CNN, “Epstein victims’ lawyers ask judges to force takedown of released files, citing ‘thousands of redaction failures'”, 2 Feb 2026, https://www.cnn.com/2026/02/02/politics/epstein-victims-demand-takedown-epstein-files; NPR, 6 Feb 2026 (search summary; the page did not extract), https://www.npr.org/2026/02/06/nx-s1-5702692/epstein-files-doj-trump-clinton-oversight.

Cross-references

  • Citizen Analyst’s production-review method: ingest, hashes, control numbers and the production log this skill writes its results into.
  • litigation-support-ediscovery-analyst: FOIA exemptions, privilege, and production formats.
  • secure-source-communication: §3 metadata stripping, §4 redaction (this skill is its full procedure), §5 source audio.
  • patriot-private-citizen-inclusion-gate: before any private person recovered from a failed redaction is named.
  • media-verification-specialist, source-credibility-scoring: authenticity of a leaked file.
  • foia-and-sunshine-laws: getting the records (requests are drafted for a human to file), and appealing a withholding.

Safety

  • Local only. No source document, production or draft goes to an online checker or converter.
  • x-ray clean is not clean. It finds text under rectangles and nothing else (§1). Run every §2 check.
  • Fail closed. “Could not check” is never “checked clean”, for make xray or for a person.
  • Never a box over text in anything we publish. Redact by removal, rewrite the file, verify the final file.
  • Recovered text is a lead. Verify it before it is reported, and say how it was obtained.
  • Escalate before publishing anything recovered from a failed redaction; never publish a victim’s, minor’s, informant’s or ungated private person’s identity from one.
  • Sealed, classified or privileged material goes to counsel first. Not legal advice.
  • Symmetry. The same checks and the same publishing test apply whoever produced the file and whoever it embarrasses.
Was this article helpful?
0 out of 5 stars
5 Stars 0%
4 Stars 0%
3 Stars 0%
2 Stars 0%
1 Stars 0%
5
Please Share Your Feedback
How Can We Improve This Article?