Numbers and Statistics Integrity
Last updated: November 15, 2026
Instructions
We use this skill on every number a piece will publish. A number can be accurately copied from a good source and still mislead: a raw count where a rate was needed, a percentage-point change written as a percent change, a “record” that is a record only in nominal dollars, a poll “lead” inside the margin of error. This skill catches those before a reader does.
It composes three existing skills and restates none of them:
fact-checkingis primary. Its §1 already lists “statistics and numbers” as a claim
type and its §3 already says “check that statistics cite their original source, not secondary reports”. This skill is the arithmetic and framing pass that runs on the numbers fact-checking extracts.
source-credibility-scoringreceives the findings. Its Dimension B (Attribution)
already deducts for “quantitative claims (dollar amounts, statistics, dates) lacking a verifiable source”. This skill’s findings go into that dimension and its neighbours (§3 below). It never produces a separate number-quality score.
claims-integrity-auditis secondary. Its Step 3 verdict scale (Verified, Supported,
Needs Context, Unsupported, Misleading, False) grades each number, and its Step 4 “Quantitative red flags” (a percentage or multiplier with no named baseline, counts that conflate entity types, cherry-picked time windows) are the fast triage.
Method, in order:
- Extract every number in the draft into the numbers register (§1), using
fact-checking §1.
- Trace each number to its original publisher (
fact-checking§3). A number with no
verifiable source is a finding before any arithmetic is checked.
- Run the checks in §2 that apply to that number. Show the arithmetic in the register.
- Grade each number with the
claims-integrity-auditverdicts and hand the findings
to source-credibility-scoring in its own finding shape (§3).
Use a different skill when:
- the question is whether a source or outlet is reliable, not whether its number is used
correctly: source-credibility-scoring (and the Source Credibility Registry);
- the number is a quotation of what someone said: Citizen Analyst’s quote-integrity check verifies the
quote first, then this skill checks the number inside it;
- the task is scoring a person, action, market or agency: the instruments (DMA, SMA, DRA,
LIA, WLM, ADA) own their own scales; this skill only checks the numbers they cite.
1. The numbers register
One row per number in the draft. The register is a working file for the editor, not reader-facing.
| # | Line | Number as written | Original source (publisher, table, date) | What it counts (unit, universe, year) | Checks run (§2) | Arithmetic shown | Verdict | Finding |
|---|---|---|---|---|---|---|---|---|
“What it counts” is the column that catches most errors: households or people, adults or all ages, fiscal or calendar year, nominal or constant dollars, a state or the nation.
2. The checks
Run every check that applies to a number. Each check names the question to ask and the source of the method.
2.1 Raw counts, rates and per-capita figures
- A count compared across places of different size needs a rate. “Most arrests in
Texas” says little until it is divided by population. To compute a per-capita figure, divide the count by the population of the same place in the same year ([The Journalist’s Resource, “Using ‘per capita’ to describe data: 4 things journalists need to know”](https://journalistsresource.org/economics/per-capita-right-wrong-journalists-tips), tip 2).
- “Per capita” means per person. A rate per 10,000 or per 100,000 is a rate, not a
per-capita figure; do not call it one (same source, tip 3).
- A per-capita figure is an average and hides the distribution. Where the spread
matters, report the median as well (same source, tips 1 and 4).
- Denominator source. For a state population, use the Census Bureau’s ACS through
scripts/ca_research.py api api.census ", which returns total population (B01003_001E) and median household income (B19013_001E) for each state named. Say which ACS vintage the denominator came from.
2.2 Small numbers and unstable rates
A rate built on a handful of events moves a lot from year to year by chance. CDC WONDER flags a death rate based on fewer than 20 deaths as “Unreliable”, because its relative standard error is 23 percent or more (CDC WONDER, Underlying Cause of Death, FAQ 7). Apply the same caution to any small-count rate: give the count beside the rate, and do not rank places on rates built from small counts.
2.3 Base rates and anecdote
One vivid case is not a trend. Ask what share of the whole the cases are, and how that share compares with the rate in the general population or the same period last year. Personal experience covers a tiny fraction of any population; that is why carefully collected statistics are needed at all (Our World in Data, “The limits of our personal experience and the value of statistics”).
2.4 Percent change and percentage-point change
- Percent change is the rate of change; percentage points are the amount of change between
two percentages. A rate that rises from 1% to 5% rose 4 percentage points, which is a 400% increase; it did not “rise 4%” (The Journalist’s Resource, “Percent change and percentage-point change: 4 tips to avoid math errors”, tips 1–2).
- Recompute every percent change from the two underlying figures:
(new − old) ÷ old × 100.
- Prefer plain words for large multiples: “doubled” rather than “rose 100%” (same source,
tip 3).
2.5 “X times more” and other multipliers
- A multiplier needs both raw figures beside it. “Three times more likely” with no base is the
claims-integrity-audit red flag “percentages or multipliers with no named baseline”.
- “Times more” and especially “times less” or “times slower” are disputed usages that readers
parse differently (Language Log, “Times more / less than”, 21 Oct 2012). Write “three times as many” (or “a third as many”) and give the two numbers.
- Do not mix forms in one comparison: “fell from 65% to half” makes the reader convert
(Online Journalism Blog, “How to (not) write about numbers”, 30 Sep 2025).
2.6 Relative risk and absolute risk
A relative risk (“doubles the risk”) without the absolute risk misleads: doubling a 1-in-a- million risk is still a 2-in-a-million risk. Report both (The Journalist’s Resource, “4 details to include in news stories about risks to health and safety”, detail 3; and “Statistics for journalists: Understanding what effect size means”).
2.7 Averages and medians
The mean is pulled by a few very large or very small values; the median is the middle value. Incomes, home prices, settlements and fines are usually skewed, so the median is usually the honest “typical” figure (The Journalist’s Resource, “Math basics for journalists: Working with averages and percentages”). Never call a median an average, or the reverse. The ACS figure api.census returns is a median household income.
2.8 Survey and sample margins of error
- Report the margin of error. A result of 44% with a margin of ±3 points means the
population value is very likely between 41% and 47% (The Journalist’s Resource, “The margin of error: 7 tips for journalists writing about polls and surveys”).
- A lead inside the margin is not a lead. Being ahead by more than the margin reported
for each candidate is not enough on its own to call the race; the margin on the gap between two candidates is larger (Pew Research Center, “Understanding the margin of error in election polls”, 2016).
- No trend inside the margin. 31% in January and 33% in July, each ±2 points, is “holding
steady”, not “rising” (Journalist’s Resource, above).
- Subgroups have larger margins than the full sample: a group of 200 inside a poll of
1,000 has a margin of roughly ±7 points (Journalist’s Resource, above; Pew, “Tips for journalists covering polls”, 2024).
- Sampling error is not all the error. Pew notes that the real margin of error is often
about double the reported one (Pew, “Key things to know about election polls in the U.S.”, 2024).
- Census survey data has margins too. ACS margins of error are published at the
90 percent confidence level; two ACS estimates differ significantly at that level only if the test statistic Z built from their standard errors is above 1.645 or below −1.645 (Census Bureau, “Instructions for Applying Statistical Testing to ACS Data”). Do not rank counties or states on ACS estimates whose margins overlap without that test.
- Watch the adjectives. Monthly jobs numbers that differ by less than their margin of
error are not “grim” one month and “stellar” the next (Journalist’s Resource, above, tip 5).
2.9 Statistical significance and practical significance
A p-value of 0.05 means a 5% probability of seeing results this “interesting” if there were no real effect (The Journalist’s Resource, “Statistical terms used in research studies: A primer for media”). It does not measure how large or important the effect is. Report the effect size (Journalist’s Resource, effect size, above), and do not write “significant” in the everyday sense about a statistical result. Correlation is not, by itself, causation (same primer): a causal claim needs a source that establishes cause, or it is a claim-support finding.
2.10 Chart axis integrity
- Bar charts start at zero. A bar encodes its value by length; a truncated baseline makes
small differences look large (Duke University Libraries, “Chart Dos and Don’ts”).
- Line charts may use a non-zero baseline when the message is change relative to a
baseline other than zero (“Truncating the Y-Axis: Threat or Menace?”, NSF Public Access Repository). Our rule on top of that: label the axis so the truncation is visible.
- Also check: both axes labelled with units; the time window is not cherry-picked (a
claims-integrity-audit structural red flag); dollar series across years are in constant dollars (§2.11); a map of counts is normalised to a rate (§2.1).
- This applies to every chart a Compose generator emits, and to any chart we republish.
2.11 Units and inflation adjustment
- Dollars from different years are not comparable until adjusted. Convert nominal to
constant dollars by multiplying by the ratio of the price index for the base year to the index for the year of the figure (BLS, “Math calculations to better utilize CPI data”; BLS, “Purchasing power and constant dollars”). BLS also publishes a CPI Inflation Calculator for annual comparisons.
scripts/ca_research.py api api.bls ""
returns CPI-U all items, U.S. city average (series CUUR0000SA0). Name the series, the base year and the months used.
- If a figure is deliberately left nominal, say so in the text: “not adjusted for inflation”.
- Check units every time: thousands or millions, fiscal or calendar year, metric or US
customary, households or people.
2.12 “Record high”, “highest ever”, “unprecedented”
A record claim needs four answers, each sourced:
- Nominal or real? A nominal-dollar record is often not a record in constant dollars
(§2.11; Federal Reserve Bank of Dallas, “Deflating nominal values to real values”).
- Count or rate? A record count in a growing population may be no record per person
(§2.1).
- Since when? “Highest since the series began” is only as strong as the series’ start
date. Name it.
- Same definition throughout? A change in how a series is measured breaks the record.
Check the publisher’s notes for definition changes.
If any answer is missing, the claim is “Needs Context” at best.
2.13 Rounding and false precision
Round to the precision the source supports. “Precision without measurement methodology” is a claims-integrity-audit red flag, and an exact figure is often harder to read than a rounded one (“almost a fifth”; Online Journalism Blog, above). Check that rounded parts still add up to the stated whole, or say they do not because of rounding.
3. Grading, and where the findings go
Grade each number with the claims-integrity-audit Step 3 verdicts. Then write each finding in the shape source-credibility-scoring already uses (gui/services/document_scorer_service.py: severity error | warning | info; dimension attribution | claim_support | framing; message; line):
| What this skill found | Dimension | Typical severity |
|---|---|---|
| A number with no verifiable original source | attribution |
error |
| A number traced only to a secondary report | attribution |
warning |
| Arithmetic that does not reproduce (wrong percent change, wrong per-capita figure) | claim_support |
error |
| A causal claim resting on a correlation | claim_support |
warning |
| Points written as percent; a lead or trend inside the margin; relative risk with no absolute; a truncated bar axis; a nominal “record” | framing |
warning, or error if the headline depends on it |
| Missing context a reader needs (no year, no denominator, mean where a median is meant) | framing |
info or warning |
The DCS stays the only score. This skill adds findings; it does not add points, and it never auto-rejects.
Highlight and ask, never rewrite. As in source-credibility-scoring, flag the span and state the corrected figure with its arithmetic; the writer or editor makes the change.
4. Tools
scripts/ca_research.py api api.census "— ACS 5-year state"
population, median household income, poverty universe, citizen voting-age population. Needs CENSUS_API_KEY; the client reports “not run” without it.
scripts/ca_research.py api api.bls "— CPI-U (inflation), unemployment rate,"
payroll employment, average hourly earnings, state unemployment.
scripts/ca_research.py extract— read the original table or report, so the number is
cited at its source.
- Computation. Do the arithmetic by hand and show it in the register. A Wolfram MCP is
planned for interactive sessions but is not part of this skill. When one is available, send it only the arithmetic and topic text, never a source’s name or an unpublished lead.
Sources
All read on 2026-11-13 through scripts/ca_research.py (search and extract).
- The Journalist’s Resource (Harvard Kennedy School Shorenstein Center): per capita; percent
and percentage-point change; margin of error (7 tips); risks to health and safety; effect size; math basics (averages and percentages); statistical terms primer. Links in §2.
- Kathleen Wickham’s “Math Tools for Journalists”, described in
Quill, “Numbers Game: Getting the Math Right”, 23 Aug 2024.
math-for-journalists tip sheets and calculators, including the BLS inflation calculator.
- Our World in Data, “The limits of our personal experience and the value of statistics”.
- Pew Research Center: margin of error in election polls (2016); tips for journalists covering
polls (2024); key things to know about election polls (2024).
- U.S. Census Bureau: statistical testing for ACS estimates (90 percent confidence level).
- CDC WONDER: the fewer-than-20 “Unreliable” flag.
- Bureau of Labor Statistics: CPI math calculations; purchasing power and constant dollars.
- Federal Reserve Bank of Dallas: deflating nominal values to real values.
- Duke University Libraries, chart dos and don’ts; “Truncating the Y-Axis: Threat or Menace?”
(NSF Public Access Repository).
- Language Log, “Times more / less than”; Online Journalism Blog, “How to (not) write about
numbers” (2025).
Cross-references
fact-checking— primary: claim extraction and tracing a statistic to its original source.source-credibility-scoring— receives this skill’s findings (Dimension B and its
neighbours); the DCS is the only score.
claims-integrity-audit— verdict scale and quantitative red flags.- Citizen Analyst’s quote-integrity check — when the number sits inside a quotation.
infographic-builder— charts and maps must pass §2.10 and §2.1.patriot-sanity-check— runs aftersource-credibility-scoring.
Safety
- Never correct a number from memory. Every corrected figure cites the source it came
from and shows the arithmetic.
- Rigor, not direction. A number that helps the piece’s argument gets the same checks as
one that hurts it. Never round, rebase or choose a time window to make a point.
- Uncertainty is news. “Too close to call” and “no change beyond the margin of error” are
findings, not failures to report.
- Highlight and ask. This skill flags; the writer or editor decides the change.
- What leaves the session. Only arithmetic and topic text go to any outside computation
service; never a source’s name or an unpublished lead.
- Not reader-facing. The register and findings are internal editorial work product.
