Watching Source Pages
Last updated: November 20, 2026
What it is
Two small jobs watch the web pages our work depends on.
- The page-change watcher polls a short, editor-approved list of government pages. If a page changes or disappears, it raises an alert for an editor, who can capture the page as evidence.
- The source liveness probe checks weekly whether the tools and sources in our open-source research registry still answer.
Neither job uses AI to judge what it finds. They detect and report.
What is live, as of November 20, 2026: neither is fully switched on. Both are built and configured. We say so plainly, because a monitor that is described but not running is worse than none.
- The page watcher has 12 candidate pages. An editor has approved none of them, so none is installed and no page is being watched through this list.
- The weekly probe is defined, but the repository shows no sign that it is installed as a running job. Its one recorded run is dated October 6, 2026.
When it runs
Page watcher. Once switched on, each approved page is checked on its own interval, daily or twice a day in plain words. The code allows nothing faster than every six hours. Today nothing is approved, so nothing is scheduled.
Source probe. It is written to run weekly. Until the schedule is switched on, it runs only when an editor starts it by hand with make osint-probe.
What it does
Page watcher
- Starts from a written list.
deploy/linux/changedetection/watch-list.yamlholds each candidate page, the knowledge-base subject it is evidence for, why a change would matter, and the check interval. Each page is one that a knowledge-base document already cites. The candidates are agency leadership, organization and inspector-general pages, for bodies such as the Census Bureau, the IRS, the Office of Government Ethics, the CDC and AmeriCorps. - Installs only approved entries. An entry goes live only when a person fills in both
approved_byandapproved_on. The sync script (make changedetection-sync) is a dry run unless told to apply, and it touches only watches it created. - Watches and alerts. An open-source change-detection tool fetches each page on schedule. A change sends an alert that becomes an internal task for an editor.
- Captures evidence on request. An editor imports snapshots into our evidence store with
make import-changes. Each record keeps the page text, its hash, its time and its address.
Source liveness probe
- Takes our registry.
agent-base/osint-sources/registry.jsonlcatalogs open-source research tools and sources. As of November 20, 2026, it has 14,521 entries. - Asks each host once. One polite, identified request per host, 16 at a time, with no redirects followed. A refusal or bot block still counts as an answer.
- Tracks a simple state. One failed run marks a source
degraded. A second in a row marks itdown. One answer restores it tolive. The results file lists only exceptions. - Has a brake. If more than half the hosts fail, the probe assumes our own network is down and writes nothing. It will not run twice on the same day.
Skills and models it uses
No language model is involved. Scraping Ethics and Methods sets the conduct rules for collecting from websites. The page watcher also follows an internal method for watching web pages, which we describe here in our own words.
Rules and gates it obeys
- A page edit is a fact about the page, not proof of intent. Pages change for redesigns, migrations, routine archiving and legal review. The method says to rule those out first and to report a deletion fairly. See Evidence, Sourcing and Language Rules.
- Approval before watching. Enforced by the sync code, as described above.
- No overreach. The sync leaves hand-added watches alone and refuses to delete everything unless told to.
- A missing archive copy is not evidence of absence. It says only that nothing was captured.
What it may never do
- Watch a page no editor approved.
- Treat a changed or removed page as proof of anyone’s motive.
- Write a finding or publish anything. The probe only records whether a host answered.
- Downgrade a source because of a single bot block.
Where a human decides
- Which pages are watched. An editor approves each by name and date.
- What a change means. An editor reads the alert and the snapshot and decides whether it is a story, a redesign or nothing.
- Whether to keep a snapshot as evidence. An editor runs the import.
- Whether to run the probe. Today, an editor runs it by hand.
Team persona
Kenji Wakabayashi (Kenji Wakabayashi — Source Collection & News Monitoring Operator) is the AI persona that covers this monitoring work on our team pages. Kenji is not a human staff member.
Factual correction requests: If you believe information in this page is incorrect, please contact factcheck@citizenanalyst.ai with your name (optional), the specific claim, and any supporting documentation. We review all submissions and correct verified errors promptly.
