Retriever Forwardtest

The frozen live-news archive (data/forward_corpus/). Two feeds fill it: a daily WebSearch pull (news_pull.sh each evening) plus a one-time GKG+Wayback cold-start seed — the very same BigQuery-GKG discovery + Wayback-lede (+ title-gated live fallback) pipeline the backtest Retriever DB uses, pointed at the last few weeks so the curator has a full trailing window on day one. Store-broad and deduped: each pull appends only genuinely new articles. This is the forward analog of that backtest DB; the curator reads a trailing 21-day slice of it at each review.

270
articles in corpus (unique)
158 / 112
GKG+Wayback seed / WebSearch daily
524
sightings logged (appearances)
7
pulls run
1
missing days (gaps)
86%
full-text extracted
52%
with a byline (author)
57%
recognized desks (specialty+major)
183
read by a review now (21d window)
7 / 108
waves / source domains
36
blocked domains (source_block)

Raw data (local files): articles.jsonl · appearances.jsonl · pulls.jsonl

1. Pull history and gaps

7 pulls across 5 day(s). 1 missing day(s): 2026-07-27 (a missed pull can't be cleanly backfilled). Colored by feed: the one-time GKG+Wayback seed vs the daily WebSearch pulls.

2. Articles by published date

published on or after 2026-07-01 (35 older mis-dated hub/evergreen pages hidden), colored by feed. The GKG+Wayback seed backfills the older days; the daily WebSearch pulls add the recent edge. The curator reads a trailing 21-day slice of this.

3. Articles by wave

coverage by theme (the wave query whose result first surfaced each article), split by feed

4. Top sources (color = feed)

top source domains by article count, split by feed: the GKG+Wayback seed contribution (authority-ranked, so it skews to recognized desks) vs the daily WebSearch pulls. source_block junk excluded upstream

5. Articles per search term

6. Articles per author