Scraper Dashboard

Monitor and control roster scrapers

Loading...
-
Total
-
Running
-
Idle
-
Broken
-
Overdue
-
Disabled

All Scrapers

Interval
The dropdown sets the base frequency. The scheduler auto-tunes the effective interval based on content changes: after 6 consecutive unchanged runs it doubles (up to 7d); when data changes it halves (down to 1h). ↑ 2d = slowed (no changes), ↓ 4h = sped up (data changed). Changing the dropdown resets auto-tuning.
Failures
Consecutive failures since last success. Resets to 0 after any successful run.
Last 24h
Number of successful runs in the past 24 hours.
Last Success
Time since the most recent successful stash.
Next Run
When the scheduler will next pick up this scraper. "Overdue" means it's past due and waiting for a worker slot.
Name State Status Interval Failures Last 24h Last Success Next Run Actions

Recent Runs

Name
Name of the scraper that ran.
State
US state abbreviation, derived from the scraper name prefix.
Stash
Whether raw data was successfully archived to S3. "ok" = saved, "validation_failed" = data failed checks, "fetch_failed" = couldn't reach the source. Click a failed row to see the error message.
Duration
Total wall-clock time for the run, including fetch, validation, and stashing.
Finished
When the run completed.
Name State Stash Duration Finished