Unit. One outcome for each declared identity and site per completed runs row. probes
stores the site and path, observed time, HTTP method and status, outcome and mechanism, Cloudflare
indicator, robots decision and rule, latency, and classification signals. Dated JSON provides an
independent representation. The database key is (run_id, site, identity).
Selection. Forty established public targets originally selected for breadth across news, commerce, docs, AI firms and controls. This is a fixed panel without population weights. The four original strings are browser, ClaudeBot, GPTBot and python-httpx. Record the declared string as an unverified request from the collector’s own address. It does not represent genuine vendor traffic.
Period and vantage. June 18 has two LAX baseline runs. July 1 starts collection on a US cloud host. The November readout isolates the 466 completed IAD-tagged observations, from July 1 through November 13; the same source IP has two separate EWR tags. There are 45 observed finish gaps longer than 7.5 hours in this cut. The series was interrupted by a storage issue on November 14. The September record has no baseline for the later identities.
Supports. Repeated comparisons for a fixed public path, vantage and string, with denominators; paired observations of site declarations and actual edge responses. Compare only completed runs.
Does not support. Verified agent/crawler traffic, causal attribution, full browser journeys, policy coverage beyond the tested path, or estimates about all sites. A challenge is a recorded outcome. The collection does not solve it. See the readout for detailed denominators. The queries are available on request.