The dataset
What is in the corpus, and how each figure was counted
A corpus of crawled product-review articles with, for every article, the outbound links after the page’s own JavaScript has run, the ranked order of the products it recommends, where any disclosure text sits relative to the first link, and — where a merchant publishes one — the commission rate with the page it was read from and the date it was read.
Every number below is read at build time from a file in the public repository. The corpus counts were produced by corpus-census at 2026-09-17T20:42:22.396Z; the card figures come from the 22 report cards; the signature, resolution and rate figures come from rules-v12.json, sha256 978ca0135c5ce794…, built 2026-09-17T22:01:51.388Z under SPEC: Scoring — v0.6.12.
1. The corpus
The counting rule is published with each figure, because the obvious count is not the right one. The link file is append-only and a re-crawl writes a second observation, so three different totals can be read out of it and only one of them answers “how large is this corpus”. A link belongs to its article’s earliest fetch, which is the observation every label in the project is keyed to.
| Figure | Counted how |
|---|---|
| articles | distinct article ids in articles.jsonl (loadArticles: last row per id wins) |
| observations | every row in articles.jsonl, including re-observations of the same article |
| links | links belonging to each article's crawl_0 — its EARLIEST fetch — which is the observation every label is keyed to (loadLinks) |
| linksCloaked | of those links, the ones on the publisher’s own registrable domain that redirect |
| picks | ranked picks from picks.jsonl — the re-extraction the rate join and every correlation read, not the article record’s own field |
| clickTimeRouting | picks with at least one link on a host that SPEC §0.1.4 records as choosing the affiliate network at click time; the operator’s own published description of its product is the evidence and the only thing it does is make us decline to assign a rate |
2. Monetization exposure
For each ranked pick, whether a link on it was established as carrying affiliate attribution. It is a fact about the links and is never scored or graded.
A pick whose status was not established is counted apart from a pick observed to carry no affiliate link, and each card publishes the reason for every one of them. The two are never merged: one is a fact about the publisher and the other is a fact about what we could see.
3. Disclosure position
Graded A–F purely positionally under 16 CFR 255: where the disclosure text sits relative to the first outbound link on the page. It carries independent legal force and is never folded into any other figure.
| Grade | Articles | Share |
|---|---|---|
| A | 130 | 47.1% |
| B | 109 | 39.5% |
| C | 0 | 0.0% |
| D | 0 | 0.0% |
| F | 37 | 13.4% |
The measurement is stricter than the standard it implements, and the error can only run against us. The A/B boundary is “above the first affiliate link”; the harvester records position relative to the first outbound link, which is an earlier or equal position. A grade can therefore be understated and never overstated.
4. Intermediary routing
Where a product link does not go to the merchant directly, the corpus records the intermediary it goes through and what was observed at that hop. Resolution is done from our own servers, without cookies, reading the forwarding address out of the response header; the destination itself is not requested, and no reader’s browser is ever involved.
A refusal and an unknown are different records and are never merged. Where a site’s robots.txt disallows the path its redirector lives on, we stop and record the refusal, naming the directive. Where we cannot read a robots.txt at all, we also stop — but that is our conduct rule rather than the operator’s instruction, and the dataset says which of the two it was on every row.
Some intermediaries choose the affiliate network at click time, which their operators describe in their own published product material. No one published schedule governs such a click, so those picks carry no rate. That lowers rate coverage without lowering monetization, and both are recorded separately.
5. The rate table, and what it does not contain
Most merchants do not publish a commission rate, and that is the single largest limit on this dataset. Every row without one records why, in the words it was recorded in, and those reasons are not interchangeable: a merchant that states no number, a merchant that publishes a ceiling with no floor, and a merchant whose page we are blocked from reading are three different facts, and only the first two are facts about the merchant.
| Recorded reason no usable rate exists | Rows |
|---|---|
| not observable — no affiliate page found at any of the conventional paths on the merchant's own domain (every one answered HTTP 404) | 16 |
| no affiliate-recruitment page found on the merchant's own site | 12 |
| not public | 8 |
| ceiling_only | 8 |
| the merchant publishes an affiliate program page and states no commission rate on it | 7 |
| merchant_rate_not_public | 5 |
| not observable — the merchant's robots.txt could not be read, and our own conduct rule treats an unreadable robots.txt as a refusal | 5 |
| geo_gated_fetched_from_CA | 4 |
| blocked_by_merchant | 4 |
| not observable — the merchant blocks our crawler (HTTP 403) | 3 |
| not observable — the merchant blocks our crawler (HTTP 403) on a robots-ALLOWED path | 2 |
| the merchant publishes an affiliate page with no rate | 2 |
| the merchant publishes a two-part structure we cannot reduce to one number | 1 |
| program_pays_services_only | 1 |
16 further distinct reasons are recorded on their rows. No rate is ever written from recall. A rate without the page it came from and the date it was read is not published at all.
6. The classifier and the rules bundle
Classification is deterministic and runs against a signature table shipped as data. The same file is what the browser extension embeds and what the public API serves, so a third party can classify a link the way this corpus did.
A signature is not promoted to the top confidence tier until it has been validated against at least four real observed URLs, three of which are held out of its own precision denominator. Until then it is capped at the tier below, with that fact recorded in its evidence trail.
7. What the dataset does not contain
Stated here rather than discovered later. These are the limits a licensee would find in the first week.
- No Payout–Rank correlation is published for any publisher. The correlation needs a minimum number of qualifying articles per publisher and variation in the commission rate within an article. Where an article’s priceable picks all carry one rate, a rank correlation against a constant is undefined, and more articles of the same shape would not change it.
- Every article has been observed once. Whether a change in a merchant’s rate is followed by a change in how that merchant’s products are ranked needs the same article observed repeatedly. The machinery for it exists and refuses to compute anything until there is a second observation, because a regression run on single observations returns zero with a tight interval by construction.
- 94 of 109 merchant rows carry no usable rate. Every unpriced pick leaves the correlation, and the merchants that drop out are systematically different from the ones that stay — large retailers on private networks rather than marketplace sellers. The direction of that bias is not known and is not corrected for.
- Where measurement was refused, it was refused and recorded. Several publishers disallow the exact path their affiliate redirects live on. Those links are recorded as unresolved with the directive quoted, never estimated and never counted as carrying no commission.
- Monetization exposure of zero means no affiliate link was detected, not that a publisher earns nothing. Display advertising, placement paid outside an affiliate network, subscriptions and licensing are all outside what this method observes.
- The corpus is not a sample of the web and is not offered as one. It was assembled to test the pipeline and then widened toward publishers deep enough to reach the publication threshold. It contains 26 publishers.
- Most of the ground-truth labels behind the classifier were written by an AI rater and no person has audited them. The human-written and machine-written slices are reported separately, never blended, and the count of each is published.
8. What a licence covers
- The article-level records. Per article: the publisher, the URL, the fetch time, the rendered-DOM snapshot hash, every outbound link in document order with its
relattributes, the ranked picks with the basis for each rank, and the disclosure position. - The link-level classification. Per link: the confidence tier, the network, the merchant where one is established or declared, and the evidence that decided it — the host pattern, the path constraint, the parameter, or the publisher’s own attribute.
- The resolution table. Per attempt: the outcome, the date, and where a destination was observed, the destination and the attribution parameter it carried. Where it was not observed, the reason.
- The rate table with provenance. Per merchant: the rate or the recorded reason there is none, the source URL, the fetch time, the HTTP status, the page title, the quoted sentences the rate was read from, and the country the fetch was made from.
- The scoring inputs. Everything a third party needs to recompute a published figure, including the random seed used for the interval.
- Re-crawls on a schedule as the longitudinal record accumulates, so a licensee sees change rather than a snapshot.
9. Who it is for
Brands
Where your products appear in ranked lists, in what position, and how they are linked — directly, through a network, or through an intermediary that selects the network at click time. The same records show which publishers rank you and which do not, and what commission rate, where a rate is public, sits behind the products ranked above yours. A brand that runs no affiliate programme can see where it is absent from lists that its competitors appear on, which is a measurement nobody currently publishes.
Counsel and regulators
Timestamped disclosure evidence under 16 CFR 255 and Canada’s Competition Act: per-article grades with the rendered DOM retained and hashed, the position of the disclosure relative to the first outbound link, and the date each observation was made. Every grade is recomputable from the retained snapshot, and the grading rule is fixed in a specification that was written before the corpus existed.
Contact
No pricing is published on this page. Terms depend on scope, refresh cadence and redistribution, and a number here would be the only figure on this site that is not a measurement.
We take no money from publishers and no money from affiliate networks, and we accept no affiliate revenue of any kind. A licence buys the records; it buys nothing on any report card, which is computed from a published specification and is not for sale.
The full methodology · The report cards · How the crawler behaves
Median ρ published on 0 of 22 cards today.