A source is not good or bad. It is usable for some claims and not for others, and the work of evaluation is to decide which, in a way that a second person can repeat. That is the whole of the discipline, and it is narrower than the words usually used for it. “Credible” is a verdict with no content. “Peer-reviewed” is a property of a document that says nothing about whether it carries the figure you need. “Reputable” is a judgement about an issuer that a reader cannot check and a reviewer cannot dispute.
The five questions below are the ones that produce a decision: where the document comes from, how it knows what it says, when what it says was true, who benefits from it being believed, and whether anything independent says the same. They are asked in that order because the order is cheap-to-expensive. Provenance takes a glance at the cover, method takes a paragraph, recency takes the date, interest takes knowing what the issuer sells, and corroboration is the only one that costs a search or a call. A source that fails at the first question is finished before the second is asked, and running them in order is what keeps the last one — the expensive one — for the sources that survive.
This piece sets out the five questions, what passes and what fails each, and then runs the test on one source in full: a document that passes three of the five and fails two. It is the ordinary case, not the exotic one. Most sources that reach a draft are like it — properly published, competently written, honestly dated, and unable to carry the sentence the writer wants to put them behind. What matters is what happens next, and the difference between a source used well and a source used as cover.
The five questions
Table 1 gives each question, what passes it, and what fails it. Read the third column as a description of the document, not of the topic: everything here is answerable by opening the file, and nothing depends on whether the subject is interesting, urgent or popular.
Table 1. The five questions of source evaluation, what passes each and what fails it. Constructed for this piece from the claim-list structure the research and editing stations work to.
| Question | What it establishes | What passes | What fails |
|---|---|---|---|
| Provenance | Who issued the document, when, and whether it is a primary account of what it describes | A named issuer, a publication date, and a text that reports the thing itself rather than a summary of it | No issuer; no date; a page that describes a report it does not contain; a document whose author is a marketing function |
| Method | How the document knows what it asserts | A stated population, a stated sample, a stated question, a stated period, and arithmetic that reconciles | “Research shows”; an unstated sample; a figure with no denominator; a method described as proprietary |
| Recency | When the document’s content was true, distinct from when it was read | An as-of period inside the window the claim describes | A date on the page that is not the date of the data; a figure carried forward from an earlier edition without saying so |
| Interest | What the issuer gains if the claim is believed | An issuer whose interest is disclosed, and a claim that does not serve it | An issuer whose conclusion is its own product; a recommendation that follows from the finding by construction |
| Corroboration | Whether anything independent states the same thing | A second document, from a different issuer and a different method, stating the claim | Five repeats of one press release; a citation circle; agreement that traces to a single origin |
The order carries an argument. Provenance and method are properties of the document; recency and interest are properties of the document in relation to the claim you want to make; and corroboration is a property of the corpus, which is why it is the one a single document can never pass on its own. A source read in isolation can be provenance-perfect and corroboration-unknown, and a draft that treats unknown as passed is how a press summary becomes an industry figure.
There is a symmetry worth noticing, because it decides how the test is used. A source that fails provenance or method is not a source for any claim; the failures are about whether the document knows anything. A source that fails recency or interest is not a source for that claim; it may be a source for a narrower one, or for a claim about the issuer rather than about the world. And a source that fails corroboration is a source carrying a claim at one remove, which is a status rather than a disqualification. Reading the five as one verdict loses all of that, and the loss is what produces the two errors that actually cost money: cutting a document that would have carried a narrower claim, and keeping one that carries nothing but looks authoritative.
Provenance, and the difference between a document and a page about a document
Provenance answers a question that takes ten seconds and is skipped in most drafts: is this the thing, or is it a description of the thing? A regulator’s guidance note is the thing. A trade publication’s article about the note is a page about it. Both can be useful; only one can carry a claim about what the regulator requires, because only one contains the requirement.
The test is mechanical. The issuer is named, the date is on the document, and the passage you want to cite appears in the text you hold rather than in a summary of a text you do not. Where the document is an edition — a handbook, a standard, a procedure — the edition and the revision number are part of the provenance, because a clause quoted from revision 4 is not a clause in revision 5, and a draft that cites “the procedure” without the revision cannot be checked against the procedure in force when the piece is read.
Two provenance failures are worth naming because they look like successes. The first is the well-designed page that summarises a report and links to a landing page rather than the report: it is dated, it has an issuer, and the figure on it cannot be opened at source. The second is the undated document that is nonetheless current — a standing guidance page, a policy that has not changed in years. Both are usable, but the first only as a secondary account, and the second only with the date it was retrieved stated beside it, because a reader who needs to know whether the guidance was in force on a particular day has nothing else to check.
Method, and the denominator
Method is the question drafts skip most often, because answering it requires reading past the finding. A figure with no denominator is not a finding. “Most teams match invoices by hand” is a claim about most teams; “of the forty-one finance staff who answered a survey sent to one association’s members, twenty-nine said they match invoices by hand” is a claim about forty-one people, and the difference between the two is the whole of the method question.
Three things pass it. A stated population, meaning who was asked or what was counted. A stated sample, meaning how many, and how they were chosen. And arithmetic that reconciles: percentages that sum, totals that match the parts, a figure that is the same figure in the summary and in the table. None of this requires a statistics education. It requires reading the table that the summary sentence was written from, which is a page further into the document than most drafts go.
What fails is not only the absent method but the unexamined one. A survey of an association’s own members is a method, and it is a good one, for a claim about those members. It is not a method for a claim about the sector, because the sample is the membership, and the membership is not the sector. That distinction is what the fourth and fifth questions then test, and it is the reason method and interest are separate questions rather than one: a document can have a perfectly described method and an interest that decides which population was asked.
Recency, and the as-of date
Recency is not “published recently”. It is “true for the period the claim describes”. A survey published last month about last year’s invoices carries a figure that is a year old, and a price list published in January carries prices that ended in March. The test is the as-of date: the date the content was current, recorded beside the claim, distinct from the date the page was read.
The failure mode is specific and quiet. A figure is correct on the day it was read, the piece says nothing about the period, and a reader six months later has no way to know whether the number describes their quarter or the one before it. The register exists to prevent exactly this: every entry carries the as-of date of the passage, so a claim can be tested against the period it was meant to describe rather than against the date it was quoted.
Recency also has a floor as well as a ceiling. A document from five years ago may be the best account of how a process was designed, and useless as an account of how it runs now; a document from last week may be the worst account of a slow-moving rule because nobody has had time to interpret it. The question is not “how old is this” but “what period does the claim I want to make describe, and is this document inside it”.
Interest, which is not the same as bias
Interest is the question that gets moralised and should not be. Every issuer has an interest; this publication’s interest is disclosed at the foot of every piece because the alternative is to pretend it has none. The question is not whether the document is interested but what its interest is, and whether the claim you want to make is the claim that interest was assembled to support.
A supplier’s own case study is a primary account of what the supplier did for one customer. It is not evidence about the market, and it is the worst available source for a claim about typical results — not because the supplier lied, but because the population is the customers who agreed to be written about. A trade association’s survey of its members is a primary account of its members; it is a weak source for a claim about the sector, and its conclusion will recommend the thing its members sell, because that is what the conclusion is for.
The practical test has two parts. Read the last paragraph of the document, which is usually where the recommendation sits, and ask whether the recommendation follows from the finding or from the issuer’s catalogue. Then ask what the document would have had to conclude if the finding had gone the other way — and whether that conclusion is a sentence the issuer could publish. A document that could only have concluded one thing is a document with an interest, and the claim it can carry is a claim about the issuer.
Interest does not disqualify. It is the reason a supplier’s own figure is often the best available evidence for the supplier’s own behaviour, and the reason a trade body’s survey is often the only count of its own members. What it disqualifies is the use: the document becomes the authority for the claim it was built to support, and a claim about the world is left carrying nothing.
Corroboration, and why five sources can be one
Corroboration is the expensive question and the one most often answered by counting. Five documents that state a figure are not five pieces of evidence if all five trace to one origin. The test is independence: a different issuer, a different method, a different population, and a chain of citation that does not close on itself.
Three shapes look like corroboration and are not. The press release and its coverage: one document and four rewrites of it. The citation circle: two documents that cite each other. The shared dataset: three analyses of the same underlying count, which is one measurement reported three times. None of these is worthless — a widely repeated figure is a fact about the discourse — but none of them raises a claim above one remove from its origin.
Genuine corroboration is rare enough that the honest default is the one the research station enforces: a claim rests on the primary document, or it is marked as corroborated at one remove, with the strength field saying so. That status is not a failure. It tells a reader precisely what to do with the sentence — trust it as a report of what a document says, and go and read that document before relying on it.
One source, five questions
Table 2 runs the test on a single source, constructed for this piece: a trade association’s survey report on how its members handle supplier invoices. It passes three questions and fails two, and the two it fails are the two that matter for the claim the draft wanted to make.
Table 2. One source tested against the five questions: a trade association’s survey of its members’ invoice handling. The source and its findings are constructed for this piece; the verdicts and their consequences are the point.
| Question | What the document shows | Verdict | What follows for the claim |
|---|---|---|---|
| Provenance | Issued by a named association, dated, published as a downloadable report; a primary account of a survey it ran itself | Pass | Can carry a claim about what the association found |
| Method | States the population (member firms in one region), the sample (forty-one finance staff), the question wording and the collection period; percentages and totals reconcile | Pass | Can carry a claim about those forty-one respondents |
| Recency | Collected in the two quarters before the document’s date, inside the period the draft’s claim describes | Pass | Can carry a claim about the current period, as of that date |
| Interest | The association’s members sell invoice-matching software, and the report’s closing recommendation is to buy it; the finding is the recommendation’s premise | Fail | Cannot be the sole authority for a claim about the sector’s need |
| Corroboration | The figure appears in five secondary accounts, all of which cite the association’s press release; no independent document states it from its own data | Fail | Cannot carry an industry-wide claim at all; carries it only at one remove |
The three passes are real and they are why the source survives. It is the only document in the set that counts anything about invoice handling among real firms; it states its population, which is more than most of the sector’s comment does; and its data are recent enough for the claim. A draft that cut it would be a draft with no count in it at all.
The two failures are real too, and they narrow the sentence rather than the source. The association asked its own members, so the finding is about its members. Its members sell matching software, so the recommendation is the premise inverted. And the figure has travelled no further than the press release, so the five accounts that repeat it are one account.
What to do with a source that passes three
Four dispositions are permitted, and the choice between them is decided by which question failed rather than by how much the writer likes the document. Table 3 sets them out.
Table 3. The four permitted dispositions for a source that passes some tests and fails others, what changes in the draft, and what the claim may then say.
| Disposition | When it applies | What happens in the draft | What the claim may say |
|---|---|---|---|
| Carry | The claim is inside everything the source passes | Cited with the passage and the as-of date | What the source found, in the source’s own population and period |
| Demote | The claim is wider than the population, the period or the strength the source can carry | Rewritten to the claim the source supports; the wider version is marked unsupported | What the respondents reported, named as respondents |
| Corroborate | The source is sound and the claim needs independence | A second document is sought from a different issuer and method; until it arrives the claim stays at one remove | The figure as reported, with its status stated |
| Cut | A question failed at provenance or method | Removed from the draft; the gap goes on the list with the document that would close it | Nothing; the sentence waits |
For the source in Table 2 the answer is carry, demote and cut, applied to three different claims drawn from one document. The claim that the association’s member firms report matching invoices by hand to a stated degree is carried as it stands, with the population in the sentence. The claim that this degree holds across the sector is demoted: rewritten to what the document supports, and the wider version marked as unsupported rather than deleted, so that a reader can see what was wanted and why it is not there. The claim that the practice is costing firms money and that software would fix it is cut — it rests on the association’s recommendation, and the recommendation rests on its members’ sales.
The discipline the table imposes is that a single document can produce three different dispositions at once, and that the draft records which is which. A source marked simply “used” carries all three claims at the strength of the strongest, which is how a caveat in paragraph fourteen becomes a fact in the executive summary.
What the disposition costs
The arithmetic is small and it is the reason the test is worth running before the draft rather than after it. Table 4 prices the pass in the research station’s own terms: one call to test a source, a rewrite for the claims that need demoting, and a cross-check on the one claim that is contested.
Table 4. Triage and repair for five sources carrying twelve claims, at the rates read from each endpoint’s 402 challenge on 31 October 2026. Call counts are planning assumptions constructed for this piece; the document parts carry no published per-call price and sit outside the total.
| Step | Calls | Unit price | Cost |
|---|---|---|---|
| Testing five sources, one call each | 5 | $0.00700 | $0.03500 |
| Rewriting two claims to what their source carries | 2 | $0.00700 | $0.01400 |
| Qwen cross-check on the contested claim | 1 | $0.00875 | $0.00875 |
| Total | 8 | $0.05775 |
Eight calls and $0.05775 covers a five-source set carrying twelve claims, which is $0.00481 a claim. Walked the other way — twelve claims, one call each, no source-level test — the same coverage is twelve calls and $0.08400, which is 2.4 times the price for the same twelve claims and no view of the sources at all. Testing at the source is cheaper because one source carries several claims, and it is better because the verdict is reusable: the second claim drawn from the same document inherits the test that has already been run.
Against the whole brief the figure is smaller still. A brief at the Researcher assembly’s counts is eleven paid calls and $0.08050, so triage and repair at eight calls adds $0.05775 to it — 72 per cent again, and the part of the work that decides whether the answer survives being read by somebody who checks. At the metered ceiling of $0.00955 the seven DeepSeek calls cost $0.06685 instead of $0.04900 — $0.01785, or 36.4 per cent, more — which brings the eight-call total to $0.07560. At the figure that circulates as DeepSeek’s price, $0.00875, those seven calls read $0.06125: $0.01225 more, or 25 per cent, and it is Qwen’s rate rather than DeepSeek’s. The 0.50 USDC credited to a workspace is seventy-one calls at $0.00700, which is eight triage-and-repair passes, with seven calls over.
What the test cannot do
Five questions produce a decision about a document. They do not produce a judgement about the world, and the temptation to read them as one is the limitation that matters here.
A source can pass all five and be wrong. Provenance, method, recency, interest and corroboration are properties of how a claim was produced and how it has travelled; a well-run survey of a population that has since changed, corroborated by a second well-run survey asking the same flawed question, passes everything and describes a world that no longer exists. The register records what a document says. Accuracy is a property of the corpus, and the corpus is the reader’s.
Corroboration cannot be tested inside the document, and often cannot be tested inside the source set. Where the only independent account of a figure is behind a paywall, in a database nobody in the workspace can reach, or in a conversation nobody has recorded, the honest status is one remove, and no amount of further reading in the same corpus changes it. The test reports the absence; it cannot fill it.
Interest is the question most easily mistaken for a character judgement, and the mistake runs both ways: a document dismissed because its issuer sells something, and a document excused because its issuer is a charity. Neither follows. What matters is the relationship between the claim and the interest, which is a reading of one document rather than a judgement about its author.
And the test is a filter, not a proof. It sorts sources into what they can carry, which is enough to write a defendable sentence and not enough to establish that the sentence is true. Where a claim has to hold up against somebody with standing — a regulator, an auditor, a court — the test is the beginning of the work rather than the end of it, and the person who signs is still the one accountable.
The decision
Run the five questions on the source set before the first claim is written into a draft, and record the verdict on each source rather than on each claim. Then let the verdict decide the sentence: carry what the source supports, demote what it nearly supports, cut what it cannot touch, and mark the one-remove cases as one-remove rather than letting a fluent sentence close the gap.
The test to apply to your own set is the one the worked example applies. Take the source your draft leans on hardest, open it, and answer five questions about the document rather than about whether you believe it. If it fails at provenance or method, you have no source for that claim and the work is to find one. If it fails at recency or interest, you have a narrower claim than you thought, and the sentence gets shorter. If it fails at corroboration, you have a claim at one remove, and saying so costs eight words. None of the four outcomes is a reason to stop writing, and all four are cheaper to reach at eight calls than to discover after the piece is out.
The agent’s part is narrow and worth stating. It reads the document into its own text, answers the five questions from what the document contains, keeps the register with the verdicts and the as-of dates, and flags what a source cannot carry. It cannot decide whether the claim matters, cannot widen a population to fit a sentence, and cannot verify a figure against a world the corpus does not contain. Those stay with the person who signs the draft.
Where this goes
- The pillar: content and knowledge: from a source you hold to a sentence you can defend — the four stations, the claim list and the verdict
- The station that keeps the register: studio.x402.press/roles/researcher — sources read into text, claims mapped to passages, the three statuses
- The station that checks the draft against it: studio.x402.press/roles/curator_editor — the citation chain, the matrix and what it will not approve
- Where the drafting is priced: studio.x402.press/roles/writer — one voice, numbered tables, model calls only
- The services: DeepSeek and Qwen — the canonical per-call prices on seven chains
- The reviews: DeepSeek and Qwen — the measured handshakes and what they do not promise
- The price ladder: inference endpoints compared — the three endpoints, the ceilings and the cost of escalation
- What the evidence has to carry: how to write a whitepaper — the argument, the evidence under each claim, and the section most of them omit
- What happens to the register next: turning source material into articles — the register, the claim map and an outline that follows the evidence
Method note
The two prices were read from the live 402 challenge on Base on 31 October 2026: 7000 atomic units under exact for a short prompt on deepseek-v4-flash, which is $0.00700, and 9550 under upto at max_tokens: 1000, which is $0.00955; qwen3.8-flash’s $0.00875 was read from its own challenge at the same settings. Every figure in Table 4 derives from those two rates, and the counts — one call a source, two rewrite calls, one cross-check — are planning assumptions rather than observations of a real triage.
The association, the survey, its forty-one respondents and the three claims drawn from it are constructed for this piece. No client, sector body, report or dataset is described, and no figure here is offered as a description of real invoice practice. The five questions are the ones the research station’s claim list and strength field are built to record; the dispositions in Table 3 are this publication’s working rules, not a standard, and where a buyer’s own procedure names different ones, theirs applies.
x402.review is written and edited by N.P. Vincent. The publication is connected to x402.press, which sells the services reviewed here; that interest is disclosed on every piece that touches a service, and it is stated here rather than left to inference.
Filed under Jobs · Tags: sources, evaluation, evidence, method
