Support macro against a model draft: two instruments, and where each one is wrong

The argument about support automation is usually conducted as a contest, and it is not one. A macro and a model draft are different instruments for different questions. The macro is a fixed answer to a question the organisation has answered before and expects to answer again in the same words. The draft is an assembly for a question nobody anticipated, built from whatever the documentation says about it. Neither replaces the other, because each is defined by a property the other does not have: the macro is knowable before it is sent, and the draft can address a question that has no macro.

Both fail, and they fail in ways that look nothing alike. The macro fails because the world moved and the text did not — it is wrong in a fixed, quotable, discoverable way, and it is only discovered when a customer acts on it. The draft fails because it is fluent about something it cannot source — it is wrong in a variable, plausible, hard-to-audit way, and it is discovered when the customer quotes it back. A support operation that can tell which of the two failures it is looking at has most of what it needs.

This piece is about the two instruments, the two failures, and the routing decision between them. It makes no claim about which produces better outcomes, because nothing in this family has been measured for outcome; the counts below are of what a reply rests on and what it costs, which are the only two things that can be checked without a customer survey.

Two instruments, defined by what they know

A macro is a stored text with a trigger. It is authored once, by somebody with the authority to state the answer, and it is sent with a click. Its strengths all follow from that: the text can be read before it is sent, because it is the same text every time; it can be reviewed by the person accountable for the policy, because reviewing it once covers every use; its marginal cost is zero, because sending a stored string costs nothing; and it is consistent, in the sense that two customers asking the same question get the same answer. Its weakness follows from the same fact: it cannot know anything about the message in front of it, so any question that differs — a different product tier, a different country, an order in a particular state — is answered with whatever the macro was written for.

A draft is assembled. Its input is the message plus the corpus, its output is a new text, and its strengths are the mirror image: it can address the question actually asked, including the parts of it that were never anticipated; it can be conditioned on the customer’s own facts if those facts are in the message or in the record; and it can carry a citation, because the passage it paraphrases is known at the moment of drafting. Its weaknesses are equally structural: it must be read every time, because the next draft will be different; its marginal cost is a metered call; and its fluent register is a poor signal of whether it is right.

Table 1. The two instruments, on the dimensions that decide where each belongs. Source: constructed for this piece; costs are the measured Base rates, 31 October 2026.

The last two rows are the ones that decide routing, and they are usually read in the wrong direction. The macro is cheap per use and expensive to maintain, because its failure is invisible until it is used; the draft is expensive per use and cheap to check, because the check is one passage. An operation that counts only the per-use line will conclude that macros are free, which is true, and that drafts are expensive, which is true, and will then be surprised by the customer who was sent a two-year-old returns policy at zero marginal cost.

The macro that no longer matches the product

The failure is specific and it has three shapes. The price moved: the macro quotes a figure that was correct when it was written. The policy changed: the returns window went from thirty days to fourteen, the fee was removed, the eligibility rule narrowed. The process changed: the macro tells the customer to reply to an email that no longer exists, or to use an address that was retired, or that a refund will appear in five days when the processor now takes ten.

In all three, the macro remains grammatically perfect, warm in tone, and signed by nobody in particular. That is what makes it dangerous: it carries the organisation’s voice and a fixed text, and a fixed text reads as policy. A draft that invents a returns window is a failure of an instrument everybody already distrusts; a macro that quotes a retired one is a failure of the system of record, sent at speed to every customer who asks.

The reason it goes unnoticed is structural rather than negligent. A macro is only wrong when it is used, and it is used by people who are not its author, in a queue that measures volume. The person who wrote it has moved on or is busy; the document that would correct it is edited by somebody else in another system; and no report compares the two. The macro does not degrade gradually — it is correct until the day the policy changes and wrong thereafter, which means the failure is a step change with no warning.

The repair is a field and a ritual. The field is a review date and an owner, held beside the text rather than in somebody’s memory. The ritual is a diff: every quarter, each macro is opened beside the passage of the policy it states, and the two are read together. A macro whose source has moved is either rewritten or retired, and a retired macro is deleted rather than left in the library, because a library that keeps its dead rows will be searched by somebody who does not know they are dead.

The arithmetic of the ritual is small and worth stating, because it is the only maintenance cost the macro has. Fifty-three macros at six minutes each is three hundred and eighteen minutes, five and a third hours, once a quarter. That is the price of the marginal cost being zero, and it is a price that is paid either in that room or by the customers who receive the stale ones.

The draft that sounds right and cites nothing

The other failure is more recent and less well understood. A message arrives that the corpus half-covers; retrieval returns a passage on the topic but not on the question; the model writes a fluent reply that reads as if it rests on the passage, and the sentence it produces is one the passage does not support. The reply is well organised, appropriately hedged in places, and specific where specificity was not available.

There are two tells, and both are checkable rather than felt. The first is in the retrieval set: is there a passage whose wording the reply follows, or is the reply an extension of a passage that is merely adjacent? The second is in the sentence itself: does it contain a number, a date, a duration or a term that does not appear in the passage it cites? A specific claim with no source is the signature of this failure, and it is a signature because specificity is what the model adds when it is asked to be helpful and has nothing to be helpful with.

The reason this failure is expensive is not that it is common; it is that it is invisible to the reader. A customer cannot tell a sourced answer from an unsourced one, because the difference is a document they do not have. The only reader who can tell is the one who opens the passage, and the only reason they will open it is that the reply carries a citation they can follow. That is the whole argument for the citation, and it is an argument about audit rather than about truth: the citation does not make the draft right, it makes the draft’s wrongness findable in a minute.

There is a second-order effect worth naming. A draft that cites nothing is not a failed citation; it is a reply with no retrieval set to check, and it is indistinguishable, in a queue, from a draft that cites properly. A desk that counts drafts by volume will attribute both to the same instrument and conclude that drafts are unreliable, when what it has is two instruments with one name and no gate between them.

A hundred questions, routed

The example is a constructed sample of one hundred questions drawn from one month’s mail, classified by what the corpus can support, and priced at the measured rate. No client, customer or product is described.

Table 2. One hundred constructed questions by what answers them, and the cost of each route at the measured DeepSeek rate. Source: constructed for this piece.

The counts reconcile: 46 plus 7 is the 53 questions where a macro is the right instrument, and 53 per cent of the sample; 19 plus 22 is the 41 that need a draft; 6 escalate; and 46 plus 7 plus 41 plus 6 is 100. The 47 paid calls are 41 drafts and 6 escalation summaries at $0.00700, which is $0.32900 — 47 per cent of the $0.70000 the same hundred questions would cost if every one of them were drafted.

Two figures in that table are the argument. The first is the seven: 13.2 per cent of the fifty-three macro uses are stale, which is to say that a method with a zero marginal cost produced seven wrong answers at no cost at all. The second is the twenty-two: questions the corpus answers and no macro covers, each of which is a macro that has not been written yet. The desk that writes those twenty-two converts a metered cost into a stored string, and it should do so with the review date already attached.

The forty-one drafts are where the citation matters. At $0.00700 a call they cost $0.28700 — twenty-two of them paraphrasing a passage and nineteen extending one with a customer’s own facts — and the only way to tell those two populations apart after the fact is the retrieval set. A desk that keeps the set can audit a sample’s drafts in about an hour. A desk that does not is relying on the queue’s own confidence, which is the instrument the failure defeats.

Set against the pillar’s measured month — 498 paid calls, $4.51575, which is $4.52 — the forty-seven calls of this sample are 9.4 per cent of the month’s calls. One deal’s document set is fifty calls at the measured rate, $0.35000, which is within a hair of this whole sample’s metered spend; the same fifty at $0.00875, Qwen’s rate rather than DeepSeek’s, would be $0.43750.

What the routing does not settle

The staleness rate is a property of change cadence, not of the macro. A library audited quarterly against a product that changes monthly will always be partly stale, and the seven is one sample of a constructed example rather than a rate anybody should expect. What is durable is the check, and the fact that the check has to be scheduled or it does not happen.

A macro used is a decision, not an outcome. The 53 per cent is a count of messages where somebody judged a macro to fit. Whether the customer was satisfied, whether the ticket closed and whether the answer was right in the sense the customer means are all outside this piece, and outside everything measured in this family. This publication has measured cost; it has measured nothing about resolution, satisfaction, deflection or response time, and it does not intend to imply a figure by leaving one out.

A citation is not a proof. A draft that cites a passage correctly can still be wrong, because the document can be wrong, and a macro that has been verified against a document inherits the same exposure. The routing decision moves the question from “is this sentence invented?” to “is this document right?”, which is the better question and still a question.

The boundary is a person’s judgement at the edge. The classification in Table 2 is a design; in a live queue, the question of whether a macro fits is decided by whoever is reading the message. Where that judgement is contested, the safe default is the draft, because a draft can be checked against a passage and a macro cannot be checked against anything but its own dated source.

The decision

Route by question class rather than by preference, and write the routing down where the person working the queue can see it.

A question the organisation has answered before, in words that should be identical every time, with a source that has been verified inside the review period: a macro. The macro carries an owner, a review date and the passage it states.

A question the corpus answers but the answer depends on the customer’s own facts — this order, this tier, this address, this date: a draft, from the corpus, carrying its citation. No macro can do this, because the macro cannot see the message.

A question the corpus does not answer: an escalation, not a draft. Six of the hundred is a corpus problem worth an afternoon; six hundred is a different organisation.

And one rule that binds all three: a macro that has passed its review date is treated as an unknown, not as a stored answer. The whole value of the macro is that it is knowable before it is sent. The day that stops being true, it is the most expensive instrument in the room.

Where this goes

• The pillar: x402.review/2026/10/31/front-office/ — the family, the six artefacts and the boundary this piece works inside
• The role: studio.x402.press/roles/customer_service — retrieval over the client’s corpus, and the reply that quotes its passage
• The service it buys: x402.press/services/deepseek/ — the canonical price, seven chains
• The service it buys: x402.press/services/qwen/ — the second model, priced per call
• The reviews: x402.review/2026/10/31/deepseek-review/ and x402.review/2026/10/31/qwen-review/ — the measured handshakes and what they do not promise

The cost of the choice

A hundred questions routed as in Table 2 cost 47 calls and $0.32900; drafted in full, the same hundred would cost $0.70000. The saving is $0.37100 on the sample, and it is bought with the five and a third hours a quarter that keeping fifty-three macros true requires.

The measured rate is $0.00700 a short-prompt DeepSeek call on Base under exact; at the metered ceiling of $0.00955 the forty-seven calls are $0.44885. One deal’s fifty calls at the measured rate are $0.35000; at $0.00875, which is Qwen’s measured price and not DeepSeek’s, they would be $0.43750, and the forty-seven calls of the sample would read $0.41125 rather than $0.32900.

Method note

Prices were read from the live 402 handshake on Base on 31 October 2026: $0.00700 for a short prompt under exact, $0.00955 for a long prompt under exact and as the upto ceiling, and $0.00875 for qwen3.8-flash under both schemes. Table 2 sums: 46 plus 7 plus 19 plus 22 plus 6 is 100, 41 plus 6 is the 47 paid calls, and 47 at $0.00700 is $0.32900; 19 at $0.00700 is $0.13300 and 22 at $0.00700 is $0.15400, which with $0.04200 for the six escalations is the same total. The fifty-three macros at six minutes each are an assumption about an audit, not an observation of one. The desk, the library, the corpus and the sample are constructed so the arithmetic can be followed; no client, customer, product or macro is described, and none is real. Nothing here measures an outcome: the counts are of questions and calls, and no resolution rate, satisfaction score, deflection figure or response time appears anywhere in the piece.

x402.review is written and edited by N.P. Vincent. The publication is connected to x402.press, which sells the services reviewed here; that interest is disclosed on every piece that touches a service, and it is stated here rather than left to inference.

Filed under Jobs · Tags: support, macros, ai, comparison