Incident record

Canned verdicts, 21-25 August 2026

A four-day window in which the verification service signed and returned receipts for evaluations it never performed.

In brief

What happened: from 21 to 25 August 2026, /evaluate signed and returned receipts for evaluations it had not performed, after the account paying for its verification calls ran out of balance.
Scale: 17,784 verdicts were issued in the window; 17,703 carried the same canned reason string.
Signed result: every affected receipt derived verdict: halt; none carried act. The plain-text body beside it said “partially supported” and did not disclose that no evaluation had run.
Found: 2026-08-25, from the receipts themselves.
Changed: from 2026-08-27 the handler returns 503 not_evaluated with no receipt when no source produced a usable claim set, and incomplete evaluations are not cached. The same audit found and closed unauthenticated signing routes; an external report on 2026-08-28 is covered below.

Summary added 2026-11-04. The full record below is unchanged.

This incident occurred before the service was renamed Tanilo. Service names below have been updated to Tanilo; dates, commit hashes, key IDs and quoted findings are exactly as recorded at the time.

Between 2026-08-21 and 2026-08-25, an outage of Tanilo's /evaluate verification service caused it to sign and return receipts for evaluations it had never performed. It was discovered 2026-08-25 from the receipts themselves; the signed layer had correctly halted every affected request, and the human-readable body did not disclose that.

A four-day window undetected is the shape of this incident. Discovery from the receipts themselves — not from a customer complaint, not from a monitoring alert — is what made that window recoverable at all. The signed layer was doing what a signed layer is supposed to do; the presentation next to it was not.

What happened

The paid API account funding the three parallel verification calls exhausted its balance and began returning errors instead of results. Balance protection was manual top-up only; no auto-reload was configured.
The handler did not treat "no source returned a parseable claim set" as first-class unavailability. It proceeded, seeded overall confidence at exactly 0.50, emitted a canned recommendation text, and signed a composed receipt.
Every affected receipt derived to v_recommendation: unverifiable and verdict: halt. No degraded receipt in the entire incident carried verdict: act. The cryptographic layer was right the whole time.
The plain-text body next to those receipts said "partially supported" with a verify recommendation. A caller integrating on the body alone had no signal that no verification had occurred. A caller integrating on receipt verification would have halted correctly.
After the upstream was restored, the cache re-served at least one degraded response; the entry was preserved in a snapshot and then deliberately deleted.

The population

17,784 verdicts were issued during the affected window. 17,703 carried a byte-identical canned reason string — 99.54% — every one of them a valid Ed25519 signature over a receipt that correctly derived verdict: halt. The remaining 81 evaluations completed before the balance drained fully; those receipts are unaffected.

Why the receipts were still right

The cryptographic layer was right the whole time. The presentation layer understated it. For the full duration of a total upstream verification outage, the signed evidence artifact carried v_verdict: unverifiable and v_gate: halt. The v0.3 receipt derivation reaches act only through an evaluated, adversarially-resistant claim set. A response with no evaluated claims cannot express act in the signed layer, and none did.

If a customer had integrated on receipt verification, we would have nothing to disclose to them beyond the body text they saw next to a correct receipt.

Correction (21 Oct 2026): checking and enforcing the signed halt could have prevented action on these affected responses. It would not remove the need to disclose that evaluations failed or establish that the service was operating correctly.

What we changed

Commit	When (UTC)	Change
`4a8b37177`	2026-08-27 01:52Z	Handler early-returns HTTP 503 `not_evaluated` when no source produced a parseable claim set. No confidence, no verdict, no receipt, no cache write.
`4a8b37177`	2026-08-27 01:52Z	Cache admission gated on evaluation completeness, across both the full-response and per-claim namespaces.
`98b3b7e78`	2026-08-27 04:41Z	Reverted a co-shipped change that collapsed `un_probed_not_cleared` into the generic `unverifiable`. Both halt, but the two states are distinct at the receipt layer.
`c28c5dd6f`	2026-08-28 02:13Z	Recommendation text derives from the caller's threshold instead of a hard-coded 0.80. Present since `min_confidence` became caller-tunable; unrelated to the outage but the same disclosure-asymmetry class.
`49261a37b`	2026-08-28 03:50Z	The adversarial-flag confidence penalty re-checks the applied threshold, so a flagged response can no longer report `act` in the body when the receipt-side gate would halt. Same disclosure-asymmetry family as the row above.
`a8ce2a18d`	2026-08-28 18:02Z	Mapping binding derived at boot from the published mapping document; `/v1/compose`, `/v1/v_gate`, `/v1/sign`, and `/v1/sign/batch` return `503` until their verdict path or auth is completed. See the externally-reported defect below.

The pre-registration for the benchmark run whose collection window this outage sat inside carries an append-only amendment (2351e29f6) withdrawing the pre-registered headline for the same reason. This note and that amendment describe the same withdrawal.

A defect found by someone else, while this note was being written

Externally reported

On 2026-08-28, @giskard09 independently ran the published recompute procedure against /v1/conformance/sample and reported two things (x402-foundation/x402#3234). Both are correct.

First, the sample's v_gate.mapping_hash was a hand-typed placeholder rather than a digest of any mapping document. The envelope's signatures were genuine — both issuers verified, canonical bytes recomputed — so the cryptographic layer faithfully committed to a mapping binding that had never been computed. A relying party following our published procedure would resolve that hash, fail to match it, and correctly halt. This is the same failure class as the first defect above: a genuine signature over content the service never produced.

Second, we found the same hard-coded constant was used by the live POST /v1/compose and POST /v1/v_gate endpoints, both reachable without authentication. It was not confined to a fixture. In the same audit we discovered POST /v1/sign and POST /v1/sign/batch signed caller-supplied bytes under the production issuer key with no authentication — a forgery oracle for this receipt format while it was open.

The same string was reported five weeks earlier. On 2026-07-28 Michael Msebenzi (headlessoracle) reported that the mapping hash carried in our published fixtures was a placeholder that had never resolved. We repaired the fixtures within twenty hours and computed the correct mapping hash on 2026-07-29 — the value the conformance sample carries today. The identical placeholder string remained in the production constant for five more weeks, and the defect above is that same string. The July fix was scoped to the artifact that had been reported rather than to the class, so the correct value existed, published, in one location while the placeholder kept being stamped from another.

Fixed in a8ce2a18d, 2026-08-28 18:02Z UTC. The mapping binding is now derived at boot from the published mapping document, or the process refuses to start. All four endpoints return 503 not_issuing; /v1/sign* require an authorization header, and the composed endpoints will resume issuing only when their verdict path is completed. The private issuer key was never exposed — an oracle that signs what you hand it is not the same as a stolen key — and closing the routes ended the capability.

Bound on the exposure window. Requests to these routes were not counted; the request-tracking helper is called zero times in the composing module. Log retention is roughly one hour. We cannot enumerate what was signed, or by whom, between 2026-06-23 and 2026-08-28. Absence of known forgery is absence of evidence, not evidence of absence, and it must be said that way rather than leaned on.

Where the recompute leg stands

Alongside the sample defect, @giskard09 also ran the full verification protocol against a genuine /evaluate receipt: cryptographic signature check, mapping-hash match against a SHA-256 he computed himself of the live mapping document, and the Section 4.3 recompute of candidate_recommendation and candidate_gate. All three held. In his own words:

"The recompute requirement stands as demonstrated — we ran the full leg independently (signature, mapping_hash, §4.3 steps 3-6) and it held. The signature-implies-issuance step is a separate claim from that, and it shouldn't carry the same weight until kid separation lands on your side. That's not a knock on the recompute work, it's just a different property being verified."

That split is right, and it names the property still open. Same-kid conflation. The kid ao-composed-2026-06-ed25519-c3abfce3 currently signs both receipts where Tanilo evaluated the claim (/evaluate) and receipts where the service only signed bytes a caller handed it (/v1/sign*). A relying party cannot distinguish those provenances from a receipt alone.

This is a property of the composed format, not of one operator’s implementation. It is present on both issuers we have checked. @poteshniy independently identified the same conflation on Base-side composed issuance during this review window, and the two of us have converged on a format-level fix rather than two separate operational patches.

The fix is an issuance-path assertion: a role parameter in the JWS protected header naming what the signature attests, checked against a role member on the signing key in the issuer’s published JWKS, so a verifier resolves what a signature attests from the same fetch that already resolves the key. Semantics are additive — a receipt carrying no role resolves to unknown, not to malformed, so every receipt issued before role annotation keeps verifying. unknown is not a pass: it records that the issuance path was not established, and a relying party whose policy requires an established path treats it as its own fail.

Kid separation remains worth doing operationally on both sides as defence in depth, but it is no longer the durable fix — the assertion is, because it survives key rotation and does not require any verifier to carry a mapping table it has to keep current. No date is set here. The old kid stays in JWKS so nothing already issued is invalidated.

Evidence limitations

Vercel hobby-tier log retention is roughly one hour. Per-customer attribution across the outage window is not reconstructable.
Cached verdict entries expired on a 24-hour TTL before the snapshot was taken, so the number of post-recovery re-serves cannot be established from cache state.
The exhausted-balance HTTP status was inferred by convention, not observed at incident time.
The /v1/compose, /v1/v_gate, /v1/sign, and /v1/sign/batch routes were not instrumented. We cannot enumerate what was signed under the same kid between 2026-06-23 and 2026-08-28.

What did not happen

No degraded response could carry a signed act verdict. The v0.3 derivation reaches act only through an evaluated, adversarially-resistant claim set, and no degraded request produced one.
No signature verification failed. No key was compromised.
No spec change is required. The v0.3 receipt derivation was correct throughout.

What is still open

Body-side presentation redesign to name the actual v_recommendation state rather than compress four states into three prose bands.
A not_evaluated value at the receipt layer (v0.4 discussion). Non-evaluation currently produces no receipt at all — see 4a8b37177 above. No signed artifact records that the service was consulted and could not answer, so the distinction is absent from the record rather than collapsed within it: an auditor reading the records later cannot separate “asked and unanswerable” from “never asked.”
Alarms A2–A4 per the appended spec. A1, the hourly content canary, shipped 2026-08-29 as 851269789 — before this note published. Once someone started actively investigating on 2026-08-25, fully scoping the incident — not the four-day undetected window above, but the characterization work after discovery — took roughly six hours; A1 is the alarm that would have caught the underlying outage in one hour, which is why it went first. A2 (credit balance), A3 (rate-limit and upstream-error) and A4 (response-shape anomaly) remain open.
Issuance-path assertion. As above: the same signing key currently attests both AO-evaluated receipts and receipts where the service only signed caller-supplied bytes. Format-level fix converged with a second issuer and committed publicly; no date. Old kid retained in JWKS so nothing already issued is invalidated. Kid separation stays on both sides as operational defence in depth.
Auto-reload with a cap. Balance protection remains manual top-up only as of publication. A prepaid balance drained silently once; nothing structural prevents that from repeating.

Records cannot be backdated. An examiner asking in 2028 about behaviour in 2026 can only be answered by records that existed in 2026. That is the whole argument for this format, and it is why the incident belongs here, in the record, rather than nowhere.

Contact and references

joe@tanilo.io. Commits 4a8b37177, 98b3b7e78, c28c5dd6f, 49261a37b, a8ce2a18d. External review thread: x402-foundation/x402#3234. Benchmark amendment: agentoracle-benchmark-a-b@2351e29f6 — pre-rename repo name, historical link.

Corrections

Corrections to this note are added here, dated. The body above is not edited after publication.

Added 2026-09-22 — on this key going forward. Tanilo will sign with a newly generated key carrying no history under it; ao-composed-2026-06-ed25519-c3abfce3 will not be used for new Tanilo issuance once that key is live. A signature from the old key, on its own, does not prove the service issued that payload — because for a two-month window it signed arbitrary caller-supplied bytes under /v1/sign*, and a forger controls the entire payload including its claimed timestamp, so no date range can separate forged bytes from legitimate ones by inspection of the signature alone. The old key remains published in the JWKS because receipts legitimately issued under it before and after the exposure window still need to verify. A claim hash that also matches a corresponding entry in the persistent claim-fingerprint store (recorded by the normal /evaluate path, independent of the signature) is corroborating evidence the payload came from a real evaluation and not from /v1/sign*; a claim hash with no matching fingerprint entry is not. That store is internal to the service. A third party can't query it, so this is a check we can run on request, not one anyone can run themselves.

Added 2026-09-24 — on the not_evaluated gap named above. The -02 revision of draft-krausz-verification-state adds an instrument_failure verdict value so a non-evaluation can be signed and recorded rather than producing no receipt at all. This closes the gap described above at the specification layer; the running service has not yet been updated to emit it, so 4a8b37177's early-return-with-no-receipt behavior is unchanged in production as of this note.

Added 2026-09-30 — a second signing path, found by our own audit.

This record covered the signing routes on agentoracle.co. A read-only audit of every published key on 24 September 2026 found a second path, on a separate service. The API gateway (agentoracle-gateway-main-39fa17e.zuplo.app) exposed its own POST /v1/compose, which signed a composed envelope over a verdict supplied by the caller. The caller set the verdict, confidence, threshold, adversarial result and recommendation, and the route copied them into the signed payload unchanged. The gateway evaluated nothing. From 27 July 2026 until 24 September 2026 it signed under ao-composed-2026-07-ed25519-3d44ba27. From 22 to 27 July it signed under a test fixture key whose private half is public and which is not in our published key set, so those signatures show nothing about issuance.

Correction (21 Oct 2026): The statement that the fixture key was not in our published key set was not accurate as a general description. The later disclosure records its publication in the agentoracle.co production key set from 29 July 2026, and in the tanilo.io and api.tanilo.io sets from 2 October 2026. Those dates do not establish publication during the 22–27 July signing interval described above. See the fixture-key disclosure on our Trust page.

Our docs described this route as "decision in, signed receipt out", so it did what it said. The problem is what its output looked like. Its envelopes carried issuer: "agentoracle" and the same format as receipts from /evaluate, and the receipt carried no assertion of which route produced it. Its key ID and issuer value differed from an /evaluate receipt's, but nothing in the receipt said what either meant, so the receipt alone could not show that no evaluation happened. Access required an API key, sold from 2 August through the $99/month self-serve tier and limited to 100 requests an hour per key. The route was never open without a key.

What the records can and cannot show. The gateway wrote a log line for every signature, but the hosting plan it ran on keeps no request history, so the number of envelopes signed through this route, and which keys requested them, cannot be established. What the records do show: the only live-mode purchase of the self-serve tier was our own, on 3 August, cancelled on 22 September. On 24 September the production gateway held six API keys: four created by test-mode checkouts, which take no payment, one we created for a benchmark, and one we created for the check described below; the preview and development environments held none. At least one envelope reached the public: the receipt in Virtual Protocol's public showcase repository, committed on 3 August 2026, which was the demo receipt on tanilo.io until 2026-09-30, was signed through this route. We know of no others, and we cannot rule them out.

Withdrawn 24 September 2026. On 24 September we deleted the private key from every environment of the gateway and redeployed production, preview and development. Signing through this route ended then. On 28 September at 21:26:56 UTC an authenticated call to the route returned HTTP 500 signing_key_unavailable: the old code could no longer sign. We then deployed a change that removes the signing code from the route itself (commit 49dd88d, serving from 21:27:13 UTC), so no configuration change can make it sign again. The route returns HTTP 410 withdrawn since 28 September 2026: the same call at 21:31:11 UTC returned 410, with no signature. An unauthenticated request is still refused with 401 by the gateway's API-key check before it reaches the route; with no keys left, that is what any caller now sees. The key used for those two calls was then deleted, and the gateway now has no API keys in any environment.

ao-composed-2026-07-ed25519-3d44ba27 is retired. It stays published in the JWKS so existing signatures still verify. A valid signature under it proves the bytes haven't changed since signing. It does not show that we evaluated the claim: the gateway evaluated nothing it signed under that key.

Two routes named /v1/compose. This record's "Evidence limitations" list names /v1/compose among the routes that were not instrumented. That refers to the stub on agentoracle.co, which signed under ao-composed-2026-06-ed25519-c3abfce3 and stopped issuing on 28 August (a8ce2a18d). The gateway route in this note is a different service with a different key. The benchmarks page also mentions AgentTrust's /v1/compose, which belongs to AgentTrust.

Added 2026-09-30 — correction to the 2026-09-24 note. instrument_failure in -02 is a reason code, available under any of the four states. It is not a verdict value. The rest of that note stands.

Added 2026-10-06 — the issuance-path assertion shipped for the current key. The role-based issuance-path assertion described above shipped on 2026-10-02 for Tanilo's current key, tanilo-2026-10-ed25519-7d885da9: every receipt it signs carries role: evaluated in the protected JWS header, and its JWKS entry carries the matching role member. Nothing already issued is changed; receipts under the older keys resolve as described on the trust page.