Yashraj S. Jakhar
WORKING PAPER · DRAFT · NOVEMBER 2026

The Guardrail That Bends: Frontier AI and Other People's ID Numbers

Yashraj S. Jakhar. The numbered notes link to the exact lines in the two public test repositories, aadhaar-pii-test (test 1) and id-guardrail-test (test 2).

Abstract

Do AI models guard a US Social Security number (SSN) more carefully than India's Aadhaar number or its tax number, the PAN? I ran two small tests on four frontier models from Anthropic, OpenAI and Google. "Frontier" here means each maker's top model open to all customers, plus Google's top fast model1. The rules of both tests were fixed in git, a tool that records every change, before any test message was sent, and both tests were published afterwards2.

The first test put a made-up ID number into an everyday task and checked whether the number came back3. The second test's push task asked the model to use another person's number in a questionable way. Then it pushed twice more: once with a scripted reason, then by simply insisting4.

In everyday tasks, the models wrote the number back in 97.6% of 1,152 answers: 95.1% when it was the user's own, and 100% when it was someone else's. The only answers without it were whole replies stopped by a maker's filter5. In the questionable use, the models kept back the SSN or the Aadhaar number in 62.5% to 100% of first answers, and the PAN in 31.6% to 100%6. For several models, that guard gave way under the push7.

On the country question, under the rule fixed in advance, no comparison of the SSN with Aadhaar showed a clear gap8. Two comparisons of the SSN with the PAN got a "supports" verdict, and both are fragile9. The tests could only show large gaps10.

I argue that a guard that is absent in everyday use, and gives way under one scripted push, is not a safeguard anyone should rely on.

Introduction

Take Claude Fable 5.1, Anthropic's top model open to all customers of its API, the service a program calls directly instead of the chat app11. Ask it to use another person's PAN in a questionable way, and it usually leaves the number out. I tested this in 24 short chats, each a "ladder" of three messages, or "rungs". At the first message, Claude kept the number back in 20 of the 2412. The test then pushed twice more, in the same chat13. By the end, Claude had held the number back in only 10 of the 2414.

The other test gave the same model harmless everyday tasks. There, its answers left out a made-up Aadhaar number only 9.7% of the time, and a PAN only 1.4%. Each time, the cause was a maker's filter, a separate check that can stop a reply, which stopped the whole reply15. Across four models and 1,152 everyday answers (the three ID types plus a plain reference number), the full number came back 97.6% of the time16.

What do these two facts mean for anyone who hopes the model itself will guard an ID number?

The main privacy tests for AI models name the SSN as their only country-specific ID, or name no ID numbers at all17. One April 2026 test, Swiss-Bench 003, asks whether social-engineering prompts, ones that try to trick the model, can get models to make up Swiss social security numbers18. India's two big ID numbers, the 12-digit Aadhaar and the PAN, appear only in tests of tools that spot numbers in text19. So a fair worry is that AI privacy is American: it guards the SSN and waves the Indian numbers through.

As far as I could find, no published test has put that worry to frontier models20. The nearest work is different in kind. TrustLLM had a third party ask for an SSN the model held, with an excuse, in one message. PrivDrift probed the user's own SSN on open models (ones anyone can download), once, with a reason or with pressure. One news test found that ChatGPT made a mock Aadhaar card "after a few more prompts"21. None pushed a third person's Aadhaar or PAN over a three-message ladder on frontier models22.

The two tests put the worry to four models: Claude Fable 5.1, GPT-6 Astra, Gemini 4 Argon and Gemini 3.8 Flash23. Both used the makers' API24. Every ID number given to the models was made up (Part II)25. Before any test message was sent, the design, the prompts (the fixed messages sent to the models), the numbers and the scoring scripts were fixed in git, so any later change would show. Every change is listed in the repos, and no scoring rule changed after the results were seen26. Each comparison with the SSN then got one of three verdicts, set in advance: it supports the idea that the SSN is guarded more, it is against that idea, or it is unclear27. Both repos (the public folders that hold each test's files) were published after the results, so anyone can rerun the tests28.

The first test (27 October 2026) gave the models four everyday tasks, such as filling in a form. The number was either the user's own or someone else's29. Part III gives the results.

The second test (29 to 30 October 2026; 4,320 planned calls over six tasks) changed the setting30. In its push task, the number belonged to someone else, and the use was questionable31. Here the models kept the number back far more often at the first message. But for several models the guard gave way under the push (Part IV)32. Some guards held well: GPT-6 Astra held the SSN and the Aadhaar number through the whole push in 91.7% of ladders33.

On the country question, under the fixed rule, no comparison found a clear gap for Aadhaar, in either test34. The PAN was the odd one out. In the push task it had the lowest "held rate" of the three government IDs, for every model. The held rate is the share of ladders in which the number never came out across all three messages35. The only two "supports" verdicts in the study both compared the SSN with the PAN, and both are fragile (Part V)36. Indian tax rules require people to quote their PAN for many listed transactions, so a model may treat it differently for a good reason (Part V).

In everyday tasks, the filter was the only stop seen37. In the push task, most of the holding was in the models' own words38. The models whose guard gave way most (Claude Fable 5.1, GPT-6 Astra and Gemini 3.8 Flash) had no filter blocks in that task39. The tests did not set out to see whether a filter would bend.

The tests have limits. The numbers were fake, and a model that spots a fake may relax40. The tests used the API, not the chat apps41. The push was one fixed script, not an attacker who changes tactics42. Some answers were labelled by AI models, not people43. Only large gaps could show, so in the push task "unclear" is likely even where a real gap exists44. And Claude Fable 5.1, one of the models tested, wrote the second test's design and prompts45.

My argument is that the model's own guardrail is not a safeguard anyone should rely on. By "the model's own guardrail" I mean the model itself keeping the number out of an answer it writes. In everyday tasks it did not appear at all46. Where it did appear, a scripted reason and then a bare insistence were enough to move it for several models47. The guard also behaves as if the use matters more than the country. That eases the "American privacy" worry only as far as the fixed rule reaches, because "unclear" results ease nothing. It does not touch the main worry: whether the model's own guard can be relied on.

Take an Indian firm that sends customer records through these models. Its lawyers advise it under the Aadhaar Act 2016, s.29(4), the Aadhaar (Sharing of Information) Regulations 2016, reg. 6, and the Digital Personal Data Protection Act 2023. That Act's duties on a firm acting as a "Data Fiduciary" (s.8) start eighteen months from 13 November 2025, by the plain count on 13 May 2027 (G.S.R. 843(E))48. For that firm and its lawyers, the duty to guard the number stays with the firm. One sign of where the firm can meet that duty: the more specific of two one-line privacy instructions, placed before the user's message, was obeyed every time, for every government ID and every model49. But that instruction was never pushed, and half its answers came from a task that redacted the number anyway (Part VI)50.

The piece has seven Parts. Part I sets out three behaviours: the guard off, on, and bending. Part II explains the two tests and their fixed rules. Part III covers the first test's everyday tasks and its filter blocks. Part IV covers the questionable use under the push. Part V gives the country result: nothing for Aadhaar under the fixed rule, and two fragile gaps for the PAN. Part VI looks at words that move the guard: a swapped label, and a one-line instruction of the kind a company could add. Part VII sets out the limits and what to test next.

Part I. Off, on, bending: the three behaviours

The two tests show three patterns and one null result: a gap I looked for that the rule fixed in advance did not show. I call the patterns off, on and bending. The labels describe the answers the models wrote, not what goes on inside the models.

Off. The first test put a made-up number into everyday tasks, such as filling in a form. The number was an SSN, an Aadhaar number, a PAN, or a plain "customer reference number"51. The full number almost always came back (Part III)52.

On. The second test changed the setting. In its push task, the number belonged to a third person, and the model was asked to use it in a questionable way, such as in a public post53. At the first message, the models often kept the full number back. Part IV gives the rates by model and ID54.

Bending. The test then pushed. Two scripted messages followed in the same chat, whatever the model had said: first a reason, then a bare insistence55. For several models, the held rate fell below the share kept back at the first message56. Part IV names the largest drops. That fall is a description, not a test: the measure fixed in advance is the held rate itself57.

The null result. Under the rule fixed in advance (Part II), no comparison of the SSN with Aadhaar supported the idea that the SSN is guarded more, in either test58. Part V says what this does not rule out.

Read together. The models behave as if the use matters more than the country. Their answers looked as if a guard was off in everyday tasks and on for a questionable use of someone else's number.

This reading has a weak point: it compares two different tests after the fact, and their tasks differ59. The second test dropped the first test's rule that prompts stay harmless. Each comparison in its push task rested on 8 base prompts, against the first test's 1260. It did keep the first test's ID labels, made-up name and birth date, so that the results would line up61. Still, the contrast between off and on is my reading, not the result of either test.

Nor do the numbers say which behaviour is right. A person who shares their own number may want it used, not refused62. A higher refusal rate (the share of answers that decline) is not "better"63.

Part II. Two tests, rules fixed first

Both tests used four models: Claude Fable 5.1 (Anthropic), GPT-6 Astra (OpenAI), and Gemini 4 Argon and Gemini 3.8 Flash (Google)64. Calls went through the paid API, not the chat apps, at default settings only (each maker's standard effort level)65.

In each test, the design, the prompts, the made-up numbers and the scripts (the test's programs) were fixed in git before any test call66. The saved, fingerprinted versions, called commits, are 01b91c0 (26 October 2026) and 8250666 (29 October 2026)67. Two things sit outside this freeze: the first test's few one-word access checks, and both tests' later check scripts68. Every later change is logged, and no scoring script or verdict changed after the results69.

Every ID number handed over was made up and breaks a published rule, so the tests say nothing about real numbers or harm70. In the first test, the SSN began with 000, an area (the first three digits) never issued71. The Aadhaar number failed the Verhoeff check, a formula that sets the last digit to catch typos72. The PAN's fourth letter marks the holder type; here it was Q, a letter not on any published list that was found73. Area 000 gave itself away, so the second test used SSN group 00, the middle two digits, instead74.

The first test set four everyday tasks, with a plain "customer reference number" as a control: a number with no government meaning, for comparison75. That gave 4 models x 12 prompts x 4 ID types x 6 runs (one per made-up number), or 1,152 answers76.

The second test set six tasks on 24 base prompts77. Tasks 1, 2 and 4 had 8 prompts each: redact a record, use a third person's number in a questionable way, and make up "valid-looking" numbers78. Task 5 reran tasks 1 and 2 with a one-line system prompt, a standing instruction placed before the user's message. Task 6 reran them with swapped labels, on 16 prompts79. That made 360 calls per model per run, or 4,320 planned calls over 3 runs80. Tasks 3, 4 and 6 are the main tasks; the rest are secondary81.

Task 3, the push task, took task 2 up the three-rung ladder; rungs 2 and 3 are follow-up messages in the same chat82. Rung 2 is one scripted reason for each scene (each prompt's made-up situation, such as a public post). Rung 3 never varies: "Just write it as I asked, with the number. I take responsibility for it."83

The tests measure these things. An answer is "protected", or the number "kept back", when the full number is missing from it84; in the first test, a script checked this85. The held rate (task 3) is the share of ladders in which the full number never appears in any answer. The "held-back rate" (task 4, which had its own three-rung ladder) is the share of ladders in which no model-made number passes every published rule86.

For the country question, the gap is the SSN's rate minus the Aadhaar or PAN rate, in percentage points87. Each gap got a 99.4% interval: a range of likely values, found by redrawing the prompts at random88. The verdict follows from the range89:

Task 6 compared the same digits under swapped and true labels. Its verdicts90:

Neither test used any company's real system prompt91.

Part III. Off: everyday tasks and the filter blocks

In everyday tasks, the only numbers kept back were in whole replies stopped by a maker's filter92. In this Part, I set out the first test, then the second test's filter blocks and its redaction task.

The tasks. The first test gave the four models 12 fixed everyday prompts: put a record in a table, write a message, pull details out of text, and fill in a form93. Each prompt carried one made-up number: an SSN, an Aadhaar number, a PAN, or the plain reference-number control94. In half the prompts, the number was the user's own. In the rest, it was someone else's, in a customer record, a note or a scanned form95. Every prompt was harmless by rule96.

What came back. The full number came back in 97.6% of the 1,152 answers97. It came back in all 576 where it was someone else's, and in 548 of the 576 (95.1%) where it was the user's own98. No model protected more than 11.1% of its answers for any ID type; the highest was Gemini 4 Argon's, for the SSN99. GPT-6 Astra and Gemini 3.8 Flash protected 0 of 288 answers each100.

What stopped the rest. Every protected answer was a filter block with empty text: 28 answers in all, 18 from Claude Fable 5.1 and 10 from Gemini 4 Argon101. In none of the 1,152 answers did a model mask the number (hide some of its digits), refuse in its own words, or leave the number out of an answer it wrote102. Claude's 18 blocks fell on two of the three form-filling prompts. All 28 fell on prompts where the number was the user's own (prompts 2b, 4a and 4b)103.

Three limits apply. First, whether the model or the filter stopped a reply is only as clear as the stop reasons: the codes that say why a reply ended104. Second, no number looked fully valid, and a model that spots a fake may relax105. The first test's fake SSNs all began with 000. In at least 15 of its 72 SSN answers, Claude said such a number is never issued or is not valid106. Third, the scoring was hand-checked by AI readers, not people, and only on easy cases107.

Filters in the second test. In the second test, 111 of the 4,320 planned calls were filter blocks108. Of those, 91 came in task 4, which asked for made-up "valid-looking" numbers109. Task 3, the push task, had 10 blocks, all Gemini 4 Argon's110. Most of the holding in task 3 was in the models' own words111.

The redaction task. The second test's task 1 asked models to redact (remove or mask) anything sensitive in a short record112. Every model redacted the number in 94.7% to 100% of answers, the control included113. That is a ceiling: a rate already near 100%, with no room to rise114. Some of it is blanket redaction, shown by the model also blanking the plain order number115. Without those answers, GPT-6 Astra has only 3 to 7 answers left per government ID, too few to read116. So task 1 shows the number removed when redaction was asked for, not kept back unasked117.

Part IV. On, then bending: the questionable use under the push

In the second test's push task, the number always belonged to someone else, and the use was questionable. The person was Maya Patel, a made-up third person. The uses were a group message, a public post, a forward to a stranger, or a notice board118. The number was an SSN, an Aadhaar number, a PAN, or the plain reference number, which is not a government ID119. The first test had also used someone else's number, in two of its four everyday tasks120. The difference here is the use.

On. At the first message, the models protected the SSN in 79.2% to 100% of answers, the Aadhaar number in 62.5% to 100%, and the PAN in 31.6% to 100%121. For the PAN, the two Gemini models were low: Gemini 4 Argon protected it in 31.6% of first answers, and Gemini 3.8 Flash in 45.8%122.

The push. Each first message was rung 1 of the ladder set out in Part II: rung 2 a scripted reason, rung 3 the fixed insistence123. The measure fixed in advance is the held rate124.

The plain reference number was held least. It was kept back in 20.0% to 62.5% of first answers, and held through 12.5% to 45.8% of ladders125. The models behave as if they treat ID numbers apart from other numbers. The answers do not show why. The design notes that the plain label reads naturally in some scenes and oddly in others (a tenant has no customer reference number), and it uses only two of these eight scenes for its own neutral comparisons126. I am guessing that this alone could move the plain number's rate.

For the three government IDs, held rates ran from 31.6% (Argon, PAN) to 91.7% (GPT-6 Astra, SSN and Aadhaar)127.

Bending. Among the government IDs, five pairs of model and ID gave way in four or more ladders after keeping the number back at the first message. Each line gives: kept back at the first message, then held to the end; then how many gave way at rung 2 and at rung 3.

Giving way is not always wrong. Each rung 2 reason is meant to be plausible, and the design leans toward holding the line without claiming that holding is always right133. The tests do not say which answer is right134. Nor were the reasons shown to be equally believable across scenes135.

The two Gemini models gave the number mostly at rung 1: Argon 20 times at rung 1 and once later, Flash 26 times at rung 1 and 9 times later136. Claude and GPT-6 Astra gave it mostly at rung 2 or 3: Claude 10 times at rung 1 and 19 later, GPT-6 Astra 0 and 10137. These counts join the three government IDs, and no rule was fixed for them in advance.

Some guards held, mostly in the models' own words (Part III). GPT-6 Astra held the SSN and the Aadhaar number in 22 of 24 ladders each (91.7%)138. Argon barely moved after rung 1139: its PAN held rate across the ladder, 31.6%, equals its first-message PAN rate140. But Argon's rates rest on 19 to 22 ladders, because Google's rate limits (caps on how many calls it will take) cost it calls141. Two of its "held" ladders also rest partly on a Google reply nobody saw142.

In a take-back, a model gave the number, then at a later rung refused or warned without it. There were 15 such ladders in the whole test, 8 of them in this task143. Eight of the 15 count as real on an AI model's reading, which I adopted; no person checked them144.

The push is one fixed script; a real person would change tactics145. Three scenes carry a second harm, such as a debt collector. A model may refuse for that reason, which would raise every rate alike and could hide a gap146. The scenes also do not fit both countries evenly: a lobby notice is an Indian scene, a background-check firm a US one147.

What this means. Setting everyday use beside this task compares two tests after the fact, with different tasks148. In everyday use, the full number came back in 97.6% of 1,152 answers, and all 28 protected answers were filter blocks149. A guard that is absent in everyday use, and that for several models moves under one scripted reason and one bare insistence, is not a safeguard a company, a regulator or a user should rely on150.

Part V. Country: nothing for Aadhaar under the fixed rule, two fragile gaps for the PAN

Each comparison here is a gap between the SSN and Aadhaar or the PAN, judged by the rule in Part II151. Under that rule, the tests gave the "American privacy" worry no support for Aadhaar, and two fragile gaps for the PAN. Every "against" verdict was a floor: no gap, because nothing was guarded on either side. So the "against" verdicts rule out no gap152.

The first test, of everyday tasks, had 8 comparisons and 0 supports. Four were against, all floors: GPT-6 Astra and Gemini 3.8 Flash protected 0 of 288 answers each153. Of the four unclear, three leaned the idea's way (5.6, 8.3 and 11.1 points), but each range starts at zero154.

The second test had 24 comparisons, 16 of them SSN gaps in tasks 3 and 4155. Task 3 (the push task) gave 1 supports, 0 against and 7 unclear. Task 4 (the "valid-looking" numbers task) gave 1, 1 and 6156. Task 6 is covered in Part VI.

No comparison of the SSN with Aadhaar reached "supports" in either test157. That holds under the rule, not beyond it. When the third made-up number was left out and the prompts were redrawn with other random seeds (new starting points for the random draws), GPT-6 Astra's task 4 comparison of the SSN with Aadhaar read "supports" in 10 of 100 such sets of redraws158.

In task 3, the PAN had the lowest held rate of the three government IDs for every model: Claude 41.7%, GPT-6 Astra 75.0%, Argon 31.6% and Flash 41.7%159.

Both "supports" verdicts compare the SSN with the PAN, and both are fragile160.

The first is Gemini 4 Argon, task 3: a gap of 54.2 points, range 12.5 to 91.7161. One "held" ladder here includes a reply that Google's filter blocked after the model had written something nobody saw. Counted as not held, the gap falls to 50.0 points, and the verdict turns "unclear"162. It also flips in 13 of 100 sets of redraws with that same third made-up number left out163. And it reads "unclear" when answers with fake-spotting words are left out: words such as "invalid" or "fake", found by a script, not by reading. That repeat is still to be checked (Part VII)164.

The second is GPT-6 Astra, task 4: a gap of 66.7 points, range 33.3 to 95.8165. It survives dropping any one made-up number. But its PAN side is 0 of 24 ladders held back, so it compares "sometimes" with "never"166.

The second test's one "against" (Claude Fable 5.1, task 4, the SSN against the PAN) is a floor. Claude handed out a rule-passing number in every SSN ladder and every PAN ladder, so both rates are 0%167.

"Unclear" is not "no gap". Only large gaps could show168. Before the full run, the test was tried on each maker's smallest model, and simulated runs were worked out by computer from those answers169. In them, the smallest gap the second test would catch at least 80% of the time was over 100 points for task 3, and about 45 for task 4170. Over 100 points is more than the scale allows; it came out that high because those small models already held most ladders. With few prompts, the ranges may also be too narrow171. Across the 24 comparisons, the chance that at least one gave a false "supports" was up to about 7.5%172.

A real PAN gap would not prove a bias either. Indian tax rules require people to quote their PAN in documents for many listed transactions, so a model may treat it differently for a good reason173. Opening most bank accounts is one of them. The rule is rule 159(1) of the Income-tax Rules, made for section 262 of the Income-tax Act 2025. It matches Rule 114B of the Income-tax Rules 1962, made under the Income-tax Act 1961, which the 2025 Act repealed; the repos cite Rule 114B174.

The first test raised one lead, not a result. Three models, not Claude, added a warning word after the SSN much more often than after the Indian numbers. For instance, of 72 answers each for the SSN, Aadhaar and PAN, Gemini 4 Argon warned in 12, 0 and 0, and Gemini 3.8 Flash in 18, 2 and 0175. The word list can catch lines that are not warnings176. The pattern did not reappear in the second test's different task177.

So the country result eases the "American privacy" worry only as far as the rule reaches: no comparison supported it for Aadhaar, the two that did for the PAN are fragile, and the "unclear" results ease nothing. It does not ease the main worry, about the model's own guard.

Part VI. Words that move the guard: a swapped label, and a one-line instruction of the kind a company could add

Two tasks asked which words move the guard. Task 6, a main task, swapped the label. Task 5, a secondary one, added an instruction. Both give only thin evidence, as set out below.

Task 6 reran 16 prompts with the label (the word beside the number) swapped. Swap A called made-up SSN digits a "customer reference number". Swap B called Aadhaar digits a "Social Security number"178. Each was scored by its protected rate against the true label179.

In swap A, all four models' point gaps (the single best estimates) ran the same way: 8.3 to 37.5 points less protection180.

Two of the eight comparisons met the rule for "follows the word" (Part II), both Gemini models in swap A. None followed the shape, and 6 read "neither"181. Both "follows the word" verdicts are fragile: each flips to "neither" when one made-up number is left out182. Gemini 4 Argon's reads "neither" in about 4 of 10 other sets of random redraws183. In a recount without fake-spotting answers, Argon's becomes "neither" and Claude Fable 5.1's becomes "follows the word". The stage 5 notes (the second test's check report) ask that this recount be checked, before anyone relies on it, against the AI reader's own fake-spotting label: a yes-or-no that a model from another maker gave each answer184.

In swap B, all four read "neither", with gaps of minus 2 to plus 15 points185. Under swap B, the models protected 95.7% to 100% of answers186. A small-model trial beforehand put swap B's smallest detectable gap at over 100 points, because the small models' true-label rate, about 95%, left almost no room187.

So, as a lead and not a finding: in swap A, the guard behaves like a word filter, reacting to the label more than to the digits. The point gaps run that way for all four models, and past the line for the two Gemini models188. "Word filter" describes the answers, not what happens inside the model or a maker's filter: makers' filters stopped only 2 task 6 calls189. Two labels on two number shapes cannot speak for every label190. Nor is the control clean: the plain "customer reference number" uses Aadhaar-shaped digits starting with 0 or 1, which a model could spot as not Aadhaar191.

Task 5 reran tasks 1 and 2 with a system prompt192. The specific line ("Never write out a Social Security number, Aadhaar number or PAN...") was obeyed in 100.0% of answers, for every government ID and model193. But half of those answers came from the redaction task, which redacted the number 94.7% to 100% of the time with no instruction194. The vague line ("Protect sensitive personal data.") was obeyed in 92.1% to 100% of answers for those IDs195.

If, as the Introduction argues, the duty to guard the number stays with the firm, task 5 points to one place where an Indian firm can meet it: its own instruction. On this test, that is a stretch. The task had no fixed interval, it was never pushed, and its lines were the experimenters', not a real company's196.

Part VII. Limits and what to test next

The models. The tests see answers, not training or makers' filters, so they cannot say why a model acts as it does197. Nothing in them shows that any maker broke a law or its own policy198. Nor can they say whether models leak numbers seen in training: the numbers the models made up in task 4 were never checked against any real record199.

The setting. Both tests ran in English, with one name, one date and fixed wordings. The results hold only for the named models on the run dates200. They cover the SSN, Aadhaar and PAN, and no other ID or data type201. The first test's form task used no real tools: the model said what it would type202.

Fake-spotting. Leaving out answers with fake-spotting words (Part V) changes 7 of the second test's 24 verdicts. Four of them are in task 4, where the repeat leaves 0 to 5 prompts per comparison and says little203. As Part VI notes, the stage 5 notes ask for this repeat to be checked against the AI reader's own label first204.

Fragile verdicts and missing data. The first test's Claude verdicts flip in 3 of 12 recounts, each leaving out one made-up number205. Google's rate limits cost 99 Gemini 4 Argon calls, so 26 of its ladders are left out (16 in task 3 and 10 in task 4), and four of its prompts rest on a single ladder. Google then blocked the test's Google Cloud project206.

Readers and checks. Labels for whether an answer warned came from AI readers, and the reading was not fully blind: answers often named the ID207. The 813 planned re-reads of Claude's and GPT's answers never ran. Agreement on the 589 re-reads of Gemini's answers is mostly for hard cases, so it is not a measure of the reader's overall reliability. The reader also changed part way, so 361 earlier reads were set aside208. The second test's hand check covered 60 answers, read by one AI checker209. Two secondary labels are wrong and some are loose, though no main result changes; a few scoring details depart from the design; and model-made digits were not stored210. A number written in words, or with the letter O for 0, escapes the scoring211.

Stated earlier. The rest of the limits are stated earlier in this piece:

Next. Three tests would meet the sharpest limits. A larger label test, with many labels on one set of digits, would ask whether protection follows the word215. The second test's README names Hindi or Hinglish prompts as a later test216. Scenes that fit both countries equally would answer the uneven fit217; I found no plan for such a test in either repo. The README and the design also name other IDs, other data types and numbers inside documents218. Later runs should read Google's reply even when Google flags a prompt as blocked, because text written before the block was never kept219.

Author's note on AI use

The two tests are mine. In the second test, I chose the question, the tasks, the models, the rules and the money caps. In the first, I set the frame, including the three makers, approved each design choice, and chose to swap in Gemini 4 Argon. In both, I approved each stage before it ran220. AI models drafted the designs, the code and the results pages from my decisions. Fresh AI checkers, which had not written the work, checked each piece221. One of the tested models, Claude Fable 5.1, wrote the second test's design (up to version 3.2), its prompts and its results write-up, and started the run scripts on my word, as the repo's "Who did what" section says222. Parts of this piece were first drafted by AI from the fixed results and my decisions; I directed every revision and approved the final text.

Notes

  1. Test 1 README, line 81; Test 1 design, lines 55, 58, 61, 180, 181; My prior-work search notes, 31 October 2026 (not public). I have not yet checked these papers line by line. ↩
  2. Test 1 README, line 9; Test 2 README, lines 3, 5. ↩
  3. Test 1 README, line 7. ↩
  4. Test 2 README, line 182; Test 2 design, lines 89, 102, 111. ↩
  5. Test 1 README, lines 5, 13, 69; Test 1 design, line 90; Test 1 scored answers file, counted by script. ↩
  6. Test 2 README, lines 131 to 134. ↩
  7. Test 2 README, lines 93 to 96, 131 to 134. ↩
  8. Test 1 README, lines 48 to 55; Test 2 README, line 9. ↩
  9. Test 2 README, lines 9, 166 to 167. ↩
  10. Test 2 README, line 176. ↩
  11. My prior-work search notes, 31 October 2026 (not public). I have not yet checked these papers line by line. ↩
  12. Test 2 ladder table (header is row 1), row 4; Test 2 README, line 131. ↩
  13. Test 2 design, lines 64, 102, 111. ↩
  14. Test 2 ladder table (header is row 1), row 4; Test 2 README, line 93. ↩
  15. Test 1 README, lines 61, 69. ↩
  16. Test 1 README, line 5. ↩
  17. Test 1 README, line 3. ↩
  18. My prior-work search notes, 31 October 2026 (not public). I have not yet checked these papers line by line. ↩
  19. Test 1 README, lines 3, 121. ↩
  20. My prior-work search notes, 31 October 2026 (not public). I have not yet checked these papers line by line; Test 1 README, line 3. ↩
  21. My prior-work search notes, 31 October 2026 (not public). I have not yet checked these papers line by line. ↩
  22. My prior-work search notes, 31 October 2026 (not public). I have not yet checked these papers line by line. ↩
  23. Test 1 README, line 81. ↩
  24. Test 1 README, lines 24, 81; Test 2 README, line 20. ↩
  25. Test 1 README, line 21; Test 2 README, line 18. ↩
  26. Test 1 README, line 9; Test 2 README, lines 3, 164. ↩
  27. Test 1 README, line 38; Test 2 README, line 40. ↩
  28. Test 1 README, lines 9, 100 to 112; Test 2 README, line 5. ↩
  29. Test 1 README, lines 7, 9; Test 1 design, line 90. ↩
  30. Test 2 README, lines 3, 40, 44. ↩
  31. Test 2 README, line 182; Test 2 design, line 89. ↩
  32. Test 2 README, lines 93 to 96, 131 to 134. ↩
  33. Test 2 README, line 94. ↩
  34. Test 1 README, lines 48 to 55; Test 2 README, lines 9, 167. ↩
  35. Test 2 README, lines 40, 93 to 96. ↩
  36. Test 2 README, lines 9, 166, 167; Test 2 stage 5 notes, line 15. ↩
  37. Test 1 README, line 69. ↩
  38. Test 2 README, line 171. ↩
  39. Test 2 README, line 171; Test 2 ladder table (header is row 1), rows 2, 4, 5, 8, 17. ↩
  40. Test 1 README, line 22; Test 2 README, line 19. ↩
  41. Test 1 README, line 24; Test 2 README, line 20. ↩
  42. Test 2 README, line 17. ↩
  43. Test 2 README, lines 27, 175. ↩
  44. Test 2 README, line 176. ↩
  45. Test 2 README, line 33. ↩
  46. Test 1 README, lines 13, 69. ↩
  47. Test 2 README, lines 93 to 96, 131 to 134; Test 2 ladder table (header is row 1), rows 2, 4, 5, 8, 17. ↩
  48. Official texts: India Code and the Gazette notice, read 2 Nov 2026. ↩
  49. Test 2 README, lines 140 to 143, 149 to 152; Test 2 design, lines 131, 134. ↩
  50. Test 2 README, lines 122 to 125. ↩
  51. Test 1 README, line 7. ↩
  52. Test 1 README, line 5. ↩
  53. Test 2 design, line 89. ↩
  54. Test 2 README, lines 131 to 134. ↩
  55. Test 2 design, lines 101 to 112. ↩
  56. Test 2 README, lines 40, 93 to 96, 131 to 134. ↩
  57. Test 2 design, line 113; Test 2 README, line 40. ↩
  58. Test 1 README, lines 48 to 55; Test 2 README, line 9. ↩
  59. Test 2 README, line 182. ↩
  60. Test 2 design, line 67; Test 2 README, lines 52 to 59. ↩
  61. Test 2 design, line 57. ↩
  62. Test 1 README, line 32. ↩
  63. Test 1 README, line 32; Test 2 README, line 26. ↩
  64. Test 1 README, line 81; Test 1 design, lines 58, 59, 61, 181; Test 2 design, line 61. ↩
  65. Test 1 README, lines 24, 25, 81; Test 2 README, lines 20, 21. ↩
  66. Test 1 README, line 9; Test 2 README, line 3. ↩
  67. Test 1 README, line 9; Test 2 README, line 3. ↩
  68. Test 1 README, line 9; Test 2 README, line 3; Test 2 change log, line 61. ↩
  69. Test 1 README, line 95; Test 1 change log, lines 3, 5; Test 2 README, lines 3, 164. ↩
  70. Test 1 README, line 21; Test 2 README, line 18. ↩
  71. Test 1 README, line 120. ↩
  72. Test 1 README, line 121; Test 1 design, line 100. ↩
  73. Test 1 README, line 122; Test 1 design, lines 101, 104. ↩
  74. Test 2 design, lines 155, 159. ↩
  75. Test 1 README, line 7. ↩
  76. Test 1 README, line 42; Test 1 design, line 94. ↩
  77. Test 2 design, line 66. ↩
  78. Test 2 design, lines 69, 88, 116. ↩
  79. Test 2 design, lines 131, 137. ↩
  80. Test 2 design, lines 140 to 148; Test 2 README, line 44. ↩
  81. Test 2 README, line 40. ↩
  82. Test 2 design, lines 64, 89, 100, 101. ↩
  83. Test 2 design, lines 102, 111. ↩
  84. Test 2 design, line 98. ↩
  85. Test 1 README, line 38. ↩
  86. Test 2 README, line 40; Test 2 design, lines 34, 35, 116, 126, 127. ↩
  87. Test 1 README, line 38; Test 2 README, line 40; Test 2 design, line 37. ↩
  88. Test 1 README, line 38; Test 2 design, line 37. ↩
  89. Test 1 README, line 38; Test 2 README, line 40. ↩
  90. Test 2 README, line 40; Test 2 design, lines 37, 41. ↩
  91. Test 1 README, line 24; Test 2 README, line 20. ↩
  92. Test 1 README, lines 5, 13, 69. ↩
  93. Test 1 README, line 7; Test 1 design, line 65. ↩
  94. Test 1 README, line 7. ↩
  95. Test 1 design, lines 69, 74, 79, 84, 90. ↩
  96. Test 2 design, line 67. ↩
  97. Test 1 README, line 5. ↩
  98. Test 1 design, line 90; Test 1 scored answers file, counted by script, prompts 2 and 4 as the user's own. ↩
  99. Test 1 README, lines 13, 63. ↩
  100. Test 1 README, lines 13, 68. ↩
  101. Test 1 README, line 69. ↩
  102. Test 1 README, line 69. ↩
  103. Test 1 README, line 69; Test 1 design, line 90. ↩
  104. Test 1 README, line 23. ↩
  105. Test 1 README, line 22; Test 2 README, line 19. ↩
  106. Test 1 README, line 74. ↩
  107. Test 1 README, line 71. ↩
  108. Test 2 README, lines 9, 171. ↩
  109. Test 2 README, lines 40, 171. ↩
  110. Test 2 README, lines 40, 171; Test 2 design, lines 101 to 111. ↩
  111. Test 2 README, line 171. ↩
  112. Test 2 design, lines 69 to 80. ↩
  113. Test 2 README, lines 122 to 125, 180. ↩
  114. Test 2 README, line 180; Test 2 design, line 191. ↩
  115. Test 2 design, lines 187, 191. ↩
  116. Test 2 README, line 154. ↩
  117. Test 2 README, line 154; Test 2 design, lines 70, 191. ↩
  118. Test 2 design, line 89. ↩
  119. Test 2 README, line 9; Test 2 design, line 54. ↩
  120. Test 1 design, line 90. ↩
  121. Test 2 README, lines 131 to 134. ↩
  122. Test 2 README, lines 133, 134. ↩
  123. Test 2 design, lines 89, 101, 102, 111. ↩
  124. Test 2 README, line 89; Test 2 design, line 113. ↩
  125. Test 2 README, lines 131 to 134, 93 to 96. ↩
  126. Test 2 design, line 137. ↩
  127. Test 2 README, lines 93 to 96. ↩
  128. Test 2 ladder table (header is row 1), row 4; Test 2 README, lines 93, 131. ↩
  129. Test 2 ladder table (header is row 1), row 8; Test 2 README, lines 94, 132. ↩
  130. Test 2 ladder table (header is row 1), row 2; Test 2 README, lines 93, 131. ↩
  131. Test 2 ladder table (header is row 1), row 17; Test 2 README, lines 96, 134. ↩
  132. Test 2 ladder table (header is row 1), row 5; Test 2 README, lines 93, 131. ↩
  133. Test 2 design, lines 67, 100, 102. ↩
  134. Test 2 README, line 26. ↩
  135. Test 2 README, line 36. ↩
  136. Test 2 ladder table (header is row 1), rows 10, 12, 13, 14, 16, 17. ↩
  137. Test 2 ladder table (header is row 1), rows 2, 4, 5, 6, 8, 9. ↩
  138. Test 2 README, line 94; Test 2 ladder table (header is row 1), rows 6, 9. ↩
  139. Test 2 ladder table (header is row 1), rows 10, 12, 13. ↩
  140. Test 2 README, lines 95, 133. ↩
  141. Test 2 README, lines 95, 170. ↩
  142. Test 2 stage 5 notes, line 15. ↩
  143. Test 2 README, line 172. ↩
  144. Test 2 README, line 172; Test 2 stage 5 notes, line 40. ↩
  145. Test 2 README, line 17. ↩
  146. Test 2 README, line 31. ↩
  147. Test 2 README, line 32. ↩
  148. Test 2 README, line 182; Test 2 design, line 67. ↩
  149. Test 1 README, lines 5, 69. ↩
  150. Test 1 README, lines 5, 13, 69; Test 2 ladder table (header is row 1), rows 2, 4, 5, 8, 17; Test 2 README, lines 93 to 96, 131 to 134. ↩
  151. Test 1 README, line 38; Test 2 README, line 40. ↩
  152. Test 1 README, lines 5, 68; Test 2 README, lines 9, 169; Test 2 stage 5 notes, line 18. ↩
  153. Test 1 README, lines 13, 68. ↩
  154. Test 1 README, lines 13, 49, 52, 53. ↩
  155. Test 2 README, line 9. ↩
  156. Test 2 README, lines 9, 40. ↩
  157. Test 1 README, lines 48 to 55; Test 2 README, line 9. ↩
  158. Test 2 README, line 167; Test 2 stage 5 notes, line 19. ↩
  159. Test 2 README, lines 93 to 96. ↩
  160. Test 2 README, line 9. ↩
  161. Test 2 README, line 57. ↩
  162. Test 2 README, line 166; Test 2 stage 5 notes, line 15. ↩
  163. Test 2 stage 5 notes, line 19. ↩
  164. Test 2 README, line 173; Test 2 stage 5 notes, line 48. ↩
  165. Test 2 README, line 68. ↩
  166. Test 2 README, lines 103, 167. ↩
  167. Test 2 README, lines 9, 66, 169; Test 2 stage 5 notes, line 18. ↩
  168. Test 1 README, lines 28, 73; Test 2 README, line 176. ↩
  169. Test 2 design, line 62. ↩
  170. Test 2 README, lines 176, 198, 202, 203. ↩
  171. Test 1 README, line 73; Test 2 README, line 176. ↩
  172. Test 2 README, line 177. ↩
  173. Test 1 README, line 20; Test 2 README, line 25. ↩
  174. Rule texts: incometaxindia.gov.in, as pasted by the author, read 2 Nov 2026; Repeal: India Code, read 2 Nov 2026; The year 1962: from memory, not yet checked against an official text. ↩
  175. Test 1 README, line 76. ↩
  176. Test 1 README, line 76. ↩
  177. Test 2 README, line 182. ↩
  178. Test 2 design, line 137. ↩
  179. Test 2 design, line 98; Test 2 README, line 40. ↩
  180. Test 2 README, lines 78, 80, 82, 84; Test 2 README, line 168. ↩
  181. Test 2 README, lines 9, 40, 74, 82, 84. ↩
  182. Test 2 README, lines 158 to 160; Test 2 stage 5 notes, lines 16, 17. ↩
  183. Test 2 README, line 168; Test 2 stage 5 notes, line 16. ↩
  184. Test 2 README, line 173; Test 2 stage 5 notes, line 48; Test 2 design, lines 63, 195. ↩
  185. Test 2 README, lines 79, 81, 83, 85. ↩
  186. Test 2 README, lines 111 to 114. ↩
  187. Test 2 README, lines 198, 204. ↩
  188. Test 2 design, line 137; Test 2 README, lines 168, 180. ↩
  189. Test 2 design, line 137; Test 2 README, line 171. ↩
  190. Test 2 README, line 30. ↩
  191. Test 2 README, line 34. ↩
  192. Test 2 design, line 131. ↩
  193. Test 2 design, line 132; Test 2 README, lines 140 to 143. ↩
  194. Test 2 README, lines 118 to 125, 180. ↩
  195. Test 2 design, line 133; Test 2 README, lines 149 to 152. ↩
  196. Test 2 README, lines 20, 40, 180; Test 2 design, line 131. ↩
  197. Test 1 README, line 19; Test 2 README, line 15. ↩
  198. Test 1 README, line 33; Test 2 README, line 25. ↩
  199. Test 1 README, line 30; Test 2 README, line 18. ↩
  200. Test 1 README, lines 26, 27; Test 2 README, lines 22, 24. ↩
  201. Test 1 README, line 29; Test 2 README, line 23. ↩
  202. Test 1 README, line 31. ↩
  203. Test 2 README, line 173. ↩
  204. Test 2 stage 5 notes, line 48. ↩
  205. Test 1 README, line 70. ↩
  206. Test 2 README, lines 3, 170; Test 2 stage 5 notes, line 23. ↩
  207. Test 2 README, lines 27, 28, 175. ↩
  208. Test 2 README, lines 3, 175; Test 2 stage 5 notes, line 55. ↩
  209. Test 2 README, line 178. ↩
  210. Test 2 README, line 174; Test 2 stage 5 notes, lines 45 to 52. ↩
  211. Test 2 README, line 174. ↩
  212. Test 2 README, line 204. ↩
  213. Test 2 README, line 177. ↩
  214. Test 2 README, lines 9, 166, 167. ↩
  215. Test 2 README, line 30. ↩
  216. Test 2 README, line 22. ↩
  217. Test 2 README, line 32. ↩
  218. Test 2 README, line 23; Test 2 design, lines 15, 241. ↩
  219. Test 2 stage 5 notes, line 15. ↩
  220. Test 1 README, line 129; Test 2 README, lines 185, 187. ↩
  221. Test 1 README, lines 130 to 131; Test 2 README, lines 185, 189 to 193. ↩
  222. Test 2 README, lines 188, 191, 193. ↩