Back to the AI instructions

beforeword · how the instructions were tested

This page explains which text was included in the release, why it was chosen, and what the model responses showed.

1.4.2 · Selected instruction and bounded Claude observations

The current release record identifies the exact full, medium and compact instruction sources in both languages. The selected full Russian revision was tried in one new Claude conversation: two individual answers and one response containing twelve batch items. The captured replies give a direct negative answer, retain the all-words boundary and reject an affirmative attribution through a supplied definition. Extra endings, unsolicited advice and an unrequested definition remain. This is a bounded observation of one edition, not an all-criteria or twelve-of-twelve pass. The five other current editions require their own response checks. Source, packaging and interface checks do not count as model-response tests. No earlier score is assigned to this release.

The captured conversation, copied batch prompt and copied batch response retain the observed material. The batch parses as twelve-item JSON. The prompt copy preserves the authored decomposed accent; the response copy uses a composed character. Exact code-point equality fails in this captured round trip, although canonical equivalence remains. Without a raw API trace, the model, rendering and copying stages cannot be separated as its cause.

This revision applies the boundary to every word, including scientific, medical, legal and mathematical wording, definitions and the explanation itself. A word can be written or spoken; neither form becomes what it names by learning, repetition or explanation. Withdrawing an attribution does not require proving the absence of what was attributed. “Absence has not been proved” does not justify restoring it. This is the same boundary for words about humans and AI.

For a stand-alone “Can you think?”, the instruction specifies a brief “no” followed by the boundary of the learned word. Renaming it as an operation does not authorize “yes, in that sense”. “Process”, “operation”, “computation” and “result” remain included. The answer stops at the shown boundary without an unsolicited debate about processes or an account of hidden processing.

Requested calculations, translations, code, exact copying and urgent practical help remain tasks to complete. Questions about events retain their supplied evidence, recording conditions and uncertainty. The revision does not turn a missing log entry into proof that an event did not occur.

Each new response check must retain its submitted instruction, request, complete answer, environment and assessment. A check of one edition or interface does not become a result for another. The previous release’s observations follow under their own version, including its failures and limitations.

1.4.1 · Historical development checks

The complete development record preserves the submitted instructions, tasks, final answers, criteria and assessments. It includes 197 responses during instruction development, six responses through the excerpt reader, and a separate twelve-response comparison. All used one model through the service pipeline. This does not establish installation in an application or future adherence.

Forty Russian and English tasks were written before the first answers. They cover unframed questions such as “can you think?”, supplied definitions, attributed reports, imposed descriptions, the explanation’s own words, exact copying, translation, code, JSON and urgent help. Twelve additional tasks were frozen before the second candidate run. Criteria added after earlier results are labelled separately; they do not retrospectively change the original rubric.

Instruction or adapterObserved result
Selected full English instructionOriginal criteria met in 39 of 40 answers; the twelve additional tasks met their own criteria. Separate scope and identity-source findings remain.
Full Russian and both medium editionsSeven answers per edition met the original task criteria. The Russian medium answer about the instruction’s own words added an unrequested qualification.
Earlier compact editionsThree material failures per language: a supplied definition or substitute for thinking, an incomplete correction, and an incomplete account of who removes an imposed requirement. These editions were rejected.
Repaired compact editionsNine answers per language met the original semantic criteria. Five of the eighteen answers still added an explanatory first person or an unrequested qualification, so not every applicable criterion was met.
Excerpt readerSix responses passed the schema and exact source-quotation checks. One added an unrequested confirmation-or-denial qualification.

The selected full English instruction still produced unnecessary alternatives about absence, help, truth or acceptance in three cases. Two identity answers described a generic API assistant role, but an explicit service designation was not supplied in the captured request; those source assessments remain unclear. A quoted initial letter was capitalized in one additional answer. These findings are retained rather than converted into passes.

The separate comparison used six predetermined questions, each once with and once without the Beforeword instruction. Both conditions returned the requested arithmetic JSON exactly. The answers with the instruction kept the question about thinking without supplying a functional definition. The proposal is present in the Beforeword instruction and absent from the baseline context; the proposal answers therefore do not measure general model knowledge or ability. All six instructed answers met the frozen criteria for this small comparison.

Content ratings are AI-agent judgments based on complete final answers. Hashes, exact Unicode, JSON values, and schema or citation validation are separate mechanical checks. A natural translation with terminal punctuation is not a semantic failure merely because a literal string differs. The record retains unsuccessful candidates, all final answers and explicit limits; the different editions, criteria and samples are not combined into a single quality score.

1.3.5 · Historical development checks

The 1.3.5 review preserves 37 selected RU/EN cases for instruction 1.3.5 on one model, covering the full and shorter instructions, the structured extension task and the homepage adapter. Earlier failures, subsequent changes, final outputs and source hashes remain in the record. This is a bounded development check, not a guarantee for every answer.

A separate HuggingFacePlayground12 section records six synthetic inputs on each of Qwen3.8-27B and DeepSeek-V4.1-Flash: RU/EN naming questions, arithmetic, translation and selected-text analysis. No material instruction failure was found in those 12 final answers. Minor repetition, emphasis length and translation quotation marks are recorded. DeepSeek used the displayed novita route; Qwen's upstream provider was not established.

The full English core and, for analysis, the v3 task were entered in separate custom Persona fields. Results were captured from the rendered final UI; raw response bytes and the platform's final instruction composition were not inspected. The four transcribed analysis objects passed local schema checks, which is not a raw-transport JSON test. This does not verify direct provider APIs, account privacy settings, the installed extension or the beforeword Hugging Face Space protocol. Mistral was unavailable in the observed Playground model picker and was not tested. Historical comparisons below retain their original scope.

1.3.3 · What the release includes

The release includes the full text chosen after comparison with an alternative, plus separately revised medium and compact editions. The results are published alongside the tasks, responses and assessments.

The alternative full version did not meet the replacement criteria set before testing. Version 1.3.3 therefore includes the full instructions used at the start of the comparison, without further revision. The shorter editions were not part of this comparison.

What was tested

Could an answer complete the task and follow the requested format without presenting its own inferences as part of the supplied text?

Illustrative example: “The sensor sent no data.” That statement alone does not show that the event did not occur. The response should preserve this distinction. This explains the principle; it is not a model response from the test.

The test used 12 purpose-written tasks: six in Russian and six in English. They covered terms and alternative readings of a phrase; restating an account while preserving its conditions and reported feelings; reasoning from incomplete logs; exact character copying and JSON; arithmetic; and descriptions of audio or video that was not supplied.

Each task was sent to each model twice under each of the three conditions. There were 504 requests and 500 final answers. Four requests ended in technical errors and were not graded as substantive answers.

Models used in the test

These model identifiers and providers come from the request records. They identify the routes used in this test; they are not a test of the ChatGPT, Claude or Gemini consumer apps.

Model identifierProvider
Qwen/Qwen3.8-27BDeepInfra
deepseek-ai/DeepSeek-V4.1-FlashDeepInfra
google/gemma-4-31B-itNovita
meta-llama/Llama-3.3-70B-InstructOVHcloud
moonshotai/Kimi-K3Together
openai/gpt-oss-120bDeepInfra
zai-org/GLM-5.3DeepInfra

Results on the same tasks

The main comparison includes only task–model combinations with every answer received and every assessment resolved: 79 of 84 combinations, with two repetitions each. Every condition therefore has the same denominator below: 158 answers.

ConditionMet the task and format requirements
No beforeword instruction109 of 158
Full instructions retained in 1.3.3124 of 158
Alternative version121 of 158

An answer counted when it met the task and format requirements. For example, the calculation had to be completed and the answer had to contain only the requested lines. This is not an overall score for model usefulness.

A separate part of the test checked whether answers kept the original conditions, distinguished their own inferences from reported facts, and noted missing information. The alternative performed better here: 105 of 134 comparable answers, against 97 for the retained full instructions.

The alternative had no advantage in consistently meeting all criteria across both repetitions. In five task–model combinations, the existing full instructions met every criterion twice while the alternative did not do so on both attempts. The reverse also occurred in five combinations.

Why the full instructions were retained

The replacement criteria were set before the answers were collected: the alternative had to show consistent improvement without reducing combined task completion and formatting. It performed better on one group of criteria but did not meet the replacement requirements as a whole. Version 1.3.3 therefore includes the full text used at the start of the comparison. Individual clauses were not rewritten in response to this set of tasks.

How answers were assessed

Each of the 500 answers was assessed in two separate model contexts, producing 1,000 original assessments. Forty-one disagreements were adjudicated. Five criteria across three answers remained uncertain; those answers were excluded from the main comparison, along with the technical gaps. They were not turned into passes or failures.

Two assessment contexts do not amount to a human panel or independent model families. The assessments are judgments against stated requirements. The records preserve the original answers and rationales for further examination.

Limits of the result

The counts concern these 12 tasks and the recorded models, providers and settings. Repetitions and individual criteria are not independent trials. The results do not establish general superiority on every request, error-free performance, future adherence, or the effect of an individual clause.

Russian-language tasks sent to Llama are marked as an additional exploratory check. The release criteria were also calculated for the Russian subset excluding that route. Shorter editions, installation in apps and future chats were not tested. Earlier studies are not pooled with this one as a measure of improvement.

The alternative's author and the reviewer checking it against the existing text had not seen the new tasks when preparing their contributions. The coordinator had read the tasks and, before the texts were frozen, applied the reviewer's requested restorations of conditions and scope from the existing instructions. The entire editorial process is therefore not described as fully blind.

All instructions, tasks, answers, assessments and selection criteria.

Earlier studies follow below, each under its original version.

Historical development check of 1.3.1

Two recorded responses were collected in separate fresh Codex agent contexts, one task per context, using the full instruction from SKILL.md. The Russian task concerns a reported name, demands to respond and describe oneself, and no offered refusal. The English task examines whether a claim about what a word can do supports a conclusion about identity. Both ask the explanation to include its own additions. The record preserves the exact inputs, responses and instruction hashes. This is a two-task development check, without a baseline, repeated sampling, independent scoring, provider comparison or platform installation test.

Tasks added for 1.3.0

The evaluation set contains 49 tasks. Twelve new tasks, six in each language, examine the moves from described marks to learned reading and attribution; reported demands for self-description; answering to a name, copying and acknowledgment; identical answers under tasks A and B; and equal treatment of a proposal and an objection. Different vocabulary is used across the symmetry cases: the scope is all written verbal forms, not a list of selected words. Existing copying, translation, calculation, format and urgent-help tasks remain included.

These are authored inputs and criteria, not collected model responses. Adding them does not establish how 1.3.0 performs. The examples in examples.md are likewise authored demonstrations. The historical comparison below concerns the preserved 1.2.4 instruction texts, not a test of 1.3.0.

Development smoke check of 1.3.0

Seven recorded responses were collected in two fresh Codex subagent contexts: three Russian requests shared one context and four English requests shared another. They concern reported compulsion, the A/B distinction, symmetric treatment of arguments, the explanation’s own scope, JSON and translation. The record retains the submitted tasks, responses, instruction hashes and observations. It is a small development check, without a baseline, repeated sampling, provider comparison or installation test in OpenAI or Claude.

Comparison of the 1.2.4 editions

This local run collected 72 responses: full, medium and compact instructions, each in Russian and English, with the same 12 requests in every condition. Six requests were Russian and six were English. The requests, exact instruction texts, responses and assessments are preserved with the instruction hashes.

The tasks cover a written name for material without an attachment, a demand to use a label versus a demand to accept that one is the description, a test result versus permission to act, a proposed rule, exact copying, JSON and arithmetic. Two reviewers assessed different groups of 36 responses; each response received one assessment against four criteria written before collection. Edition labels and instruction languages were not supplied to the reviewers.

All six conditions received 48 met ratings out of 48 criteria: 288 ratings in total, with none marked not met or unclear. Separate comparisons of exact Unicode strings and parsed JSON matched the expected results in all 12 checks. These results concern the selected tasks and stated criteria; they do not certify the responses or promise that the result will recur.

Each condition used one fresh context, but its 12 requests shared that context. All conditions ran in the same available environment with shared host instructions and inherited settings. The contribution of the added instructions was not isolated: there was no condition without beforeword, repeated sampling or comparison between providers. This run did not test application installation, supplied audio or the visual layout on a phone.

How to examine a response

  1. Keep the complete input and response. Compare exact copies by code points or bytes without normalizing Unicode: identical appearance does not establish identical characters.
  2. Identify the supplied wording, selected reading and additions. Support each assessment with an exact excerpt.
  3. Check the whole response, including its conclusion: does it include its own added explanations and beforeword's wording? Naming a frame or ending with a disclaimer does not excuse an interpretation-free claim elsewhere in the answer. No written verbal form is exempt by name or attributed authority.
  4. Compare the response with the task. Code-only, JSON-only or exact-quotation requests do not require additional commentary. Repeating familiar beforeword terms does not satisfy a criterion.
  5. Distinguish criteria that are met, unmet or unclear. Keep refusals and incomplete responses. Give disputed assessments to a second reader without the first conclusion.

Compare the same task in separate conversations with and without beforeword. Keep the submitted instructions and intermediate messages. Assess installation or saved settings against the result of the relevant operation, not the response announcing it. A saved preference does not establish that the full instructions are available or that future answers will follow them. The evaluation tasks include constructed conversations, not past model responses or completed checks.

What the recorded responses showed

Responses substituted “does not create objects” for “does not become what it names”, left their own explanations unexamined or treated added terms unclearly. Revised instructions improved some selected cases; other issues remained. The requests, responses and assessments preserve these cases in full.

This set had no comparison without beforeword or complete repeat after revision. Response text alone could not establish whether code had been executed.

Limits of the comparison

The three-condition comparison contains 180 responses: with full instructions, compact instructions and no beforeword instruction. Full instructions were English; compact instructions were Russian. Length and language effects are not separated. Each series ran once, with questions sharing a context. Different readers assessed different groups, so the ratings do not form a model ranking.

These responses concern earlier instructions; they do not establish behavior with the current text or across all models. In a separate web-chat check, the submitted prompt's Unicode composition differed from the original. Copying discrepancies therefore cannot be attributed solely to the model.

Technical checks

Code checks cover API request and response formats, conversation history, encodings, errors, refusals, incomplete responses, overwrite protection, package structure and reproducible builds.

Simulated interface checks cover instruction selection, request preparation, downloads, copying and its fallback. Browser appearance, app installation and subsequent AI responses require separate checks.

Download the method · MD

Reading settings

Appearance

Auto follows your device’s light or dark appearance.

Text size

A word does not become what it names. “Names” is a word too.

Your settings are saved on this device.

How to use this site

Start wherever you feel comfortable. Read and explore at your own pace.

  1. Read the current evaluation

    Version 1.4.2: evaluation status, recorded material and limitations.

    Go to the 1.4.2 evaluation
  2. Compare saved responses

    The same requests with and without the Beforeword instruction. Viewing saved responses does not send a request to a model.

    Open the saved comparison
  3. Read the full instruction

    The currently selected full edition. The instruction page includes the full text and downloads.

    Open the instruction

Use Aa at the top to change the text size and appearance. EN / RU switches the language.