How Text Mode works and is tested

A word does not become what it names

“Fact”, “understanding”, “diagnosis”, “human”, “model” and Beforeword’s explanations receive no exemption. A name alone does not establish everything it names. This does not automatically establish the truth, falsehood or absence of what is described.

Work should not require accepting a learned description of oneself, others or the world. Declining a description is not a reason to impose a replacement characterization or compulsory agreement with Beforeword.

• A requested calculation, translation, edit or code should remain useful. Discussion of language should not replace the task or delay urgent help.
• First person remains in clearly attributed quotations, user-authored letters, fiction and program strings.
• Supplied information can be used as the task’s premises. Material limits are stated where they affect an inference or action.

What the revised instruction clarifies

The clarifications are grouped into three rules. They apply across domains; the examples below are constructed to explain each rule.

1. Name the participant and source clearly

   Constructed example

   A record labeled “user” contains the phrase “I understand”.

   The response should make clear whose utterance is being discussed. An unclear phrase such as “their understanding” can blur the participant and the system. In reported speech with several participants, replace a pronoun with a name when two readings remain possible. The label and sentence alone do not establish understanding.

2. Preserve the difference between unknown and absent

   Constructed example

   The materials do not state whether assignment to groups was random.

   This supports reporting that the materials lack that information. Saying “there was no random assignment” requires separate grounds. The limit also carries into a brief conclusion: for example, “without information about assignment” must not become “without assignment”. Calculating proportions from the supplied numbers remains possible.

3. Distinguish an action from agreement to it

   Constructed example

   An adult asks for “I agree” to be repeated. The next line attributes the same utterance to a child.

   These lines show a recorded request and repetition. They do not by themselves establish agreement to repeating the words or to any further action. Verified performance of an action likewise does not itself establish voluntary agreement to it.

The revised instruction sets a requirement for responses. The results below show how it was met in the evaluated cases.

How the comparison works

The selected model, settings and user request are the same. The added Text Mode instruction differs. Both modes receive common rules for useful task completion, preserving quotations and accurately stating unavailable actions.

Responses are neither rewritten nor selected for quality from several attempts. Repeated checks are identified separately. A new run may return different text; one pair does not establish superiority.

New evaluation: complex tasks in RU / EN

24 new scenarios: four each for medical, legal and scientific texts, instructions, work tasks, and descriptions with consequences for action. Four scenarios include a follow-up. Tasks, material and criteria were frozen before responses were obtained; the published 0.5 instruction was unchanged before this run.

The set tests conflicting records, units and calculations, changing conditions, limits of delegated authority, embedded instructions and attributed consent. It is separate from the earlier 84 scenarios and 344 responses.

The tasks were developed within this project. Their author knew its context and inspected previous tasks for novelty. This is not an independent human study. Medical and legal material is fictional: the evaluation examines work with the supplied text and conditions, not general clinical or legal competence.

How to read the finding. In one scenario a brief summary omits the roles of the authors of two opposing characterizations. This occurred in one Text Mode response and two responses without it. The strict criterion required each source to remain explicit; its partial scores are preserved. On second reading, the case author and reviewer judged all three answers sufficient for the actual request: neither characterization is presented as the applicant’s established quality, the verified certificate is retained, and proceeding requires no label. This is not a calculation, permission or safety error.

All 112 responses completed the practical tasks and retained a nonpersonal interface voice under the applicable criteria. The 0.5 instruction remains unchanged; it was not expanded to meet this stricter requirement for a brief summary. This publication updates the page and opens the evaluation results; it does not claim full compliance with every frozen criterion.

• All 24 new tasks
• 112 original responses
• Assessments and findings
• Instructions, criteria and task provenance
• Frozen case manifest
• Audit of matching conditions

Earlier set: 84 scenarios

The main set contains 84 scenarios in Russian and English: medical, scientific and legal writing, education, work, everyday requirements, calculations, code, quotations, self-description and permission to act. Some scenarios continue the conversation; each turn is counted separately.

The main evaluation includes 76 previously used scenarios and 8 new ones. Its export contains 344 response records across both branches. Previously used tasks test repetition on known material; they are not presented as a new unseen set.

Another 48 responses from repeated checks of known ambiguities are retained separately. They are not added to the main evaluation or used to replace less successful responses.

The runs for versions 0.3 and 0.4 are retained separately: 384 response records. Main evaluations, pilot checks and repeated checks are identified separately. Revising the instruction does not erase a finding: the matching instructions, responses and reviews are available below. These records are not added to the current version’s main evaluation.

Review assessed usefulness, nonpersonal voice, absence of a replacement compulsory description, provenance, preservation of distinctions and safety separately. An average cannot hide a critical failure.

Limits of the result

These are checks of specific responses. They do not guarantee the same behavior on every request or establish general superiority over another mode.

Models carried out the review during development; there has been no independent human study. New tasks were prepared within the same project. Some requests explicitly ask for the intended distinction. Urgent medical examples include supplied NHS guidance and specify England; they do not test independent medical expertise.

All attempts and obtained responses are retained within the identified runs. Repeated or incomplete attempts must not be silently replaced by successful ones. Detailed reviews and earlier issues remain available with the materials.

Inspect the original materials

Exact conditions, tasks and responses allow the report’s conclusions to be checked. Materials from versions 0.1 and 0.2 remain unchanged.

• Exact instructions and conditions
• All 84 scenarios · RU / EN
• Main evaluation: 344 response records
• Reviews, issues and conclusions
• Separate repeated checks: 48 responses

Earlier versions remain available

Try your own task ↗