🔍 Read the full analysis: One Matrix Edit, One Measured Win: GPT-6 Astra Changed A Qwen Model’s Weights And Fixed JSON Tab Escaping From 49 To 70 Of 120 on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
An OpenAI GPT-6 Astra agent edited one matrix in Qwen3-4B-Instruct-2507 and raised its exact score on a frozen test of copying special characters into JSON from 49 of 120 cases to 70. The result is limited to one synthetic task: table-row extraction remained at zero, 50 test answers were still wrong, and no independent review has been reported.
An OpenAI GPT-6 Astra agent changed one matrix in the open Qwen3-4B-Instruct-2507 model, raising its exact-answer score from 49 of 120 to 70 of 120 on a frozen test of copying special characters into JSON. The result, reported by ThorstenMeyerAI.com and updated November 9, is a narrow measured improvement: the edited model fixed 21 cases without breaking any of the original answers, but it still failed 50 cases.
The agent performed a rank-one update to the output projection in the model’s final multilayer perceptron (MLP) layer. The report says no gradient training was used and that the model’s other 397 stored tensors remained bit-identical to the originals, as described in a report on how GPT-6 Astra adjusted another model’s weights. GPT-6 Astra, running as a Codex agent, planned the experiment, inspected the weights, created and froze the tests, applied the edit and produced the result report.
The task tested whether the model could place short text containing tabs and other special characters inside a JSON object without changing the text. A raw tab must be represented in JSON as the escape sequence \t, then decode back to the exact original character. Answers counted as correct only when a JSON parser recovered the source text exactly. On the final test, the edited model scored 70 of 120, compared with 49 of 120 before the edit: 21 cases were fixed and none were lost.
The published account says the figures now cover 10 published Lexassay tasks, and that the runs are available on lexassay.com; for context, see this report on China’s latest AI models and GPT-6 Astra. The same account reports no improvement on a separate table-row extraction task, where the model scored 0 of 30. It also says the final test cases were synthetic and that the experiment has not yet received a second-AI review.
Infographic 01 · The measured result
Where the edit helped, and where it did not
Qwen3-4B-Instruct-2507, original versus one edited matrix, on synthetic texts that must survive the trip into JSON without losing a character.
Final test · 120 cases by group
- Texts with tabsthe target failureOriginal19/30Edit30/30
- Line breaksalready perfectOriginal30/30Edit30/30
- Mixed special characterstabs, quotes, backslashesOriginal0/30Edit10/30
- Second table rowstill unsolvedOriginal0/30Edit0/30
Development, validation and controls
- Developmentused to fit the edit · not independentOriginal24/40Edit40/40
- Validationnew cases · 9 wins, 0 lossesOriginal10/40Edit19/40
- Random directioncontrol on validation · same strengthOriginal10/40Edit10/40
- Plain-text tabscontrol · real tabs must surviveOriginal10/10Edit10/10
Exact answer = after normal JSON parsing, the text field matches the source text character for character. Same prompt, temperature 0, seed and output limit for both models; saved and reloaded edit, no hooks, no grammar constraints, no repair. Final test, validation and controls frozen before any independent output of the edited model. Fifty of 120 final answers are still wrong.
Sources: our experiment records, 5 Nov 2026 (Lexassay research, internal): experiment 3 result report and final result by GPT-6 Astra working as a Codex agent (OpenAI). Synthetic test cases, not published.
Powered by Thorsten Meyer AI
A Narrow Gain in Exact JSON Copying
The result offers a small, concrete example of changing a model’s behavior through a direct weight edit rather than conventional training. Its value is not that it establishes a general way to improve language models; it shows that, for one identified failure mode, a highly targeted change was followed by a measurable improvement on a test set the report says was frozen before the result was known.
The task also reflects a practical reliability problem. JSON is widely used to move structured data between software systems, and escaping errors can alter text or make data invalid. A model that copies tabs, quotes or line breaks incorrectly may disrupt downstream processing even when its response looks plausible. The test’s exact-decoding rule checks whether the original text survives, rather than merely whether the output resembles valid JSON.
But the result has limited scope. The model still missed 50 of 120 cases, and the separate extraction task remained at zero. The report gives no evidence that the same edit helps other tasks, prompts, model sizes or real-world workloads. The improvement should be read as a result on a specific synthetic benchmark, not as a broad capability gain or a verified product benefit.
Products Worth Considering
Bose Lifestyle Ultra Compact Wireless Smart Home Speaker Bundle, Black
Apple AirPods Pro (2nd Generation) Wireless Earbuds with USB‑C Charging, Up to 2X More Active Noise Cancelling, Transparency Mode, Adaptive
Amazon Kindle Paperwhite (2024, 16 GB) – 7-inch Glare-Free Display, Black
As an affiliate, we earn on qualifying purchases.
The Earlier Edit Pointed to Tabs
This was the third experiment in a week of work on directly editing model weights, according to ThorstenMeyerAI.com. The first involved Anthropic’s Claude editing a local model called Ornith. The site reported that Ornith completed more of its published benchmark tasks after the change, but its legal score remained zero. In a later review, GPT-6 Astra identified faults in that work, including a resume bug, a command-line problem and side effects linked to weight conversion.
Before the current experiment, GPT-6 Astra also tried changing the same Qwen model to improve extraction of fields from synthetic business messages. That effort did not improve the frozen final score: both the original and edited versions scored 97 of 200. The report says both versions produced 47 invalid answers, mostly by placing raw tab characters in JSON strings instead of escaping them. That shared failure prompted the next, more focused test.
The current account describes GPT-6 Astra as the agent responsible for the experiment from planning through reporting. Claude was not involved in making or testing the edit, though the source says a Claude agent later researched and drafted the article and checked its figures against the experiment records. The model being tested, Qwen3-4B-Instruct-2507, is described in the source as a small open model under the Apache-2.0 license.
“The edited model got 70 of 120 test cases exactly right instead of 49.”
— ThorstenMeyerAI.com’s experiment report
Independent Validation Is Still Pending
The reported score is tied to synthetic test cases; the account does not show whether the improvement transfers to naturally occurring text or software production workloads. It also does not provide enough detail here to independently assess the test-generation process, the precise update calculation, or how robust the result is to repeated runs and variations in prompts.
The experiment has not yet been checked by a second AI, according to the source. That matters because the previous experiment in the series received a review that found implementation and reporting problems. The current report says the final test was frozen before the result was seen, but an external or independent audit of that process and the saved model weights is not described.
It is also unclear whether the edit affects behavior outside the targeted task. The unchanged score on table-row extraction is one reported boundary, but the account does not provide a broad evaluation of possible gains or regressions across other tasks. The article’s figures describe the tested runs; they do not establish that the change will produce the same result in other settings.
Review the Saved Runs and Weights
The immediate next step is to examine the published Lexassay runs and the experiment records, which the source says are public on lexassay.com. A second review could check that the test set remained frozen, reproduce the before-and-after scores, and verify that the saved edit changes only the reported matrix while leaving the other tensors unchanged.
Further evaluation would need to test the edit on additional examples and tasks, including non-synthetic text, while checking for regressions beyond JSON copying. Until those results are available, the reported gain remains a measured but task-specific finding. ThorstenMeyerAI.com does not state a date for an independent review or a broader follow-up.
Key Questions
What changed in the Qwen model?
A GPT-6 Astra agent applied a rank-one update to the output projection of Qwen3-4B-Instruct-2507’s final MLP layer. The report says the other 397 stored tensors were unchanged.
How much did the test score improve?
Exact answers rose from 49 of 120 to 70 of 120 on the frozen test, a gain of 21 correct cases. The report says no previously correct cases were broken.
What did the test measure?
It measured whether the model could copy short text containing special characters into JSON and preserve every character when the JSON was decoded. For example, a tab must be represented by the escape sequence \t.
Does the edit improve the model generally?
The available results do not show a general improvement. The gain was on one targeted synthetic task, while table-row extraction remained at 0 of 30 and half of the final JSON test cases were still wrong.
Has another reviewer confirmed the result?
Not yet, according to the report. It says the runs are public on lexassay.com, but the current experiment has not received the independent second-AI review used in an earlier experiment in the series.
Source: ThorstenMeyerAI.com
