🔍 Read the full analysis: Testing Weight-Space Programming: Claude, GPT-6 Astra, And Zero Liability on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
A home experiment used a Claude-written tool to remove a looping direction from Ornith 1.5 35B-A3B’s weights. On the 10 published Lexassay tasks, the edited model completed all 10 answers, up from 4 in the same comparison setup, but its legal score remained zero. GPT-6 Astra reviewed the code and claims and identified faults; the results are limited by a small benchmark and one run per setting.
A home experiment reported by ThorstenMeyerAI.com found that a targeted edit to the weights of local AI model Ornith 1.5 35B-A3B stopped it from leaving some legal-benchmark answers unfinished, but did not improve its score on the law questions. Across the 10 published Lexassay tasks, the edited model completed all 10 in the same comparison setup, up from four, while its legal score remained zero.
The experiment was inspired by MIT CSAIL research on “weight-space programming,” but it did not reproduce the MIT method. According to Thorsten Meyer’s account, a Claude Opus 5.5 session wrote code that ran the model, measured an internal direction associated with looping, and removed that direction at half strength from the output weights of 27 of 40 layers. The process did not train the model on new legal material, and Claude did not read and edit the weights directly by reasoning about them.
The edited model’s 10 answers contained 55 wrong or invented citations, according to the report. On a set of 50 checked content criteria, it met seven, compared with three before editing. The reported legal score nevertheless stayed at zero. The result suggests the intervention addressed a problem with completing answers, not a demonstrated ability to give accurate legal advice.
The report also describes a separate check by OpenAI’s GPT-6 Astra in Codex. Astra reviewed the code, results and claims and found faults, Meyer wrote. It was an independent reviewer from another company, not a second experiment. The source says the runs, answers, grades and citation checks are public on Lexassay; all figures in the updated account refer to its 10 published tasks.
Infographic 01 · What we changed
One loop direction, weakened at half strength
Inspired by the MIT post, but a simpler, classic technique (directional ablation): code written by a Claude Opus 5.5 session measured one loop direction by running the model and projected it out of the weights. No training, and no AI reading weights as text.
The six steps
- Write test material that is not the benchmark: 24 legal development questions plus 8 general controls, 0 overlap with benchmark prompts
- Find the true loop onset at token level: from there, at least 90% of the next 1,500 positions repeat earlier 32-token sequences
- Run the model and measure one loop direction per layer: average internal signal inside the loop minus the reasoning just before it
- Weaken it in the weights:
W ← W − α·r·(rᵀW)on the MLP output matrices of all 256 experts and the shared expert - Bake an edited copy; the original files stay untouched and the edited weights stay internal
- Benchmark original and edit on the same local setup, graded blind by three Claude agents; figures cover the 10 published tasks
Quick test: share of pure copying
Restart 20 tokens before a known loop onset, 8 cases, 3,000 tokens. Lower is better.
The cost of the edit
- Full strength: 23 of 24 legal questions finished, but control answers broke: tokens dropped out of short formulas, and month counting went wrong
- Half strength: 23 of 24 finished, 9 of 9 checked control facts right, and one German control question was answered in Chinese once
- Before the edit: 8 of 24 legal development questions did not finish; all 8 controls did
One run per setting. The loop detector was revised three times during the experiment, so the development numbers come from a moving yardstick. The direction is a statistical description, not a single “loop neuron”.
Sources: our experiment records, 4–5 Nov 2026 (executing Claude Opus 5.5 session; review by GPT-6 Astra in Codex); Ornith-1.5-35B-A3B model card.
Powered by Thorsten Meyer AI
Completion Improved, Legal Accuracy Did Not
The outcome separates two kinds of model failure that can look similar to users. A model that repeats itself or fails to finish may be helped by an intervention targeting that behavior. But finishing an answer is not the same as answering correctly: here, the reported completion count rose while the legal score remained at zero and the edited answers included many citation errors.
That distinction matters for anyone considering weight edits as a way to improve deployed AI systems, especially in high-stakes settings such as legal research. A more fluent or complete response can appear more useful without being more reliable. The report’s results do not establish that the edit improves performance beyond this model and test, and they offer no basis for treating the edited model as a legal tool.
The experiment also shows why evaluations should track more than whether a model produces an answer. Completion, factual correctness, citation validity and performance on unseen questions measure different things. A change that improves one measure can leave another unchanged—or obscure continuing errors if readers focus only on the more visible improvement.
Products Worth Considering
DJI Osmo Action 4 Essential Combo Waterproof 4K Action Camera
Bose Lifestyle Ultra Compact Wireless Smart Home Speaker Bundle, Black
Apple AirPods Pro (2nd Generation) Wireless Earbuds with USB‑C Charging, Up to 2X More Active Noise Cancelling, Transparency Mode, Adaptive
As an affiliate, we earn on qualifying purchases.
How the Local Test Differed From MIT
The MIT CSAIL work cited in the report was conducted by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. It described a general-purpose model examining another network’s weights and directly changing them without a conventional training loop. In a Qwen example, the researchers attributed looping to a direction in the model’s internal signals and reported subtracting it from a small portion of the weights. Meyer’s account gives the Qwen model’s maths result as rising from 7.9% to 48.1%.
The home test used a more established approach, directional ablation: run the model, record internal activity, calculate a direction associated with a behavior, then project that direction out of selected weights. The report says this kind of single-direction intervention has been studied before, including work on refusal behavior. The test asked whether such an edit could address a loop problem in a model evaluated on law, not whether it could teach law.
Lexassay asks models 15 questions about German, U.S. and European Union law, without internet access or tools. Ten tasks are published and five are hidden, according to the source. Ornith was selected because the report found that it looped on some tasks: two answers did not arrive before the 62,000-token limit. The source says the model was designed for coding and agent work, rather than law.
“The edit repairs finishing, not legal knowledge.”
— Thorsten Meyer, ThorstenMeyerAI.com
Limits of the Ten-Task Results
The findings are based on one run per setting and only the 10 published tasks, not the five hidden Lexassay questions. The source also says the loop detector changed during the experiment and that three graders came from the same model family. Those limitations leave uncertainty about how repeatable the results are and how well the grading reflects performance on other questions.
The reported comparison depends on setup. The unchanged model completed nine of 10 tasks when run with sampling settings recommended by its model card and no weight edit. The source also says an earlier published run of the unchanged model completed eight of 10, while it completed four in the setup used for the direct comparison with the edit. Because these local setups were not directly comparable, the report cautions against treating the numbers as a clean general estimate of the edit’s effect.
The account says GPT-6 Astra identified faults in the review, but does not specify those faults in the supplied material. It also does not establish whether the edit would help other models, hold up across repeated trials, or improve results on the hidden tasks. There is no reported evidence here of improved legal accuracy.
The source says the experiment’s answers, grades and citation checks are publicly available on Lexassay, alongside the original run, model-card sampling run, edited-model run and edit control. Readers can examine the underlying outputs and compare the different settings rather than relying only on the summary figures.
The next meaningful test would be repeated runs under clearly matched settings, followed by evaluation on the five hidden tasks or another benchmark the experiment did not use to guide the edit. The source does not report that such testing has been completed or announce a schedule for it. Until further results are available, the finding remains a limited demonstration that this particular edit coincided with more completed answers in one setup—not evidence that the model became more capable at law.
Key Questions
What changed in the edited model?
A Claude-written tool measured an internal direction associated with looping and removed it at half strength from the output weights of 27 of Ornith’s 40 layers. The model was not given new legal training data.
Did the edit improve the model’s legal score?
No. The report says the score on the 10 published Lexassay tasks remained at zero, despite the model completing more answers in the direct comparison.
Was this the same method used by MIT CSAIL?
No. The MIT work described a model directly reasoning about another model’s weights. The home test used directional ablation, a method based on running the model and calculating an internal direction to remove.
How strong is the evidence?
It is limited: the report describes one run per setting, 10 published tasks, a changing loop detector and grading by three models from the same family. It also notes that the compared local setups were not directly comparable.
Can the edited model be used for legal advice?
The report does not support that use. It says the legal score stayed at zero and the edited answers included 55 wrong or invented citations. The article also states that the experiment is not legal advice.
Source: ThorstenMeyerAI.com
