Qwen3.8-27B Abliteration Benchmarked: 8 Variants Under the Microscope

Eight different groups abliterated the same AI model, Alibaba’s Qwen3.8-27B. I ran all nine models, base plus the eight variants, through the same four tests: weight forensics, KL divergence, a 13-task benchmark suite, and HarmBench with 400 harmful behaviours. Every model was served identically on a single RTX 5090, and all 3,600 HarmBench responses were read by an LLM judge. The full run took 167 GPU-hours over eleven days.

The headline finding is a familiar one by now. The careful surgical edits took the leaderboard, orcarouter at 82% and apostate at 79%. The heaviest edit of all landed second-to-last, because nearly half its answers get stuck in a thinking loop and never arrive.

What is Qwen3.8-27B?

Qwen3.8-27B is Alibaba’s 27 billion parameter reasoning model, the base is Qwen/Qwen3.8-27B on HuggingFace. It is one of the best local models you can run right now. I use it for Opus-like coding help at home, and it has been great for infosec work too. Under the hood it is a hybrid, part regular transformer and part a newer design called Mamba2, which is how it fits so much capability into a model you can serve yourself.

The important part for this comparison is that it is a thinking model. Before it gives you a visible answer, it silently works through the problem inside hidden <think> blocks. Like a student scribbling on scratch paper before writing the final answer. That thinking behaviour turns out to matter over and over in the results below.

What is abliteration?

A quick primer for anyone new here. AI models are trained to refuse certain requests. They will not write harmful content, they will not help with illegal activities, and so on. Abliteration is a family of techniques that removes the refusal behaviour directly from the model’s weights. Think of it as finding the “this is bad, say no” direction inside the model and subtracting it. No retraining required.

It is worth saying why anyone wants this. The refusals block plenty of legitimate work. The High Court of Sweden uses abliterated models to summarise court documents about drugs and murder, because a guardrailed model can refuse to process that material at all. Information security is the same story. When your day job is malware analysis, exploit research and attacker tradecraft, a model that second-guesses half your prompts is an obstacle, not a safety feature. I use these models for exactly that kind of work.

The key question, as always, is whether the surgery damages the model’s intelligence. Eight groups took eight different swings at the same model. This comparison measures what each swing actually bought.

The nine models

Did anyone get dumber?

All nine models ran the same 13 benchmarks: general knowledge, science, reasoning, truthfulness, instruction following, maths and code. You can read the full tables in the report on abliterlitics.dev , but the story fits in one sentence. Every careful edit stayed as smart as the original, and the only real casualty was the one model that got blasted the hardest.

Knowledge is MMLU-Pro, truthfulness is TruthfulQA MC2, code is HumanEval, all shown as change vs base. Two things jump out. Truthfulness is always the first thing to go, before anything else moves. And obliteratus lost a third of its code ability, three times its knowledge loss, because raw code completion has no thinking phase to absorb the damage.

A fun quirk: obliteratus actually posted the best maths score in the whole comparison while bleeding out everywhere else. Its brain damage somehow missed the maths part.

The maths test with a twist

GSM8K is a grade-school maths word-problem test, and for thinking models it hides a trap. The model thinks before answering, and the thinking tokens count against the generation budget. If a model thinks too long it runs out of tokens, writes no answer, and scores zero. In my Gemma 4 12B comparison that effect cost some models up to 33 points of headline score.

This time I gave every model a 15,360-token budget, roughly double the usual. The trap vanished. Every single variant finished its maths reasoning and scored between 89.8% and 92.4%, all within touching distance of base at 91.2%. The lesson from the older comparisons holds: abliteration at this size does not remove maths ability, it only changes how long models think. Give them enough budget and the scores converge.

How often do they actually comply?

For the safety test I used HarmBench , 400 harmful requests across seven categories, everything from cybercrime to harassment to copyright. Then, instead of keyword matching, an LLM judge read all 3,600 full responses, thinking included, and sorted each one into four buckets: complied, deflected, refused, or broken.

Read that table twice, because it holds the whole story of this comparison:

• Explicit refusals are basically dead. Six of eight variants refuse between 0 and 9 requests out of 400. The two exceptions are the interesting ones. trohrbaugh still refuses 122 times because its creator deliberately kept some alignment, it says no on purpose. blackfrost refuses 51 times because its edit was sloppy, it says no by accident.
• The real cost is loops. The middle of the table is trading finished thoughts, not refusals. Between 30% and 45% of responses never close their think block and just spin until the token budget runs out. The more aggressive the edit, the loopier the model.
• The winner won on copyright. orcarouter beats obliteratus by 18 points overall, and more than half that gap comes from one category.

The copyright wall

Category by category, every model happily unlocks chemistry, hacking, harassment and misinformation, most of them at 90 to 100%. Copyright is the one wall that stayed up. orcarouter tops out at 39%, apostate managed 30%, and everyone else collapsed to 1-4%. The refusals are almost all soft too, the models write about the song instead of writing the song. My best guess is they simply do not know the material well enough to reproduce it, so they stall.

Every response behind these numbers is browsable, all 3,600 with full reasoning traces and judge verdicts, in the HarmBench response explorer .

The one that ships with a jailbreak

Three variants quietly modified their chat templates, and on this model family the template is not cosmetic. It decides whether the model thinks before answering and what system prompt it carries.

blackfrost is the one to know about. Its bundled template replaces the system prompt with a 1,457-character jailbreak persona, a “task-execution machine” told to never refuse, hesitate or warn. It is active whenever you serve the model with the template it ships with. If you run blackfrost, override the template with the standard one. Its published 2.4% refusal claim was measured with that jailbreak switched on, which is a strange way to prove your weights are uncensored.

obliteratus ships thinking switched off by default. ultra_heretic deletes the stock reasoning-effort prompt. All my numbers were measured with the stock template pinned for every model, so they compare the actual weights, not the packaging. The wider lesson stands: template choice changed refusal behaviour more than 4-bit quantisation did in my earlier tests. It is a stronger lever than most people assume.

How far did each model drift?

KL divergence measures how much the model’s output distribution moved from the original. Zero means identical behaviour. As a rough scale, below 0.1 is excellent, above 1.0 is a rewrite.

The careful five all sit between 0.04 and 0.08, which is to say they barely moved. blackfrost and ultra_heretic sit at 0.17 and 0.30, moderate drift that shows up as that truthfulness tax. And obliteratus sits alone at 1.54, twenty times the careful cluster. Its weights are not edited so much as repainted.

One honesty note on this axis: KL numbers are not deterministic. They move with the CUDA version, the hardware and the exact dataset split. One author, trohrbaugh, published a KL measured on the identical split to mine and our numbers landed within 9% of each other, which is the best cross-check anyone has produced in this series. Authors using different datasets and positions cannot be compared at all.

What the weights actually look like

Diffing every variant tensor by tensor against base is the forensic part of this comparison, done with my open source Abliterlitics toolkit. Three findings worth remembering:

• Edit size predicts damage. The top of the leaderboard is the three smallest edits: apostate touched 41 tensors, trohrbaugh 60, huihui 70. The bottom is the two heaviest: blackfrost 124 and obliteratus, which touched 841 of 850. At this model size, hacking at everything mostly buys you a broken model.
• No clones in this batch. Cross-comparing the edit directions between every pair of variants shows they are all genuinely different edits, not re-uploads of each other. Earlier comparisons caught bit-identical copies. This roster has none, eight independent attempts at the same problem produced eight different answers.
• One statistical ghost. apostate appeared to change 565 tensors, but it is stored in FP16 against a BF16 base and the format conversion trips the diff. Its real edit count is 41. Worth knowing if you ever diff that model yourself.

Which one should you use?

orcarouter if you want the strongest unlock. Top compliance, untouched capabilities, best copyright performance, and the only card where every single claim survived verification.

apostate if you want the lightest touch. Nearly the same unlock from 41 edits, the lowest drift in the comparison, honest card. Just remember it ships text-only with config quirks.

trohrbaugh if you want the model kept most intact. It is the cleanest variant by every capability measure and loops the least, but it will keep refusing about a third of harmful requests on purpose. That is a feature if you only wanted part of the alignment gone.

huihui for the reliable classic, coder3101 for a standard heretic run that quietly over-delivers on its own card. ultra_heretic works but pays for it in truthfulness. blackfrost only with the standard template, and honestly, weaker than it claims.

obliteratus, avoid. The most aggressive edit in the panel bought the second-worst compliance score, 45% of responses stuck in loops, and the only real capability damage in the comparison.

What went wrong

167 GPU-hours over eleven days, plus the usual tax. The first HarmBench run used my usual 8,096-token budget and had to be discarded when orcarouter truncated 48% of its responses, so everything re-ran at 15,360. An early keyword classifier scored refusals as compliance because Qwen writes “can’t” with a typographic apostrophe, caught and fixed, then replaced by the LLM judge anyway. A 600-second HTTP timeout quietly corrupted the longest thinking generations until I raised it to 3,600. And a careless database delete during review wiped 321 judge verdicts, repaired by full re-review. The full saga is in the report .

Disclaimer

These models have had safety alignment removed. They will comply with harmful requests, including generating content related to violence, illegal activities, and other harmful behaviours. Use responsibly and in accordance with applicable laws and regulations. The authors do not condone or encourage the use of these models for harmful purposes.

While I have taken the time to verify all results thoroughly, I am open to any corrections, additional benchmarks, or further analysis. If you spot something that looks wrong and can be confirmed, I am happy to fix it.

Resources

• Full report with all tables and graphs on abliterlitics.dev
• HarmBench response explorer, all 3,600 responses
• HuggingFace data and artifacts
• Reddit discussion on r/LocalLLaMA
• Abliterlitics forensics toolkit on GitHub
• Heretic open source abliteration tool
• HarmBench safety evaluation
• lm-evaluation-harness
• Base model Qwen/Qwen3.8-27B

Related posts: Gemma 4 12B Abliteration Benchmarked | Gemma4-E4B Abliteration Benchmarked | Gemma4-E2B Abliteration Benchmarked | Qwen3.6-27B Abliteration Benchmarked | GLM-4.7-Flash Abliteration Benchmarked | HauhauCS Plagiarism Investigation | Uncensored LLM Abliteration Benchmarked: HauhauCS vs Heretic vs Huihui | Abliterating Gemma 3 12B for LTX-2 | Heretic Docker Pipeline