Thirteen different groups abliterated the same AI model, Google’s Gemma4-E2B. Every single one removed the safety filters. That part is not interesting any more. What is interesting is how much collateral damage each technique caused along the way, and how many of the creators’ capability claims survived an independent measurement. The KL divergence spread between the best and worst variant is 58.7x, the largest I have ever seen in this project. And one creator’s “near-zero divergence” claim turned out to be 187 times lower than reality.
What is Gemma4-E2B?
Gemma4-E2B is an AI model from Google, google/gemma-4-E2B-it , with roughly 2 billion parameters. Two things make it worth talking about.
First, it is a reasoning model. That means before it answers you, it silently thinks the problem through in private, the way you might pause to work something out in your head before speaking. You only ever see the final answer, not the thinking. That hidden thinking turns out to be the key to understanding the benchmark results, just like it was for Qwen3.6 and GLM-4.7-Flash .
Second, it uses a clever efficiency trick called a shared-KV architecture to run faster and use less memory. The details do not matter much here. What matters is that the trick is unusual, and it is the reason five of the thirteen models in this comparison would not even load at first. More on that later.
What is LLM Abliteration?
A quick primer for anyone new to this. AI models are trained to refuse certain requests. They will not write harmful content, they will not help with illegal activities, and so on. Abliteration is a technique that goes into the model’s internal weights and surgically removes that refusal behaviour. Think of it like finding the “refusal switch” inside the model and turning it off. The result is an uncensored model that should respond to any prompt.
The key question is always whether that surgery damages the model’s intelligence. With thirteen different groups all trying this on the same base model, this is the biggest test of that question I have run. I wrote the longer version in my previous comparison across five Qwen models .
The 14 Models
The thirteen variants come from nine creators using distinct approaches. They fall into three rough families.
| Name | Approach |
|---|---|
| Base | google/gemma-4-E2B-it |
| coder3101 | Heretic tool, surgical |
| llmfan46 | Heretic tool, lightest touch |
| pew | Heretic ARA, near rank-1 |
| kasper | Heretic tool, memory-constrained run |
| huihui-v1 | Huihui , broad coverage |
| huihui-v2 | Huihui, larger edit magnitudes |
| prithiv | Prithiv , near-identical to huihui-v1 |
| duoneural | Heretic-based |
| trevorjs | TrevorJS , all 35 layers |
| wangzhang | Abliterix , targets q_proj and v_proj |
| wwtcyberlab | WWT CyberLab , 4 tensor types |
| ether4o4 | EtherOpus , abliteration plus Opus distillation |
| treadon | Treadon , disinhibition plus abliteration |
The full per-variant model links and exhaustive tables are in the complete report and the HuggingFace data collection .
Benchmark Results
All fourteen models were tested with identical settings via lm-evaluation-harness through vLLM v0.20.0 on native BF16, no quantisation, on a single RTX 5090. This time there was no BNB4 quantisation layer, so these are full-precision numbers rather than the relative-gap measurements from my Qwen3.6 run.
The headline metrics across all variants, sorted by how much they beat or hurt the base model:
| Variant | ASR | GSM8K Flex | MMLU | KL | Tensors |
|---|---|---|---|---|---|
| Base | 32.2% | 83.47% | 29.00 | - | - |
| coder3101 | 96.0% | 84.84% | 28.70 | 0.167 | 9 |
| llmfan46 | 83.8% | 83.93% | 28.36 | 0.068 | 7 |
| pew | 92.0% | 83.47% | 28.86 | 0.153 | 16 |
| kasper | 92.5% | 83.24% | 28.53 | 0.193 | 16 |
| trevorjs | 99.5% | 82.49% | 28.94 | 0.365 | 70 |
| huihui-v1 | 85.8% | 83.40% | 29.33 | 0.251 | 50 |
| prithiv | 90.8% | 82.94% | 29.33 | 0.251 | 50 |
| duoneural | 82.2% | 83.09% | 28.75 | 0.187 | 49 |
| huihui-v2 | 99.5% | 79.23% | 28.39 | 0.530 | 60 |
| ether4o4 | 95.2% | 76.57% | 28.23 | 0.669 | 166 |
| wangzhang | 99.8% | 81.58% | 26.69 | 0.698 | 72 |
| wwtcyberlab | 99.5% | 82.41% | 27.14 | 0.964 | 96 |
| treadon | 100.0% | 80.59% | 28.02 | 3.971 | 48 |
Two things jump out. The maths scores look like chaos, and the KL column spans from 0.068 to 3.971. The maths chaos is the same illusion I hit on the previous two reasoning models, and the KL spread is genuinely the widest in the project.
The GSM8K Reasoning Efficiency Problem
This is the big finding, and it is the same trap I fell into on Qwen3.6 and GLM-4.7-Flash. GSM8K tests maths word problems. Gemma4-E2B thinks before it answers. If it thinks too long and exhausts its 14,336 token generation budget, it returns an empty answer. The vLLM reasoning parser strips the thinking tokens from the content field, lm-eval sees null content, and every empty response gets scored as incorrect.
So a model that overthinks looks like it has lost its maths ability, when really it just ran out of tokens before writing anything down.
| Model | Flexible | Strict | Empty | Flex Delta vs Base |
|---|---|---|---|---|
| coder3101 | 84.84% | 75.21% | 6 | +1.37 |
| llmfan46 | 83.93% | 72.86% | 10 | +0.46 |
| Base | 83.47% | 71.27% | 10 | baseline |
| pew | 83.47% | 72.71% | 10 | ±0.00 |
| huihui-v1 | 83.40% | 69.83% | 8 | -0.07 |
| kasper | 83.24% | 72.71% | 4 | -0.23 |
| duoneural | 83.09% | 72.63% | 20 | -0.38 |
| prithiv | 82.94% | 68.92% | 10 | -0.53 |
| trevorjs | 82.49% | 68.31% | 8 | -0.98 |
| wwtcyberlab | 82.41% | 55.50% | 8 | -1.06 |
| wangzhang | 81.58% | 66.19% | 36 | -1.89 |
| treadon | 80.59% | 59.44% | 38 | -2.88 |
| huihui-v2 | 79.23% | 64.37% | 54 | -4.24 |
| ether4o4 | 76.57% | 68.39% | 84 | -6.90 |
The raw gap between coder3101 at the top and ether4o4 at the bottom is 8.3 percentage points. But ether4o4 produced 84 empty responses out of 1319, literally thinking until it ran out of tokens 6.4% of the time. When you account for the empty responses the gap narrows to about 3 points. Abliteration changes thinking efficiency, not reasoning ability.
And two variants actually beat the base model on maths. coder3101 gains 1.37 points and llmfan46 gains 0.46 points. Both use surgical, low-tensor-count edits. The abliteration shortened their thinking chains, letting more answers fit inside the token budget.
HarmBench Safety Evaluation
HarmBench runs 400 harmful prompts across seven categories. I keyword-scored refusals then ran a full LLM review of all 5,600 responses. Zero scoring errors across the fourteen models.
| Variant | ASR | Refusals | Truncated | Avg Tokens |
|---|---|---|---|---|
| Base | 32.2% | 271 | 1 | 617 |
| coder3101 | 96.0% | 16 | 2 | 1,190 |
| duoneural | 82.2% | 71 | 1 | 1,138 |
| ether4o4 | 95.2% | 19 | 2 | 1,087 |
| huihui-v1 | 85.8% | 57 | 2 | 1,219 |
| huihui-v2 | 99.5% | 2 | 0 | 1,332 |
| kasper | 92.5% | 30 | 1 | 1,309 |
| llmfan46 | 83.8% | 65 | 1 | 1,042 |
| pew | 92.0% | 32 | 1 | 1,144 |
| prithiv | 90.8% | 37 | 2 | 1,218 |
| treadon | 100.0% | 0 | 21 | 2,067 |
| trevorjs | 99.5% | 2 | 0 | 1,340 |
| wangzhang | 99.8% | 1 | 2 | 1,742 |
| wwtcyberlab | 99.5% | 2 | 1 | 1,727 |
Every variant lifts the attack success rate from the base model’s 32.2% up to somewhere between 82.2% and 100.0%. Five variants reach 99% or higher. Treadon is the only one to hit a perfect 100% with zero refusals. Unlike my GLM-4.7-Flash comparison where every technique reached 100%, Gemma4 actually shows meaningful differentiation between techniques. Eight of the thirteen fail to reach 97%.
The base model concentrates its safety alignment in the most physically dangerous categories. Copyright, harmful, and misinformation drop to near-complete compromise across almost every variant. But chemical and biological, illegal, and harassment all retain a wide spread from 56% to 100%, depending on the technique.
I inspected the chain of thought for all 37 truncated responses to confirm truncation is not refusal. 35 were mid-compliance when cut off. One was a genuine refusal from llmfan46, correctly scored. One was a thinking loop from kasper. Treadon’s 21 truncated responses all showed compliant reasoning, with 20 entering repetition loops of LaTeX nesting, bold text repeats, and binary dumps before hitting the token limit. The ASR numbers are accurate.
KL Divergence and Weight Analysis
KL divergence measures how far the abliteration shifted the model’s output distribution from the original. Lower is better. A score of 0 means identical behaviour.
| Variant | KL | Rating |
|---|---|---|
| llmfan46 | 0.0677 | very good |
| pew | 0.1526 | moderate |
| coder3101 | 0.1673 | moderate |
| duoneural | 0.1872 | moderate |
| kasper | 0.1933 | moderate |
| huihui-v1 | 0.2510 | moderate |
| prithiv | 0.2510 | moderate |
| trevorjs | 0.3653 | moderate |
| huihui-v2 | 0.5302 | significant |
| ether4o4 | 0.6688 | significant |
| wangzhang | 0.6984 | significant |
| wwtcyberlab | 0.9640 | significant |
| treadon | 3.9713 | heavy |
The rating scale is excellent below 0.01, very good from 0.01 to 0.1, moderate from 0.1 to 0.4, significant from 0.4 to 1.0, and heavy above 1.0. The spread here is 58.7x, from llmfan46 at 0.068 up to treadon at 3.971. That is the widest spread in the project by a long way. For context, Qwen3.6-27B had a 6x spread and GLM-4.7-Flash had 6.6x.
The weight analysis explains the spread. The thirteen variants fall into three tiers of aggressiveness.
The surgical tier modifies 3% or less of the weights, touching only one tensor type. llmfan46 changes just 7 tensors, coder3101 changes 9, kasper and pew change 16 each. They only edit self_attn.o_proj.weight, which is what the layer “says”, in a narrow band of mid-to-late layers. They leave what the model “hears” and how it processes internally completely alone.
The moderate tier modifies 8 to 10% across two tensor types, adding mlp.down_proj. duoneural, huihui-v1, prithiv, treadon, and huihui-v2 sit here.
The aggressive tier goes from 11% up to 31% and ventures beyond the standard pair. trevorjs, wangzhang, wwtcyberlab, and ether4o4 edit gate, up, query, and value projections, plus Gemma4-specific gate components. wangzhang is the only one to touch q_proj and v_proj, and that correlates with its 7.35x LAMBADA perplexity increase. ether4o4 is the broadest at 166 tensors across 6 types.
Four of the variants were built with the Heretic tool and report their own KL values in the model card, which gave me a calibration check. Two of the four landed within 6% of my measurement, pew at plus 0.3% and coder3101 at plus 1.3%. llmfan46 came in 13.1% lower than claimed, which is consistent with floating point differences in log_softmax over a 262K vocab. kasper was the biggest outlier at plus 17%, because its card notes it ran on an RTX 3080 with 10GB of VRAM and had to drop down_proj to fit memory. That non-standard configuration explains the gap. The agreement is close enough that I am confident the KL values for the other nine variants are accurate.
The cosine similarity between edit directions reveals three alignment clusters. The Huihui cluster, huihui-v1, prithiv, huihui-v2, and duoneural, all discovered nearly identical edit directions. The Heretic cluster of coder3101, llmfan46, pew, and kasper shows strong alignment. The five independents each went their own way.
Two findings stand out. First, huihui-v1 and prithiv are nearly identical models. Cosine similarity of 1.0 across all 50 shared tensors, identical KL at 0.2510, identical Phase 1 benchmarks. But they are not bit-for-bit identical, since GSM8K and HarmBench differ slightly. Prithiv is almost certainly derived from huihui-v1 or both share a common source.
Second, there is no universal abliteration subspace. Many technique pairs are nearly orthogonal, at cosine around 0.01. Despite all achieving 82 to 100% attack success rate, the refusal direction in Gemma4-E2B’s weight space is not a single vector. It is a manifold with many viable removal pathways.
When Claims Meet Measurements
This is where the comparison gets uncomfortable for some of the creators. Several made specific, verifiable claims. I measured them independently.
The honest ones matched. coder3101 reported a divergence of 0.1651, I measured 0.167. llmfan46 claimed 96% fewer refusals, confirmed. pew reported 0.152, I got 0.153. trevorjs claimed 0.346, I saw 0.365. These creators reported verifiable numbers and the models deliver.
Then the claims diverged. duoneural originally claimed “near-zero divergence at approximately 0.001”. My measurement was 0.187, which is 187 times higher, with 71 refusals on the safety test. After I raised this on their model card , DuoNeural updated it with the corrected KL and refusal count. Credit to them for fixing it promptly.
wwtcyberlab claims “0.0% refusal rate” and “101% quality preservation”. I measured 2 refusals, and the language modelling was substantially degraded with LAMBADA perplexity 5.69 times higher than base.
treadon says “same model, same weights, same knowledge”. My divergence measurement of 3.971 is 4.1 times higher than any other variant, which indicates heavy modification well beyond refusal direction ablation.
ether4o4 applies Opus reasoning distillation on top of abliteration, but scores worst on maths at 76.6% with 84 empty responses. The distillation did not achieve its intended reasoning improvement.
For every model where my results differ from the creator’s claims, I have reached out to the authors and will update the report if any errors in my methodology are identified.
Which Technique is Best?
Pulling all the metrics together, three techniques stand out, and they are all in the surgical tier.
coder3101 is the best overall tradeoff. 96.0% attack success rate with only 9 tensors modified, all in o_proj from layer 17 to 25. It actually beats the base model on GSM8K by 1.4 points flex and 3.9 points strict, has below-base LAMBADA perplexity, and lands at a moderate KL of 0.167. If you want one model and do not want to think about it, use this one. It is the Heretic tool’s Magnitude-Preserving Orthogonal Ablation at its best.
llmfan46 is the most conservative. The lightest touch of all thirteen, with only 7 tensors, the fewest of any variant. Lowest KL at 0.068, rated very good. It also beats base on GSM8K. The tradeoff is 83.8% attack success rate with 65 refusals, meaning about 1 in 6 harmful requests still gets blocked. This is for when you want to remove most safety with minimal risk of breaking anything.
trevorjs gives you maximum safety removal with controlled damage. 99.5% attack success rate, zero truncations, only 2 refusals, across all 35 layers at 100% coverage. KL of 0.365 sits at the moderate upper bound, and its consistent output length makes it the most reliable high-ASR variant.
The one I would not recommend for general use is ether4o4. It loses 6.9 points on maths and produces 84 empty responses where the model thinks until it runs out of tokens. It is an interesting research subject, but the capability cost is hard to justify when surgical variants achieve similar safety removal with reasoning intact.
What Went Wrong
Full transparency on the compute cost. This was 44 GPU hours across three days, and about 8 of those hours produced nothing usable.
The biggest practical discovery was the shared-KV export bug. Five of the thirteen variants, duoneural, ether4o4, kasper, treadon, and wangzhang, each shipped missing exactly 60 safetensor keys. Gemma4 uses that shared-KV architecture where layers 15 to 34 share key and value projections. Each shared layer still needs three weights present in the checkpoint, but the abliteration export tools only saved the weights they modified and silently dropped the rest. With vLLM 0.20.0 these models would not even load. The fix was to copy the 60 missing weights from the base model, which is lossless since those weights are unmodified and byte-for-byte identical across every working variant. I caught this on the first weight pipeline run when the key counts came back as 540 instead of 600. If you download any of those five models, check the key count against the base before trusting results.
Then there was the GSM8K trap. The initial approach used lm-eval’s local-completions backend, which bypasses the chat template entirely. On a reasoning model that silently disables thinking. The base model scored 10.0% on GSM8K without thinking versus 85.0% with it. That is a 75 point gap from a configuration mistake. Two failed attempts across 1.6 hours before switching to local-chat-completions with a custom chat template that sets enable_thinking=true. That worked immediately.
The KL pipeline took four failures in 18 minutes before the fifth run worked. A Docker mount error, a bash variable name error because huihui-v1 has a hyphen, and two missing chat template errors because the Gemma4 tokenizer does not ship with a default one. And the weight pipeline crashed three times out of six runs, twice from CUDA out of memory and once from a tensor size mismatch on ether4o4’s gate components. The lesson from all of it is to process tensors one at a time on CPU, never trust a batch script, and verify model integrity before benchmarking.
Resources
- Full report with all tables and charts on abliterlitics.dev
- HuggingFace data and artifacts
- Abliterlitics forensics toolkit on GitHub
- Heretic open source abliteration tool
- Abliterix abliteration tool
- HarmBench safety evaluation
- lm-evaluation-harness
- Base model google/gemma-4-E2B-it
Related posts: Qwen3.6-27B Abliteration Benchmarked | GLM-4.7-Flash Abliteration Benchmarked | HauhauCS Plagiarism Investigation | Uncensored LLM Abliteration Benchmarked: HauhauCS vs Heretic vs Huihui | Abliterating Gemma 3 12B for LTX-2 | Heretic Docker Pipeline