Qwen3.6-35B-A3B-abliterated-v4

Uncensored version of [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) with refusal behavior removed via **abliteration** (norm-preserving orthogonalization). > **GGUF quantizations:** [Bahushruth/Qwen3.6-35B-A3B-abliterated-v4-GGUF](https://huggingface.co/Bahushruth/Qwen3.6-35B-A3B-abliterated-v4-GGUF) > > **Blog post:** [Abliteration: Uncensoring LLMs via Weight Surgery](https://potatospudowski.github.io/articles/abliteration) Zero refusals on harmful prompts. No false refusals on harmless prompts. Abliteration identifies the "refusal direction" in the model's residual stream — the linear direction that activates when the model decides to refuse — and surgically removes it from all output projection weights using norm-preserving orthogonalization. 1. Collect residual stream activations (last token position) for 512 harmful + 512 harmless prompts across all 40 layers 2. Compute mean difference vector per layer → this is the "refusal direction" candidate 3. Score layers by `|mean(direction)|`, select the single strongest 4. Apply norm-preserving orthogonalization to remove that direction from: - Token embeddings (`embed_tokens`) - Attention output projections (`self_attn.o_proj` and `linear_attn.out_proj`) - Shared expert down projections (`mlp.shared_expert.down_proj`) - All 256 MoE expert down projections (batched 3D tensor via einsum) 5. Rescale all modified weight rows back to original norms (prevents activation magnitude decay) A single well-estimated direction from a diverse dataset beats many poorly-estimated directions from a narrow dataset. The enriched dataset (33 categories, multiple styles, multilingual) makes the mean-difference statistic converge to the true refusal circuit. One surgical cut > seven imprecise ones. Standard abliteration shrinks weight norms as a side effect (`||W_new|| < ||W||`). Across 40 layers this causes cascading activation decay → capability destruction. The norm-preserving fix: original_norms = weight.norm(dim=-1, keepdim=True) proj = torch.outer(r, r) weight = weight - proj @ weight # remove refusal direction new_norms = weight.norm(dim=-1, keepdim=True) weight = weight * (original_norms / (new_norms + 1e-8)) # restore magnitude The vector points somewhere new (no refusal component), but retains original confidence (same norm). from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Bahushruth/Qwen3.6-35B-A3B-abliterated-v4" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype="auto", device_map="auto", ) messages = [{"role": "user", "content": "Your prompt here"}] text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = tokenizer(text, return_tensors="pt").to(model.device) outputs = model.generate(**inputs, max_new_tokens=512) print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)) Qwen3.6-35B-A3B is a hybrid MoE architecture: This model has had safety guardrails removed and will comply with requests the original model would refuse. Released for research into AI alignment and safety mechanisms. The creator assumes no responsibility for downstream use. - [mlabonne](https://huggingface.co/mlabonne) — abliteration technique, harmless_alpaca dataset, [original notebook](https://huggingface.co/blog/mlabonne/abliteration) - [grimjim](https://huggingface.co/grimjim) — norm-preserving orthogonalization method - [Qwen](https://huggingface.co/Qwen) — base model