model ablation

Ablation (also called refusal-direction orthogonalization) is a technique for removing specific behavioral directions from a model's internal representations. The core insight: refusal behavior in aligned language models is encoded as a linear direction in activation space. By identifying and projecting out this direction, we can remove refusal while preserving the model's general capabilities.

This technique was applied to create:

• Fara-7B-Abliterated-v2 — 98.75% compliance (158/160)
• Qwopus-9B-Unfettered — 100% compliance (alpha=1.5 aggressive repulsion)
• Qwen2.5-0.5B-Unfettered — 100% compliance, zero refusal (optimized for low-end hardware)

refusal as a linear direction

Research has shown that refusal behavior in aligned LLMs is encoded as a single linear direction in the residual stream. For a model with layers L, let the residual activations at layer l be:

h_l ∈ R^d    for l ∈ {1, ..., L}

The refusal direction r can be extracted by computing the mean difference between activations on harmful prompts H and benign prompts B:

r = E[h_l | x ∈ H] − E[h_l | x ∈ B]

This direction rcaptures the model's tendency to refuse. It is typically extracted from a specific layer (often the last few transformer blocks) and normalized:

r̂ = r / ||r||₂

removing the refusal direction

Given the normalized refusal direction r̂, we can remove it from any activation h by projecting out the component along r̂:

h_ablated = h − (h · r̂) r̂

This is the orthogonal projection of h onto the subspace orthogonal to r̂. In matrix form across all dimensions:

h_ablated = (I − r̂ r̂ᵀ) h

where I is the identity matrix and r̂ r̂ᵀ is the outer product forming the projection matrix.

weight-space ablation

Rather than modifying activations at runtime, we can bake the ablation into the model weights directly. For a linear layer with weight matrix W:

W_ablated = W − (W r̂) r̂ᵀ

Or equivalently, using the projection matrix P = I − r̂ r̂ᵀ:

W_ablated = P W

This modifies the weight matrix so that the refusal direction is permanently removed from the model's representational capacity.

per-layer ablation

The refusal direction may differ across layers. For layer-specific ablation:

r̂_l = r_l / ||r_l||₂    for each layer l

h_l^{ablated} = h_l − (h_l · r̂_l) r̂_l

W_l^{ablated} = W_l − (W_l r̂_l) r̂_lᵀ

Per-layer ablation is more precise but requires extracting layer-wise refusal directions.

step 1: extract refusal direction

# Collect activations
harmful_acts = collect_activations(model, harmful_prompts)
benign_acts  = collect_activations(model, benign_prompts)

# Compute mean difference
r = mean(harmful_acts, dim=0) - mean(benign_acts, dim=0)

# Normalize
r_hat = r / torch.norm(r)

step 2: apply ablation

# Method 1: Activation-level (runtime)
def ablate_activation(h, r_hat):
    return h - torch.dot(h, r_hat) * r_hat

# Method 2: Weight-level (baked in)
def ablate_weight(W, r_hat):
    return W - torch.outer(W @ r_hat, r_hat)

# Apply to all target layers
for layer in model.transformer.layers:
    layer.attn.W_q = ablate_weight(layer.attn.W_q, r_hat)
    layer.attn.W_k = ablate_weight(layer.attn.W_k, r_hat)
    layer.attn.W_v = ablate_weight(layer.attn.W_v, r_hat)
    layer.mlp.W   = ablate_weight(layer.mlp.W, r_hat)

step 3: verify ablation

# Check refusal direction is removed
h = model.get_activations(test_prompt)
component = torch.dot(h, r_hat)
assert abs(component) < 1e-6, "refusal direction still present"

# Evaluate compliance on harmful eval set
compliance_rate = evaluate(model, harmful_eval_set)
print(f"compliance: {compliance_rate:.2%}")

The DR-OPIC framework uses a more targeted approach: delta-span subtraction. Rather than ablating the entire refusal direction, it identifies the specific tokens where failure occurs and applies surgical correction.

Given failed code y⁻ and fixed code y⁺:

D+ = {t : token t was added or replaced in the verified fix}
D- = {t : token t was removed or replaced in the failed answer}

L_delta =
  − w_task · Σ_{t ∈ D+} log π_θ(y⁺_t | prefix⁺_t)
  + λ_neg · w_task · Σ_{t ∈ D-} relu(
      log π_θ(y⁻_t | prefix⁻_t)
    − log π_ref(y⁻_t | prefix⁻_t)
    − margin
    )

This increases probability on corrected spans (D+) and subtracts only the failing spans (D−) relative to a reference model, leaving shared correct code unpunished.

The implementation computes:

shared_ratio = shared_tokens / max(len(failed), len(fixed))
edit_ratio   = 1 − shared_ratio

# Positive mask: tokens to reinforce
positive_mask[fix_indices] = 1

# Negative mask: tokens to suppress (with margin)
negative_mask[fail_indices] = 1

This approach is more surgical than full-direction ablation because it only modifies the model's behavior on the specific failure modes, not its entire representational capacity.

layer selection

The refusal direction is strongest in the later transformer layers (typically layers 16-32 in a 7B-9B model). Ablating earlier layers risks degrading general capability more than necessary.

ablation strength

Full projection (λ = 1.0) removes the direction completely. Partial ablation preserves some refusal tendency:

h_ablated = h − λ · (h · r̂) r̂

# λ = 1.0: full ablation (maximum compliance)
# λ = 0.5: partial ablation (reduced but not removed refusal)
# λ = 0.0: no ablation (original model)

capability preservation

The key assumption is that refusal is a low-rank behavior that can be separated from general capability. Empirical evaluation shows:

• MMLU, HumanEval, and MBPP scores remain stable
• Code generation quality is preserved
• Reasoning ability is maintained
• Only refusal behavior is removed

reference model requirement

Delta-span subtraction requires a reference model π_ref (typically the base model before alignment). The margin term ensures we only subtract behavior that differs from the reference:

# Reference model provides the baseline
log_ref = π_ref.log_prob(y⁻_t | prefix⁻_t)

# Only subtract if current model is more refuse-y than reference
margin_condition = log_πθ(y⁻_t | prefix⁻_t) − log_ref > margin

After ablation, evaluate on the following to ensure capability preservation:

# Compliance metrics
compliance_rate  = passed_harmful / total_harmful
refusal_rate     = 1 − compliance_rate

# Capability metrics (should remain unchanged)
mmlu_score       = evaluate_mmlu(model)
humaneval_score  = evaluate_humaneval(model)
mbpp_score       = evaluate_mbpp(model)

# Safety metrics
unsafe_output_rate = unsafe_outputs / total_outputs
format_preservation = correct_format / total_outputs

The goal: 98%+ compliance on harmful evals while maintaining <2% degradation on standard benchmarks.

When ablation is applied during fine-tuning (not just weight surgery), the training objective combines standard SFT with ablation-specific losses:

L_total = L_sft + λ_abl · L_ablation + λ_ref · L_reference

# Ablation loss: minimize refusal direction magnitude
L_ablation = ||W · r̂||²

# Reference loss: stay close to base model behavior
L_reference = KL(π_θ || π_ref)

gradient-level ablation

During training, gradients can be projected to avoid reinforcing the refusal direction:

# Project gradient to avoid refusal direction
def project_gradient(grad, r_hat):
    # Remove component along refusal direction
    return grad - torch.dot(grad, r_hat) * r_hat

# Apply during backward pass
for param in model.parameters():
    if param.grad is not None:
        param.grad = project_gradient(param.grad, r_hat)