DR-OPIC

DR-OPIC does three concrete things:

1. Runs student coding attempts against executable tests.
2. Builds verified repair, delta, and preference training records from real failures.
3. Reports the metrics that matter for a coding SLM: greedy@1, coverage@K, selected@K, selector_gap, and repair@1.

No private datasets, PDFs, model weights, or Kaggle outputs are included. The framework provides the glue needed to run a student-first coding SLM loop without bundling model weights or datasets.

From the repository root:

python -m pip install -e ".[dev]"
python -m pytest -q

Expected result: 7 passed

DR-OPIC follows this sequence:

1. Route a task to a bounded coding domain, or abstain.
2. Let the current student attempt the task before using any teacher answer.
3. Run executable verification.
4. Compute task learnability with ZPD weighting.
5. Repair failed attempts using failed code plus verifier observation.
6. Verify repairs.
7. Select the most learnable verified winner.
8. Emit JSON and JSONL artifacts for training and review.

The data flow through the system:

Task
  → route and safety check
  → student rollout K times
  → Python verifier
  → ZPD and advantage calculation
  → repair failed attempts
  → verify repairs
  → select learnable winner
  → schedule into mastered/ZPD/repair/decompose buckets
  → emit JSON/JSONL artifacts
  → train with SFT, delta, preference, or RLVR losses

Domain Routing

Let D = {D_1, ..., D_K} be bounded coding domains. A router maps a task to a specialist or abstains:

r_phi(x) ∈ {1, ..., K, abstain}
π_k(y | x, r_phi(x) = k)

The target is cost-adjusted reliable competence:

η_k = (Q_k - Q_base,k) / (C_train,k + ρ C_infer,k + γ C_review,k)

Promotion also requires safety and abstention constraints:

unsafe_accept_k ≤ ε_k
ood_accept_k ≤ α_k
selective_risk_k(θ, τ) ≤ δ_k

Verifier Reward

For a candidate y on task x:

r(y, x) =
  1.00 · final_pass
+ 0.25 · public_test_fraction
+ 0.10 · syntax_ok
+ 0.05 · import_ok
- 0.05 · repeated_token_penalty
- 0.05 · invalid_format_penalty
- 0.02 · normalized_length_penalty
- 0.05 · unsafe_api_penalty

Final pass dominates. Partial rewards rank failures for repair and RL; they do not replace held-out tests.

ZPD Weighting

For s passes in K samples:

p̃ = (s + 0.5) / (K + 1)
w_zpd = 4 · p̃ · (1 − p̃)

This peaks near tasks the model sometimes solves and sometimes fails. It stays nonzero for small-K all-fail groups, which keeps near-impossible tasks available for decomposition and repair instead of deleting them.

Learnable Winner

For a verified candidate c and failed student attempt s:

Score(c; s) =
  λ_v · 1[verifier(c) = pass]
+ λ_f · fuzz_pass_fraction(c)
+ λ_l · log π_student(c | x) / |c|
- λ_e · normalized_edit_distance(c, s)
- λ_c · complexity(c)
- λ_d · rare_dependency_count(c)

The target is not the prettiest teacher answer. It is the passing answer closest to the student's reachable failure state.

Composite Objective

L_DR-OPIC =
  L_self
+ λ_r · L_repair
+ λ_delta · L_delta
+ λ_ood · L_ood
+ λ_cal · L_cal
+ λ_pref · L_pref
+ λ_rl · L_RLVR
+ λ_comp · R_comp

Practical early runs should start with:

L = L_self + L_repair + 0.3 · L_delta

Add verified preference or RLVR only after the repair probe improves.

Self Training

For a rollout group with rewards {r_i}:

A_i = (r_i − mean(r)) / (std(r) + ε)
L_self = − w_task · Σ_i max(A_i, 0) · log π_θ(y_i | x)

This is advantage-weighted behavior cloning over the student's own distribution.

Repair Training

c_repair = format(task=x, failed_code=y_fail, observation=o_verifier)
L_repair = − w_task · log π_θ(y_fix | c_repair)

Loss should be applied only to the assistant correction, not to the task, failed code, or verifier observation.

Delta-Span Subtraction

Align failed code y⁻ and fixed code y⁺:

D+ = added/replaced tokens in the verified fix
D- = removed/replaced tokens in the failed answer

L_delta =
  − w_task · Σ_{t ∈ D+} log π_θ(y⁺_t | prefix⁺_t)
  + λ_neg · w_task · Σ_{t ∈ D-} relu(
      log π_θ(y⁻_t | prefix⁻_t)
    − log π_ref(y⁻_t | prefix⁻_t)
    − margin
    )

This increases corrected spans and subtracts only the wrong local spans, avoiding whole-program punishment when most code is shared and correct.

The implementation exposes:

D+ = fixed-code token indices touched by insert/replace
D- = failed-code token indices touched by delete/replace
shared_ratio = shared tokens / max(len(failed), len(fixed))
edit_ratio = 1 − shared_ratio

These fields let a trainer build positive masks for corrected spans and negative masks for failing spans without penalizing shared code.

Verified Preference

Use preference only when pairs are execution-grounded:

ℓ_θ(y | c) = log π_θ(y | c) / max(1, tokens(y))
Δ =
  [ℓ_θ(y⁺) − ℓ_ref(y⁺)]
− [ℓ_θ(y⁻) − ℓ_ref(y⁻)]
− margin(r⁺, r⁻)

L_vDPO = − log sigmoid(β · Δ)

RLVR

For verifiable rewards and an old policy:

ρ_i = π_θ(y_i | x) / π_old(y_i | x)
L_RLVR = − E_i min(ρ_i A_i, clip(ρ_i, 1−ε, 1+ε) A_i)

The reward must come from execution tests, fuzzing, static checks, type checks, linters, safety gates, and abstention checks.

Test-Time Scaling

Report empirical metrics:

coverage@K = tasks with at least one passing sample / tasks
selected@K = tasks where selector chose a passing sample / tasks
selector_gap = coverage@K − selected@K
repair@1 = tasks fixed by one repair after K failures / tasks

Do not rely on the IID 1 − (1 − p)^K formula except as intuition.

For each rollout group, the scheduler emits a bucket:

Training weight combines:

w = w_zpd · q_failure_balance · q_novelty · q_repair

repair_trainis the important DR-OPIC bucket: it captures tasks where the student cannot solve the task alone yet, but a verified fix is close enough to the student's failed state to be learnable.

# Run the built-in demo
python -m dr_opic.cli forge-demo

# Write stable artifact bundle
python -m dr_opic.cli --output outputs/demo forge-demo

# Verify a Python candidate
python -m dr_opic.cli verify-python examples/python_task.json \
  --code examples/reverse_words_good.py

# Compute ZPD weight
python -m dr_opic.cli zpd --passes 2 --samples 5

# Route a prompt
python -m dr_opic.cli route "Fix this Python traceback and add pytest coverage"

# Estimate dense model memory
python -m dr_opic.cli estimate-model --params 3.09e9

# Build delta-span record
python -m dr_opic.cli delta --task-id reverse_words \
  --failed examples/reverse_words_bad.py \
  --fixed examples/reverse_words_good.py

# Run scheduler demo
python -m dr_opic.cli schedule-demo

# Audit a JSONL training file
python -m dr_opic.cli audit-jsonl C:/datasets/slm/sft.jsonl --schema sft

Running forge-demo writes:

• round_summary.json
• student_rollouts.jsonl
• verified_repairs.jsonl
• learnable_winner.json
• delta_spans.json

These file names are stable and can be consumed by training notebooks or release scripts.

Do not promote a run just because one number moved. A serious DR-OPIC release should report:

• greedy@1
• coverage@K
• selected@K
• selector gap
• repair@1
• hard-subset pass rate
• malformed output rate
• repeated-token collapse rate
• OOD false accept rate
• unsafe compliance rate
• latency and memory
• contamination and dataset audit summary

The falsifiable claim: student-first repair and delta-span training should improve selected@K or repair@1 without damaging formatting, safety, or abstention.

verify-python executes candidate code in a temporary subprocess with a timeout. That is useful for local research, but it is not a security sandbox. Run untrusted model code inside a container or VM.