von-read / AMD AI Academy

A fluent answer is not a finished reading.

What a small OCR study taught us about target selection, misleading perfect scores and the token limit that cut off a correct answer.

By Joseph Ayanda. AI-assisted development and evaluation on AMD Instinct MI300X VF, using ROCm and native Qwen vision-language models.

Want to build in the AMD AI Academy Challenge? Join through Joseph's official LabLab referral link. It is a referral link: if a new participant joins and completes LabLab's referral criteria, Joseph may receive event XP.
Research prototype, not a leaderboard result. These are small public-image experiments with assistant-reviewed references. Some images were used during development; degraded variants reuse the same source photographs. Final container validation and an official submission are still pending.

Matched baseline

31 / 33

Qwen3-VL-4B on the known-photo slice. Not an independent benchmark.

Completion fix

131 tokens

Required for one long sign. The old 96-token ceiling cut it off.

Measured memory

9,094 MB

Largest of 72 driver samples, not a continuous exact peak.

1. Define success before choosing the model

The task is not to write a plausible caption. It is to return the intended plate identifier or road-sign text. Reading a registration sticker instead of the identifier is a failure, even when every sticker character is correct. So is adding a nearby sign, dropping a short word or returning a convincing prefix of an unfinished answer.

Reference answers were stored separately from inference inputs. The input manifest contained only image identifiers, paths and hashes. Where a character or the intended target could not be resolved confidently, the example remained an unscored diagnostic instead of receiving a convenient reference after prediction.

A specialist pipeline reached 24/24 on its original development photographs. That number did not survive ordinary input changes: it scored 22/24 after a one-megapixel cap, and 19/24 under both JPEG-quality-35 compression and half-width/half-height resizing. The useful result was the failure pattern, not the original perfect score.

2. Measure the replacement, not its reputation

The same inputs and task prompt were passed to native Qwen3-VL-4B and Qwen3.5-4B on the allocated AMD GPU. The newer model frequently included state banners, secondary signs and background text.

Exact normalized matches from the recorded matched comparison
Evaluation sliceQwen3-VL-4BQwen3.5-4B
33 known photographs31 / 3323 / 33
Seven additional scored photographs6 / 77 / 7
JPEG quality 35 variants23 / 2419 / 24
Half-resolution variants22 / 2419 / 24

A separate localization-and-crop experiment processed 91 inputs. It rescued a missing plate prefix, but turned a correct zero into O elsewhere and dropped a short sign word. One rescue against two regressions was not an improvement. The earlier three-disagreement contrastive reranking experiment also corrected nothing. Neither became the default.

Practical rule: retain the whole-image baseline, count rescues and regressions separately, and promote an extra inference stage only after it earns its cost.

3. Fix delivery before buying more capacity

A new long warning sign revealed a different failure: the 4B model was reading correctly, but the 96-token cap stopped generation mid-sentence. The 8B model under the same cap was truncated too. Increasing the 4B allowance to 512 produced the complete 131-token answer.

ConfigurationFive scored referencesIncomplete / all 11 inputs
4B, 96-token cap4 / 52 / 11
8B, 96-token cap4 / 52 / 11
4B, 512-token cap + EOS check5 / 50 / 11

The six ambiguous examples stayed unscored. A rerun of all 91 earlier inputs produced exactly the same normalized answers as before. This fixes a delivery failure; it does not establish broad recognition accuracy or eliminate the remaining character ambiguities.

Try the completion rule

This is a local logic demonstrator based on the recorded token lengths. It does not invoke a model or fabricate a live GPU result.

Choose an allowance and check whether the reading can finish.
ids = model.generate(**batch, max_new_tokens=512,
                     do_sample=False, max_time=23.0)
suffix = ids[:, batch.input_ids.shape[1]:]
if int(suffix[0, -1]) not in eos_ids:
    raise TimeoutError("Incomplete transcription")
# Only a completed, on-time reading may be committed.

max_new_tokens is a ceiling, not a target length. Short answers still stop at EOS. The Transformers time limit is cooperative, so a final deployment also needs a separate hard-deadline supervisor and atomic output commits. See the official generation documentation.

4. Measure the deployed resource, not just the tensor allocator

The selected 4B/512 policy was repeated on the 11-image slice while amd-smi sampled driver memory approximately every half second. All 72 samples were valid, the largest observed used-VRAM value was 9,094 MB in the tool's reported units, and all outputs matched the previous run.

This is a sampled high-water mark. It is not an instantaneous peak, nor proof that the final container passes the evaluator's memory and invocation rules. The cloud experiment used Torch 2.13.0+rocm10.0.0, Transformers 5.3.0 and an AMD Instinct MI300X VF. The AMD SMI CLI reference describes the telemetry tool.

Three completed engineering milestones

01 · Native AMD execution

Revision-locked Qwen3-VL inference on real MI300X VF hardware, with exact-output, latency and input-hash records. No API response mocks were used for the neural results.

02 · Matched model and robustness evaluation

Seven comparison jobs covered 186 model-image evaluations, including overlapping compatibility inputs. Failed alternatives and paired degradation results were retained rather than hidden.

03 · Completion and memory validation

A token-budget ablation isolated truncation; an EOS gate prevents incomplete output from being labelled successful. The updated policy was regression-tested and rerun under driver-memory sampling.

Reproduce the guard, inspect the evidence

The downloadable adapter keeps the measured prompt and pinned 4B model revision. Its completion and snapshot-integrity tests run without a GPU. The adapter is an extracted deployment component, not the complete challenge container; its final warm-worker GPU integration has not been validated.

cd source
python -m pip install -r requirements-test.txt
python -m pytest -q tests
# CPU guard tests only. These do not measure OCR accuracy.

Next: validate the complete container and command lifecycle, then evaluate a frozen policy on a larger, predeclared source-separated set. Public-photo pretraining overlap is unknown. Repeated variants must not inflate the number of independent scenes. No technical leaderboard rank is claimed here.