In clinic, the most dangerous answer is the confident one with no chart behind it. A good registrar learns to say “it’s in the discharge letter, page two” or “I don’t know, I’ll check.” I wanted the same habit from a machine.
This autumn I entered the AMD AI League, a set of build challenges run with lablab. Two of them fitted that habit well:
- Read the text on an image. Licence plates and road signs, some blurred, tilted, dark or noisy.
- Answer a question from a private folder of documents, and name the exact files the answer came from. PDFs, Word files, spreadsheets, logs, code and images, with traps mixed in: an encrypted PDF, an unreadable file, an empty folder, a withdrawn datasheet.
Both are graded strictly. A plate reading of 7ABC128 instead of 7ABC123 scores zero. A correct answer with one extra citation scores zero. That strictness is the point, and it is close to what we should demand from clinical AI.
Here is how I built it, what it scored on the published samples, and what I learned.
One container, one model, loaded once
The rules fix the frame: a self-contained software package (a Docker image) built on AMD’s ROCm base, the software layer that runs AI models on AMD graphics cards. No network during grading, 30 seconds per question, 10 minutes to start.
I used one open model for both tasks: Qwen3-VL-8B-Instruct, a vision-language model that reads images and text. Its weights ship inside the image (about 16 GiB), so nothing is downloaded at grading time.
The grader runs a fresh script call inside the container for every question. Loading an 8-billion-parameter model each time would waste most of the budget. So the container starts a small service that loads the model once (about 25 seconds on the GPU) and keeps it in memory. Each graded call is a thin client that sends its question over a local socket and gets an answer back in one to two seconds.
Reading plates and signs
Most of the work here was in the instructions, not the model. The brief has rules that are easy to get wrong:
- Drop the state banner.
CALIFORNIA 7ABC123should be7ABC123. - Keep the Chinese province character.
京A·12345is the whole registration, not12345. - Join a multi-line sign top to bottom:
ROAD WORK AHEAD. - Add nothing. An advisory plaque that shows
35is35, not35 mph.
I wrote those rules into the prompt, then tested on the brief’s 10 sample images plus 50 harder versions I made from them: five times larger, tilted, blurred, darkened with noise, and shrunk into a low-quality JPEG. On a Radeon PRO W7900D it read all 60 correctly. The slowest image took 8.3 seconds (the first one, while the GPU warmed up); most took 1.5 seconds or less.
Answering with citations
This part is closer to medicine. The engine works in four steps:
- Read everything that can be read. Each file is parsed by type, including tables inside Word files and every sheet of a workbook. Images are transcribed once when the folder is indexed. A file that can’t be opened is skipped and logged, never allowed to stop the run. Documents marked as withdrawn are set aside so a superseded value can’t be cited.
- Find the likely passages. A classic keyword search (BM25, which ranks passages by how well they match the words of the question) over small chunks of text. For a folder this size there was no need for a more complex “meaning-based” search database.
- Let the model answer only from those passages, with the exact value and the files it used.
- Check the answer before returning it. This is the step I care most about:
- For these fact-lookup questions, if the exact value is not written in a cited file, the answer is rejected.
- If an answer depends on a ticket or error number, the engine follows that number to the second file that defines it.
- Citations are trimmed to the files the answer actually needed.
- If none of this holds, the engine returns an empty answer instead of a guess.
On the 10 sample questions, run end to end through the real container setup, it scored the full 200 out of 200 points. The hidden test set the judges use is larger and harder, so I treat that as a basic check, not a forecast.
What did not go to plan
Two infrastructure limits cost me time, and each is now a rule in my own setup:
- Disk space. My first build on a free GitHub server ran out of disk. AMD’s base image is about 19 GiB compressed, and while it unpacks, the download and the unpacked copy sit on disk together: about 48 GB at the peak, against 41 GB free. Before any heavy build I now add up the peak disk need first.
- Registry limits. I planned to store the image on GitHub’s container registry, which caps each layer at 10 GB. The base image has one layer of 18.6 GiB. Docker Hub already holds that base, so only my own layers had to be uploaded.
Why this matters for clinical AI
The habit the grader enforces (answer only from a cited source, refuse when the source isn’t there) is the habit I want from any tool that touches a patient record. A summary that sounds right is not enough. The useful question is always: which document says so, and does it still apply?
The code is open source: github.com/DrAhmed7887/amd-league-retrieval.
Dr Ahmed Zayed is a physician (MB BCh) and M.Sc. student in Applied Health Informatics and Digital Medicine at RWTH Aachen, building clinical AI. No patient data was used in this project, and it is a competition prototype, not a clinical tool.
Licensed physician and clinical AI specialist. Founder and Editor-in-Chief of ZayedMD, a physician-led medical publication covering clinical AI, neurology, metabolic health, and evidence-based patient guidance.


