AI Assistant for Lab Parameters - LLMs in Laboratory Diagnostics

We built an AI-powered parameter assistant for medcom labcomplete®. When doctors order lab tests, the system suggests suitable parameters, based on medical context, order history, and billing rules. In this post, I describe the technical decisions, the architecture, and what we learned along the way.

Starting point

When ordering lab tests, the doctor chooses from hundreds of possible parameters. Which of them make sense depends on several factors:

The patient's age and gender, the doctor's specialty
Insurance status (not every parameter can be billed to every insurer)
Order history over the last six months
Available sample material (for add-on orders, only certain parameters can be run on the existing sample)

In addition, there are billing rules, LOINC codes, and parameter incompatibilities. Given the sheer number of parameters and their often cryptic abbreviations, doctors often do not even know what the lab actually offers and which parameters would be relevant to their clinical question. Until now, this knowledge has lived in the heads of experienced lab staff.

This is exactly where the assistant comes in: the doctor enters a clinical question or symptoms – e.g. “Suspicion of iron deficiency and thyroid dysfunction” – and receives suitable parameter suggestions. For each suggestion, the system provides a rationale: which parameter, why it is relevant, and what it measures.

What the system does – and does not do

The assistant is integrated into the existing ordering interface and suggests parameters in real time. It knows all available lab parameters, including LOINC codes, synonyms, and billing rules, and takes the context of the current order into account.

Important: The system is not a medical device. It does not diagnose or make medical decisions. The final choice is up to the doctor. Questions outside the defined scope, e.g. “Does the patient have diabetes?”, are rejected. The model is strictly limited to parameter suggestions.

Model choice

The choice of language model was the decision with the biggest impact on the outcome.

First attempt: open-source model. Result:

Hallucinations – the model invented parameter names that did not exist, despite a hardened system prompt
Medical context was often ignored (specialty, insurance status)
Time to First Token (TTFT) sometimes over a minute
No token caching available

For a system where false suggestions can have consequences, that was not acceptable.

Second attempt: commercial model via an EU-hosted managed service. Result:

No hallucinations in parameter names
Reliable context understanding
Time to First Token in the low-second range
Reliable structured JSON output

The requirement was that the model run exclusively on EU infrastructure. Direct API calls to US endpoints were not an option.

Token prices are higher for the commercial model than for the open-source alternative – but it pays off: no hallucinations, no manual corrections, no rework. The cost per request is measured in cents. Token caching in particular helps a lot: in multi-turn chats, the system prompt and parameter context are cached, which noticeably reduces the cost of follow-up requests.

I deliberately do not name specific providers or models here – they change quickly, and the lessons from this project remain valid regardless. What matters is the evaluation process, not the label on the model.

Privacy

No personal data goes to the language model. No name, no social security number, no address. The model receives only anonymised context data: age, gender, insurance status. All LLM communication runs over EU infrastructure.

All patient data and critical data remain on our Exoscale infrastructure in Austria and are never sent to the AI provider. Before the LLM call, the backend extracts only the anonymised context data – everything else stays on Exoscale.

It was designed this way from the start, not retrofitted.

Architecture

The main components:

Frontend: React with Ant Design X
Backend: Spring Boot with Spring AI for LLM integration
Streaming: Server-Sent Events – the response is streamed in real time
Rate limiting: per user, implemented with Redis
Feedback: thumbs up/down with optional free text, written to MLflow
Observability: OpenTelemetry traces → MLflow + Grafana dashboard (token consumption, cost per referrer, latency, etc.)

Why MLflow instead of Langfuse?

We deliberately chose MLflow over Langfuse. The reason: infrastructure overhead. In self-hosted operation, Langfuse requires Redis, PostgreSQL, ClickHouse, and S3 – a considerable amount of work for operations and maintenance. For our use case, MLflow runs with a single PostgreSQL database. That fits our principle of introducing complexity only where it clearly pays off.

Context instead of RAG

We do not use retrieval-augmented generation. For each request, all relevant parameters including LOINC codes, synonyms, and AI descriptions are packed into the context. This works because the data is in a compact format. The context consists of three layers:

Global system prompt
Lab-specific extension of the system prompt
Parameter-specific context (description, LOINC, synonyms, short/long name)

At around 1,000 lab parameters, the context is about 50,000 tokens. That fits entirely in the context window – so RAG was not necessary. This greatly simplified the architecture.

Quality assurance with automated evaluations

We built an automated eval pipeline:

Defined test scenarios with expected results
Automated tests against the live endpoint
Scorers validate the responses for correctness and completeness
Model-as-a-judge: a second model assesses the quality
All results are versioned in MLflow and are comparable across model versions

The system prompt was refined over several iterations, with each change automatically tested against the eval suite.

Timeline

It took two to three weeks from prototype to production:

Week 1: prototype with LLM integration, initial eval scenarios
Week 2: streaming UI, feedback, rate limiting, observability
Week 3: beta rollout to selected testers, prompt fine-tuning with real feedback

This was followed by a gradual rollout: beta testers first, then more users.

What I would do differently

Start model evaluation earlier. The detour through the open-source model cost time. Next time, I would evaluate this earlier and more broadly.
Build the feedback loop earlier. Real user feedback is extremely valuable – the sooner it comes in, the better.
Observability from the first commit. We only added token consumption per user to monitoring quite late. In a system with variable context sizes (500 to 1,000+ parameters), the cost per request is not constant – that needs to be made visible from the start, otherwise it will come back to bite you during rollout.

Conclusion

Two to three weeks from prototype to production – that sounds fast, but it was only possible because of careful planning up front. The architecture decisions (no RAG, no fine-tuning, EU-only hosting, MLflow instead of Langfuse) were fixed before the first line of code was written. Without that clear picture, we would have kept rebuilding along the way.

What made it work: the right model choice based on systematic evaluation, privacy as an architectural principle from day one, automated evals as a safety net, and deliberately avoiding unnecessary complexity.

// next step

Got a similar project in mind?

In a free initial call we look at your situation and tell you what's realistic and what the next step looks like.

Request a consultation or book a call directly →

// about the author

Alexander J. Gassner, MSc

Founder and managing director of agsolutions, with more than 15 years in software development (MSc Software Engineering, FH Hagenberg). Builds and runs business-critical software from requirements to operations: Kotlin, Spring Boot and React in the code, Kubernetes, Pulumi and GitOps in operations, as an Exoscale Certified Solution Architect.