The problem with many hospitals and many formats
IHX is a healthcare technology platform working between hospitals and insurers. A good deal of what it needs from hospitals still arrives as email.
Every hospital sends that email differently. Systems, formats and communication channels vary, so data that means the same thing can look completely different from one sender to the next. Handling it by hand is slow, and at volume it is not realistic.
The existing APIs also needed attention. They were tightly coupled, which made new requirements slower to add. The goal was one dependable, scalable way to bring hospital data in, along with APIs that would be easier to change.
What we knew before we built
No before-and-after baseline is published for this project, so this page does not offer an improvement figure. What can be stated is the context the work sat in.
- 30+
- Insurers IHX states it connects to, as shown on its site in November 2026
- 1
- Schema that every hospital's emails had to map into
- 2
- AWS services in the asynchronous path: SQS and Lambda
That set three requirements for the work. Emails in many formats had to come out as one consistent set of data. Sudden surges in volume could not overwhelm processing. And a failed extraction had to be retried safely, without creating duplicates.
The approach we chose
We separated ingestion from processing, and gave each hospital format its own extractor rather than building one parser for everything:
- Modular extractors. Python parsers that adapt to each data format and normalize the output into one schema.
- Asynchronous processing. AWS SQS queues incoming work, and AWS Lambda runs extraction and transformation.
- Refactored APIs. Legacy FastAPI endpoints analyzed, optimized and reorganized along modular design principles.
- Observability. Custom metrics and structured logs for extraction, latency and API health.
A hospital email moves through four stages:
- Queue. The incoming email is placed on an SQS queue, separating ingestion from processing.
- Extract. A Lambda function runs the extractor that matches the sender's format.
- Normalize. The extracted data is mapped to the unified schema, with idempotency checks to prevent duplicates.
- Serve. The data is available to the platform through the modernized FastAPI layer.
What we built
Our contribution was the extraction pipeline, the asynchronous processing and the API modernization.
Email extraction from multiple hospital systems
A Python-based pipeline handles emails from varied hospital communication channels and transforms them into a unified data model.
Refactored legacy APIs
Older FastAPI endpoints were analyzed, optimized and refactored to match changing product requirements, making them easier to read, scale and maintain.
Asynchronous data processing
AWS SQS and Lambda handle messages asynchronously, which spreads the workload and helps performance hold up as data volumes vary.
Error handling and reliability
Retry mechanisms, idempotency checks and structured logging support reliability, fault tolerance and transparency during processing.
Monitoring and observability
Custom metrics and logging track extraction success rates, processing latency and API health, which makes troubleshooting faster.
Every hospital can send data its own way, as long as it all comes out the same on the other side.
The guardrails around medical data
These emails carry medical communication data, and the platform depends on it arriving correctly. Care in both directions was part of the design from the start.
- No duplicates on retry. Idempotency checks mean that retrying a failed step does not create a second record.
- Retries for transient failures. Failed processing is retried, rather than lost.
- Structured logs. Processing is logged in a consistent structure, so that problems can be traced.
- Health metrics. Extraction success rates, latency and API health are measured, so that changes in behavior are visible.
- A clear view of compliance. How health data is protected, stored and retained depends on IHX, its contracts and the applicable law. We do not describe the pipeline as compliant in itself.
What was reported
The project delivered email extraction across multiple hospital systems, asynchronous processing and refactored APIs. The effects below are what the project team reported; none of them is a measured result:
- Ingestion. Automated extraction was reported to reduce manual intervention. No figure is published here.
- Scalability. The SQS and Lambda architecture was reported to handle variable workloads efficiently.
- Reliability. Error handling and monitoring were reported to reduce extraction failures and downtime. No figure is published here.
- Delivery. Refactored APIs were reported to make new requirements quicker to integrate.
What stays hard
Four challenges shaped this work, and any team building healthcare data ingestion should plan for them. Each came with a specific solution.
- Inconsistent formats and protocols. Hospitals send data in many different ways. We designed modular extractors and parsers that adapt to each format and normalize the output into a unified schema.
- Data surges. Volume does not arrive evenly. We used asynchronous workflows on AWS SQS and Lambda to separate ingestion from processing.
- Tightly coupled legacy APIs. Older code was hard to change safely. We applied modular design principles and clean refactoring to improve maintainability and reduce technical debt.
- Reliable data flow. Extraction can fail for many reasons. We added retries, idempotency logic and structured error logging.