FHIR and DICOM for Multimodal AI: Connecting EHR, PACS, Imaging

Posted in AI Healthcare

Hospitals in the United States are spending heavily on imaging AI, and a good portion of that investment is stalling at the integration layer. The U.S. AI in medical imaging market sat at $524 million in 2024 and is expected to grow at 33% annually through 2030, but the infrastructure connecting imaging systems to clinical records has not kept pace. Most radiology AI tools run inside a PACS and never see the patient’s EHR. Most EHR integrated platforms have no live connection to imaging data. For multimodal AI, that disconnect makes the technology far less useful than it should be. This blog walks through what a working FHIR-DICOM integration actually looks like, what makes it hard, and what health systems and digital health vendors need to do differently.

What are FHIR and DICOM Standards

FHIR: Handles Clinical Data Exchange

• FHIR (Fast Healthcare Interoperability Resources) is HL7’s standard for moving patient data between systems over the web. It works on the same principles as any modern REST API. 
• Clinical data, such as diagnoses, medications, lab results, care plans, etc., is organized into discrete resource types and made available through queryable API endpoints.
• In the United States, FHIR adoption has been pushed forward by the ONC’s 21st Century Cures Act, which requires certified EHR systems to expose FHIR R4 APIs. 
• That means most health systems already have some level of FHIR infrastructure in place, even if it is not being used beyond the minimum compliance requirement.
• What FHIR was not designed for is large binary files. A 400-slice CT scan is not the kind of data that moves cleanly through a REST API built for structured text records. That is the gap DICOM fills.

DICOM: Works on Medical Images

• DICOM (Digital Imaging and Communications in Medicine) is the format that every medical image in a hospital is stored in. 
• When a radiologist reads a chest CT or a cardiologist reviews an echo, they are looking at DICOM files retrieved from a PACS software. 
• What makes DICOM integration useful is its metadata. Every DICOM file carries structured tags: patient identifiers, the date of the study, the body part scanned, the equipment used, and the acquisition parameters. 
• That metadata is clinically meaningful and directly relevant to any AI model working with the image.
• The issue is that all of this is stored inside a format completely separate from the patient’s EHR. A DICOM file knows the patient’s MRN but has no direct connection to their problem list, their lab history, or their current medications. Without deliberate integration work, that connection never gets made.

Why Disconnected Systems Are a Real Clinical Problem

The gap between imaging data and clinical data has direct effects on care quality and on the usefulness of any AI tool present in this environment.

An AI model trained to detect early-stage lung nodules on CT will produce better results when it knows whether the patient is a current smoker, has a family history of lung cancer, or is being monitored as part of a high-risk screening program. None of that information lives in the DICOM file. Without a connection to the EHR, the model works with incomplete inputs.

For Radiologists: 

Workflow friction is a significant roadblock. Reading a scan without the patient’s clinical context means toggling between the PACS viewer and the EHR, often in separate windows on separate systems, manually piecing together a picture that should be automatic.

For Health System IT & Operations Leaders: 

The main problem is duplication. Radiology AI solutions in healthcare generate findings, and those need to get into the EHR as structured data, not just as a PDF attached to a result. When the integration is not there, a person manually bridges the gap. That step adds time and introduces errors.

How to Build a Working FHIR + DICOM Integration

Converting DICOM Metadata to FHIR

1. The first component is a DICOM gateway to watch your PACS for new studies, extract the relevant metadata from the DICOM header, and convert it into a FHIR ImagingStudy resource.
2. That resource gets posted to a FHIR server where it sits alongside the patient’s other clinical data. This step requires mapping DICOM tags to FHIR fields. 
3. Most of the standard tags, PatientID, StudyDate, Modality, BodyPartExamined, and StudyInstanceUID, have clear FHIR equivalents. 

The complexity comes from inconsistencies in how different scanner vendors implement DICOM, particularly on older equipment. Any production-grade adapter needs validation rules to catch and handle those inconsistencies before they create bad data downstream.

Using the FHIR ImagingStudy Resource Correctly

FHIR R4 includes ImagingStudy as a first-class resource type, specifically designed to represent imaging data within the FHIR model. 

1. It captures study, series, and instance-level details and includes DICOMweb endpoint references, URLs that point back to the actual images in the PACS or vendor-neutral archive.
2. When ImagingStudy is implemented properly, any system with FHIR access can find a patient’s imaging history alongside their labs, medications, and clinical notes, all through the same API. 
3. That is the foundation multimodal AI tools need.

DICOMweb is the retrieval layer that makes this work at the image level. WADO-RS lets systems fetch actual DICOM instances over HTTP. QIDO-RS lets them search studies and series without downloading files. STOW-RS handles storage. 

Together, these three services allow AI tools to retrieve specific images on demand without going through proprietary PACS interfaces.

Connecting PACS & EHR Through a Common Integration Layer

Once DICOM metadata is in FHIR format, connecting it to the EHR becomes a more standard integration problem. Most major EHR platforms in the United States, like Epic, expose FHIR R4 APIs that external systems can read from and write to.

An integration layer sits in the middle and coordinates data flow. This can be a general-purpose integration engine like Mirth Connect or Rhapsody, a cloud-based health data platform like Azure Health Data Services or Google Cloud Healthcare API, or a custom service built and maintained internally. The right choice depends on your existing infrastructure and your team’s capacity.

The goal is a single point of access where an AI tool can send a patient identifier and get back a complete, normalized data set without needing to manage separate connections to both systems.

Three Clinical Scenarios Where This Integration Delivers Real Value

Radiology Worklist Management

• A radiology department at a mid-sized U.S. health system might move 600 to 800 imaging studies per day. Without a clinical context, AI triage tools can only prioritize based on imaging findings. 
• With a FHIR connection, they can factor in the patient’s current location, their acuity score, the care team’s flags, and whether the study is a follow-up or a new finding. 
• Studies that need immediate attention surface to the top. Routine follow-ups move down. Radiologists spend their time where it matters most.

Oncology Treatment Monitoring

• Tracking a cancer patient’s response to treatment requires comparing serial imaging studies over months or sometimes years, while accounting for changes in their labs, their medications, and their clinical status. 
• When imaging data and clinical data are connected through FHIR, an AI platform can automate a significant portion of that comparison and present it in a structured summary before a tumor board meeting. 
• Physicians still make the decisions, but they are not spending the first 20 minutes of every meeting pulling data from three different systems.

Emergency Stroke Response

• In stroke care, treatment decisions need to happen in minutes. 
• An AI tool with access to both CT perfusion imaging and the patient’s EHR record can check contraindications and flag the relevant clinical factors for the physician faster than any manual process. 
• Every minute that saves translates directly into better patient outcomes.

The Integration Challenges Organizations Actually Run Into

Inconsistent DICOM Implementations Across Vendors

DICOM is a standard, but every modality vendor implements it with some variation. Tags that should be populated are sometimes missing. Values that should be standardized are sometimes free-text. 

Facilities running a mix of scanner generations will encounter these inconsistencies regularly. The integration layer needs to handle them without failing silently, because bad metadata that makes it into the FHIR server will eventually corrupt downstream AI inputs.

Access Control Across System Boundaries

Moving patient data between PACS, EHR, and an AI platform means PHI is crossing multiple system boundaries. Every one of those handoffs needs to comply with HIPAA at encryption in transit and rest, role-based access using SMART on FHIR, audit logs for every data access event, and valid BAAs with every vendor in the chain. 

This is not a difficult set of requirements to meet, but it has to be designed in from the start. Retrofitting access controls onto an existing integration is significantly more expensive than building them in initially.

Avoiding Proprietary API Dependencies

PACS vendors often offer proprietary APIs that are faster to implement and easier to use than their DICOMweb interfaces. 

The short-term appeal is real, but the long-term cost is vendor lock-in integrations that break when contracts change, cannot connect to new AI tools, and require renegotiation every time the technology stack evolves. Building on open standards, DICOMweb for images, FHIR for clinical data, adds some upfront complexity but keeps your options open.

What Health Systems and Vendors Need to Prioritize

For health system technology leaders, the most productive starting point is an honest assessment of your current state. 

Ponder over: 

What FHIR version does your EHR expose? 

Does your PACS have a DICOMweb interface, and is it turned on? 

Is there any current data flow between your imaging systems and your clinical data platform? 

Most organizations find significant gaps at this stage, and knowing where they are makes the integration project far easier to scope.

From there, implementing the FHIR ImagingStudy resource is a practical first project with clear deliverables. It does not require deploying an AI tool; it creates the foundation that makes AI tools deployable later, and it has standalone value for any clinician who currently has to log into two systems to read a scan in context.

Multimodal AI Integration with Imaging Platform | Folio3 Digital Health

Folio3 Digital Health supports healthcare organizations in integrating multimodal AI technology into their medical imaging platforms. We design and deploy PACS imaging systems that securely manage data from all major modalities. These systems provide instant, DICOM-compliant access to imaging studies and centralized repositories that support secure sharing across facilities. We offer HIPAA-compliant structure & Epic integrations that make imaging platforms go beyond traditional PACS capabilities. 

Closing Note 

FHIR integration and DICOM are not competing against each other. They cover different parts of the data problem and were never meant to replace each other. FHIR gives clinical systems a common language for structured patient data. DICOM gives imaging systems a reliable format for storing and transmitting scans; multimodal AI in healthcare needs both.

The organizations seeing real results from imaging AI today are not necessarily the ones with the most sophisticated models. They are the ones that did the infrastructure work first — built the DICOM-to-FHIR bridge, implemented ImagingStudy correctly, connected their PACS to their EHR through a standards-based integration layer — and then deployed AI tools into an environment that was actually ready for them.

That infrastructure work is not glamorous. It does not generate press releases. But without it, the investment in AI tools produces far less than it should.

Frequently Asked Questions

What is the core difference between FHIR and DICOM? 

FHIR is a standard for exchanging structured clinical data: patient records, diagnoses, medications, lab results, through web APIs. DICOM is a format for storing and transmitting medical images and their associated metadata. 

What is DICOMweb, and how does it fit into a FHIR integration? 

DICOMweb is a set of RESTful web services, WADO-RS for image retrieval, QIDO-RS for search, and STOW-RS for storage, that allow DICOM images to be accessed over standard HTTP. It is how modern systems retrieve imaging data without going through proprietary PACS interfaces. When combined with FHIR ImagingStudy references, DICOMweb gives AI tools a clean, standards-based way to fetch images on demand.

How does HIPAA compliance apply to FHIR-DICOM integrations? 

Any integration that moves PHI between a PACS, an EHR, and an AI platform must meet HIPAA requirements at each data transfer point. That means encryption in transit and at rest, access controls through SMART on FHIR, audit logging for every data access event, and valid Business Associate Agreements with every vendor involved. These requirements are well-understood and manageable, but they need to be scoped into the integration design from the start rather than added afterward.

How long does a FHIR-DICOM integration typically take to build? 

A focused integration connecting a single EHR and PACS to an AI platform generally takes three to six months for a well-staffed team. Organizations with multiple PACS systems, older modalities with inconsistent DICOM implementations, or complex EHR configurations should plan for longer. The most reliable way to reduce timeline surprises is to audit your current interoperability posture before the project starts. Most organizations find issues they were not aware of once they look closely.

About the Author