Another quarter gone by, another HAPI-FHIR release.

HAPI FHIR 5.5.0 (Codename: Pangolin) brings a whole bunch of great new features, bugfixes, and more.

Highlights of this release are shown below. See the Changelog for a complete list. There will be a live Webinar (recording available on-demand afterward) on August 18 2021. Details available here: https://www.smilecdr.com/quarterly-product-release-webinar-reminder

Security Changes

• Resolved multiple vulnerabilities by updating dependencies.

General Client/Server/Parser Changes

• Added configuration to enable/disable various scheduled tasks.
• Support for qualified * for _include and _revinclude has been added, e.g. _include=Observation:*.
• Flyway migration used to enforce order by default. This has been changed so now the default behaviour is out of order migrations are permitted. Strict order can be enforced via the new strict-order flag if required.

CLI Tool changes:

• Support for multiple header-passthrough option using -hp or --header-passthroughparameter has been added to hapi-fhir-cli commands: example-data-uploader, export-conceptmap-to-csv, import-csv-to-conceptmap and upload-terminology

JPA Server General Changes

• DELETE _expunge=true has been converted to a Spring Batch job.
• Added a new setting to JpaStorageSettings called Tag Versioning Mode, which determines how tags are maintained.
• A new tag storage mode called Inline Tag Mode tas been added. In this mode, all tags are stored directly in the serialized resource body in the database, instead of using dedicated tables. This has significant performance advantages when storing resources with many distinct tags (i.e. many tags that are unique to each resource, as opposed to being reused across multiple resources).
• A new interceptor has been addeed to the JPA server called ForceOffsetSearchModeInterceptor. This interceptor forces all searches to be offset searches, instead of relying on the query cache.
• Added a new $reindex operation which creates a Spring Batch job to reindex selected resources.

JPA Server Performance Changes

• A new setting has been added to the JpaStorageSettings that allows the maximum number of _include and _revinclude resources to be added to a single search page result. In addition, the include/revinclue processor have been redesigned to avoid accidentally overloading the server if an include/revinclude would return unexpected massive amounts of data.
• When performing non-query cache JPA searches (i.e. searches with Cache-Control: no-store) the loading of _include and _revinclude will now factor the maximum include count."
• Subscriptions will no longer be triggered on unversioned changes.
• A new JpaStorageSettings setting called Mass Ingestion Mode has been added. This mode enables rapid data ingestion by skipping a number of unnecessary checks during backloading.
• FHIR Transactions with writes in them will now aggressively pre-fetch as many entities as possible at the start of processing. This reduces the number of database roundtrips.
• A new interceptor has been added for JPA servers that uses semaphores to avoid multiple concurrent FHIR transactions from trying to create/update the same resource at the same time. This can improve overall performance when writing many concurrent transactions since it avoids the need for retries.

JPA Server Partitioning Changes

• PatientIdPartitionInterceptor now supports conditional creates of resources where the resource is in the patient compartment but the conditional URL does not contain a patietn reference.
• The $evaluate-measure now works on a partitioned server.

Terminology Server and Validation Changes

• Added support for ICD-10-CM.
• A new interceptor ValidationMessageSuppressingInterceptor has been added. This interceptor can be used to selectively suppress specific vaLidation messages.
• Support for LOINC 2.70 has been added.
• Fixed a bug where ValueSet expansion did not correctly preserve order if multiple codes were included in a single inclusion block.
• A new Validation Support Module has been added that can use NPM Packages to supply validation conformance artifacts programatically to the validator."

JPA Server MDM Enhancements

• Too many MDM candidates matching could result in an OutOfMemoryError. Candidate matching is now limited to the value of IMdmSettings.getCandidateSearchLimit(), default 10000.
• Searches for mdm-expanded references such as Observation?subject:mdm=123 were getting denied by access rules that did not recognize the :mdm suffix. This has been corrected.
• $mdm-submit operation was only submitting 100 resources and then stopping. It now correctly submits all requested resources.

It's time for another release of HAPI FHIR.

HAPI FHIR 5.4.0 (Codename: Pangolin) brings a whole bunch of great new features, bugfixes, and more.

Highlights of this release are shown below. See the Changelog for a complete list. There will be a live Webinar (recording available on-demand afterward) on May 20 2021. Details available here: https://www.smilecdr.com/quarterly-product-release-webinar-reminder

Security Changes

• A resource exhaustion vulnerability in the HAPI FHIR JPA server was corrected. Learn more about CVE-2021-32053 here. Thanks to Zachary Minneker at Security Innovation for reporting!

General Client/Server/Parser Changes

• HAPI FHIR now supports OpenAPI (aka Swagger). See here for an example.
• Normalization and Standardization interceptors have been added. These can be used to normalize selected data fields according to configurable rules prior to storage.
• Contained resources can now reference to containing resources, as allowed in the FHIR Specification. Previously this direction was blocked and contained resources with no incoming reference from the containing resource were automatically stripped, as this style was not permitted in early versions of the FHIR specification. In addition, contained resource order will now be preserved during parsing round-trips.
• New interceptors have been added that can automatically map terminology in response resources using HAPI FHIR terminology services, returning configurable canonical terminology in the response payload.
• Support for the FHIR Prefer: handling=lenient header has been added via an optional interceptor.
• The automatic CapabiityStatement generation has been completely rewritten for R4+ servers. CapabilityStatements now include many new data elements, such as supported profiles, revincludes, resource level operations, and more.
• Token Search Parameters in GraphQL expressions are now correctly parsed.

JDK Changes

• HAPI FHIR now supports JDK 16, and this version is used to execute our CI test suite in order to ensure continued compliance. The minimum Java version required in order to use HAPI FHIR remains JDK 8. This may be updated to JDK 11 in an upcoming release, as many of the libraries we use are now either contemplating or have already completed an upgrade to JDK 11 as a minimum requirement.

JPA Server General Changes

• Support for the _list search parameter has been added to the JPA server
• Support for the :contained modifier has been added, allowing searches to select from data in contained resources found within the resource being searches. Note that this feature is disabled by default and must be enabled if needed.
• The JPA server now supports persisting FHIR extensions in Resource.meta
• Bulk Export now supports Patient- and Group- based exports
• Auto-created reference target placeholder resources now include an extension and an identifier if one is known
• A profiling effort led to improvements in performance when processing large FHIR Transaction bundles
• Resources imported into a repository via NPM Packages will now attempt to preserve the resource ID defined in the source package.

JPA Server Performance Changes

• Searches with only a single search parameter now generate a more streamlined SQL expression (one unnecessary JOIN was removed), improving performance.
• A new header X-Upsert-Extistence-Check (note there is a typo in the name, this will be addressed in the next release of HAPI FHIR! Please be aware if you are planning on using this feature) can be added which avoids existence checks when using client assigned IDs to create new records. This can speed up performance.

JPA Server Partitioning Changes

• Resource Reindexing is now supported on partitioned servers.
• FHIR Bulk Export is now supported on partitioned servers (note that this operation is run at the system level and includes data from all partitions. Future enhancements may allow for more nuanced exports on partitioned servers.)

Terminology Server and Validation Changes

• ValueSet expansion can now optionally return codes in the same hierarchy that they are defined in their source CodeSystem.
• Validation can now be configured to return only a warning when a code is found from a CodeSystem that is unknown/unavailable to the validator.

JPA Server MDM Enhancements

• A new search mdm-expansion syntax has been added to FHIR searches on MDM-enabled servers. For example Observation?patient:mdm=Patient/123 can be used to search for Observation resources belonging to Patient/123 but also to other MDM-linked patient records.
• MDM matching rules can now use FHIRPath expressions as selection criteria.
• A new syntax has been added to Group Bulk Export that allows MDM matching to be used to include matches in the group to export.
• MDM matching rules can now match on extensions, checking the URL and Value for equality.
• A new NUMERIC matcher has been added, allowing matching using numeric values.

It's August, so it's time for our next quarterly relese: HAPI FHIR 5.2.0 (Codename: Numbat).

Security Notice:

• Security Issue CVE-2020-24301: An XSS vulnerability has been fixed in the testpage overlay project. This issue affects only the testpage overlay module, but users of this module should upgrade immediately. Note that this issue is addressed in HAPI FHIR 5.1.0 (as well as 5.2.0+) so users who have already upgraded to HAPI FHIR 5.1.0 do not need to upgrade again to resolve this issue. It is listed here as we experienced a delay in obtaining a CVE number.

Major New Features:

• The JPA SearchBuilder (which turns FHIR searches into SQL statements to be executed by the database) has been completely rewritten to not use Hibernate. This allows for much more efficient SQL to be generated in some cases. For some specific queries on a very large test repository running on Postgresql this new search builder performed 10x faster. Note that this new module is enabled by default in HAPI FHIR 5.2.0 but can be disabled via a JpaStorageSettings setting. It is disabled by default in Smile CDR 2020.11.R01 but will be enabled by default in the next major release.

• Support for RDF Turtle encoding has been added, finally bringing native support for the 3rd official FHIR encoding to HAPI FHIR. This support was contributed by Josh Collins and Eric Prud'hommeaux of the company Janeiro Digital. We greatly appreciate the contribution! To see an example of the RDF encoding: hapi.fhir.org/baseR4/Patient?_format=rdf

Terminology Enhancements:

• Integration with remote terminology services has been improved so that required bindings to closed valuesets are no longer delegated to the remote terminology server. This improves performance since there is no need for remote services in this case.

• The CodeSystem/$validate-code operation has been implemented for R4+ JPA servers.

• The JPA Terminology Server is now version aware, meaning that multiple versions of a single CodeSystem can now be stored in a single FHIR terminology server repository. ValueSet expansion, CodeSystem lookup, and ConceptMap translation are all now fully version aware. Note that implementing support for fully versioned terminology is mostly complete, but some validation operations may still not work. This should be completed by our next major release.

• ValueSet expansion with filtering (e.g. using the filter parameter on the $expand operation) has now been implemented in such a way that it fully supports filtering on pre-expanded ValueSets, including using offsets and counts. This is a major improvement for people building picker UIs leveraging the $expand operation.

EMPI Improvements:

• Identifier matchers have been added, providing native FHIR support for matching on resource identifiers

• The $empi-clear operation performance has been greatly improved

Other Notable Improvements:

• A new combined "delete+expunge" mode has been added to the DELETE operation in the JPA server. This mode deletes resources and expunges (physically deletes) them in a single fast operation. Note that with this mode must be enabled, and completely bypasses interceptor hooks notifying registered listeners that data is being deletes and expunged. It is several orders of magnitude faster when deleting large sets of data, and is generally intended for test scenarios.

• The Package Server module now supports installing non-conformance resources from packages.

• The _typeFilter parameter has been implemented for the $bulk-export module.

As always, see the changelog for a full list of changes.

Thanks to everyone who contributed to this release!