The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• FlywayDB (JPA): 8.4.4 -> 8.5.0
• Postgresql (JPA): 42.3.2 -> 42.3.3 (Addresses WS-2022-0080)

The HAPI FHIR server GraphQL endpoints now support GraphQL introspection, making them much easier to use with GraphQL-capable IDEs.

The SearchNarrowingInterceptor can now narrow searches to require a token:in or token:not-in parameter.

Performance for JPA Server ValueSet expansion has been significantly optimized in order to minimize database lookups, especially with large expansions.

SearchNarrowingInterceptor can now be used to automatically narrow searches to include a code:in or code:not-in expression, for mandating that results must be in a specified list of codes.

Support has been added to the JPA server for token :not-in queries. Similar to :not queries, resources will currently be considered to match if any codes in the relevant resource field are not found in the given ValueSet (as opposed to matching if all codes are not in the given ValueSet).

Added logs that identify the resource that failed the validation check during package installation, and describe the reason for the failure.

Support has now (finally!) been added for the FHIR Bulk Import ($import) operation. This operation is the first operation to leverage the new Batch2 framework.

A new batch operation framework for executing long running background jobs has been created. This new framework is called 'Batch2', and will eventually replace Spring Batch. This framework is intended to be much more resilient to failures as well as much more paralellized than Spring Batch jobs.

It is now possible to register multiple IConsentService implementations against a single ConsentInterceptor.

A consent service implementation that enforces search narrowing rules specified by the SearchNarrowingInterceptor has been added. This can be used to narrow results from searches in cases where this can't be done using search URL manipulation. See ResultSet Narrowing for more information.

When validating a code, a helpful error message is now returned if the user has supplied a ValueSet URL where a CodeSystem URL should be used, since this is a common mistake.

Previously there was no way to recreate freetext indexes for terminology entities. A new CLI operation, reindex-terminology now exists for this purpose.

Improve logging so it is clear when a new search parameter has been loaded into the cache.

The JPA server terminology service can now process IS-A filters in ValueSet expansion on servers with Hibernate Search disabled.

The server now supports date searches with the NOT_EQUALS (ne) prefix.

When using ApacheProxyAddressStrategy as a server address strategy, the Forward header will now be respected. Thanks to Thomas Papke for the pull request!

Added a helper method to BundleUtil, to support the creation of a new bundle entry and to set the value for one of the fields.

Added a new pointcut: STORAGE_PRESTORAGE_CLIENT_ASSIGNED_ID which is invoked when a user attempts to create a resource with a client-assigned ID.

Group Bulk Export (e.g. Group/123/$export) now additionally supports Organization and Practitioner as valid _type parameters. This works internally by querying using a _has parameter

The resource JSON can now be stored and retrieved in the Lucene/Elasticsearch index. This enables some queries to provide results without using the database. This is enabled via DaoConfig.setStoreResourceInLuceneIndex()

When using JPA persistence with Hibernate Search (Lucene or Elasticsearch), simple FHIR queries that can be satisfied completely by Hibernate Search no longer query the database. Before, every search involved the database, even when not needed. This is faster when there are many results.

When updating or reindexing a resource on a JPA server where indexing search param presence is enabled (i.e. support for the :missing modifier), updates will try to reuse existing database rows where possible instead of deleting one row and adding another. This should cause a slight performance boost systems with this feature enabled.

Reverting the change to treat canonical references as local when setting the flag getTreatBaseUrlsAsLocal(). This was added to address a limitation in _include that did not process references by canonical URLs. The _include search operation now includes canonical references without special configuration.

Previously, during RepositoryValidation, we would perform validation on resources created via DaoConfig's setAutoCreatePlaceholderReferenceTargets property. This caused validation failures on placeholder resources as they do not conform to any profile. This has been changed, and Repository Validation will not occur on any resource containing an extension with URL http://hapifhir.io/fhir/StructureDefinition/resource-placeholder`.

Several classes and interfaces related to the $expunge operation have moved from the hapi-fhir-jpaserver-base project to the hapi-fhir-storage project.

Method signatures on several interfaces and classes related to the $expunge operation have changed to support the case where the primary identifier of a resource is not necessarily a Long.

The XML and JSON Parsers are now encoding narratives of contained resources. Narratives were skipped before for contained resources. This was a rule which was only valid in STU1 (https://www.hl7.org/fhir/DSTU1/narrative.html), and was removed in DSTU2+

Include property regex operation was not working when expanding ValueSet. This is now fixed

A regression in HAPI FHIR 5.5.0 meant that very large transactions where the bundle contained over 1000 distinct client-assigned resource IDs could fail on MSSQL and Oracle due to SQL parameter count limitations.

An occasional concurrency failure in AuthorizationInterceptor has been resolved. Thanks to Martin Visser for reporting and providing a reproducible test case!

When performing a search with a _revinclude. the results sometimes incorrectly included resources that were reverse included by other search parameters with the same name. Thanks to GitHub user @vivektk84 for reporting and to Jean-Francois Briere for proposing a fix.

When searching for date search parameters on Postgres, ordinals could sometimes be represented as strings, causing a search failure. This has been corrected.

A race condition in the Subscription processor meant that a Subscription could fail to register right away if it was modified immediately after being created. Note that this issue only affects rapid changes, and only caused the subscription to be unregistered for a maximum of one minute after its initial creation so the impact of this issue is expected to be low.

When cross-partition reference Mode is used, the rest-hook subscriptions on a partition enabled server would cause a NPE. Cause of this is from the reloading of the subscription when the server is restarted. This issue has been fixed. Also fixed issue with revinclude for rest-hook subscription not working.

ValueSet pre-expansion was failing when number of concepts was larger than configured BooleanQuery.maxClauseCount value (default is 1024). This is now fixed.

Fixed a bug when searching for the resource persistent id in the partitioned environment. This happens when the Cross-Partition Reference Mode is set to ALLOWED_UNQUALIFIED and it is searching for the wrong partition id and resulted in resource persistent id not found.

Updated the FHIR core libs to 5.6.36 to support fix for funcReplace in FHIRPathEngine.

A regression in HAPI FHIR 5.7.0 meant that when UnknownCodeSystemWarningValidationSupport was configured for WARNING behaviour, validating a field with an implicit code system could incorrectly result in an error.

Reindexing jobs were not respected the passed in date range in SearchParams. We now take date these date ranges into account when running re-indexing jobs.

Previously, or expressions were not being properly validated in FHIRPath in STU3 due to a bug with expression path splitting. This has been corrected.

When doing package upload, all resources were filtered by status=active. That is inappropriate for some types. For Subscription, we need to check if the value is requested, for DocumentReference and Communication other values outside the expected active and not active also exist. This change adds a check for those types of resources other than the one normally done for active matching value.

Previously the Fhir parser would only set the resource type on the resource ID for resources which implemented IDomainResource. This caused a few resource types to be missed. This has been corrected, and resource type is now set on the id element for all IBaseResource instances instead.

Previously, allowing any client assigned ID by setting ClientIdStrategyEnum to ANY threw an exception when creating CodeSystem resources. This has been corrected.

Supporting expansion of ValueSet include/exclude system URI expressed in canonical format during validation.

Previously if partitioning was enabled, the URL returned from $export-poll-status provided a location in the wrong partition. This has been fixed and all Bulk Exports will be stored in the DEFAULT partition.

TerserUtil.mergeAllFields() threw an exception when cloning contained resources. This has been corrected.

Added a new setting to BinaryStorageInterceptor which allows you to disable binary de-externalization during resource reads.

Previously, Bulk export jobs automatically cleared the collection after a hardcoded 2 hour time period from start of the job. This is now configurable via a new DaoConfig property, setBulkExportFileRetentionPeriodHours(). If this is set to a value of 0 or below, then the collections will never be expired.

The Spring Framework library was upgraded to version 5.3.18 in order to avoid depending on a version known to be vulnerable to CVE-2022-22965, known as Spring4Shell. HAPI FHIR is not believed to be vulnerable to this issue, but the library has been bumped as a precaution.

Bump the version of Spring Core to 5.3.18 to avoid CVE-2022-22963

Previously, during RepositoryValidation, we would perform validation on resources created via DaoConfig's setAutoCreatePlaceholderReferenceTargets property. This caused validation failures on placeholder resources as they do not conform to any profile. This has been changed, and Repository Validation will not occur on any resource containing an extension with URL http://hapifhir.io/fhir/StructureDefinition/resource-placeholder`.

The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• log4j-api (JPA): 2.11.1 -> 2.17.1 (Addresses CVE-2021-44228 - HAPI FHIR was not vulnerable to this issue but this upgrade avoids unnecessary OWASP scan notices)
• SLF4j (All): 1.7.30 -> 1.7.33
• Logback (All): 1.2.8 -> 1.2.10
• Commons-IO (All): 2.8.0 -> 2.11.0
• Jackson (All): 2.13.0 -> 2.13.1
• Guava (All): 30.1.1-jre -> 31.0.1-jre
• JDOM (XML Patch Support): 2.0.6 -> 2.0.6.1 (Addresses CVE-2021-33813)
• Spring (JPA): 5.3.7 -> 5.3.15
• Spring (JPA): 5.3.7 -> 5.3.15
• Spring-Data (JPA): 2.5.0 -> 2.6.1
• Hibernate ORM (JPA): 5.4.30.Final -> 5.6.2.Final
• Flyway (JPA): 6.5.4 -> 8.4.1
• Sqlbuilder (JPA): 3.0.1 -> 3.0.2
• H2 (JPA): 1.4.200 -> 2.1.210 (Note that this change requires the use of the HapiFhirH2Dialect instead of the built-in Hibernate H2Dialect due to Hibernate issue HHH-15002
• Commons-DBCP2 (JPA): .8.0 -> .8.0 -> 2.9.0
• Swagger-Models (OpenAPI Support): 2.1.7 -> 2.1.12
• Thymeleaf (Testpage Overlay): 3.0.12.RELEASE -> 3.0.14.RELEASE (Addresses CVE-2021-43466)
• Commons-CLI (CLI): 1.4 -> 1.5.0
• JANSI (CLI): 2.3.2 -> 2.4.0
• Jetty Server (CLI): 9.4.43.v20210629 -> 9.4.44.v20210927
• Spring Boot (Boot): 2.5.0 -> 2.6.2
• Swagger UI (OpenAPI): 3.46.0 -> 4.1.3
• Resteasy (JAX-RS): 4.0.0.Beta3 -> 5.0.2.Final
• Postgresql (JPA): 42.3.1 -> 42.3.2
• Spring Security Oauth2(Oauth): 2.0.2.RELEASE -> 2.0.17.RELEASE

In the JPA server, the token :of-type modifier is now supported. This is an optional feature and must be explicitly enabled (default is disabled).

Added partition support for subscriptions. Subscriptions will now only match resource from the same partition

New configuration option added to validate bundle resources concurrently. Also new configuration added to skip validation of contained resources.

Added support for cross-partition subscriptions. Subscription in the default partition can now listen to resource changes from all partitions

The resource contents are optionally added to the lucene index so $lastn queries can be satisfied without database access. This is activated by the DaoConfig 'StoreResourceInLuceneIndex' property.

Added http://hapifhir.io/fhir/StructureDefinition/subscription-delivery-retry-count extension that can be provided to a subscription to define a specific retry strategy (retry retry-count number of times before giving up).

Provided a Remote Terminology Service implementation for the $validate-code Operation.

Add the concept of messageKey to the BaseResourceMessage class. This is to support brokers which permit key-based partitioning.

Allow Rest Hook subscriptions to be configured to send delete requests.

Added ability to load Implementation Guide packages from filesystem by supporting the file:/ syntax of url.

A redundant set of hash calculations in the JPA server was eliminated.

A number of minor optimizations have been added to the JsonParser serializer module as well as to the transaction processor. These optimizations lead to a significant performance improvement when processing large transaction bundles (i.e. transaction bundles containing a larger number of entries).

A new configuration option has been added to ParserOptions called setAutoContainReferenceTargetsWithNoId. This setting disables the automatic creation of contained resources in cases where the target/contained resource is set to a Reference instance but not actually added to the Resource.contained list. Disabling this automatic containing yields a minor performance improvement in systems that do not need this functionality.

Improved validation performance by switching validation serialization from XML to JSON.

Significantly improved $delete-expunge performance by adding database indexes, and filtering needed foreign keys to delete by resource type.

A new JPA setting has been added to DaoConfig settings called Inline Resource Text Below Size. When this setting is set to a positive value, any resources with a total serialized length smaller than the given number will be stored as an inline VARCHAR2 text value on the HFJ_RES_VER table, instead of using an external LOB column. This improves read/write performance (often by a significant amount) at the expense of a slightly larger amount of disk usage.

Improves the performance of the query for searching by chained search parameter when the Index Contained Resources feature is enabled.

Improved performance of validating resources when skip contained resources is enabled.

Previously in configuring MDM, you were only allowed to set a single eidSystem which was global regardless of how many resource types you were performing MDM on. This has been changed. That field is now deprecated, and a new field called eidSystems (a json object) should be used instead. This will allow you to have granular control over which resource types use which EIDs. See the documentation of eidSystems for more information.

Fixed a regression where packages would fail to load if HAPI-FHIR was operating in DATABASE binary storage mode.

These changes are related to bumping the dependency on org.hl7.fhir.core to version 5.3.0. The main updates that resulted in the majority of the changes in this PR were the addition of the interface IValidationPolicyAdvisor, and the refactoring of some of the main validation classes (for cleaning purposes). Also, many of the error messages were updated, so the resulting tests in HAPI had to be modified to reflect these message changes. In regard to the new interface IValidationPolicyAdvisor, by implementing this interface and passing it through the validation context in HAPI to the InstanceValidator in the core libraries, users can now control the behavior of the validator when validating references, contained resources, and coded content. Previously, only references were controlled in this way, and users controlled this by overriding the validation fetcher. Test cases were added in the validation test cases repository to demonstrate it's functionality. In particular, the two test cases contained-resource-bad-id, and contained-resource-bad-id-ignore. The definitions and expected outcomes of these test cases are in the manifest.json in that repository.

Add email configuration to MailSvc constructor

Removed the following modules from the HAPI-FHIR project: hapi-fhir-testpage-interceptor, hapi-fhir-structures-dstu, hapi-fhir-oauth2, hapi-fhir-narrativegenerator.

Calling the $document operation previously omitted the fullUrl of the bundle entries. This has been corrected.

When a subscription fails to activate (either on the first attempt or on start up), hapi-fhir will log the exception, set the subscription to ERROR state, and continue on. This is to prevent hanging on startup for errors that cannot be resolved by infinite retries.

$member-match operation was allowed to be invoked by GET when it shouldn't have. That is fixed. Operation can only be invoked by POST request

Using the delivery option: Payload Search Result Mode for rest-rook subscriptions on a partition enabled server would cause a NPE. This issue has been fixed.

Code System deletion background tasks were taking over a day to complete on very large CodeSystems for PostgreSQL, SQL Server and Oracle databases. That was improved now taking less than an hour in all three platforms

RequestValidatingInteceptor incorrectly prevented GraphQL requests from being submitted using the HTTP POST form of the GraphQL operation.

Updated UnknownCodeSystemWarningValidationSupport to allow the throwing of warnings if configured to do so.

Resource links were previously not being consistently created in cases where references were versioned and pointing to recently auto-created placeholder resources.

Fixed a serious performance issue with the $reindex operation.

In servers with a large number of StructureDefinition resources loaded, occasionally a call for the CapabilityStatement could take a very long time to return. This has been corrected.

$everything operation returns a 500 error when querying for a page when _getpagesoffset is greater than or equal to 300. This has been corrected.

MDM was throwing a NullPointerException when upgrading a match from POSSIBLE_MATCH to MATCH. This has been corrected.

Fixed bug that caused queries that checked Elasticsearch when setting _total=accurate searches to return all values when using _content or _text parameters

Terminology reindexing job was launching the wrong process, so terminology reindexing was never launched. This has been fixed.

Concurrent validation failed with StringIndexOutOfBoundsException when the validation location did not contain a '.'. This has been corrected.

In rare cases where patient ID String happened to have hashCode equal to Integer.MIN_VALUE, PatientIdPartitionInterceptor would generate an invalid partition ID. This has been fixed.

When $mdm-clear operation batch was split into multiple threads, ResourceVersionConflictExceptions were being thrown. This issue has been fixed.

Conformance validation should happen once per client-endpoint, no matter the number of threads executing the requests, but was happening once per client-endpoint-thread. This is now fixed.

Previously, $binary-access-read and other operations that return neither method outcomes nor resources were causing no invocation of the SERVER_OUTGOING_RESPONSE pointcut. This has been corrected, and those operations will now correctly invoke SERVER_OUTGOING_RESPONSE.

GraphQL searches cannot be executed when number of matching results exceeds default page size. This has been corrected.

In HAPI FHIR 5.6.0, a regression meant that JPA server FHIR transactions containing two identical conditional operations (for example, a transaction with two conditional create operations where the conditional URL was identical for both) could result in resources being saved in an inconsistent state. This scenario will now be result in the server returning an HTTP 400 error instead. Note that there is no meaning defined in the FHIR specification for duplicate entries in a FHIR transaction bundle, so it has never been recommended to do this.

The JPA server will now validate _include and _revinclude statements before beginning search processing. This prevents an ugly JPA error when some invalid params are used.

Enhanced Lucene Indexing failed when indexing contained resources. Contained resources are not indexed in Lucene/Elasticsearch, but this no longer causes an exception.

Chained searches will handle common search parameters correctly when the Index Contained Resources configuration parameter is enabled.

When cross-partition reference Mode is used, the rest-hook subscriptions on a partition enabled server would cause a NPE. Cause of this is from the reloading of the subscription when the server is restarted. This issue has been fixed. Also fixed issue with revinclude for rest-hook subscription not working.

Bulk export permissions were over-permissive in RuleBulkExportImpl. This has been corrected.

In the JPA server, searching using the token :text modifier will no longer index and match values in Identifier.type.text. Previously we indexed this element, but the FHIR specification does not actually show this field as being indexed for this modifier, and intuitively it doesn't make sense to do so.

Bump Spring Core dependency to remove Spring4Shell vulnerability.

Fixed a regression where packages would fail to load if HAPI-FHIR was operating in DATABASE binary storage mode.

Updated UnknownCodeSystemWarningValidationSupport to allow the throwing of warnings if configured to do so.

Resource links were previously not being consistently created in cases where references were versioned and pointing to recently auto-created placeholder resources.

Fixed a serious performance issue with the $reindex operation.

$everything operation returns a 500 error when querying for a page when _getpagesoffset is greater than or equal to 300. This has been corrected.

MDM was throwing a NullPointerException when upgrading a match from POSSIBLE_MATCH to MATCH. This has been corrected.

Added mdm support on $everything operation for patients by adding _mdm=true query parameter. Eg: /Patient/123/$everything?_mdm=true.

Suport for annotation scanning in synthetic types such as weld proxies has been added. Thanks to GitHub user @tarekmamdouh for the pull request!

FHIR bundle batch is now processed in parallel by default and is configurable by DaoConfig.

Updated handling of MDM_AFTER_PERSISTED_RESOURCE_CHECKED pointcut to include additional MDM related info.

Allows to upload not-current version of LOINC ValueSet(s).

Added displayLanguage support for CodeSystem $lookup operation to filter out designation by language.

Fixed a regression which causes transactions with multiple identical ifNoneExist clauses to create duplicate data.

Two improvements have been made to the connection to Elasticsearch. First, null username and password values are now permitted. Second, multiple hosts are now permitted via the setHosts() method on the ElasticHibernatePropertiesBuilder, allowing you to connect to multiple elasticsearch clusters at once. Thanks to Dušan Marković for the contribution!

Add new RuleBuilder options which allow you to specify additional resources and search parameters which match a given compartment. More explanations of the enhancements can be found in the documentation.

When performing ValueSet expansions and ValueSet-based code validations, the HAPI FHIR terminology service will allow ValueSets to be expanded if they contain an enumeration of codes, even if the corresponding CodeSystem resource can not be found. Response messages have also been improved to give better insight into whether a precalculated expansion was used. In addition, a new operation called $invalidate-expansion has been added that allows for manual invalidation of previously calculated ValueSet expansions.

Lucene/Elasticsearch indexing has been extended to string, token, and reference parameters. This can be enabled by the new setAdvancedLuceneIndexing() property of DaoConfig.

Open up the visibility of some methods in the generation of the Open API definition files to allow extenders to add support for OIDC authorization.

Previously, when a search query explicitly includes a search parameter that is for the same resource type but a different resource instance from the one(s) specified on the authorized list, the search narrowing interceptor would include both search parameters in the final query, resulting in an empty bundle being returned to the caller. Now, such a call will result in a 403 Forbidden error, making it more clear why no resources were returned.

Added documentation for $partition-management-read-partition. Added $partition-management-list-partitions operation and documentation.

A new parser that naticaly handles NDJSON format (automatically converting to/from FHIR Bundle resources in order to represent the collection being parsed/serialized) as been added. Thanks to Ben Li-Sauerwine for the contribution!

Added $mdm-create-link operation and documentation.

Inline match URL searches (e.g. search URLs for conditional creates, conditional updates, etc.) are now subject to the same security and access control checks as other searches.

A new parameter, _id has been added to the Patient/$everything type-level operation. This allows you to pass in multiple patients as arguments, e.g. [base]/Patient/$everything?_id=1,2,3. This call would retrieve everything for patients with IDs 1,2, and 3.

Previously, chained searches were not able to traverse reference fields within contained resources. This enhancement adds the ability to traverse the reference fields of contained resources when those fields refer to discrete resources.

LOINC copyright notice is now taken from CodeSystem.copyright entry from loinc.xml input file.

Previously, only contained resources that are referenced directly by the containing resource were being indexed. This enhancement indexes the fields of contained resources that are referenced by other contained resources and uses these new indices in chained searches. Note: in order to make use of this new capability, it must be enabled via a configuration parameter and the repository must be re-indexed.

Further enhances the features added by issue 3100 to allow chained searches across any combination of discrete and contained references.

Subscription criteria in the HAPI FHIR JPA server now supports an optional alternate syntax of [*] (all resources of all types) and [resourcename,resourcename,...] (all resources of the given types. Note that no search parameters may be specitied with this syntax.

Added a functionality to deny unknown extensions.

Provided a Remote Terminology Service implementation for the $lookup Operation.

When performing a FHIR transaction using the JPA server where the transaction contains many identical inline match URLs (as is the case with recent versions of Synthea), HAPI FHIR will now avoid repeateed identical lookups while processing the transaction.

When ingesting Synthea data (or simnilarly structured transaction bundles) into a JPA server, a redundant resource ID lookup has been optimized out. This should particularly speed up larger sized transactions.

Support for the _language search parameter has been dropped from the JPA server. This search parameter was specified in FHIR DSTU1 but was dropped in later versions. It is rarely used in practice and imposes an indexing cost, so it has now been removed. A custom search parameter may be used in order to achieve the same functionality if needed.

HAPI-FHIR provides indexing on Canonical Types as references. However, the option to treat absolute references as local was being ignored for those indexed canonicals. This has been corrected. Now, if you have set getTreatBaseUrlsAsLocal() and HAPI-FHIR detects a canonical which starts with such a url, that prefix will be stripped, and indexing will occur normally.

Change getPathItem method in OpenApiInterceptor from private to protected

The $mdm-clear operation has been changed to use Spring Batch.

During transactions, any resources that were PUT or POSTed with a conditional URL now receive extra validation. There is now a final storage step which ensures that the stored resource actually matches the conditional URL.

This PR eliminates the search coordinator threadpool, and executes searches synchronously on the HTTP client thread. The idea of using a separate pool was supposed to help improve server scalability, but ultimately created false bottlenecks and reduced the utility of monitoring infrastructure so it has been eliminated.

Prevent _expunge and _cascade from being used on the same DELETE operation

Removed loinc.xml file from application, which was used as a fallback if not provided when uploading a CodeSystem. it is now required as input for LOINC CodeSystem upload.

The Saxon-HE library has been removed as a mandatory dependency from the converter library. Thanks to Jing Tang for the contribution!

This issue involves searching for a resource with a DATE parameter that is specified at only the YEAR level of precision. When searching at a higher level of precision, no results are matched. This issue is fixed now.

The SearchParameter canonical URLs exported by the JPA server have been adjusted to match the URLs specified in the FHIR specification.

Previously, when using the Expunge Everything operation, caches could retain old invalid values. This has been corrected. Thanks to Ben Li-Sauerwine for the fix!

Certain ValueSet validation/expansion operations failed with a 'no transaction' error on Postgresql. This has been corrected. Thanks to @tyfoni-systematic for reporting!

The :not modifier does not currently work for observations with multiple codes for the search. This is fixed.

Fixed a bug wherein an NPE could be thrown by the MDM module interceptor if an incoming resource had a tag with no system.

Open API docs failed to load if a custom context path is set. This has been corrected.

Fixed a bug wherein includes were not being included on revincludes.

Previously, validation against bcp47 (urn:ietf:bcp:47) as a language would fail validation if the region was absent. This has been fixed, and the validate operation will now correctly validate simple languages, e.g. nl instead of requiring nl-DE or nl-NL

$lookup operation cache was based on system and code, it becomes a defect after adding displayLanguage support. Problem is now fixed.

No resource returned when search with percent sign. Problem is now fixed

Fixed issue where the processing of queries like Procedure?patient= before a cache search would cause the parameter key to be removed. Additionally, ensured that requests like Procedure?patient= cause HTTP 400 Bad Request instead of HTTP 500 Internal Error.

Added a new DaoConfig setting called setElasticSearchIndexPrefix(String prefix) which will cause Hibernate search to prefix all of its tables with the provided value.

Previously, the system would only traverse references to discrete resources while performing a chained search. This fix adds support for traversing references to contained resources as well, with the limitation that the reference to the contained resource must be the last reference in the chain.

CLI smileutil help {command} returns Unknown command which should return the usage of command. This has been corrected.

Fixed a bug where two identical tags in parallel entries being created in a batch would fail.

CodeSystem version is copied to ValueSet.compose.include.version on loinc terminology upload to support versioned ValueSet expansion.

Fixes migration of VARBINARY columns on MS SQLServer.

A new customized Hibernate dialect is now used for HAPI FHIR JPA on Postgresql Database. This dialect uses the PG oid datatype instead of text for CLOB columns, avoiding CLOBs being seen as abandoned by the Postgresql VACUUMLO tool.

Fixes a bug that was causing a null pointer exception to be thrown during database migrations that add or drop indexes.

Fixed a bug in processing large batch requests containing many modifying entries. PUT/POST/DELETE operations now occur sequentially, instead of in parallel.

When using deferred model scanning in highly parallelized environments, a crash could sometimes occur during parse/serialize operations.

Update documentation for $mdm-match

Fixes a regression caused by an earlier release, which would cause the $everything operation to throw an SQL error when run under Postgresql.

$binary-access-write operation returns a DocumentReference with no content. This has been corrected.

MySQL 5.7 maps BLOB columns to LONGVARBINARY and the migrator did not take this into account. This has been fixed.

When performing paging queries over large result sets in the JPA server (> 1000 results), SQL could be generated that violated the Oracle RDBMS limit of 1000 parameters. This has been resolved.

Previously, the package registry would not work correctly when externalized binary storage was enabled. This has been corrected.

The BaseResourceModifiedJson class had a redundant field called myId containing the payload ID. This has been removed.

Bump Spring Core dependency to 5.3.18 to remove vulnerability.

Chained searches will handle common search parameters correctly when the Index Contained Resources configuration parameter is enabled.

Improves the performance of the query for searching by chained search parameter when the Index Contained Resources feature is enabled.

Fixed a serious performance issue with the $reindex operation.

Previously, chained searches were not able to traverse reference fields within contained resources. This enhancement adds the ability to traverse the reference fields of contained resources when those fields refer to discrete resources.

Previously, only contained resources that are referenced directly by the containing resource were being indexed. This enhancement indexes the fields of contained resources that are referenced by other contained resources and uses these new indices in chained searches. Note: in order to make use of this new capability, it must be enabled via a configuration parameter and the repository must be re-indexed.

Further enhances the features added by issue 3100 to allow chained searches across any combination of discrete and contained references.

No resource returned when search with percent sign. Problem is now fixed

Previously, the system would only traverse references to discrete resources while performing a chained search. This fix adds support for traversing references to contained resources as well, with the limitation that the reference to the contained resource must be the last reference in the chain.

Fixed a regression which causes transactions with multiple identical ifNoneExist clauses to create duplicate data.

The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• Spring (JPA): 5.3.6 -> 5.3.7
• Spring Boot (JPA Starter): 2.4.4 -> 2.5.0
• Jetty (CLI): 9.4.39.v20210325 -> 9.4.42.v20210604

Pagination returned incorrect offset and count in the previous link of the last page when total element count was one more than multiple of page size. Problem is now fixed

Settings have been added to the JPA Server DaoConfig to enable/disable various individual kinds of scheduled tasks.

When performing a conditional create operation on a JPA server, the system will now verify that the conditional URL actually matches the data supplied in the resource body, and aborts the conditional create if it does not.

Support has been added to the JPA server for _include and _revinclude where the value is a qualified star, e.g. _include=Observation:*.

A new interceptor ValidationMessageSuppressingInterceptor has been added. This interceptor can be used to selectively suppress specific vaLidation messages.

A new setting has been added to the DaoConfig that allows the maximum number of _include and _revinclude resources to be added to a single search page result. In addition, the include/revinclue processor have been redesigned to avoid accidentally overloading the server if an include/revinclude would return unexpected massive amounts of data.

When performing non-query cache JPA searches (i.e. searches with Cache-Control: no-store) the loading of _include and _revinclude will now factor the maximum include count.

A new config option has been added to the DaoConfig that causes generated SQL statements to account for potential null values in HAPI FHIR JPA date index rows. Nulls are no longer ever used in this table after HAPI FHIR 5.3.0, but legacy data may still have nulls.

A new DaoConfig setting called Mass Ingestion Mode has been added. This mode enables rapid data ingestion by skipping a number of unnecessary checks during backloading.

A new Pointcut has been added that is invoked when a new Bulk Export is initiated.

The JPA server terminology uploader now supports uploading ICD-10-CM (US Edition) using the native format for that vocabulary.

AuthorizationInterceptor can now be used to authorize bulk export requests

Support for LOINC 2.70 has been added.

PatientIdPartitionInterceptor now supports conditional creates of resources where the resource is in the patient compartment but the conditional URL does not contain a patietn reference.

The $evaluate-measure now works on a partitioned server.

A new interceptor has been added for JPA servers that uses semaphores to avoid multiple concurrent FHIR transactions from trying to create/update the same resource at the same time. This can improve overall performance when writing many concurrent transactions since it avoids the need for retries.

A new tag storage mode called Inline Tag Mode tas been added. In this mode, all tags are stored directly in the serialized resource body in the database, instead of using dedicated tables. This has significant performance advantages when storing resources with many distinct tags (i.e. many tags that are unique to each resource, as opposed to being reused across multiple resources).

A new interceptor has been addeed to the JPA server called ForceOffsetSearchModeInterceptor. This interceptor forces all searches to be offset searches, instead of relying on the query cache. This means that FHIR search operations will never result in any database write, which can be good for highly concurrent servers.

A new JPA partitioning interceptor PatientIdPartitionInterceptor has been added. This interceptor uses the ID of the patient associated with any resources in the patient compartment to generate a consistent partition ID.

$mdm-query-links and $mdm-duplicate-golden-resources now enforce paging via parameters _offset and _count. More details can be found in the MDM Operations documentation.

Support for multiple header-passthrough option using -hp or --header-passthrough parameter has been added to hapi-fhir-cli commands: example-data-uploader, export-conceptmap-to-csv, import-csv-to-conceptmap and upload-terminology

A new Validation Support Module has been added that can use NPM Packages to supply validation conformance artifacts programatcally to the validator.

Add 'loinc.codesystem.make.current' property to upload-terminology command to allow loading LOINC version without becoming current.

Allowed the optional inclusion of the LOINC Consumer Names archive in addition to the main LOINC distribution. If it is supplied, the consumer names CSV file will be scanned, and all consumer names will be added to uploaded Concepts as additional designations

Allowed the optional inclusion of the LOINC Linguistic Variants archive in addition to the main LOINC distribution. If it is supplied, all linguistic variants files will be scanned, and all translations will be added to uploaded Concepts as additional designations

Upgrade net.java.dev.jna to run docker tests on Mac arm64 M1 machines

The :text Search Parameter modifier now searches by word boundary of the text content as opposed to only searching at the start of the text when using Lucene/Elasticsearch indexing. Add * to match word prefixes (e.g. weig* will match weight).

Added new $reindex operation with similar syntax to $delete-expunge that creates a spring-batch job to reindex selected resources. $mark-all-resources-for-reindexing and $perform-reindexing-pass are now deprecated, and will likely be removed in a future release.

Replace existing email implementation code with SimpleJavaMail library.

Modified the behaviour of the :mdm param qualifier. Previously, it used to only resolve IDs if the resource ID was a source resource. Now, MDM expansion will work if you pass it the ID of a golden resource instead.

FHIR Transaction duplicate record checks are now performed without any database interactions or SQL statements, reducing the processing load associated with FHIR transactions by at least a small amount.

Conditional URL lookups in the JPA server will now explicitly specify a maximum fetch size of 2, avoiding fetching more data that won't be used inadvertently in some situations.

FHIR transactions in the JPA server that perform writes will now aggressively pre-fetch as many entities as possible at the very start of transaction processing. This can drastically reduce the number of round-trips, especially as the number of resources in a transaction gets bigger.

A new setting has been added to the DaoConfig called Tag Versioning Mode. This setting controls whether a single collection of tags/profiles/security labels is maintained across all versions of a single resource, or whether each version of the resource maintains its own independent collection. Previously each version always maintained an independent collection, which is useful sometimes, but is often not useful and can affect performance.

When performing a conditional create/update/delete on a JPA server, if the match URL contained a plus character, this character was interpreted as a space (per legacy URL encoding rules) even though this has proven to not be the intended behaviour in real life applications. Plus characters will now be treated literally as a plus character in these URLs.

Bulk import batch jobs are now activated in a local scheduled task, making bulk import jobs better able to take advantage of large clusters.

DELETE _expunge=true has been converted to use Spring Batch. It now simply returns the jobId of the Spring Batch job while the job continues to run in the background. A new operation called $expunge-delete has been added to provide more fine-grained control of the delete expunge operation. This operation accepts an ordered list of URLs to be delete expunged and an optional batch-size parameter that will be used to perform the delete expunge. If no batch size is specified in the operation, then the value of DaoConfig.getExpungeBatchSize() is used.

The ConceptMap.group.element.display storage size limit has been increased to 500 characters.

The bulk export request length limit has been increased to 1024 characters.

If two authorization compartments apply to the same targets and share the same compartment name, then instead of creating a new compartment, the rule builder now adds the new owner to the list of owners in the existing compartment.

When operating in partitioned mode, the interceptor pointcut STORAGE_PARTITION_IDENTIFY_CREATE will now be invoked to determine the partition to use for create with client-assigned IDs. Previously the STORAGE_PARTITION_IDENTIFY_READ pointcut was invoked, which was confusing and potentially unexpected.

Flyway migration used to enforce order by default. This has been changed so now the default behaviour is out of order migrations are permitted. Strict order can be enforced via the new strict-order flag if required.

Identifier maximum length increased from 200 to 500. This specifically applies to table HFJ_IDX_CMP_STRING_UNIQ.

When the contents of a package are corrupt, the error messages now identify the corrupt element

When performing a FHIR transaction containing a conditional create, references to that resource were inadvertently replaced with contained references."

A concurrency error was fixed when using client assigned IDs on a highly concurrent server with resource deletion disabled.

A null-pointer exception was fixed when a ResponseTerminologyDisplayInterceptor is registered and a search or read response returns a resource with code value that in turn returns a null code lookup.

Subscription notifications will no longer be triggered by default in response to changes that do not increment the resource version (e.g. $meta-add and $meta-delete). A new DaoConfig setting has been added to make this configurable.

When myDaoConfig.setDefaultTotalMode(SearchTotalModeEnum.ACCURATE) and there are zero search results on an _id search, An Index Out of Bounds error was thrown. This has been corrected.

Fixes the problem that FHIR package IDs were incorrectly treated as case sensitive when being loaded, causing loads to fail when dependencies were declared with a different case than in the package itself.

Constraint errors were not always auto-retried even when configured to do so on certain platforms (particularly Postgresql) where constraint names are auto converted to lower case. Thanks to Bruno Hedman for the pull request!

When searching by source, if deleted resources are matched, the search returned an incorrect size. This has been corrected.

The _filter search parameter was incorrectly included in the server capability statement if it was disabled on the server. This has been corrected.

ValueSet expansion did not correctly preserve the order if multiple codes were included in a single inclusion block.

Too many MDM candidates matching could result in an OutOfMemoryError. Candidate matching is now limited to the value of IMdmSettings.getCandidateSearchLimit(), default 10000.

A regression caused the JPA Server History operation to not return paging links in responses. This has been corrected.

Added null checks to MDM resource interceptor in order to avoid NPEs.

The SQL generated for the _profile search parameter did not use all of the columns on the tag index table, resulting on poor performance on MySQL. This has been corrected.

The internal cache for tags, concepts, and others had longer or shorter expiry times than was intended. This has been corrected.

When searching for ExplanationOfBenefit?patient=123,456, the compartment authorization interceptor was treating '123,456' as a single id rather than as a list of ids. This has been corrected.

A regression was introduced in 2760 where a READ compartment could get collapsed into a WRITE compartment. This has been corrected.

Searches for mdm-expanded references such as Observation?subject:mdm=123 were getting denied by access rules that did not recognize the :mdm suffix. This has been corrected.

$mdm-submit operation was only submitting 100 resources and then stopping. It now correctly submits all requested resources.

When providing links for placeholder creation, DaoResourceLinkResolver expects just a single 'identifier=value' param, but it can be additional data, s.a. tags, extra identifiers, etc.

When initiating a FHIR bulk export, if more than one _typeFilter parameter was supplied only the first one was respected. This has been corrected.

Loading packages would fail when partitioning was enabled with unnamed partitions. This has been fixed.

An issue in the FHIRPath evaluator prevented Encounters from being stored when using the new PatientIdPartitionInterceptor. This has been corrected.

Mdm failed to load if any mdm subscription had been deleted, e.g. if $expunge expungeEverything had been run on the server. This has been corrected.

When Client Id Strategy is set to NOT_ALLOWED, permit system requests to create resources, e.g. SearchParameter and Subscription resources required by the system.

Add DSTU3 Support To UploadTerminologyCommand.

Fixed a bug in transaction bundle processing, specifically for bundles which contained both a conditional create, and a resource which relied on this conditional create as a reference. This would cause the referring resource to generate a contained resource instead of appropriately referencing the existing patient.

Fixed a bug wherein an NPE could be thrown by the MDM module interceptor if an incoming resource had a tag with no system.

Fixed a bug where the search results cache was ignoring the value of _contained parameter when assigning a cache key. This was causing queries run in a short period of time to return wrong cached results if one query used _contained=true and the other did not.

Addressed the following CVE report by bumping the minor version for Jetty in the root POM:

• CVE-2021-34429

Support for LOINC 2.70 has been added.

The $evaluate-measure now works on a partitioned server.

A new tag storage mode called Inline Tag Mode tas been added. In this mode, all tags are stored directly in the serialized resource body in the database, instead of using dedicated tables. This has significant performance advantages when storing resources with many distinct tags (i.e. many tags that are unique to each resource, as opposed to being reused across multiple resources).

FHIR transactions in the JPA server that perform writes will now aggressively pre-fetch as many entities as possible at the very start of transaction processing. This can drastically reduce the number of round-trips, especially as the number of resources in a transaction gets bigger.

A new setting has been added to the DaoConfig called Tag Versioning Mode. This setting controls whether a single collection of tags/profiles/security labels is maintained across all versions of a single resource, or whether each version of the resource maintains its own independent collection. Previously each version always maintained an independent collection, which is useful sometimes, but is often not useful and can affect performance.

DELETE _expunge=true has been converted to use Spring Batch. It now simply returns the jobId of the Spring Batch job while the job continues to run in the background. A new operation called $expunge-delete has been added to provide more fine-grained control of the delete expunge operation. This operation accepts an ordered list of URLs to be delete expunged and an optional batch-size parameter that will be used to perform the delete expunge. If no batch size is specified in the operation, then the value of DaoConfig.getExpungeBatchSize() is used.

The ConceptMap.group.element.display storage size limit has been increased to 500 characters.

If two authorization compartments apply to the same targets and share the same compartment name, then instead of creating a new compartment, the rule builder now adds the new owner to the list of owners in the existing compartment.

Flyway migration used to enforce order by default. This has been changed so now the default behaviour is out of order migrations are permitted. Strict order can be enforced via the new strict-order flag if required.

ValueSet expansion did not correctly preserve the order if multiple codes were included in a single inclusion block.

Too many MDM candidates matching could result in an OutOfMemoryError. Candidate matching is now limited to the value of IMdmSettings.getCandidateSearchLimit(), default 10000.

A regression caused the JPA Server History operation to not return paging links in responses. This has been corrected.

Added null checks to MDM resource interceptor in order to avoid NPEs.

The SQL generated for the _profile search parameter did not use all of the columns on the tag index table, resulting on poor performance on MySQL. This has been corrected.

A regression was introduced in 2760 where a READ compartment could get collapsed into a WRITE compartment. This has been corrected.

Searches for mdm-expanded references such as Observation?subject:mdm=123 were getting denied by access rules that did not recognize the :mdm suffix. This has been corrected.

Pagination returned incorrect offset and count in the previous link of the last page when total element count was one more than multiple of page size. Problem is now fixed

Settings have been added to the JPA Server DaoConfig to enable/disable various individual kinds of scheduled tasks.

When performing a conditional create operation on a JPA server, the system will now verify that the conditional URL actually matches the data supplied in the resource body, and aborts the conditional create if it does not.

Support has been added to the JPA server for _include and _revinclude where the value is a qualified star, e.g. _include=Observation:*.

A new interceptor ValidationMessageSuppressingInterceptor has been added. This interceptor can be used to selectively suppress specific vaLidation messages.

A new setting has been added to the DaoConfig that allows the maximum number of _include and _revinclude resources to be added to a single search page result. In addition, the include/revinclue processor have been redesigned to avoid accidentally overloading the server if an include/revinclude would return unexpected massive amounts of data.

When performing non-query cache JPA searches (i.e. searches with Cache-Control: no-store) the loading of _include and _revinclude will now factor the maximum include count.

A new config option has been added to the DaoConfig that causes generated SQL statements to account for potential null values in HAPI FHIR JPA date index rows. Nulls are no longer ever used in this table after HAPI FHIR 5.3.0, but legacy data may still have nulls.

A new DaoConfig setting called Mass Ingestion Mode has been added. This mode enables rapid data ingestion by skipping a number of unnecessary checks during backloading.

A new Pointcut has been added that is invoked when a new Bulk Export is initiated.

The JPA server terminology uploader now supports uploading ICD-10-CM (US Edition) using the native format for that vocabulary.

AuthorizationInterceptor can now be used to authorize bulk export requests

FHIR Transaction duplicate record checks are now performed without any database interactions or SQL statements, reducing the processing load associated with FHIR transactions by at least a small amount.

Conditional URL lookups in the JPA server will now explicitly specify a maximum fetch size of 2, avoiding fetching more data that won't be used inadvertently in some situations.

When performing a conditional create/update/delete on a JPA server, if the match URL contained a plus character, this character was interpreted as a space (per legacy URL encoding rules) even though this has proven to not be the intended behaviour in real life applications. Plus characters will now be treated literally as a plus character in these URLs.

Bulk import batch jobs are now activated in a local scheduled task, making bulk import jobs better able to take advantage of large clusters.

When performing a FHIR transaction containing a conditional create, references to that resource were inadvertently replaced with contained references."

A concurrency error was fixed when using client assigned IDs on a highly concurrent server with resource deletion disabled.

A null-pointer exception was fixed when a ResponseTerminologyDisplayInterceptor is registered and a search or read response returns a resource with code value that in turn returns a null code lookup.

Subscription notifications will no longer be triggered by default in response to changes that do not increment the resource version (e.g. $meta-add and $meta-delete). A new DaoConfig setting has been added to make this configurable.

When myDaoConfig.setDefaultTotalMode(SearchTotalModeEnum.ACCURATE) and there are zero search results on an _id search, An Index Out of Bounds error was thrown. This has been corrected.

Fixes the problem that FHIR package IDs were incorrectly treated as case sensitive when being loaded, causing loads to fail when dependencies were declared with a different case than in the package itself.

Constraint errors were not always auto-retried even when configured to do so on certain platforms (particularly Postgresql) where constraint names are auto converted to lower case. Thanks to Bruno Hedman for the pull request!

When searching by source, if deleted resources are matched, the search returned an incorrect size. This has been corrected.

The _filter search parameter was incorrectly included in the server capability statement if it was disabled on the server. This has been corrected.

The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• Commons-Lang3 (Core): 3.9 -> 3.12.0
• Commons-Text (Core): 1.7 -> 1.9
• Commons-Codec (Core): 1.14 -> 1.15
• Commons-IO (Core): 2.6 -> 2.8.0
• Guava (Core): 30.1-jre -> 30.1.1-jre
• Jackson (Core): 2.12.1 -> 2.12.3
• Woodstox (Core): 6.2.3 -> 6.2.5
• Apache Jena (Core/RDF): 3.16.0 -> 3.17.0
• Gson (JPA): 2.8.5 -> 2.8.6
• Caffeine (JPA): 2.7.0 -> 3.0.1
• Hibernate (JPA): 5.4.26.Final -> 5.4.30.Final
• Hibernate Search (JPA): 6.0.0.Final -> 6.0.3.Final
• Spring (JPA): 5.3.3 -> 5.3.6
• Spring Batch (JPA): 4.2.3.RELEASE -> 4.3.2
• Spring Data (JPA): 2.4.2 -> 2.5.0
• Commons DBCP2 (JPA): 2.7.0 -> 2.8.0
• ElasticSearch Client (JPA): 7.10.2 -> 7.12.1
• Thymeleaf (Testpage Overlay): 3.0.11.RELEASE -> 3.0.12.RELEASE
• JAnsi (CLI): 2.1.1 -> 2.3.2
• JArchivelib (CLI): 1.0.0 -> 1.1.0

Support for the FHIR _list search parameter.

Terminology storage is now skipped for placeholder ValueSet and ConceptMap resources.

When a REST server returns a failure because no method matched the request parameters, the resulting OperationOutcome will now correctly set the issue type to not-supported. Thanks to Jari Maijenburg for the pull request!

Group Bulk exports are now supported. You can export all data for a Group of Patients via the /Group/[id]/$export endpoint for any resource type which contains a patient compartment. The _typeFilter and _since criteria are currently not supported at this level, but may eventually be

Optionally support '_contained' resource search by enabling the indexing on the contained resources in the ModelConfig.

When using the JPA server in partitioned mode with a partition interceptor, the interceptor is now called even for resource types that can not be placed in a non-default partition (e.g. SearchParameter, CodeSystem, etc.). The interceptor may return null or default in this case, but can include a non-null partition date if needed.

In MDM matching rules, support has been added for using FHIRPath expressions instead of Resource Path expressions via the fhirPath field in a field matcher

Support has been added for MDM expansion during Group bulk export. Calling a group export via /Group/123/$export?_mdm=true will cause Bulk Export to not only match group members, but also any MDM-matched patients, and their related golden record patients

Support has been added to the JPA server for indexing and searching using the _contained parameter, which allows searching using chained parameters that chain into contained resources. This feature is disabled by default but can be enabled via a setting on the ModelConfig object.

Support has been added for patient level Bulk export. This can be done via the /Patient/$export endpoint. Also, support has been added for setting Cache-Control header to no-cache for Bulk Export requests.

Auto-created placeholder reference targets now have an extension with the URL http://hapifhir.io/fhir/StructureDefinition/resource-placeholder and a value of true.

Adds interceptors for the following functionality:

• Data normalization (n11n) - removing unwanted characters (control, etc. as defined by the requirements)
• Data standardization (s13n) - normalizing data by ensuring word spacing and character cases are uniform
• Data validation - making sure that addresses / emails are validated

Added matching based on extension, when given the path to a fhir resource the matcher will take the extensions and match if the url and string value are the same

Two new server interceptors have been added that can be used to map codes and populate code display names respectively using the server terminology services.

When using the _mdm parameter during Group Bulk Export, resources written out will now contain an extension with the url https://hapifhir.org/associated-patient-golden-resource/ identifying which golden resource the target resource belongs to.

A new server interceptor has been added that allows servers to implement lenient search mode, where unknown search parameters are ignored if an optional HTTP Prefer header is provided.

The server generated CapabilityStatment will now include supported Profile declarations for FHIR R4+.

Add support for :mdm search parameter qualifier on reference search parameters. Details about enabling this feature can be found in the documentation.

The server generated CapabilityStatement now reflects whether RDF/Turtle is supported by the server. In addition, the ResponseHighlightingInterceptor will now provide some TTL support.

The automatically generated CapabilityStatement for R4+ will now incude the list of supported revinclude values.

A new Validation Support Module has been added called UnknownCodeSystemWarningValidationSupport. This module allows validation to produce a warning but not an error if a code being validated references an unknown code system.

A new optional parameter has been added to the ValueSet/$expand operation. When provided a value of true, the operation will include the concept hierarchy in the expansion response.

Add new pointcut STORAGE_TRANSACTION_PROCESSED, which fires after all operations in a transaction have executed.

It is now possible t create narrative generator templates that apply to any custom strucures including custom extension structures.

Added new NUMERIC mdm matcher for matching phone numbers. Also added NUMERIC phonetic encoder to support adding NUMERIC encoded search parameter (e.g. if searching for matching phone numbers is required by mdm candidate searching).

A new interceptor called OpenApiInterceptor has been added. This interceptor can be registered against FHIR Servers to automatically add support for OpenAPI / Swagger.

A new header called X-Upsert-Extistence-Check can now be added to JPA server Create with Client-Assigned ID operations (aka Upsert) in order to improve performance when loading data that is known to not exist by skipping the resource existence check.

When using the JPA server in offset mode, the count+offset information is now passed to the SQL query, resulting in better performance. Thanks to Tuomo Ala-Vannesluoma for the pull request!

When automatically creating a placeholder reference that is set to auto-populate identifiers, logic has been improved. If the reference does not contain an identifier, but the inline match URL does, the identifier found in the match URL will be added to the target. If both are populated, they will both be added to the target.

When resources are created using package load, the new resources will use the same IDs as were provided in with the resource definitions in the package, if they exist. If the ids are numeric, a prefix of 'npm-' will be added.

Support for validating BCP-47 (language) codes against the FHIR Languages and All-Languages ValueSets has been improved.

The MSSQL-specific index definition for the ForcedId table in the JPA server has been enhanced to include an INCLUDE() clause, which should significantly improve performance.

It is now possible to declare database migrations as being mandatory even when running in schema initialization mode.

A regression in HAPI FHIR 5.3.0 resulted in concurrent searches being executed in a sequential (and not parallel) fashion in some circumstances.

Several optimizations have been made to the JPA server transaction processor that should result in improved performance, particularly when processing large transaction Bundles, such as transactions containing many entries, or transactions containing very large entries.

The HAPI FHIR parser will now preserve the order of contained resources across round-trip parse/serialize passes.

The generated search SQL statements have been optimized for simple JPA server searches containing only one parameter. In this case, an unnecessary JOIN has been removed.

A new PID-to-forced-ID cache, and a new optional Match-URL-to-PID cache have been added. These can improve write performance when doing large loads.

DaoConfig setting for Populate Identifier In Auto Created Placeholder Reference Targets now defaults to true.

PointCutLatch now works with IPointcut instead of Pointcut.

Custom Search Parameters may now have FHIRPath expressions up to 500 characters long, up from 200.

The ConsentInterceptor no longer fully runs on calls to /metadata or during the $meta operation.

Added a new configuration option to DaoConfig, setInternalSynchronousSearchSize(), this controls the loadSynchronousUpTo() during internal operations such as delete with expunge, and certain CodeSystem searches.

An SQL syntax error was corrected when using the JPA server terminology services to pre-expand valuesets when using Oracle Database.

When storing resources in the JPA server, extensions in Resource.meta were not preserved, nor were any contents in Bundle.entry.resource.meta. Both of these things are now correctly persisted and returned. Thanks to Sean McIlvenna for reporting!

A crash was fixed when performing a FHIR read on a partitioned server, where the requested ID is not known. Thanks to Umberto Cappellini for reporting!

Supplementary characters in the parameters might cause errors when calculating hash values

A NullPointerException was corrected when indexing resources containing an indexed Period field that had a start but not an end defined.

When using the Generic Client, search invocations where the parameters are supplied using .where(Map) did not include search modifiers such as :exact in the search URL. Thanks to GitHub user @granadacoder for reporting this issue!

HasParam#doGetQueryParameterQualifier() returned a malformed modifier. For example, the modifier for _has:Observation:patient:code=123 was returned as Observation:code:123 when it should be :Observation:patient:code. This has been corrected.

References from a contained resource to the containing resource are now possible in the JPA server.

When performainfg a search using a date search parameter, invalid values (e.g. date=foo) resulted in a confusing error message. This has been improved.

Running a manual reindex of data failed on a partitioned server with an interceptor error. This has been corrected. Thanks to Ajay Shekar for reporting!

An NPE was fixed when performing highly concurrent system requests while using the ResourceVersionConflictResolutionStrategy interceptor.

the create-package command of the HAPI FHIR CLI was not correctly adding the fhirVersions section to the generated package.json. This has been fixed

A database deadlock in Postgresql was observed when uploading large terminology CodeSystems using deferred uploading. Thanks to Tyge Folke Nielsen for reporting and suggesting a fix!

An incorrect path caused the select2 library to fail to load in the HAPI FHIR testpage overlay modue. Thanks to Ari Ruotsalainen for reporting!

Fixed issues with application of survivorship rules when matching golden record to a single resource

Added Operation section to Conformance Statement.

An issue with compartment definitions in R5 models was fixed. This issue caused some authorization rules to reject valid requests. Thanks to Patrick Palacin for reporting!

When issuing a request for a specific Resource and also specifying an _include param, the referenced resource is not returned when there is only 1 version of the referenced resource available. When there are more than 1 versions available, the referenced resource is returned in the response bundle.

An issue with package installer involving logical StructureDefinition resources was fixed. Package registry will no longer attempt to generate a snapshot for logical StructureDefinition resources if one is not already provided in the resource definition.

When issuing a request for a specific Resource and also specifying an _include param, the proper historical referenced resource is not returned when there are more than 1 versions of the referenced resource available, after the reference has been changed from the original version 1 to some other version. When there are more than 1 versions available, and the referring resource had previously referred to version 1 but now refers to version 4, the resource returned in the response bundle is for version 1.

Fixed a bug which would cause Bulk Export to fail when run in a partitioned environment.

Added support for deleting resources to BundleBuilder via method addTransactionDeleteEntry.

When running the $apply-codesystem-delta-add operation, code properties were not correctly saved. Thanks to Hanan Awwad for the pull request!

In the JPA server, if a tag was defined with the exact same system and code as a security label on a different resource, the tag would be incorrectly filed as a security label (and vice versa). This has been corrected.

When creating resources in a JPA server under highly concurrent conditions, creating a new tag across multiple threads could result in a race condition leaving an invalid entry in the tag cache. This resulted in new instances of this tag being unavailable for creation.

When performing a search via GraphQL, token search parameters were not properly parsed. Thanks to Jari Maijenburg for the pull request!

Certain calls to the $evaluate-measure operation could result in a NullPointerException. This is now corrected.

A vulnerability in the FHIR History operation was resolved. When running HAPI FHIR JPA server on a large database (i.e. containing a large number of resources), if a malicious user performs a large number of concurrent FHIR History (_history) operations, an expensive COUNT() statement can consume all available database resources and ultimately trigger resource exhaustion and disable the server. A huge thanks to Zachary Minneker at Security Innovation who discovered and submitted a responsible disclosure of this issue.

When using Auto-Version references in the JPA server, if an auto-versioned reference within a FHIR transaction pointed to a resource that did not actually change during the transaction (e.g. an update/PUT where the resource body was unchanged from the existing version), the reference would point to an incremented version number even though none existed. This has been corrected.

The new Match URL cache suffered from potential cache poisoning if multiple threads performed a condiitonal create operation at the same time.

The id of ValueSet resources in ValueSet expansions was null. This has been corrected. The id of the expanded value set is now the same as the id of the value set that was expanded.

Fixed a bug where delete with expunge would throw a LazyInitializationException if there were delete conflicts, and the expunge batch size was smaller than the available list of resources to delete.

The Testpage Overlay now suppresses authorization headers from the output headers. Thanks to Tuomo Ala-Vannesluoma for the pull request!

Addressed the following CVE report by bumping the minor version for Jetty in the root POM:

• CVE-2020-27223

A NullPointerException was corrected when indexing resources containing an indexed Period field that had a start but not an end defined.

When issuing a request for a specific Resource and also specifying an _include param, the referenced resource is not returned when there is only 1 version of the referenced resource available. When there are more than 1 versions available, the referenced resource is returned in the response bundle.

When issuing a request for a specific Resource and also specifying an _include param, the proper historical referenced resource is not returned when there are more than 1 versions of the referenced resource available, after the reference has been changed from the original version 1 to some other version. When there are more than 1 versions available, and the referring resource had previously referred to version 1 but now refers to version 4, the resource returned in the response bundle is for version 1.

When using the JPA server in partitioned mode with a partition interceptor, the interceptor is now called even for resource types that can not be placed in a non-default partition (e.g. SearchParameter, CodeSystem, etc.). The interceptor may return null or default in this case, but can include a non-null partition date if needed.

The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• SLF4j (All Modules): 1.7.28 -> 1.7.30
• Jackson (All Modules): 2.11.2 -> 2.12.1
• Woodstox (XML FHIR Parser): 4.4.1 -> 6.2.3 (Note that the Maven groupId has changed from org.codehaus.woodstox to com.fasterxml.woodstox and the Maven artifactId has changed from woodstox-core-asl to woodstox-core for this library)
• Spring (JPA): 5.2.3.RELEASE -> 5.2.9.RELEASE
• Datasource-Proxy (JPA): 1.5.1 -> 1.7
• Apache Commons Collections4 (JPA): 4.3 -> 4.4
• Jetty (JPA Starter): 9.4.30.v20200611 -> 9.4.35.v20201120
• Guava (JP): 29.0-jre -> 30.1-jre
• Hibernate ORM (JPA Server): 5.4.22.FINAL -> 5.4.26.FINAL
• Spring (JPA Server): 5.2.9.RELEASE -> 5.3.3
• Spring Data (JPA Server): 2.2.0.RELEASE -> 2.4.2
• Hibernate Search (JPA Server): 5.11.5.FINAL -> 6.0.0.Final
• Lucene(HAPI FHIR JPA Server): 5.5.5 -> 8.7.0
• Spring Boot (JPA Starter): 2.2.6.RELEASE -> 2.4.1
• JANSI (CLI): 1.18 -> 2.1.1
• PH-Schematron (SCH Validator): 5.2.0 -> 5.6.5
• PH-Commons (SCH Validator): 5.3.8 -> 9.5.4

Two new switches have neen added to FhirInstanceValidator to suppress optional warning messages. Thanks to Anders Havn for the pull request!

Redesigning the Enterprise Master Patient Index solution to a Master Data Management solution. The new MDM solution supports other FHIR resources where EMPI only allowed Person resource to be used. For example, if MDM is occurring on a patient, we will create a new Patient, and tag that patient as a Golden Record. This means that several things have changed:

• Provider methods that pointed to type of Person are now server-level operations in which you specify a resource type.
• Link updating and querying no longer rely on Person IDs, but instead on arbitrary resource ids, depending on the resource type you are referring to in MDM.
• Change to the EMPI config to require a list of mdmTypes.

• hapi-fhir-server-empi and hapi-fhir-jpaserver-empi Maven projects were renamed to hapi-fhir-server-mdm and hapi-fhir-jpaserver-mdm
• All classes were refactored to use correct terms, e.g. Golden Resource in place of Person
• Message channel was renamed from empi to mdm
• Subscriptions were renamed to mdm-RESOURCE_TYPE, where RESOURCE_TYPE is an MDM type configured in mdmTypes section of the configuration file
• Configuration file was renamed from empi-rules.json to mdm-rules.json
• Log file was changed from empi-troubleshooting.log to mdm-troubleshooting.log

When a unique index SearchParameter violation is blocked, the error message will now include the ID of the relevant SearchParameter, in order to make troubleshooting easier.

Added a new IResourceChangeListenerRegistry service and modified SearchParamRegistry and SubscriptionRegistry to use it. This service contains an in-memory list of all registered {@link IResourceChangeListener} instances along with their caches and other details needed to maintain those caches. Register an {@link IResourceChangeListener} instance with this service to be notified when resources you care about are changed. This service quickly notifies listeners of changes that happened on the local process and also eventually notifies listeners of changes that were made by remote processes.

It is now possible for read operations (read/history/search/etc) in a partitioned server to read across more than one partition if the partitioning interceptor indicates multiple partitions.

Non release (i.e. SNAPSHOT) builds of HAPI FHIR will now include the Git revision hash as well as the build date in the version string that is logged on initialization, and included in the default server X-Powered-By string. Release builds are not affected by this change.

The error message returned by the transaction processor has been improved for the case where a transaction uses an unsupported/disabled resource types.

When performing a conditional create/update containing the email search parameter, any + characters will now be interpreted as actually being a plus symbol instead of being unescaped into a space character. This is technically a deviation from how URLs should be parsed, but allows for a sensible behaviour in a spot where no spaces are allowed.

It is now possible to use a parameter of type IBaseResource or IBaseBundle as the parameter on a @Transaction method in a plain server.

Optionally supports storage and search in canonical form of the quantity value which is defined by 'http://unitsofmeasure.org'; please check ModelConfig for the configuration. No changes were made to the existing behaviour.

A new interceptor called the Repository Validating Interceptor has been added. This new interceptor allows a HAPI FHIR JPA Server to declare rules about mandatory profiles that all resources stored to the server must declare conformance and/or correctly validate against.

The interceptor framework will now recognize and invoke @Hook methods that have an access level of public, protected, or default. Previously only public methods were recognized.

When performing a ValueSet expansion where the valueset to be expanded includes a display name for concepts it is explicitly including, this display name will be propagated to the expansion if no other display is available. Thanks to Hanan Awwad for the contribution!

A new utility called FHIRPathResourceGeneratorR4 has been added. This class can be used to build and populate a resource model object using FHIRPath expressions. Thanks to Marcel P for the contribution!

The JPA server has a new setting on the ModelConfig bean called "AutoVersionReferencesAtPaths". Using this setting, the server can be configured to add the current target resource version ID to any resource references found in a resource being stored. In addition, a new setting has been added to the JPA ModelConfig bean that allows _include statements to respect versioned references, and actually include the correct version for the reference.

Added support for the $evaluate-measure Operation as part of adding CQL support.

The JPA server generated SQL for date search parameters has been streamlined to avoid the use of redundant OR expressions that slow down performance in some cases.

Updates to Hibernate Search require a full reindexing of all indexed fulltext data, which is held in Lucene or Elasticsearch. Users using elasticsearch for fulltext indexing must upgrade to Elasticsearch 7.10.0.

The resource IDs for several LOINC resources were corrected. Thanks to Steven Wagers for the pull request!

The experimental TransactionBuilder helper class has been renamed to BundleBuilder as it now has utility methods for working with other types of Bundles too.

In the JPA server. the SQL datatype used to index quantities has been changed from NUMBER(19,2) to double precision (or equivalents depending on platform). This improves the query support for ssearching on very small quantities.

The @ResourceDef annotation has been marked as inheritable, so it does not need to be explicitly added to custom structures that extend built-in structures. Thanks to Marcelo Avancini for the pull request!

ElasticsearchHibernatePropertiesBuilder will now reject any REST url which is protocol-aware. Protocol information should be set in the protocol field of the builder.

Expanding a ValueSet using a filter now evaluates the display with left-matching by string token, case-insensitive.

Loading a package without a description was causing a null pointer exception. This has been fixed.

The CodeSystem/$subsumes operation inadvertantly reversed the meanings of the CodeA and CodeB parameters, resulting in subsumes and subsumed-by responses being reversed. This has been corrected. Thanks to Rob Hausam for reporting!

ApacheRestfulClientFactory now uses system properties for proxy configuration. Thanks to Vladimir Nemergut for the pull request!

When performing a ValueSet expansion with a filter a large pre-expanded ValueSet (more than 1000 codes), the filter failed to find concepts appearing after the first thousand. This has been corrected.

The new JPA SearchBuilder failed to perform FHIR Searches on Oracle DB with an invalid SQL error. This has been corrected.

When performing a JPA server search on a partitioned server, searches with only the _id parameter and no other parameters did not include the partition selector in the generated SQL, resulting in leakage across partitions. Thanks to GitHub user @jtheory for reporting!

The recent EMPI project accidentally caused a split package in ca.uhn.fhir.rest.server. This has been corrected. Thanks to Bill Denton for the pull request!

When using the validator from within the JPA server, validating CapabilityStatement resources failed with an error when trying to load linked SearchParameter resources. This has been corrected.

When using a partitioned JPA server, auto-create placeholder targets did not work if the partition interceptor was registered against the server (e.g. for a multitenancy configuration). This has been corrected. Thanks to Rob Whelan for reporting!

Sorting of search results was not working for MySQL, MSSQL and MariaDB due to recent changes made to handle sorting of nullable columns. This has now been fixed.

The new optimized SQL Generator introduced in HAPI FHIR 5.2.0 did not correctly bind variables for SQL Server queries, making the search functionality unusable. This has been corrected.

A database index in the JPA server was added in HAPI FHIR 5.2.0 and Smile CDR 2020.11 that exceeded the maximum index length in MySQL, preventing server upgrades on that database platform. This has been corrected.

Attempts to load IG packs when partitioning was enabled, resulted in nullpointer exceptions. This has now been fixed and IG packs and conformance resources will be loaded to the DEFAULT partition.

An incorrect HTML resource path led to a utility JavaScript library failing to load in the TestPage Overlay module. Thanks to Alejandro Medina for the pull request!

When performing a FHIR create using the HAPI FHIR client, if the payload is a Bundle resource the individual resources in the Bundle had their IDs removed by the client during payload serialization. This has been corrected.

A recent change inadvertently caused an issue with DB migration utility such that it would attempt to drop indexes even when the dry-run option was specified. This has been fixed.

The BulkExportDaoSvc bean was forgotten from the BaseConfig context. It has been added.

In the JPA server, HumanName.name.text was not being indexed and therefore was not searchable. This has been corrected.

The $expand filter parameter was not matching the ValueSet display value in all cases. E.g. a ValueSet with name 'abc def ghi' would match 'abc def' and 'def' but not 'def ghi'. This has been corrected so the ValueSet will match the filter if any substring of the ValueSet display value matches the $expand filter.

Unrecognized search param prefix strings were silently discarded. This is now an error.

A bug in the InMemoryResourceMatcher caused AND clauses to be treated as OR clauses. Thanks to Jari Maijenburg for the report and pull request!

When using the Consent Interceptor, the startOperation method was not invoked for search paging requests. This has been corrected. Thanks to Tue Toft Nørgård for reporting!

As of version 2.69, the LOINC Top2000CommonLabResultsSi.csv and Top2000CommonLabResultsUs.csv became optional. The Terminology Loader Service has been updated to reflect this change.

An important security issue with the JPA Server was solved. This issue applies only to JPA servers running in partitioned mode. When performing searches on a partitioned server, search results from previously cached searches against different partitions may be returned, potentially leaking data across partitions. This issue has been resolved.

Remove support for the upload-igpack command. This command is no longer supported, as the IGPack format has been withdrawn and replaced with the FHIR NPM Package specification, which is also supported by HAPI FHIR

A recent change inadvertently caused an issue with DB migration utility such that it would attempt to drop indexes even when the dry-run option was specified. This has been fixed.