The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• SLF4j (All): 1.7.33 -> 2.0.3
• Logback (All): 1.2.10 -> 1.4.4
• Woodstox-Core (XML Parser): 6.3.1 -> 6.4.0
• log4j-to-slf4j (JPA): 2.17.1 -> 2.19.0
• Jetty (CLI): 9.4.48.v20220622 -> 10.0.12
• Spring Boot: 2.7.4 -> 2.7.5
• Graphql-Java (OpenAPI): 17.4 -> 19.2

• javax.annotation:javax.annotation-api:1.3.2 -> jakarta.annotation:jakarta.annotation-api:1.3.3 – This change brings HAPI FHIR up to the new permanent Maven dependency name for this library. Note that this new version does not change the actual package structure of the included classes from javax. to jakarta., it is only a change in the Maven packaging.
• javax.transaction:javax.transaction-api:1.3.2 -> jakarta.transaction:jakarta.transaction-api:1.3.5 – This change brings HAPI FHIR up to the new permanent Maven dependency name for this library. Note that this new version does not change the actual package structure of the included classes from javax. to jakarta., it is only a change in the Maven packaging.

Extend $member-match to store Consent resource when there is a matching patient.

In-memory caching provided by Caffeine has been refactored into using s ServiceLoader pattern in order to improve Android compatibility. Thanks to Vitor Pamplona for the pull request!

When indexing CodeSystem content in the terminology service, an unintended busy-wait cycle sometimes caused the CPU to be thrashed while waiting for a batch job to complete. This has been resolved.

In some cases in the JPA server, CodeSystem updating failed and left the server with a hung reindex job that never completes. This has been corrected. Thanks to GitHub user @tyfoni-systematic for the bug report!

The internal LinkedChannelQueue implementation used for Batch2 processing and Subscription delivery now uses automated retries if a message processing failure occurs in order to improve resiliency.

When validating terminology as a part of resource validation or by directly calling the $validate-code operations, an incorrect caching routine sometimes caused incorrect cached results to be returned in cases where display names are being validated and multiple display names were validated for the same system+code combination. This has been corrected.

The HFJ_RESOURCE table has a new column FHIR_ID which always contains the FHIR resource id, both client-assigned and server-assigned. This is parallel storage to allow gradual upgrade away from the current HFJ_FORCED_ID join table, which should improve performance in a future release.

The JPA server can now handle indexing search parameters of type token and reference where the value is a CodeableReference (a new datatype added in R5).

The JPA server can now handle indexing search parameters of type number where the value is a Range, e.g. RiskAssessment.probability.

The NPM package installer did not support installing on R4B repositories. Thanks to Jens Kristian Villadsen for the pull request!

When updating resources using a FHIR transaction in the JPA server, if the client instructs the server to include the resource body in the response, any tags that have been carried forward from previous versions of the resource are now included in the response.

The BundleBuilder now supports adding conditional DELETE operations, PATCH operations, and conditional PATCH operations to a transaction bundle.

When performing create/update/patch/delete operations against the JPA server, the response OperationOutcome will now include additional details about the outcome of the operation. This includes:

• For updates, the message will indicate the the update did not contain any changes (i.e. a No-op)
• For conditional creates/updates/deletes, the message will indicate whether the conditional URL matched any existing resources and the outcome of the operation.
• A new coding has been added to the OperationOutcome.issue.details.coding containing a machine processable equivalent to the outcome.

The JPA server implementation of the $process-message operation has been extracted into a standalone provider called ProcessMessageProvider which can be registered individually on servers that have provided an implementation of the backing operation. Because this operation can not be implemented in a generic was in HAPI FHIR (it assumes business-specific processing logic) it does not make sense to include it by default.

Before DB table joins were LEFT OUTER by default, which was causing some queries to skip table indexes, making them very slow. This was more noticeable in H2 database, but affected all DB types in different measures. This has been fixed by using INNER joins, unless specific use case requires otherwise.

Before, When exporting a group including patients referenced in observations, observations were missed from output if Search Parameters Indexing was enabled. This is now fixed.

When using the client LoggingInterceptor with OkHttp clients, the request URL was not correctly logged. Thanks to Roel Scholten for the pull request!

Previously, when creating a bundle, if multiple resources matched the conditional URL provided in the request, it would process the request using the last resource found as a match. An error is now thrown when multiple resources are found

Previously, an ExecutionException was thrown when $expunge operations were used with a small batch size and a low thread count due to a race condition involving the expunge limit. This has been corrected.

The ConceptMap/$translate operation did not work for transation against a ConceptMap resource ID if the ID was non-numeric. This has been fixed. Thanks to Panayiotis Savva for the report!

A race condition in the Cache factory has been resolved. This issue could cause strange errors such as NoSuchElementException if caches are created in multiple threads.

The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• Woodstox (Base): 6.2.5 -> 6.3.1
• Spring (JPA): 5.3.20 -> 5.3.23
• Hibernate-Core (JPA): 5.6.2 -> 5.6.12
• Postgresql JDBC (JPA): 42.4.1 -> 42.5.0
• H2 (JPA): 2.1.212 -> 2.1.214
• Caffeine (JPA): 2.9.1 -> 3.1.1
• Commons-Text (JPA and Testpage Overlay): 1.9.0 -> 1.10.0 (Addresses CVE-2022-42889)
• Spring Boot (Boot): 2.6.7 -> 2.7.4
• Jackson Databind: 2.13.2.2 -> 2.13.4.1
• Snakeyaml : 1.31 -> 1.33
• Graphql-Java : 17.3 -> 17.4

Added a new optional parameter to the upload-terminology operation of the HAPI-FHIR CLI. you can pass the -s or --size parameter to specify the maximum size that will be transmitted to the server, before a local file reference is used. This parameter can be filled in using human-readable format, for example: upload-terminology -s "1GB" will permit zip files up to 1 gigabyte, and anything larger than that would default to using a local file reference.

Advanced Lucene indexing now fully supports composite SearchParameters.

Previously, DELETE request type is not supported for any operations. DELETE is now supported, and is enabled for operation $export-poll-status to allow cancellation of jobs

Provided the ability to have the NPM package installer skip installing a package if it is already installed and matches the version requested. This can be controlled by the reloadExisting attribute in PackageInstallationSpec. It defaults to true, which is the existing behaviour. Thanks to Craig McClendon (@XcrigX) for the contribution!

Added support for AWS OpenSearch to Fulltext Search. If an AWS Region is configured, HAPI-FHIR will assume you intend to connect to an AWS-managed OpenSearch instance, and will use Amazon's DefaultAwsCredentialsProviderChain to authenticate against it. If both username and password are provided, HAPI-FHIR will attempt to use them as a static credentials provider.

When using ForcedOffsetSearchModeInterceptor, any synchronous searches initiated programmatically (i.e. through the internal java API, not the REST API) will not be modified. This prevents issues when a java call requests a synchronous search larger than the default offset search page size.

Previously, Patient Bulk Export only supported endpoint [fhir base]/Patient/$export, which exports all patients. Now, Patient Export can be done at the instance level, following this format: [fhir base]/Patient/[id]/$export, which will export only the records for one patient. Additionally, added support for the patient parameter in Patient Bulk Export, which is another way to get the records of only one patient.

Added support for the :text modifier on string search parameters. This corrects an issue when using Elastic/Lucene indexing enabled where prefix match was used instead.

LOINC terminology upload process was enhanced to consider 24 additional properties which were defined in loinc.csv file but not uploaded.

A new built-in server interceptor called the InteractionBlockingInterceptor has been added. This interceptor allows individual operations to be included/excluded from a RestfulServer's exported capabilities.

The OpenApi generator now allows additional CSS customization for the Swagger UI page, as well as the option to disable resource type pages.

Previously, Group Bulk Export did not support the inclusion of resources referenced in the resources in the patient compartment. This is now supported.

LOINC terminology upload process was enhanced by loading MAP_TO properties defined in MapTo.csv input file to TermConcept(s).

Added new attribute for the @Operation annotation to define the operation's canonical URL. This canonical URL value will populate the operation definition in the CapabilityStatement resource.

Previously, the number of resources per binary file in bulk export was a static 1000. This is now configurable by a new DaoConfig property called 'setBulkExportFileMaximumCapacity()', and the default value is 1000 resources per file.

A new interceptor pointcut STORAGE_TRANSACTION_PROCESSING has been added. Hooks for this pointcut can examine and modify FHIR transaction bundles being processed by the JPA server before processing starts.

Extend $member-match to validate matched patient against family name and birthdate

For SNOMED CT, upload-terminology now supports both Canadian and International edition's file names for the SCT Description File

Support has been added for FHIR R4B (4.3.0). See the R4B Documentation for more information on what this means.

By default, if the $export operation receives a request that is identical to one that has been recently processed, it will attempt to reuse the batch job from the former request. A new configuration parameter has been introduced that disables this behavior and forces a new batch job on every call.

$mdm-submit can now be run as a batch job, which will return a job ID, and can be polled for status. This can be accomplished by sending a Prefer: respond-async header with the request.

A remove method has been added to the Batch2 job registry. This will allow for dynamic job registration in the future.

Added new System Property called 'CLEAR_LOCK_TABLE_WITH_DESCRIPTION' that when set to the uuid of a lock record, will clear that lock record before attempting to insert a new one.

When using SearchNarrowingInterceptor, FHIR batch operations with a large number of conditional create/update entries exhibited very slow performance due to an unnecessary nested loop. This has been corrected.

Processing for _include and _revinclude parameters in the JPA server has been streamlined, which should improve performance on systems where includes are heavily used.

All Spring Batch dependencies and services have been removed. Async processing has fully migrated to Batch 2.

The Partition Interceptor has been refactored out of the JPA module in order to facilitate future use in other modules. No functional changes have been made.

Removed Flyway database migration engine. The migration table still tracks successful and failed migrations to determine which migrations need to be run at startup. Database migrations no longer need to run differently when using an older database version.

Changed Minimum Size (bytes) in FHIR Binary Storage of the persistence module from an integer to a long.

Cascading deletes don't work correctly if multiple threads initiate a delete at the same time. Either the resource won't be found or there will be a collision on inserting the new version. This changes fixes the problem by better handling these conditions to either ignore an already deleted resource or to keep retrying in a new inner transaction..

With Elasticsearch configured, including terminology, an exception was raised while expanding a ValueSet with more than 10,000 concepts. This has now been fixed.

Initial page loading has been optimized to reduce the number of prefetched resources. This should improve the speed of initial search queries in many cases.

Previously, Celsius and Fahrenheit temperature quantities were not normalized. This is now fixed. This change requires reindexing of resources containing Celsius or Fahrenheit temperature quantities.

With Elastic/Lucene indexing enabled search for numbers or quantities was not always using the specified ranges, because the number of significant figures was not properly calculated. This is now fixed.

When serializing references where the reference target has been instantiated using a resource object and not a reference string, the serialization was omitted when the reference appeared in the resource metadata section. This has been corrected.

Providing a meaningful response message when deleting non-existing or already deleted resources.

Previously, unexpected response status code is not handled by the import-poll-status operation. Now, it is fixed and throwing an error message.

During bulk import, fetching resource files from a client server would fail if the server response specified a content-type that was not application/fhir+json. Validation has been loosened to accept content-type values of text/plain and application/json.

Fixing bug where searching with a target resource parameter (Coverage:payor:Patient) as value to an _include parameter would fail with a 500 response.

Previously, if the Endpoint Base URL is set to something different from the default value, the URL that export-poll-status returned is incorrect. After correcting the export-poll-status URL, the binary file URL returned is also incorrect. This error has also been fixed and the URLs that are returned from $export and $export-poll-status will not contain the extra path from 'Fixed Value for Endpoint Base URL'.

Previously, when a client would provide a requestId within the source uri of a Meta.source, the provided requestId would get discarded and replaced by an id generated by the system. This has been corrected. And now it depends on configuration.

Previously, using the import-csv-to-conceptmap command in the CLI successfully created ConceptMap resources without a ConceptMap.status element, which is against the FHIR specification. This has been fixed by adding a required option for status for the command.

Fast-tracking batch jobs that produced only one chunk has been rewritten to use Quartz triggerJob. This will ensure that at most one thread is updating job status at a time. Also jobs that had FAILED, ERRORED, or been CANCELLED could be accidentally set back to IN_PROGRESS; this has been corrected.

Previously, if a FullTextSearchSvcImpl was defined, but was disabled via configuration, there could be data loss when reindexing due to transaction rollbacks. This has been corrected. Thanks to @dyoung-work for the fix!

Previously, the :nickname qualifier only worked with the predefined name and given SearchParameters. This has been fixed and now the :nickname qualifier can be used with any string SearchParameters.

A regression was introduced in 6.1.0 which caused bulk export jobs to not default to the correct output format when the _outputFormat parameter was omitted. This behaviour has been fixed, and if omitted, will now default to the only legal value application/fhir+ndjson.

Previously, bulk export for Group type with _typeFilter did not apply the filter if it was for the patients, and returned all members of the group. This has now been fixed, and the filter will apply.

Previously when ValueSets were pre-expanded after loinc terminology upload, expansion was failing with an exception for each ValueSet with more than 10,000 properties. This problem has been fixed. This fix changed some freetext mappings (definitions about how resources are freetext indexed) for terminology resources, which requires reindexing those resources. To do this use the reindex-terminology command.

Added support for -l parameter for providing a local validation profile in the HAPI FHIR CLI.

Previously, when executing a '[base]/_history' search, '_since' and '_at' shared the same behaviour. When a user searched for the date between the records' updated date with '_at', the record of '_at' time was not returned. This has been corrected. '_since' query parameter works as it previously did, and the '_at' query parameter returns the record of '_at' time.

In HAPI-FHIR 6.1.0, a regression was introduced into bulk export causing exports beyond the first one to fail in strange ways. This has been corrected.

Previously, creating a DSTU3 SearchParameter with an expression that does not start with a resource type would throw an error. This has been corrected.

When running HAPI FHIR inside intellij with runtime Nonnull assertions enabled, a sequencing issue in OperationMethodBinding could cause a null pointer exception. This has been corrected.

Fixed a Group Bulk Export bug in which the group members would not be expanded correctly.

Fixed a Group Bulk Export bug which was causing it to fail to return resources due to an incorrect search.

Fixed a bug in Group Bulk Export where the server would crash in oracle due to too many clauses.

Fixed a bug in Group Bulk Export: If a group member was part of multiple groups , it was causing other groups to be included during Group Bulk Export, if the Group resource type was specified. Now, when doing an export on a specific group, and you elect to return Group resources, only the called Group will be returned, regardless of cross-membership.

Previously, when the upload-terminology command was used to upload a terminology file with endpoint validation enabled, a validation error occurred due to a missing file content type. This has been fixed by specifying the file content type of the uploaded file.

Search for strings with :text qualifier was not performing advanced search. This has been corrected.

There was a bug in content-type negotiation when reading Binary resources. Previously, when a client requested a Binary resource and with an Accept header that matched the contentType of the stored resource, the server would return an XML representation of the Binary resource. This has been fixed, and a request with a matching Accept header will receive the stored binary data directly as the requested content type.

Fixed the $poll-export-status endpoint so that when a job is complete, this endpoint now correctly includes the request and requiresAccessToken attributes.

When enabling validation on a fhir_endpoint module, a bundle with a Measure resource referencing a Library resource or a MeasureReport resource referencing a Measure, validation will fail with an IllegalArgumentException complaining about the Library or Measure resource, respectively. The fix ensures that Library and Measure resources are handled correctly and all of validation can proceed with a report of all success and errors.

Fixed a bug where the $everything operation on Patient instances and the Patient type was not correctly propagating the transactional semantics. This was causing callers to not be in a transactional context.

Fixed a bug where /Patient/$export would fail if _type=Patient parameter was not included.

Modified BinaryAccessProvider to use a safer method of checking the contents of an input stream. Thanks to @ttntrifork for the fix!

A bug prevented the DSTU2 JPA server Conformance.implementation.description field from being populated in some cases. This has been corrected.

Previously, the error segment of the $poll-export-status operation was missing. This has now been added.

Filter by range for numeric extension does not work after update. hfj_spidx_number record deleted after PUT update on SearchParameter value for Resource

A previous fix resulted in Bulk Export files containing mixed resource types, which is not allowed in the bulk data access IG. This has been corrected.

A previous fix resulted in Bulk Export files containing duplicate resources, which is not allowed in the bulk data access IG. This has been corrected.

Fixed issue where adding a sort parameter to a query would return an incomplete result set.

MDM messages were using the resource id as a message key when it should be using the EID as a partition hash key. This could lead to duplicate golden resources on systems using Kafka as a message broker.

Previously, enabling 'indexMissingFields' and 'advancedHSearchIndexing' functionalities would cause creating a Patient resource to return HTTP 500. Upon further investigation, the same happened for creating an Observation. This has been fixed.

In the JPA server, when a resource is being updated, the response will now include any tags or security labels which were not present in the request but were carried forward from the previous version of the resource.

In the JPA server, when deleting a resource the associated tags were previously deleted even though the FHIR specification states that tags are independent of FHIR versioning. After this fix, resource tags and security labels will remain when a resource is deleted. They can be fetched using the $meta operation against the deleted resource, and will remain if the resource is brought back in a subsequent update.

A transaction scoping bug in Batch2 has been resolved, preventing a crash on Postgres. We have also standardized on the Spring @Transactional annotation, and removed use of the equivalent javax.transaction annotation.

Two issues with reverse chaining (i.e. the _has search parameter) have been addressed: * Searching for a reverse chain with a target search parameter of _id did not work correctly, e.g. Patient?_has:Observation:subject:_id=Patient/123 * Searching with a combination of a forward chain and a reverse chain did not work correctly if indexing contained resources was enabled, e.g. Observation?subject._has:Group:member:_id=Group/123

Upon hitting a subscription delivery failure, we currently log the failing payload which could be considered PHI. Resource content is no longer written to logs on subscription failure.

Documentation was added for reindex-terminology command.

Previously when updating a phonetic search parameter, any existing resource will not have its search parameter String updated upon reindex if the normalized String is the same letter as under the old algorithm (ex JN to JAN). Searching on the new normalized String was failing to return results. This has been corrected.

Loading us-core IG was raising UnprocessableEntityException: HAPI-2131: Can't process submitted SearchParameter as it is overlapping an existing one. This problem has been fixed.

When storing a blob, the database blob binary storage service may store the blob size as being much smaller than the actual blob size. This issue has been fixed.

When triggering a batch export, the number of resources processed is returned as zero. This is now fixed to return the total number of resources processed across a single or multiple bundles.

Previously, performing $validate operation on resource update and sending the resource as a parameter in the request body would result in an error stating that the resource was missing id. This has been fixed by allowing the resource to be processed correctly when sent as a parameter.

Support has been added for clustered database upgrades in the new (non-FlyWay) migrator codebase. Only one database migration is permitted at once.

When DaoConfig UpdateWithHistoryRewriteEnabled is enabled, the package loader throws a NullPointerException. This has been corrected.

Previously, when using _offset, the queries will result in short pages, and repeats results on different pages. This has now been fixed.

Bulk Group export was failing to export Patient resources when Client ID mode was set to: ANY. This has been fixed

Previously to improve performance, if the total number of resources was less than the _getpageoffset, the results would default to last resource offset. This is especially evident when requests are consecutive resulting in one entry being displayed in some requests. This issue is now fixed.

Generating Bundle with resources was setting entry.request.url as absolute url when it should be relative. This has been fixed

Performing a bulk export with an _outputParam value encoded in a GET request URL that contains a '+' (ex: 'application/fhir+ndjson') will result in a 400 because the '+' is replaced with a ' '. After this fix the '+' will remain in the parameter value.

Previously during Bulk Export, if no _type parameter was provided, an error would be thrown. This has been changed, and if the _type parameter is omitted, Bulk Export will default to all registered types.

Fixed a bug which caused a failure when combining a Consent Interceptor with version conversion via the Accept header.

Previously, when creating a DocumentReference with an Attachment containing a URL over 254 characters an error was thrown. This has been corrected and now an Attachment URL can be up to 500 characters.

Batch2 jobs were incorrectly prevented from transitioning from ERRORED to COMPLETE status.

Previously, Bulk Export jobs were always reused, even if completed. Now, jobs are only reused if an identical job is already running, and has not yet completed or failed.

Previously, SearchParameters with identical codes and bases could be created. This has been corrected. If a SearchParameter is submitted which is a duplicate, it will be rejected.

Previously, if the $reindex operation failed with a ResourceVersionConflictException the related batch job would fail. This has been corrected by adding 10 retry attempts for transactions that have failed with a ResourceVersionConflictException during the $reindex operation. In addition, the ResourceIdListStep was submitting one more resource than expected (i.e. 1001 records processed during a $reindex operation if only 1000 Resources were in the database). This has been corrected.

Database migration steps were failing with Oracle 19C. This has been fixed by allowing the database engine to skip dropping non-existent indexes.

The ActionRequestDetails class has been dropped (it has been deprecated since HAPI FHIR 4.0.0). This class was used as a parameter to the SERVER_INCOMING_REQUEST_PRE_HANDLED interceptor pointcut, but can be replaced in any existing client code with RequestDetails. This change also removes an undocumented behaviour where the JPA server internally invoked the SERVER_INCOMING_REQUEST_PRE_HANDLED a second time from within various processing methods. This behaviour caused performance problems for some interceptors (e.g. SearchNarrowingInterceptor) and no longer offers any benefit so it is being removed.

The interceptor system has now deprecated the concept of ThreadLocal interceptors. This feature was added for an anticipated use case, but has never seen any real use that we are aware of and removing it should provide a minor performance improvement to the interceptor registry.

MDM messages were using the resource id as a message key when it should be using the EID as a partition hash key. This could lead to duplicate golden resources on systems using Kafka as a message broker.

Added support for AWS OpenSearch to Fulltext Search. If an AWS Region is configured, HAPI-FHIR will assume you intend to connect to an AWS-managed OpenSearch instance, and will use Amazon's DefaultAwsCredentialsProviderChain to authenticate against it. If both username and password are provided, HAPI-FHIR will attempt to use them as a static credentials provider.

In HAPI-FHIR 6.1.0, a regression was introduced into bulk export causing exports beyond the first one to fail in strange ways. This has been corrected.

Fixed a Group Bulk Export bug in which the group members would not be expanded correctly.

Fixed a Group Bulk Export bug which was causing it to fail to return resources due to an incorrect search.

Fixed a bug in Group Bulk Export where the server would crash in oracle due to too many clauses.

Fixed a bug in Group Bulk Export: If a group member was part of multiple groups , it was causing other groups to be included during Group Bulk Export, if the Group resource type was specified. Now, when doing an export on a specific group, and you elect to return Group resources, only the called Group will be returned, regardless of cross-membership.

A regression was introduced in 6.1.0 which caused bulk export jobs to not default to the correct output format when the _outputFormat parameter was omitted. This behaviour has been fixed, and if omitted, will now default to the only legal value application/fhir+ndjson.

Previously, bulk export for Group type with _typeFilter did not apply the filter if it was for the patients, and returned all members of the group. This has now been fixed, and the filter will apply.

The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• 9.4.44.v20210927 -> 9.4.48.v20220622 (Addresses CVE-2022-2047)

Refactored mdm classes to use ResourcePersistenceId rather than prematurely converting to longs

Previously, the http://hapifhir.io/fhir/StructureDefinition/subscription-send-delete-messages extension on REST-HOOK subscription channel element was only valid for R4. This has been expanded to support DSTU3 and DSTU2.

The SearchParameterMap query normalizer did not support _include and _revinclude parameters with the :recurse qualifier. This has been corrected. Thanks to Augustas Vilčinskas for the PR!

Experimental support for CockroachDB has been added to the JPA server. Thanks to Joe Shook for the contribution!

The ValueSet/$expand operation can now optionally support the displayLanugage parameter. Thanks to Gjergj Sheldija for the pull request!

Add support for the :in and :not-in qualifiers for use in SMART on FHIR v2 granular scope definition.

If a gated Batch job step and all prior steps produced only a single chunk of work, notify the next step immediately instead of waiting for the scheduled job maintenance to advance it.

Extended Lucene/Elasticsearch indexing of search parameters now supports _tag, _profile and _security.

Currently neither HAPI-FHIR nor the FHIR specification supports modifying historical version of a resource. This implementation adds the feature Update with History Rewrite to HAPI-FHIR. It can be accessed via the IGenericClient's updateHistoryRewrite method

The rule system has been extended to support optional fhir-expressions that narrow the application of a rule. A matching RuleFilteringConsentService implements post-query support for this filtering using an in-memory matcher.

Added index for table MPI_LINK on attributes MATCH_RESULT, TARGET_PID and VERSION

Added support to Bulk Export for FHIR Response Terminology Translation. It will use the same mappings as the Translation Interceptor.

Adding some test cases for CQL measures on immunization as well as some testing methods to support easier changes of CQL in unit tests.

If provided, the practitioner parameter for $evaluate-measure is now also be passed on to measures of type population (not only for type subject-list).

Added support for loading the International version of ICD-10. Thanks to kaicode for the contribution!

Added operations to batch2 api for searching job instances.

Batch2 job definitions can now optionally provide an error handler callback that will be called when a job instance fails, errors or is cancelled.

JPA server patch operation has been moved to the hapi-fhir-storage project, for reuse in other backends.

Previously, the HAPI FHIR CLI commands that made HTTP requests could only be used for HTTP endpoints. This feature adds support for HTTPS endpoints by using TLS authentication for ExampleDataUploader, ExportConceptMapToCsvCommand, ImportCsvToConceptMapCommand, ReindexTerminologyCommand and UploadTerminologyCommand.

The InMemoryMatcher used by Subscription matching not supports the token :not modifier. The :not-in modifier was also corrected for cases of multiple values in a , separated or-list.

Adding pointcut MDM_BEFORE_PERSISTED_RESOURCE_CHECKED allowing for the customization of source resource before MDM processing.

Updating the IJobCoordinator API to have a pageable request.

The 'SUBSCRIPTION_BEFORE_MESSAGE_DELIVERY' pointcut now supports a new parameter, ResourceModifiedJsonMessage. This permits interceptor implementers to modify the outgoing envelope before it is sent off.

Providing a specialized exception encapsulating the mechanism for reporting TokenParam validation failure when searching resources on parameter _tag and _security. The new exception was introduced primarily for reusability purposes.

Add Sort parameter to IJobCoordinator pageable request for fetching all jobs.

Added support for the _type parameter has been added when using the $everything operation.

Chained searches have been sped up for configurations where the Index Contained Resources feature is enabled.

Converted mdm clear batch job from Spring Batch to Batch2

When the Mark Resources for Reindexing after SearchParameter change configuration parameter is enabled, SMILE will use the Batch 2 framework to perform the reindexing operation.

The IAuthRuleTester api has changed and now uses a parameter object for flexibility.

The Delete Expunge operation has been moved from Spring Batch to Batch 2.

The Legacy Search Builder has been removed in favour of the new Search Builder. The DaoConfig will retain its setter for deprecation purposes, but that will also be removed after the next release.

Fixed an issue where FindCandidateByLinkSvc still uses long to store persistent ids rather than ResourcePersistentId, also fixed an issue where MdmLinkCreateSvc was not setting the resource type of MdmLinks properly

Previously, R5 Appointment resources would fail to save in JPA servers due to a path splitting problem during Search Parameter extraction. This has been corrected.

Previously, PATCH operations that were contained in a transaction bundle would not return response entries. This has been corrected.

Search with tag parameters using :below qualifier was not working. This is now fixed.

Previously, the $expunge operation at the system level was always available, regardless of configuration parameter settings. Now, the system-level $expunge operation requires that the Expunge Enabled parameter is enabled. Additionally, the expungeEverything option of the operation requires that the Allow Multiple Delete Enabled parameter is enabled.

Nickname matching only matched formal names to a nickname. This has been broadened to match nicknames to other nicknames for the same formal name.

Previously, the RuleBuilder's rules surrounding Group Bulk Export would return failures too early in the case of multiple permissions. This has been corrected, and the rule will no longer prematurely return a DENY verdict, instead opting to delegate to future rules.

Removed a strict dependency on a FullTextSVC. This was causing HAPI to fail to boot when Lucene/ElasticSearch was not enabled.

MS SQL Standard Edition does not support the ONLINE=ON feature for index creation, and failed during upgrades to 6.0. Upgrades now detect these limitations and avoid this feature when unsupported.

Fhir patch operation returning 500 when patching in a transaction with a resource query in property request.url.

Previously subscriptions in a partition with the id null will be matched against incoming resources from all partitions. Changed to subscriptions will only match against incoming resources in the partition the subscription exists in unless cross partition subscription is enabled and the subscription has the appropriate extension.

A change in H2's new version caused resources >1mb in size to fail to correctly store. This has been corrected. Thanks to Patrick Werner for the report and pull request!

Moved the http://hapifhir.io/fhir/StructureDefinition/subscription-send-delete-messages from the Subscription to the Subscription's Channel element.

hapi now populates issue.detail with the messageID from the core validator, org.hl7.fhir.core was updated to the latest version (5.6.48) Thanks to Patrick Werner for the feature request and pull request!

Previously, deleted resources with client generated ids were being included in the bundle total when searching by _id. This has been corrected by adding functionality to optionally filter out deleted resources when resolving forced ids to persistent ids.

Previously, Group Bulk Export would expand into other groups, due to the nature of Group.member being part of the patient search compartment. This has been fixed, and now, Group Bulk Export will only ever export the Group resource specified in the request, regardless of patient membership.

Previously, using the http://hapifhir.io/fhir/StructureDefinition/subscription-send-delete-messages extension on a subscription, along with a subscription that used a Search Expression criteria (e.g. Patient?gender=male) would cause a failure during an attempt to send the delete. This has been corrected.

Codes with the Seventh Character will be dynamically generated while uploading the ICD10-CM diagnosis code file, and the corresponding extended description can be extract using the code.

Changing the Unknown resource type error message to include only resource provider classes and exclude plain providers classes.

Server was blocking updates with Forbidden 403 that included a resource version in the url without the X-Rewrite-History header. This has been corrected.

Fixed the $expunge operation that expunge everything response will return the correct number of dropped resources.

The patient-list type was renamed to subject-list for a CQL measure in R4.

Previously, the DSTU3 Conformance provider was not including searchInclude or searchRevInclude results in the conformance response. This has been corrected.

Fixed a bug where NPE occurs while updating references for posting a bundle with duplicate resource in conditional create and another conditional verb (delete).

Fixed a regression in 6.0.0 which caused the SearchNarrowingInterceptor to occcasionally be applied to non-search operations.

It was possible under certain race conditions for the batch2 completion handler to be called twice. This has been corrected.

Batch2 will have a standard error handling that will fail the job if it fails a chunk processing for more than 3 times. Further, added better validation to reindex job to disallow bad urls.

A batch2 state change regression was introduced recently that resulted in batch2 jobs not being properly completed. This has been corrected.

FHIR queries using date sort were ignoring seconds and milliseconds when using lucene/elasticsearch SearchParameter indexing. This has been corrected.

Previously, if a FHIR patch was performed on a soft deleted resource, the patch was successfully applied and the resource was undeleted and populated with only the data contained in the patch. This has been fixed so that patch on deleted resources now issues a HTTP 410 response.

Previously, the FHIRPath PATCH operation type add was replacing the entire element as opposed to adding the new element to the existing element. This has been corrected.

For Lucene/Elastic previously we were using scrolled results even when performing synchronous searches. That has now been fixed.

When starting hapi-fhir from starter application, BeanDefinitionOverrideException was thrown with message: Invalid bean definition with name loadIdsStep. This issue is now fixed.

Elastic/Lucene documentation was updated to indicate that full reindex is required when enabling storing resource bodies in indexes when indexed resources exist.

Migrate existing Term Code System Delete and Term Code System Delete Version jobs to batch 2 framework

Previously when a Subscription was created using the package installer and with partitioning enabled, there was an error reading the partition name inside the SubscriptionRegisteringSubscriber. This fix checks incoming Subscription requests for a RequestPartitionId with a list of partition names containing null values and uses RequestPartitionId#defaultPartition() to obtain the default partition instead.

Previously, when doing a GET operation with an _include as a bundle entry could occasionally miss returning some results. This has been corrected, and queries like '/Medication?_include=Medication:ingredient' will now correctly include the relevant target resources.

Migrated Bulk Import Pull to Batch2 Framework

Previously, command prompt was not returned after initiating bulk import operation and even when the importing was completed. This has been fixed, and the command prompt will return after the uploading process finished.

Previously a lucene/elastic enabled search including offset=0 and count > 50 was returning only 50 resources this has now been fixed.

Previously a lucene/elastic enabled sorted search including offset greater than zero was not sorting results. This has now been fixed.

Previously, when modifying one of the common search parameters, if the Mark Resources For Reindexing Upon Search Parameter Change configuration parameter was enabled, an invalid reindex batch job request would be created. This has been fixed, and the batch job request will contain no URL constraints, causing all resources to be reindexed.

Previously, if an intermediate step of a gated batch job produced no output data, an exception would be thrown and processing of the step would abort, leaving the batch job in an unrecoverable state. This has been fixed.

Previously, if an error was thrown after the outgoing response stream had started being written to, an OperationOutcome was not being returned. Instead, an HTML servlet error was being thrown. This has been corrected.

Prevent creating overlapping SearchParameter resources which would lead to random search behaviors.

Removed a bug with subscriptions that supported sending deletes. Previously, if there was a previously-registered subscription which did not support deletes, that would shortcircuit processing and cause subsequent subscriptions not to fire. This has been corrected.

Searches using code:in and code:not-in will now expand the valueset up to the Maximum Expansion Size defined in the DaoConfig.

Fixed bug where creating a new resource in partitioned mode using a PUT operation invoked pointcut STORAGE_PARTITION_IDENTIFY_READ during validation. This caused errors because read interceptors (listing on Pointcut.STORAGE_PARTITION_IDENTIFY_READ) and write interceptors (listening on Pointcut.STORAGE_PARTITION_IDENTIFY_CREATE) could return different partitions (say, all vs default). Now, only the CREATE pointcut will be invoked, and the same partition will be used for any reads during UPDATE.

Previously, if Fulltext Search was enabled, and a $delete-expunge job was run, it could leave orphaned documents in the Fulltext index. This has been corrected.

Fixed a bug where repeated calls to the same Bulk Export request would not return the cached result, but would instead start a new Bulk Export job.

When multiple permissions exist for a user, granting access to the same compartment for different owners, the permissions will be collapsed into one rule. Previously, if these permissions had filters, only the the filter of the first permission in the list would be applied, but it would apply to all of the owners. This has been fixed by turning off the collapse function for permissions with filters, converting each one to a separate rule.

When offset searches are enforced on the server, not all of the search parameters were loading and this caused a NPE. Now an ERROR will be logged instead.

Fixed an issue with $delete-expunge that resulted in SQL errors on large amounts of deleted data

Fixing bug where searching with a target resource parameter (Coverage:payor:Patient) as value to an _include parameter would fail with a 500 response.

Previously, the http://hapifhir.io/fhir/StructureDefinition/subscription-send-delete-messages extension on REST-HOOK subscription channel element was only valid for R4. This has been expanded to support DSTU3 and DSTU2.

Previously, using the http://hapifhir.io/fhir/StructureDefinition/subscription-send-delete-messages extension on a subscription, along with a subscription that used a Search Expression criteria (e.g. Patient?gender=male) would cause a failure during an attempt to send the delete. This has been corrected.

Previously, the RuleBuilder's rules surrounding Group Bulk Export would return failures too early in the case of multiple permissions. This has been corrected, and the rule will no longer prematurely return a DENY verdict, instead opting to delegate to future rules.

Removed a strict dependency on a FullTextSVC. This was causing HAPI to fail to boot when Lucene/ElasticSearch was not enabled.

The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• FlywayDB (JPA): 8.4.4 -> 8.5.0
• Postgresql (JPA): 42.3.2 -> 42.3.3 (Addresses WS-2022-0080)

The HAPI FHIR server GraphQL endpoints now support GraphQL introspection, making them much easier to use with GraphQL-capable IDEs.

SearchNarrowingInterceptor can now be used to automatically narrow searches to include a code:in or code:not-in expression, for mandating that results must be in a specified list of codes.

Performance for JPA Server ValueSet expansion has been significantly optimized in order to minimize database lookups, especially with large expansions.

The SearchNarrowingInterceptor can now narrow searches to require a token:in or token:not-in parameter.

Support has been added to the JPA server for token :not-in queries. Similar to :not queries, resources will currently be considered to match if any codes in the relevant resource field are not found in the given ValueSet (as opposed to matching if all codes are not in the given ValueSet).

Added logs that identify the resource that failed the validation check during package installation, and describe the reason for the failure.

Support has now (finally!) been added for the FHIR Bulk Import ($import) operation. This operation is the first operation to leverage the new Batch2 framework.

A new batch operation framework for executing long running background jobs has been created. This new framework is called 'Batch2', and will eventually replace Spring Batch. This framework is intended to be much more resilient to failures as well as much more paralellized than Spring Batch jobs.

It is now possible to register multiple IConsentService implementations against a single ConsentInterceptor.

When validating a code, a helpful error message is now returned if the user has supplied a ValueSet URL where a CodeSystem URL should be used, since this is a common mistake.

A consent service implementation that enforces search narrowing rules specified by the SearchNarrowingInterceptor has been added. This can be used to narrow results from searches in cases where this can't be done using search URL manipulation. See ResultSet Narrowing for more information.

Provided a Remote Terminology Service implementation for the $translate operation.

Previously there was no way to recreate freetext indexes for terminology entities. A new CLI operation, reindex-terminology now exists for this purpose.

Improve logging so it is clear when a new search parameter has been loaded into the cache.

The JPA server terminology service can now process IS-A filters in ValueSet expansion on servers with Hibernate Search disabled.

The server now supports date searches with the NOT_EQUALS (ne) prefix.

When using ApacheProxyAddressStrategy as a server address strategy, the Forward header will now be respected. Thanks to Thomas Papke for the pull request!

Added a helper method to BundleUtil, to support the creation of a new bundle entry and to set the value for one of the fields.

Added a new multi-column index on the HFJ_RESOURCE table indexing the columns RES_TYPE, RES_DELETED_AT, RES_UPDATED, PARTITION_ID and RES_ID and removed the existing single-column index on the RES_TYPE column. This new index will improve the performance of the $reindex operation and will be useful for some other queries a well.

Added a new pointcut: STORAGE_PRESTORAGE_CLIENT_ASSIGNED_ID which is invoked when a user attempts to create a resource with a client-assigned ID.

Group Bulk Export (e.g. Group/123/$export) now additionally supports Organization and Practitioner as valid _type parameters. This works internally by querying using a _has parameter

Added search parameter modifier :nickname that can be used with 'name' or 'given' search parameters. E.g. ?Patient?given:nickname=Kenny will match a patient with the given name Kenneth. Also added MDM matching algorithm named NICKNAME that matches based this.

The resource JSON can now be stored and retrieved in the Lucene/Elasticsearch index. This enables some queries to provide results without using the database. This is enabled via DaoConfig.setStoreResourceInLuceneIndex()

When using JPA persistence with Hibernate Search (Lucene or Elasticsearch), simple FHIR queries that can be satisfied completely by Hibernate Search no longer query the database. Before, every search involved the database, even when not needed. This is faster when there are many results.

When updating or reindexing a resource on a JPA server where indexing search param presence is enabled (i.e. support for the :missing modifier), updates will try to reuse existing database rows where possible instead of deleting one row and adding another. This should cause a slight performance boost systems with this feature enabled.

Reverting the change to treat canonical references as local when setting the flag getTreatBaseUrlsAsLocal(). This was added to address a limitation in _include that did not process references by canonical URLs. The _include search operation now includes canonical references without special configuration.

Previously, during RepositoryValidation, we would perform validation on resources created via DaoConfig's setAutoCreatePlaceholderReferenceTargets property. This caused validation failures on placeholder resources as they do not conform to any profile. This has been changed, and Repository Validation will not occur on any resource containing an extension with URL http://hapifhir.io/fhir/StructureDefinition/resource-placeholder`.

Changing the MDM logging to contain scores for each applied matcher field. Deleting summary score when creating MDM link.

Several classes and interfaces related to the $expunge operation have moved from the hapi-fhir-jpaserver-base project to the hapi-fhir-storage project.

Ensure that migration steps do not timeout. Adding an index, and other migration steps can take exceed the default connection timeout. This has been changed - migration steps now have no timeout and will run until completion.

Method signatures on several interfaces and classes related to the $expunge operation have changed to support the case where the primary identifier of a resource is not necessarily a Long.

Modified operation method binding so that later registered custom providers take precedence over earlier registered ones. In particular, this means that custom operations can now override built-in operations.

The normalized and exact string database indexes have been changed to provide faster string searches.

The tag indexes have been changed to provide faster tag searches.

Hibernate search has been updated to 6.1.4, which adds support for Elasticsearch 7.16.X. Previous installations running on 7.10.X will continue to work normally, and this change does not require you to upgrade your elasticsearch service.

The XML and JSON Parsers are now encoding narratives of contained resources. Narratives were skipped before for contained resources. This was a rule which was only valid in STU1 (https://www.hl7.org/fhir/DSTU1/narrative.html), and was removed in DSTU2+

Include property regex operation was not working when expanding ValueSet. This is now fixed

A regression in HAPI FHIR 5.5.0 meant that very large transactions where the bundle contained over 1000 distinct client-assigned resource IDs could fail on MSSQL and Oracle due to SQL parameter count limitations.

An occasional concurrency failure in AuthorizationInterceptor has been resolved. Thanks to Martin Visser for reporting and providing a reproducible test case!

When performing a search with a _revinclude. the results sometimes incorrectly included resources that were reverse included by other search parameters with the same name. Thanks to GitHub user @vivektk84 for reporting and to Jean-Francois Briere for proposing a fix.

When searching for date search parameters on Postgres, ordinals could sometimes be represented as strings, causing a search failure. This has been corrected.

A race condition in the Subscription processor meant that a Subscription could fail to register right away if it was modified immediately after being created. Note that this issue only affects rapid changes, and only caused the subscription to be unregistered for a maximum of one minute after its initial creation so the impact of this issue is expected to be low.

When cross-partition reference Mode is used, the rest-hook subscriptions on a partition enabled server would cause a NPE. Cause of this is from the reloading of the subscription when the server is restarted. This issue has been fixed. Also fixed issue with revinclude for rest-hook subscription not working.

ValueSet pre-expansion was failing when number of concepts was larger than configured BooleanQuery.maxClauseCount value (default is 1024). This is now fixed.

Fixed a bug when searching for the resource persistent id in the partitioned environment. This happens when the Cross-Partition Reference Mode is set to ALLOWED_UNQUALIFIED and it is searching for the wrong partition id and resulted in resource persistent id not found.

Updated the FHIR core libs to 5.6.36 to support fix for funcReplace in FHIRPathEngine.

A regression in HAPI FHIR 5.7.0 meant that when UnknownCodeSystemWarningValidationSupport was configured for WARNING behaviour, validating a field with an implicit code system could incorrectly result in an error.

Reindexing jobs were not respected the passed in date range in SearchParams. We now take date these date ranges into account when running re-indexing jobs.

Previously, or expressions were not being properly validated in FHIRPath in STU3 due to a bug with expression path splitting. This has been corrected.

When doing package upload, all resources were filtered by status=active. That is inappropriate for some types. For Subscription, we need to check if the value is requested, for DocumentReference and Communication other values outside the expected active and not active also exist. This change adds a check for those types of resources other than the one normally done for active matching value.

Previously the Fhir parser would only set the resource type on the resource ID for resources which implemented IDomainResource. This caused a few resource types to be missed. This has been corrected, and resource type is now set on the id element for all IBaseResource instances instead.

Previously, allowing any client assigned ID by setting ClientIdStrategyEnum to ANY threw an exception when creating CodeSystem resources. This has been corrected.

Supporting expansion of ValueSet include/exclude system URI expressed in canonical format during validation.

Previously if partitioning was enabled, the URL returned from $export-poll-status provided a location in the wrong partition. This has been fixed and all Bulk Exports will be stored in the DEFAULT partition.

TerserUtil.mergeAllFields() threw an exception when cloning contained resources. This has been corrected.

Added a new setting to BinaryStorageInterceptor which allows you to disable binary de-externalization during resource reads.

Previously, Bulk export jobs automatically cleared the collection after a hardcoded 2 hour time period from start of the job. This is now configurable via a new DaoConfig property, setBulkExportFileRetentionPeriodHours(). If this is set to a value of 0 or below, then the collections will never be expired.

The tag index migration failed on Postres and Oracle. These have been fixed.

Mdm was not excluding NO_MATCH from golden-resource candidates in eid mode. This caused mdm to produce an error when a Patient eid is changed after that patient's link was updated to NO_MATCH. This has been corrected

Unmodified string searches and string :contains search were incorrectly case-sensitive when configured with recent versions of Lucene/Elasticsearch. This has been corrected.

While converting the reindexing job to the new batch framework, a regression of #3441 was introduced. Reindexing jobs were not respecting the passed in _lastUpdated parameter. We now take date these date ranges into account when running re-indexing jobs.

When searching with the _lastUpdated parameter and using the ne prefix, search would fail with HAPI-1928 error. This has been fixed.

Command-line log output now only sends colour commands if output is being printed to a console. Otherwise, (e.g. if output is redirected to a file) the log output will not contain any special colour escape characters.

New batch job implementation (batch2) were staying on IN_PROGRESS status after being cancelled. That is now fixed. After cancellation status is changed to CANCELLED.

Previously, it was possible to update a resource with wrong tenantID. This issue has been fixed.

The Spring Framework library was upgraded to version 5.3.18 in order to avoid depending on a version known to be vulnerable to CVE-2022-22965, known as Spring4Shell. HAPI FHIR is not believed to be vulnerable to this issue, but the library has been bumped as a precaution.

A regression in HAPI FHIR 5.5.0 meant that very large transactions where the bundle contained over 1000 distinct client-assigned resource IDs could fail on MSSQL and Oracle due to SQL parameter count limitations.

Previously subscriptions in a partition with the id null will be matched against incoming resources from all partitions. Changed to subscriptions will only match against incoming resources in the partition the subscription exists in unless cross partition subscription is enabled and the subscription has the appropriate extension.

This version specifically modifies reindex to support moving data from the RES_TEXT to the RES_TEXT_VC column in the HFJ_RES_VER table. This is especially important for PostgreSQL users, as the RES_TEXT column only has an addressable space of about 4 billion resources. Any installation that exceeds this amount of resources stored in the RES_TEXT will experience that the software hangs on attempting to store new resources. In order to avoid this, you should use the DaoConfig#setInlineResourceTextBelowSize setting, and set it to a large non-zero value. This will cause PostgreSQL to not store the resource text as a LOB, but instead as a VARCHAR field. By default, this field has length 4000, but you can and should update it by following the documentation here.

UPDATE: This change has breen removed. Previous content was: 'Modify reindexing to migrate data HFJ_RES_VER data from the RES_TEXT column to the RES_TEXT_VC if the resource's size falls inside of the configuration defined in DaoConfig's getInlineResourceTextBelowSize property.'

Bump the version of Spring Core to 5.3.18 to avoid CVE-2022-22963

Previously, during RepositoryValidation, we would perform validation on resources created via DaoConfig's setAutoCreatePlaceholderReferenceTargets property. This caused validation failures on placeholder resources as they do not conform to any profile. This has been changed, and Repository Validation will not occur on any resource containing an extension with URL http://hapifhir.io/fhir/StructureDefinition/resource-placeholder`.

The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• log4j-api (JPA): 2.11.1 -> 2.17.1 (Addresses CVE-2021-44228 - HAPI FHIR was not vulnerable to this issue but this upgrade avoids unnecessary OWASP scan notices)
• SLF4j (All): 1.7.30 -> 1.7.33
• Logback (All): 1.2.8 -> 1.2.10
• Commons-IO (All): 2.8.0 -> 2.11.0
• Jackson (All): 2.13.0 -> 2.13.1
• Guava (All): 30.1.1-jre -> 31.0.1-jre
• JDOM (XML Patch Support): 2.0.6 -> 2.0.6.1 (Addresses CVE-2021-33813)
• Spring (JPA): 5.3.7 -> 5.3.15
• Spring (JPA): 5.3.7 -> 5.3.15
• Spring-Data (JPA): 2.5.0 -> 2.6.1
• Hibernate ORM (JPA): 5.4.30.Final -> 5.6.2.Final
• Flyway (JPA): 6.5.4 -> 8.4.1
• Sqlbuilder (JPA): 3.0.1 -> 3.0.2
• H2 (JPA): 1.4.200 -> 2.1.210 (Note that this change requires the use of the HapiFhirH2Dialect instead of the built-in Hibernate H2Dialect due to Hibernate issue HHH-15002
• Commons-DBCP2 (JPA): .8.0 -> .8.0 -> 2.9.0
• Swagger-Models (OpenAPI Support): 2.1.7 -> 2.1.12
• Thymeleaf (Testpage Overlay): 3.0.12.RELEASE -> 3.0.14.RELEASE (Addresses CVE-2021-43466)
• Commons-CLI (CLI): 1.4 -> 1.5.0
• JANSI (CLI): 2.3.2 -> 2.4.0
• Jetty Server (CLI): 9.4.43.v20210629 -> 9.4.44.v20210927
• Spring Boot (Boot): 2.5.0 -> 2.6.2
• Swagger UI (OpenAPI): 3.46.0 -> 4.1.3
• Resteasy (JAX-RS): 4.0.0.Beta3 -> 5.0.2.Final
• Postgresql (JPA): 42.3.1 -> 42.3.2
• Spring Security Oauth2(Oauth): 2.0.2.RELEASE -> 2.0.17.RELEASE

In the JPA server, the token :of-type modifier is now supported. This is an optional feature and must be explicitly enabled (default is disabled).

Added partition support for subscriptions. Subscriptions will now only match resource from the same partition

New configuration option added to validate bundle resources concurrently. Also new configuration added to skip validation of contained resources.

Added support for cross-partition subscriptions. Subscription in the default partition can now listen to resource changes from all partitions

The resource contents are optionally added to the lucene index so $lastn queries can be satisfied without database access. This is activated by the DaoConfig 'StoreResourceInLuceneIndex' property.

Added http://hapifhir.io/fhir/StructureDefinition/subscription-delivery-retry-count extension that can be provided to a subscription to define a specific retry strategy (retry retry-count number of times before giving up).

Provided a Remote Terminology Service implementation for the $validate-code Operation.

Add the concept of messageKey to the BaseResourceMessage class. This is to support brokers which permit key-based partitioning.

Allow Rest Hook subscriptions to be configured to send delete requests.

Added ability to load Implementation Guide packages from filesystem by supporting the file:/ syntax of url.

A redundant set of hash calculations in the JPA server was eliminated.

A new configuration option has been added to ParserOptions called setAutoContainReferenceTargetsWithNoId. This setting disables the automatic creation of contained resources in cases where the target/contained resource is set to a Reference instance but not actually added to the Resource.contained list. Disabling this automatic containing yields a minor performance improvement in systems that do not need this functionality.

A number of minor optimizations have been added to the JsonParser serializer module as well as to the transaction processor. These optimizations lead to a significant performance improvement when processing large transaction bundles (i.e. transaction bundles containing a larger number of entries).

Improved validation performance by switching validation serialization from XML to JSON.

Significantly improved $delete-expunge performance by adding database indexes, and filtering needed foreign keys to delete by resource type.

A new JPA setting has been added to DaoConfig settings called Inline Resource Text Below Size. When this setting is set to a positive value, any resources with a total serialized length smaller than the given number will be stored as an inline VARCHAR2 text value on the HFJ_RES_VER table, instead of using an external LOB column. This improves read/write performance (often by a significant amount) at the expense of a slightly larger amount of disk usage.

Improves the performance of the query for searching by chained search parameter when the Index Contained Resources feature is enabled.

Improved performance of validating resources when skip contained resources is enabled.

Previously in configuring MDM, you were only allowed to set a single eidSystem which was global regardless of how many resource types you were performing MDM on. This has been changed. That field is now deprecated, and a new field called eidSystems (a json object) should be used instead. This will allow you to have granular control over which resource types use which EIDs. See the documentation of eidSystems for more information.

Fixed a regression where packages would fail to load if HAPI-FHIR was operating in DATABASE binary storage mode.

These changes are related to bumping the dependency on org.hl7.fhir.core to version 5.3.0. The main updates that resulted in the majority of the changes in this PR were the addition of the interface IValidationPolicyAdvisor, and the refactoring of some of the main validation classes (for cleaning purposes). Also, many of the error messages were updated, so the resulting tests in HAPI had to be modified to reflect these message changes. In regard to the new interface IValidationPolicyAdvisor, by implementing this interface and passing it through the validation context in HAPI to the InstanceValidator in the core libraries, users can now control the behavior of the validator when validating references, contained resources, and coded content. Previously, only references were controlled in this way, and users controlled this by overriding the validation fetcher. Test cases were added in the validation test cases repository to demonstrate it's functionality. In particular, the two test cases contained-resource-bad-id, and contained-resource-bad-id-ignore. The definitions and expected outcomes of these test cases are in the manifest.json in that repository.

Add email configuration to MailSvc constructor

Removed the following modules from the HAPI-FHIR project: hapi-fhir-testpage-interceptor, hapi-fhir-structures-dstu, hapi-fhir-oauth2, hapi-fhir-narrativegenerator.

Calling the $document operation previously omitted the fullUrl of the bundle entries. This has been corrected.

When a subscription fails to activate (either on the first attempt or on start up), hapi-fhir will log the exception, set the subscription to ERROR state, and continue on. This is to prevent hanging on startup for errors that cannot be resolved by infinite retries.

$member-match operation was allowed to be invoked by GET when it shouldn't have. That is fixed. Operation can only be invoked by POST request

Using the delivery option: Payload Search Result Mode for rest-rook subscriptions on a partition enabled server would cause a NPE. This issue has been fixed.

Code System deletion background tasks were taking over a day to complete on very large CodeSystems for PostgreSQL, SQL Server and Oracle databases. That was improved now taking less than an hour in all three platforms

RequestValidatingInteceptor incorrectly prevented GraphQL requests from being submitted using the HTTP POST form of the GraphQL operation.

Updated UnknownCodeSystemWarningValidationSupport to allow the throwing of warnings if configured to do so.

Resource links were previously not being consistently created in cases where references were versioned and pointing to recently auto-created placeholder resources.

Fixed a serious performance issue with the $reindex operation.

In servers with a large number of StructureDefinition resources loaded, occasionally a call for the CapabilityStatement could take a very long time to return. This has been corrected.

$everything operation returns a 500 error when querying for a page when _getpagesoffset is greater than or equal to 300. This has been corrected.

MDM was throwing a NullPointerException when upgrading a match from POSSIBLE_MATCH to MATCH. This has been corrected.

Fixed bug that caused queries that checked Elasticsearch when setting _total=accurate searches to return all values when using _content or _text parameters

Terminology reindexing job was launching the wrong process, so terminology reindexing was never launched. This has been fixed.

Concurrent validation failed with StringIndexOutOfBoundsException when the validation location did not contain a '.'. This has been corrected.

In rare cases where patient ID String happened to have hashCode equal to Integer.MIN_VALUE, PatientIdPartitionInterceptor would generate an invalid partition ID. This has been fixed.

When $mdm-clear operation batch was split into multiple threads, ResourceVersionConflictExceptions were being thrown. This issue has been fixed.

Conformance validation should happen once per client-endpoint, no matter the number of threads executing the requests, but was happening once per client-endpoint-thread. This is now fixed.

Previously, $binary-access-read and other operations that return neither method outcomes nor resources were causing no invocation of the SERVER_OUTGOING_RESPONSE pointcut. This has been corrected, and those operations will now correctly invoke SERVER_OUTGOING_RESPONSE.

GraphQL searches cannot be executed when number of matching results exceeds default page size. This has been corrected.

In HAPI FHIR 5.6.0, a regression meant that JPA server FHIR transactions containing two identical conditional operations (for example, a transaction with two conditional create operations where the conditional URL was identical for both) could result in resources being saved in an inconsistent state. This scenario will now be result in the server returning an HTTP 400 error instead. Note that there is no meaning defined in the FHIR specification for duplicate entries in a FHIR transaction bundle, so it has never been recommended to do this.

The JPA server will now validate _include and _revinclude statements before beginning search processing. This prevents an ugly JPA error when some invalid params are used.

Enhanced Lucene Indexing failed when indexing contained resources. Contained resources are not indexed in Lucene/Elasticsearch, but this no longer causes an exception.

Chained searches will handle common search parameters correctly when the Index Contained Resources configuration parameter is enabled.

When cross-partition reference Mode is used, the rest-hook subscriptions on a partition enabled server would cause a NPE. Cause of this is from the reloading of the subscription when the server is restarted. This issue has been fixed. Also fixed issue with revinclude for rest-hook subscription not working.

Bulk export permissions were over-permissive in RuleBulkExportImpl. This has been corrected.

In the JPA server, searching using the token :text modifier will no longer index and match values in Identifier.type.text. Previously we indexed this element, but the FHIR specification does not actually show this field as being indexed for this modifier, and intuitively it doesn't make sense to do so.

An occasional concurrency failure in AuthorizationInterceptor has been resolved. Thanks to Martin Visser for reporting and providing a reproducible test case!

Bump Spring Core dependency to remove Spring4Shell vulnerability.

Bump Spring Core dependency to 5.3.18 to remove vulnerability.

Chained searches will handle common search parameters correctly when the Index Contained Resources configuration parameter is enabled.

Improves the performance of the query for searching by chained search parameter when the Index Contained Resources feature is enabled.

The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• Spring (JPA): 5.3.6 -> 5.3.7
• Spring Boot (JPA Starter): 2.4.4 -> 2.5.0
• Jetty (CLI): 9.4.39.v20210325 -> 9.4.42.v20210604

Pagination returned incorrect offset and count in the previous link of the last page when total element count was one more than multiple of page size. Problem is now fixed

Settings have been added to the JPA Server DaoConfig to enable/disable various individual kinds of scheduled tasks.

When performing a conditional create operation on a JPA server, the system will now verify that the conditional URL actually matches the data supplied in the resource body, and aborts the conditional create if it does not.

Support has been added to the JPA server for _include and _revinclude where the value is a qualified star, e.g. _include=Observation:*.

A new interceptor ValidationMessageSuppressingInterceptor has been added. This interceptor can be used to selectively suppress specific vaLidation messages.

A new config option has been added to the DaoConfig that causes generated SQL statements to account for potential null values in HAPI FHIR JPA date index rows. Nulls are no longer ever used in this table after HAPI FHIR 5.3.0, but legacy data may still have nulls.

When performing non-query cache JPA searches (i.e. searches with Cache-Control: no-store) the loading of _include and _revinclude will now factor the maximum include count.

A new setting has been added to the DaoConfig that allows the maximum number of _include and _revinclude resources to be added to a single search page result. In addition, the include/revinclue processor have been redesigned to avoid accidentally overloading the server if an include/revinclude would return unexpected massive amounts of data.

A new DaoConfig setting called Mass Ingestion Mode has been added. This mode enables rapid data ingestion by skipping a number of unnecessary checks during backloading.

A new Pointcut has been added that is invoked when a new Bulk Export is initiated.

The JPA server terminology uploader now supports uploading ICD-10-CM (US Edition) using the native format for that vocabulary.

AuthorizationInterceptor can now be used to authorize bulk export requests

Support for LOINC 2.70 has been added.

PatientIdPartitionInterceptor now supports conditional creates of resources where the resource is in the patient compartment but the conditional URL does not contain a patietn reference.

The $evaluate-measure now works on a partitioned server.

A new interceptor has been added for JPA servers that uses semaphores to avoid multiple concurrent FHIR transactions from trying to create/update the same resource at the same time. This can improve overall performance when writing many concurrent transactions since it avoids the need for retries.

A new tag storage mode called Inline Tag Mode tas been added. In this mode, all tags are stored directly in the serialized resource body in the database, instead of using dedicated tables. This has significant performance advantages when storing resources with many distinct tags (i.e. many tags that are unique to each resource, as opposed to being reused across multiple resources).

A new interceptor has been addeed to the JPA server called ForceOffsetSearchModeInterceptor. This interceptor forces all searches to be offset searches, instead of relying on the query cache. This means that FHIR search operations will never result in any database write, which can be good for highly concurrent servers.

A new JPA partitioning interceptor PatientIdPartitionInterceptor has been added. This interceptor uses the ID of the patient associated with any resources in the patient compartment to generate a consistent partition ID.

$mdm-query-links and $mdm-duplicate-golden-resources now enforce paging via parameters _offset and _count. More details can be found in the MDM Operations documentation.

Support for multiple header-passthrough option using -hp or --header-passthrough parameter has been added to hapi-fhir-cli commands: example-data-uploader, export-conceptmap-to-csv, import-csv-to-conceptmap and upload-terminology

A new Validation Support Module has been added that can use NPM Packages to supply validation conformance artifacts programatcally to the validator.

Add 'loinc.codesystem.make.current' property to upload-terminology command to allow loading LOINC version without becoming current.

Allowed the optional inclusion of the LOINC Consumer Names archive in addition to the main LOINC distribution. If it is supplied, the consumer names CSV file will be scanned, and all consumer names will be added to uploaded Concepts as additional designations

Allowed the optional inclusion of the LOINC Linguistic Variants archive in addition to the main LOINC distribution. If it is supplied, all linguistic variants files will be scanned, and all translations will be added to uploaded Concepts as additional designations

Upgrade net.java.dev.jna to run docker tests on Mac arm64 M1 machines

The :text Search Parameter modifier now searches by word boundary of the text content as opposed to only searching at the start of the text when using Lucene/Elasticsearch indexing. Add * to match word prefixes (e.g. weig* will match weight).

Added new $reindex operation with similar syntax to $delete-expunge that creates a spring-batch job to reindex selected resources. $mark-all-resources-for-reindexing and $perform-reindexing-pass are now deprecated, and will likely be removed in a future release.

Replace existing email implementation code with SimpleJavaMail library.

Modified the behaviour of the :mdm param qualifier. Previously, it used to only resolve IDs if the resource ID was a source resource. Now, MDM expansion will work if you pass it the ID of a golden resource instead.

Conditional URL lookups in the JPA server will now explicitly specify a maximum fetch size of 2, avoiding fetching more data that won't be used inadvertently in some situations.

FHIR Transaction duplicate record checks are now performed without any database interactions or SQL statements, reducing the processing load associated with FHIR transactions by at least a small amount.

FHIR transactions in the JPA server that perform writes will now aggressively pre-fetch as many entities as possible at the very start of transaction processing. This can drastically reduce the number of round-trips, especially as the number of resources in a transaction gets bigger.

A new setting has been added to the DaoConfig called Tag Versioning Mode. This setting controls whether a single collection of tags/profiles/security labels is maintained across all versions of a single resource, or whether each version of the resource maintains its own independent collection. Previously each version always maintained an independent collection, which is useful sometimes, but is often not useful and can affect performance.

When performing a conditional create/update/delete on a JPA server, if the match URL contained a plus character, this character was interpreted as a space (per legacy URL encoding rules) even though this has proven to not be the intended behaviour in real life applications. Plus characters will now be treated literally as a plus character in these URLs.

Bulk import batch jobs are now activated in a local scheduled task, making bulk import jobs better able to take advantage of large clusters.

DELETE _expunge=true has been converted to use Spring Batch. It now simply returns the jobId of the Spring Batch job while the job continues to run in the background. A new operation called $expunge-delete has been added to provide more fine-grained control of the delete expunge operation. This operation accepts an ordered list of URLs to be delete expunged and an optional batch-size parameter that will be used to perform the delete expunge. If no batch size is specified in the operation, then the value of DaoConfig.getExpungeBatchSize() is used.

The ConceptMap.group.element.display storage size limit has been increased to 500 characters.

The bulk export request length limit has been increased to 1024 characters.

If two authorization compartments apply to the same targets and share the same compartment name, then instead of creating a new compartment, the rule builder now adds the new owner to the list of owners in the existing compartment.

When operating in partitioned mode, the interceptor pointcut STORAGE_PARTITION_IDENTIFY_CREATE will now be invoked to determine the partition to use for create with client-assigned IDs. Previously the STORAGE_PARTITION_IDENTIFY_READ pointcut was invoked, which was confusing and potentially unexpected.

Flyway migration used to enforce order by default. This has been changed so now the default behaviour is out of order migrations are permitted. Strict order can be enforced via the new strict-order flag if required.

Identifier maximum length increased from 200 to 500. This specifically applies to table HFJ_IDX_CMP_STRING_UNIQ.

When the contents of a package are corrupt, the error messages now identify the corrupt element

When performing a FHIR transaction containing a conditional create, references to that resource were inadvertently replaced with contained references."

A concurrency error was fixed when using client assigned IDs on a highly concurrent server with resource deletion disabled.

A null-pointer exception was fixed when a ResponseTerminologyDisplayInterceptor is registered and a search or read response returns a resource with code value that in turn returns a null code lookup.

Subscription notifications will no longer be triggered by default in response to changes that do not increment the resource version (e.g. $meta-add and $meta-delete). A new DaoConfig setting has been added to make this configurable.

When myDaoConfig.setDefaultTotalMode(SearchTotalModeEnum.ACCURATE) and there are zero search results on an _id search, An Index Out of Bounds error was thrown. This has been corrected.

Fixes the problem that FHIR package IDs were incorrectly treated as case sensitive when being loaded, causing loads to fail when dependencies were declared with a different case than in the package itself.

Constraint errors were not always auto-retried even when configured to do so on certain platforms (particularly Postgresql) where constraint names are auto converted to lower case. Thanks to Bruno Hedman for the pull request!

When searching by source, if deleted resources are matched, the search returned an incorrect size. This has been corrected.

The _filter search parameter was incorrectly included in the server capability statement if it was disabled on the server. This has been corrected.

ValueSet expansion did not correctly preserve the order if multiple codes were included in a single inclusion block.

Too many MDM candidates matching could result in an OutOfMemoryError. Candidate matching is now limited to the value of IMdmSettings.getCandidateSearchLimit(), default 10000.

A regression caused the JPA Server History operation to not return paging links in responses. This has been corrected.

Added null checks to MDM resource interceptor in order to avoid NPEs.

The SQL generated for the _profile search parameter did not use all of the columns on the tag index table, resulting on poor performance on MySQL. This has been corrected.

The internal cache for tags, concepts, and others had longer or shorter expiry times than was intended. This has been corrected.

When searching for ExplanationOfBenefit?patient=123,456, the compartment authorization interceptor was treating '123,456' as a single id rather than as a list of ids. This has been corrected.

A regression was introduced in 2760 where a READ compartment could get collapsed into a WRITE compartment. This has been corrected.

Searches for mdm-expanded references such as Observation?subject:mdm=123 were getting denied by access rules that did not recognize the :mdm suffix. This has been corrected.

$mdm-submit operation was only submitting 100 resources and then stopping. It now correctly submits all requested resources.

When providing links for placeholder creation, DaoResourceLinkResolver expects just a single 'identifier=value' param, but it can be additional data, s.a. tags, extra identifiers, etc.

When initiating a FHIR bulk export, if more than one _typeFilter parameter was supplied only the first one was respected. This has been corrected.

Loading packages would fail when partitioning was enabled with unnamed partitions. This has been fixed.

An issue in the FHIRPath evaluator prevented Encounters from being stored when using the new PatientIdPartitionInterceptor. This has been corrected.

Mdm failed to load if any mdm subscription had been deleted, e.g. if $expunge expungeEverything had been run on the server. This has been corrected.

When Client Id Strategy is set to NOT_ALLOWED, permit system requests to create resources, e.g. SearchParameter and Subscription resources required by the system.

Add DSTU3 Support To UploadTerminologyCommand.

Fixed a bug in transaction bundle processing, specifically for bundles which contained both a conditional create, and a resource which relied on this conditional create as a reference. This would cause the referring resource to generate a contained resource instead of appropriately referencing the existing patient.

Fixed a bug wherein an NPE could be thrown by the MDM module interceptor if an incoming resource had a tag with no system.

Fixed a bug where the search results cache was ignoring the value of _contained parameter when assigning a cache key. This was causing queries run in a short period of time to return wrong cached results if one query used _contained=true and the other did not.

Addressed the following CVE report by bumping the minor version for Jetty in the root POM:

• CVE-2021-34429