The version of a few dependencies have been bumped to more recent versions (dependent HAPI modules listed in brackets):

• org.hl7.fhir.core (Base): 6.3.18 -> 6.3.25
• Bower/Moment.js (hapi-fhir-testpage-overlay): 2.27.0 -> 2.29.4
• htmlunit (Base): 3.9.0 -> 3.11.0
• Elasticsearch (Base): 8.11.1 -> 8.14.3

A new Pointcut called MDM_SUBMIT_PRE_MESSAGE_DELIVERY has been added. If you wish to customize the ResourceModifiedJsonMessage sent to the broker, you can do so by implementing this Pointcut, and returning ResourceModifiedJsonMessage.

Added a configuration setting for the TTL of HTTP connections to IRestfulClientFactory. The following implementations have been updated to respect this new setting: 1. ApacheRestfulClientFactory 2. OkHttpRestfulClientFactory 3. HapiFhirCliRestfulClientFactory Thanks to Alex Kopp and Alex Cote for the contribution!

Batch instance ID and chunk ID have been added to the logging context so that they can be automatically added to batch-related messages in the log.

The method FhirTerser.cloneInto() has been enhanced to copy the resource field of a Reference, in addition to the fields that are accessible via the structure definition.

The STORAGE_PARTITION_DELETED pointcut has been added and will be called upon deleting a partition using the $partition-management-delete-partition operation.

A new configuration option, PartitionSettings#setPartitionIdsInPrimaryKeys(boolean) configures the query engine to include the partitioning column in search query joins.

Upgrade the Clinical Reasoning module to the latest release of 3.13.0. This update comes with several changes and feature enhancements to CPG and dQM clinical-reasoning operations. Please review associated ticket and upgrade.md for detailed list of changes.

Remote Terminology validation has been enhanced to support output parameter issues for the $validate-code operation.

Add plumbing for combining IConsentServices with different vote tally strategies

When using the FHIR TerserUtil to merge two resource, if one resource has real data in a particular field, and the other resource has a data-absent-reason extension in the same field, the real data will be given precedence in the merged resource, and the extension will be ignored.

A new experimental JPA setting has been added to JpaStorageSettings which causes searches for token SearchParameters to include a predicate on the HASH_IDENTITY column even if it is not needed because other hashes are in use.

A cache has been added to the validation services layer which results in improved validation performance. Thanks to Max Bureck for the contribution!

A synchronization choke point was removed from the model object initialization code, reducing the risk of multi-thread contention.

Date searches using equality would perform badly as the query planner does not know that our LOW_VALUE columns are always < HIGH_VALUE columns, and HIGH_VALUE is always > LOW_VALUE columns. These queries have been fixed to account for this.

Upgrading to Jakarta had caused a problem with the version of java-simple-mail that was in use. This has been updated to be conformant with the Jakarta Mail APIs. Thanks to Thomas Papke(@thopap) for the contribution!

The CachingValidationSupport cache for concept translations will now keep up to 500000 translations instead of the previous 5000. This will be made configurable in a future release.

A resource leak during database migration on Oracle could cause a failure ORA-01000 maximum open cursors for session. This has been corrected. Thanks to Jonas Beyer for the contribution!

Previously, searches combining the _text query parameter (using Lucene/Elasticsearch) with query parameters using the database (e.g. identifier or date) could miss matches when more than 500 results match the _text query parameter. This has been fixed, but may be slow if many results match the _text query and must be checked against the database parameters.

Previously, when disabling the 'Enforce Referential Integrity on Write' setting, referential integrity was still partially enforced when posting a resource with an invalid reference. This has now been fixed.

The PatientIdPartitionInterceptor could on rare occasion select the incorrect partition for a resource. This has been corrected. In order for the wrong partition to be selected, the following three things need to be true: 1) there are multiple values of a patient compartment for a resource (see https://hl7.org/fhir/R4/compartmentdefinition-patient.html) 2) a patient compartment value is a non-Patient reference 3) the search parameter of the incorrect value needs to come alphabetically before the search parameter of the correct value. For example, if a QuestionnaireResponse has subject Patient/123 and author Organization/456, then since 'author' appears ahead of 'subject' alphabetically it would incorrectly determine the partition. The fix changed the partition selection so that it now only matches on Patient references.

Previously, when a extension was passed in as a part of the CDS hooks request, it would result in a 400 service not found. This behaviour has now been fixed.

Updated the Reindex Batch2 job to allow for an additional step that will check to ensure that no pending 'reindex' work is needed. This was done to prevent a bug in which value set expansion would not return all the existing CodeSystem Concepts after a reindex call, due to some of the concepts being deferred to future job runs. As such, $reindex operations on CodeSystems will no longer result in incorrect value set expansion when such an expansion is called 'too soon' after a $reindex operation.

Previously, a specific migration task was using the TRIM() function, which does not exist in MSSQL 2012. This was causing migrations targeting MSSQL 2012 to fail. This has been corrected and replaced with usage of a combination of LTRIM() and RTRIM(). Thanks to Primož Delopst at Better for the contribution!

Previously, when having StorageSettings#getIndexMissingFields() == IndexEnabledEnum.DISABLED (default value) and attempting to search with the missing qualifier against a resource type with multiple search parameters of type reference, the returned results would be incorrect. This has been fixed.

Previously, defining a unique combo Search Parameter with the DateTime component and submitting multiple resources with the same dateTime element (e.g. Observation.effectiveDateTime) resulted in duplicate resource creation. This has been fixed.

Fixed a bug in migrations when using Postgres when using the non-default schema. If a migration attempted to drop a primary key, the generated SQL would only ever target the public schema. This has been corrected, and the current schema is now used, with public as a fallback. Thanks to Adrienne Sox for the contribution!

After upgrading org.hl7.fhir.core from 6.1.2.2 to 6.3.11, the $validate-code operation stopped returning an error for invalid codes using remote terminology. This has been fixed.

A crash while executing a search with named _include parameters on MSSQL has been fixed. Thanks to Craig McClendon for the pull request!

Searches that combined full-text searching (i.e. _text or _content) with other search parameters could fail to return all results if we encountered 1600 matches against the full-text index where none of them match the rest of the query. This has now been fixed.

Hibernate Search Fulltext fields which were unused have been removed from indexing. This will reduce storage usage in Lucene and Elasticsearch. The fields that were removed are: myNarrativeTextEdgeNGram, myNarrativeTextNGram, myNarrativeTextPhonetic, myContentTextEdgeNGram, myContentTextNGram, myContentTextPhonetic.

Searches that combined full-text searching (i.e. _text or _content) with other search parameters could fail to return all results if we encountered 1600 matches against the full-text index where none of them match the rest of the query. This has now been fixed.

Remote Terminology validation has been enhanced to support output parameter issues for the $validate-code operation.

After upgrading org.hl7.fhir.core from 6.1.2.2 to 6.3.11, the $validate-code operation stopped returning an error for invalid codes using remote terminology. This has been fixed.

This release updates the org.hl7.fhir core dependency up to 6.3.23, in order to patch CVE-2024-45294.

Previously, defining a unique combo Search Parameter with the DateTime component and submitting multiple resources with the same dateTime element (e.g. Observation.effectiveDateTime) resulted in duplicate resource creation. This has been fixed.

Previously, searches combining the _text query parameter (using Lucene/Elasticsearch) with query parameters using the database (e.g. identifier or date) could miss matches when more than 500 results match the _text query parameter. This has been fixed, but may be slow if many results match the _text query and must be checked against the database parameters.

When encoding resources in summary mode, it is now possible to override the built-in list of summary elements, by adding additional elements and/or by removing elements from the default list. This can be done for an individual parser instance, or globally using the ParserOptions object available from the FhirContext.

Update Clinical Reasoning version to v3.6.0

Added the entire WorkChunk to the StepExecutionDetails of a Batch Job.

Remote Terminology Service can now return subproperty fields for a CodeSystem lookup operation. This can be done in DSTU3 and R4. R5 is not yet implemented.

Generated IPS documents will now include a bundle profile declaration.

A new utility method has been added to BundleUtil which converts a FHIR Bundle containing resources (e.g. a search result bundle) into a FHIR transaction bundle which could be used to upload those resources to a server.

Previously, since hapi-fhir 7.0, when retrieving the consecutive pages of a Bundle resource search operation using a client with read permissions for Bundle resources, the request would fail with a 403 Forbidden error. This has been fixed.

Added support for filtering on CodeSystem defined properties when using database (hibernate) filtering. The following ValueSet filters can be used to filter on CodeSystem defined properties: EQUAL, EXISTS, IN, NOTIN

When processing a batch job, OpenTelemetry spans named hapifhir.batch_job.execute are now generated by the worker threads. These spans have the following span attributes related to the batch job: hapifhir.batch_job.definition_id, hapifhir.batch_job.definition_version, hapifhir.batch_job.instance_id, hapifhir.batch_job.step_id, hapifhir.batch_job.chunk_id.

When performing a search for a resource which uses versioned canonical references to another resource (e.g. a QuestionnaireResponse with a questionnaire reference of http://example.org/my-questionnaire|1.0), the server failed to process _include directives which should bring in these references. Includes will now fetch all versions of a canonical reference, which is still not perfect but is an improvement over the current behaviour. A future update may add more targeted behaviour to address including versioned reference targets, but this will require a significant re-architecture of the loading module.

Previously, when partitioning is enabled, the reindex job could only be run against one partition. The reindex job parameters can now accept a list of partitions or all partitions (list of RequestPartitionId or RequestPartitionId.allPartitions). In the future, the functionality can be extended to other bulk operations run via batch2 jobs (e.g. $delete-expunge, $export).

When populated, the search score field will now be included in the entries of a response Bundle.

When uploading an invalid CodeSystem to the JPA server containing duplicate codes, the server responded with an unhelpful error message referring to a database constraint error. This has been fixed so that a more informative error message is returned.

Subscriptions now support the evaluation use of FhirPath criteria and the use of the variables %current and %previous. Thanks to Jens Villadsen (@jkiddo) for the contribution!

Allow overriding RestfulServer's contextPath determination by overriding IServerAddressStrategy. Thanks to Alex Kopp (@alexrkopp) for the contribution!

Added support for :contains parameter qualifier on the _text and _content Search Parameters. When using Hibernate Search, this will cause the search to perform an substring match on the provided value. Documentation can be found here.

The server-generated CapabilityStatement will now use the server name defined by RestfulServer#setServerName(..) instead of the hardcoded string HAPI FHIR. Thanks to Renaud Subiger for the pull request!

Added paging support for $everything operation in synchronous search mode.

Previously, if a unique or non-unique combo SeaerchParameter was defined, it would not be used by searches if any parameter contained multiple OR clauses (e.g. Patient?family=simpson&given=homer,marge). Such searches will now use the combo index table, which should result in much more performant searches in some cases.

It is now possible to save TermConceptDesignations with values over 2,000 characters.

Added the target resource partitionId and partitionDate to the resourceLink table.

The MDM candidate search will now be skipped if data for required search parameters is not present. Thanks to Adrienne Sox for the pull request!

Indexing for unique combo Search Parameters has been modified so that a hash value is now stored. This hash value is not yet used in searching or enforcing uniqueness, but will be in the future in order to reduce the space required to store the indexes and the current size limitation on unique indexes.

When unique and non-unique combo parameters are in use on a server, FHIR Transaction and Reindex Job performance has been optimized by pre-fetching all existing combo index rows for a large batch of resources in a single database operation. This should yield a meaningful performance improvement on such systems.

Indexing for non-unique combo Search Parameters has been improved, using a new hash-based index that should perform significantly better in many circumstances.

A new configuration option, StorageSettings#setIndexStorageOptimized(boolean) has been added. If enabled, the server will not write data to the SP_NAME, RES_TYPE, SP_UPDATED columns for all HFJ_SPIDX_xxx tables. This can help reduce the overall storage size on servers where HFJ_SPIDX tables are expected to have a large amount of data.

The in-memory search matcher module (used for Subscriptions and other places where a resource needs to be evaluated against an existing pool of search parameter expressions) has been optimized so that it doesn't evaluate unnecessary search parameters while analyzing the resource. Thanks to Ahmet Melih Aydoğdu for the pull request!

Database migrations that add or drop an index no longer lock tables when running on Azure Sql Server.

Extract Subscription Settings from Storage Settings so they can be managed independently.

This brings the org.hl7.fhir.core dependency to version 6.3.11.

Previously, CDS hook extensions needed to be encoded as strings. This has been changed so that extensions are now properly provided as inline JSON.

An prototype interface to abstract data access across different types of FHIR repositories (e.g. remote REST, local JPA) has been added to the hapi-fhir-base project. Implementations of this interface will follow in future HAPI releases, and it will continue to evolve as it's validated through implementation.

The $reindex operation could potentially initiate a reindex job without any urls provided in the parameters. We now internally generate a list of urls out of all the supported resource types and attempt to reindex found resources of each type separately. As a result, each reindex (batch2) job chunk will be always associated with a url.

Removed the validation error of the max query parameter on the $lastn operation. Now, the $lastn operation can be invoked with the max query parameter. Contribution by Gijs Groenewegen (@thetrueoneshots).

The CSS file used by the OpenApiInterceptor to serve up the Swagger UI component inadvertently blocked the authorization button evem when it was wanted. This has been fixed. Thanks Jesse Bonzo for the contribution!

In the case where a resource was serialized, deserialized, copied and reserialized it resulted in duplication of contained resources. This has been corrected.

The IPS narrative generator previously created narratives for each Composition.section.text entry, but also combined these section narratives into a single concatenated narrative for Composition.text. This caused validation issues as the narratives then contained duplicate IDs, and also wasted space. This has been removed.

Previously, when adding multiple resources and defining a golden resource using MDM, the golden resource's tags were removed. This has been fixed

Previously, updating a tokenParam with a value greater than 200 characters would raise a SQLException. This issue has been fixed.

Remove unnecessary call to deleteAllSearchParams when $expunge is called on an already-deleted resource.

Bulk export was failing with a HAPI-2222 error with the persistence module configured for R5. This has been fixed.

An UnsupportedOperationException occurred when validating R5 MHD bundles using the HAPI FHIR validator. Thanks to Renaud Subiger for contributing a fix!

A regression in HAPI FHIR 6.4.0 meant that ther JPA server schema migrator ran all tasks even when the database was initially empty and the schema was being initialized by script. This did not produce any incorrect results, but did impact the amount of time taken to initialize an empty database. This has been corrected.

A number of columns in the JPA schema use primitive types (and therefore can never have a null value) but aren't marked as non-null.

Delete expunge targeted to resources (not everything) that were created with IfNoneExists URL followed by recreation of said resources was failing with HAPI-0550 due to the fact that the corresponding search url records were not being deleted. This has been fixed.

Fixed a regression that prevented rebuilding golden resources with mdm survivorship rules when resources have non-numeric fhir ids in a partitioned system.

Previously, when a MDM rule tried to perform a phonetic match by HumanName (eg. Patient.name), a ClassCastException was thrown. This has now been fixed.

An overly verbose log in the JPA server about deleting stale searches has been reduced to DEBUG level.

Previously, queries with chained would fail to sort correctly with lucene and full text searches enabled. This has been fixed.

The default use of the Accept-Charset header has been removed as its use is deprecated. Thanks to Jens Villadsen for the suggestion and pull request!

The JSON Parser failed to parse alternate names (e.g. _family) containing extensions if the corresponding regular named element (e.g. family) was not present. Thanks to Stefan Lindström for the pull request!

Previously, subscription criteria on R4 backport subscriptions were not validated by the SubscriptionValidatingInterceptor. This has been corrected.

Batch2WorkChunkEntity poll_attempt count is a nullable field, but was incorrectly declared as a primitive. This has been fixed.

Fixed an issue where MDM json links were unserializable due to using the IResourcePersistenceId objects. This has been fixed, and IResourcePersistenceId objects will not be serialized or returned to users.

Previously, when creating a custom SearchParameter for a document Bundle that references an entry resource through the composition (e.g. Bundle.entry[0].resource.as(Composition).subject.resolve().as(Patient).identifier), the search using that parameter would not return the matching document Bundle resources, despite the fullUrl of the entry matching the composite reference. This problem does not exist if the reference match can be done against the entry id. However an id is not required so the match should be done against the fullUrl. This issue has been fixed.

Bulk export was failing with a HAPI-1775 error with the persistence module configured for DSTU3. This has been fixed.

Corrected a regression in valueset expansion. If a valueset with a non-standard property in the filter used the is-a operation, the caller would receive HAPI-2526. This has been corrected.

Fixed a bug in the MdmSurvivorshipSvcImpl that caused resourceType/id (as opposed to just id) to be passed on to the IIdHelperService.

Expanding a ValueSet on a property using a filter operator that is not supported (ISA, NOTISA, etc) would throw an exception and fail expansion. This has been fixed.

In an MDM enabled system with multi-delete enabled, deleting both the final source resource and it's linked golden resource at the same time results in an error being thrown. This has been fixed.

Previously, HAPI-FHIR schedulers were automatically suffixing the Quartz scheduler's org.quartz.scheduler.instanceName value to a unique autogenerated value based on the provided instance name. This has been corrected, and now the instance name provided to the BaseHapiFhirScheduler is used directly, with no unique suffix added. This is in line with the Quartz Documentation related to defining the scheduler instance name for those who run schedulers in a cluster.

The updated Spring Boot was not compatible with jetty version 12.0.3. Jetty has been updated to 12.0.9, fixing this issue.

Fixed a bug in search where requesting a count with HSearch indexing and FilterParameter enabled and using the _filter parameter would result in inaccurate results being returned. This happened because the count query would use an incorrect set of parameters to find the count, and the regular search when then try and ensure its results matched the count query (which it couldn't because it had different parameters).

Previously, attempting to store resources with common identifies but different partitions would. This has been fixed by adding a new configuration key defaulting to false to allow storing resources with duplicate identifiers across partitions. This new feature can be activated by calling PartitionSettings.setConditionalCreateDuplicateIdentifiersEnabled()

Two indexes introduced in HAPI-FHIR 6.6.0, IDX_SP_URI_HASH_IDENTITY_V2 and IDX_SP_URI_HASH_URI_V2 were previously created as unique indexes. This has caused issues on SQL Server due to the way that a filtered index is created. The unique clause was not necessary to this index, and has been removed.

The maximum length of SP_VALUE_EXACT and SP_VALUE_NORMALIZED in the HFJ_SPIDX_STRING table has been increased from 200 to 768. This allows for longer :contains searches. If you have previously stored values longer than 200 characters in String search parameters and want to perform a :contains search on the new longer maximum size, a reindex of the affected data is required.

The meta.profile element on resources was not being respected as canonical (ie, allowing for a version to be appended, http://example.com/StructureDefinition/abc|1.0.0), and was thus being ignored during validation. This has been fixed.

Fixed an issue where doing a cache refresh with advanced Hibernate Search enabled would result in an infinite loop of cache refresh -> search for StructureDefinition -> cache refresh, etc

Previously, using _text and _content searches in Hibernate Search in R5 was not supported. This issue has been fixed.

Rolled back org.apache.derby dependency to one that supports JREs 11 - 17.

Fixed a bug in SchemaMigrator that would considered skippable migration tasks as 'failed' migrations.

Added PATCH operation to BundleEntryTransactionMethodEnumTest in order to fully support transaction bundle operations.

The latest migration for HFJ_RES_SEARCH_URL introduced in 6033 does not fully respect dry-run. This has been fixed.

Removing deprecated thymeleaf syntax from existing templates, so as to avoid deprecation warnings in logs.

Before being processed, subscriptions would be read out of the database all at once. This lead to massive memory consumption if there were a lot of them. This has now been changed to use batching as a means of mitigating this problem.

Address care-gaps bug for handling missing improvement notation validation, and group level measure scoring definition. Bug fix for evaluate-measure subject-list where contained List resource was setting invalid references. Bumping to latest version of clinical reasoning 3.8.

Fix for regression of searches for canonical uris using a version (eg: http://example.com|1.2.3).

Fixed a bug where booting the JPA Server Starter would get a duplicate bean error. This has been corrected. Thanks to @subigre for the fix!

Previously, the package installer wouldn't create a composite SearchParameter resource if the SearchParameter resource didn't have an expression element at the root level. This has now been fixed by making SearchParameter validation in package installer consistent with the DAO level validations.

Searching or conditional creating/updating with a timestamp with an offset containing '+' fails with HAPI-1883. For example: 'Observation?date=2024-07-08T20:47:12.123+03:30' This has been fixed.

Previously, executing the '$validate' operation on a resource instance could result in an HTTP 400 Bad Request instead of an HTTP 200 OK response with a list of validation issues. This has been fixed.

IAnyResource _id search parameter was missing path property value, which resulted in extractor not working when standard search parameters were instantiated from defined context. This has been fixed, and also _LastUpdated, _tag, _profile, and _security parameter definitions were added to the class.

Previously, when retrieving a resource which may contain other resources, such as a document Bundle, if a ConsentService's willSeeResource returned AUTHORIZED or REJECT on this parent resource, the willSeeResource was still being called for the child resources. This has now been fixed so that if a consent service returns AUTHORIZED or REJECT for a parent resource, willSeeResource is not called for the child resources.

A bug with $everything operation was discovered when trying to search using hibernate search, this change makes all $everything operation rely on database search until hibernate search fully supports the operation.

Fixed a regression in 7.2.0 which caused systems using FILESYSTEM binary storage mode to be unable to read metadata documents that had been previously stored on disk.

Previously, if you upgraded from any older HAPI version to 6.6.0 or later, the SEARCH_UUID column length still showed as 36 despite it being updated to have a length of 48. This has now been fixed.

Previously, on MSSQL, two resources with IDs that are identical except for case (ex: Patient1 vs. patient1) would be considered to have the same ID because the database collation is case insensitive (SQL_Latin1_General_CP1_CI_AS). Among other things, this would manifest itself when trying to delete and re-create one of the resources. This has been fixed with a migration step that makes the collation on the resource ID case sensitive (SQL_Latin1_General_CP1_CS_AS).

Previously, the resource $validate operation would return a 404 when the associated profile uses a ValueSet that has multiple includes referencing Remote Terminology CodeSystem resources. This has been fixed to return a 200 with issues instead.

Previously, if you created a resource with some conditional url, but then submitted a transaction bundle that a) updated the resource to not match the condition anymore and b) create a resource with the (same) condition a unique index violation would result. This has been fixed.

Index IDX_IDXCMBTOKNU_HASHC on table HFJ_IDX_CMB_TOK_NU's migration is now marked as online (concurrent).

Previously, $apply-codesystem-delta-add and $apply-codesystem-delta-remove operations were failing with a 500 Server Error when invoked with a CodeSystem Resource payload that had a concept without a display element. This has now been fixed so that concepts without display field is accepted, as display element is not required.

When JPA servers are configured to always require a new database transaction when switching partitions, the server will now correctly identify the correct partition for FHIR transaction operations, and fail the operation if multiple partitions would be required.

Previously, the $reindex operation would fail when using a custom partitioning interceptor which decides the partition based on the resource type in the request. This has been fixed, such that we avoid retrieving the resource type from the request, rather we use the urls provided as parameters to the operation to determine the partitions.

Previously, a Subscription not marked as a cross-partition subscription could listen to incoming resources from other partitions. This issue is fixed.

A regression was temporarily introduced which caused searches by _lastUpdated to fail with a NullPointerException when using Lucene as the backing search engine. This has been corrected

The Derby JARs have been removed from this release. The older versions have multiple high-severity vulnerabilities against them, and the newer versions force usage of Java 21, which HAPI-FHIR does not yet support. If you wish to continue to use derby, you will have to provide the derby, derbyclient,derbynet, derbyshared, and derbytools jars on your classpath.

Added paging support for $everything operation in synchronous search mode.

Previously, searches combining the _text query parameter (using Lucene/Elasticsearch) with query parameters using the database (e.g. identifier or date) could miss matches when more than 500 results match the _text query parameter. This has been fixed, but may be slow if many results match the _text query and must be checked against the database parameters.

Added support for :contains parameter qualifier on the _text and _content Search Parameters. When using Hibernate Search, this will cause the search to perform an substring match on the provided value. Documentation can be found here.

Database migrations that add or drop an index no longer lock tables when running on Azure Sql Server.

Fixed a bug in search where requesting a count with HSearch indexing and FilterParameter enabled and using the _filter parameter would result in inaccurate results being returned. This happened because the count query would use an incorrect set of parameters to find the count, and the regular search when then try and ensure its results matched the count query (which it couldn't because it had different parameters).

Fixed an issue where doing a cache refresh with advanced Hibernate Search enabled would result in an infinite loop of cache refresh -> search for StructureDefinition -> cache refresh, etc

Previously, using _text and _content searches in Hibernate Search in R5 was not supported. This issue has been fixed.

A bug with $everything operation was discovered when trying to search using hibernate search, this change makes all $everything operation rely on database search until hibernate search fully supports the operation.

Fixed a regression in 7.2.0 which caused systems using FILESYSTEM binary storage mode to be unable to read metadata documents that had been previously stored on disk.

Previously, queries with chained would fail to sort correctly with lucene and full text searches enabled. This has been fixed.

The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• Apache Commons Compress (Base): 1.21 -> 1.26.0

Before, message broker always used JacksonMessageConverter. Added a configuration option that allows to disable it so other converters can be configured depending on the message type.

Several enhancements have been made to the International Patient Summary generator based on feedback from implementers:

• New methods have been added to the IIpsGenerationStrategy allowing resources for any or all sections to be fetched from a source other than the FHIR repository.
• The IpsSectionEnum class has been removed and replaced in any user-facing APIs with references to SectionRegistry.Section. This makes it much easier to extend or replace the section registry with custom sections not defined in the universal IPS implementation guide.
• Captions have been removed from narrative section tables, and replaced with H5 tags directly above the table. This results in an easier to read display since the table title will appear above the table instead of below it.
• The IPS narrative generator built in templates will now omit tables when the template specified multiple tables and the specific table would have no resources.

The JPA WebSocket delivery mechanism now supports the content delivery mode. Thanks to Artiom Darie for the contribution!

The SearchNarrowingInterceptor can now optionally be configured to also apply URL narrowing to conditional URLs used by conditional create/update/delete/patch operations, both as raw HTTP transactions as well as within FHIR transaction Bundles.

Added another state to the Batch2 work chunk state machine: READY. This work chunk state will be the initial state on creation. Once queued for delivery, they will transition to QUEUED. The exception is for ReductionStep chunks (because reduction steps are not read off of the queue, but executed by the maintenance job inline.

Update to the 3.2.0 release of the Clinical Reasoning Module. This includes the following changes:

• Updated the Clinical Reasoning documentation.
• Added support for additional parameters on operations.
• Added StructureDefinition/$questionnaire operation.
• Add ability to generate PlanDefinition/$apply results with unique ids.
• Resolved issues with Questionnaire item generation during PlanDefinition/$apply.
• Resolved issues with some request resources not generated correctly from ActivityDefinition/$apply

Added new POLL_WAITING state for WorkChunks in batch jobs. Also added RetryChunkLaterException for jobs that have steps that need to be retried at a later time (can be provided optionally to exception). If a step throws this new exception, it will be set with the new POLL_WAITING status and retried at a later time.

Change the implementation of CDS on FHIR to use the Auto Prefetch functionality and to no longer pass the fhirServer from the request into the dataEndpoint parameter of $apply.

Add support to _sort for chained composition Bundle SearchParameters

A new setting in JpaStorageSettings enforces a maximum file size for Bulk Export output files, as well as work chunks creating during processing. This setting has a default value of 100 MB.

The ConceptMap/$translate operation will include targets with an equivalence code of unmatched in the response regardless of whether the target has a code.

Added another state to the Batch2 work chunk state machine: GATE_WAITING. This work chunk state will be the initial state on creation for gated jobs. Once all chunks are completed for the previous step, they will transition to READY.

Added a new pointcut: STORAGE_PRE_INITIATE_BULK_EXPORT. This pointcut is meant to be called explicitly before STORAGE_INITIATE_BULK_EXPORT, so that parameter manipulation that needs to be done before permission checks (currently done using STORAGE_INITIATE_BULK_EXPORT) can safely be done without risk of affecting permission checks/ authorization.

Add additional Clinical Reasoning operations $collect-data and $data-requirements to HAPI-FHIR to expand capability for clinical reasoning module users. These operations will assist users in identifying data requirements for evaluating a measure, and what data was used for a measure evaluation

If using an OpenTelemetry agent, a span is now generated for each interceptor method call. The span is named as 'hapifhir.interceptor' and it has the following attributes about the interceptor: 'hapifhir.interceptor.pointcut_name','hapifhir.interceptor.class_name', 'hapifhir.interceptor.method_name'

Enhance RuleBuilder code to support multiple instance IDs.

As part of the migration from LOB, provided the capability to force persisting data to LOB columns. The default behavior is to not persist in lob columns.

The MDM_POST_MERGE_GOLDEN_RESOURCES now supports an additional parameter, of type ca.uhn.fhir.mdm.model.MdmTransactionContext. Thanks to Jens Villadsen for the contribution.

In the JPA server, several database columns related to Batch2 jobs and searching have been reworked so that they no will longer use LOB datatypes going forward. This is a significant advantage on Postgresql databases as it removes a significant use of the inefficient pg_largeobject table, and should yield performance boosts for MSSQL as well.

Migration of remaining database columns still using the LOB datatypes. This change effectively cuts all ties with the inefficient pg_largeobject table.

The IPS $summary generation API has been overhauled to make it more flexible for future use cases. Specifically, the section registry has been removed and folded into the generation strategy, and support has been added for non-JPA sources of data. This is a breaking change to the API, and implementers will need to update their code. This updated API incorporates community feedback, and should now be considered a stable API for IPS generation.

Extracted methods out of ResourceProviderFactory into ObservableSupplierSet so that functionality can be used by other services. Unit tests revealed a cleanup bug in MdmProviderLoader that is fixed in this MR.

The HFJ_FORCED_ID table is no longer used.

Fixed behaviour of the _list query parameter. Now it returns exclusively resources that are members of the given list. Thanks to Jens Villadsen (@jkiddo) for the contribution!

Previously, the fullUrl for resources in _history bundles was not generated correctly when using a client provided id. The same problem started to happen for the resources with server generated ids more recently (after 6.9.10). This has now been fixed

When processing a FHIR transaction in the JPA server, an identifier containing a system that has no value but has an extension present could cause a NullPointerException. This has been corrected.

Previously, creating an XML encoded FHIR resource with a decimal element that has a leading plus sign value would result in JsonParseException during the read operation from the database. Thus, making it impossible to retrieve or modify such resources. This has been fixed.

Added support for sorting on a chained location.near search. This allows you to sort location by nearness via a chained search. Thanks to Nicolai Gjøderum (@nigtrifork) for the contribution!

Avoid lock contention by refreshing SearchParameter cache in a new transaction.

Previously, when performing a FHIR search using a non-chained relative reference (returns entire resource) with a server assigned id, it ignores the invalid resourceType in the parameter value and proceeds with the id based lookup. e.g. GET /MedicationAdministration?context=abc/1352 returns Encounter/1352. This has been fixed.

The IPS Generator will no longer replace resource IDs with placeholder IDs in the resulting bundle by default, although this can be overridden in the generation strategy object.

The BundleBuilder utility class will no longer include the /_version/xxx portion of the resource ID in the Bundle.entry.fullUrl it generates, as the FHIR specification states that this should be omitted.

Previously, a DELETE on a specific URL search string would always attempt to delete no matter the number of resolved resources. This has been fixed by adding a storage setting to enforce a threshold for resolved resources, above which the DELETE operation will fail to execute with HAPI-2496.

Previously, invoking search URLs containing ':identifier' would result in a HAPI-1250 error complaining about an invalid resource type. This has been fixed by returning a clearer error message for this specific condition: HAPI-2498.

Previously, with validation active, when a user POSTed a resource with a meta profile with a non-existent StructureDefinition URL, then POSTed the StructureDefinition, POSTing the same or another patient with that same meta profile URL would still fail with a VALIDATION_VAL_PROFILE_UNKNOWN_NOT_POLICY validation error. This has been fixed.

Previously, a Patch operation would fail when adding a complex extension, i.e. an extension comprised of another extension. This issue has been fixed.

System-level and Type-level History operations on the JPA server (i.e. _history) could sometimes contain duplicates or miss entries when a large number of matching resources on the server had identical update timestamps. This has been corrected.

An incorrect migration script caused a failure when upgrading to HAPI FHIR 7.0.0 on PostgreSQL if the database was not in the public schema. Thanks to GitHub user @pano-smals for the contribution!

The recommended constructor was not present on hibernate dialects provided by HAPI FHIR, leading to warnings during startup, and failures in some cases. This has been corrected. Thanks to GitHub user @pano-smals for the contribution!

Previously, using the FhirTerser to process an Extension with an Enumeration would fail. This has been fixed.

Previously, transactions that had entries with Bundle requests failed with HAPI-0339: Can not handle transaction with nested resource of type Bundle. This has been fixed and now transactions are permitted that contain entries with Bundle requests that have a specified url (i.e. POST to /Bundle), but are rejected if no url is specified (i.e. nested transactions or batches).

A scheduled job to clean up the Search URL table used to enforce uniqueness of conditional create/update jobs was created as a local job and not a clustered job. This has been fixed.

Previously, complex _has then chain then _has searches would fail to return results when expected (ex: 'Practitioner?_has:ExplanationOfBenefit:care-team:coverage.payor._has:List:item:_id=list1') This has been fixed.

Fixed behaviour of the _language query parameter. Now it is picked up as search parameter in the resource provider and filters accordingly. Thanks to Jens Villadsen (@jkiddo) for the contribution!

Batch2 jobs with reduction steps didn't fire the completion handler. This has been fixed.

Batch2 jobs with reduction steps didn't fire the completion handler with the up-to-date job status. This has been fixed.

Previously, the casing of the X-Request-ID header key was not retained in the corresponding response. This has been fixed.

Support for the _language parameter was added but it was not able to be used by clients of a JPA server because the _language parameter was not added to the resource providers. Additionally, no error message was returned when language support was disabled and a search with _language was performed. This has been fixed.

Previously, the 'planDefinition' resource parameter for the Dstu3 version of PlanDefinition/$apply was incorrectly defined as an R4 resource. This issue has been fixed.

Unifying the code paths for Patient type export and Patient instance export. These paths should be the same, since type is defined by spec, but instance is just 'syntactic sugar' on top of that spec (and so should be the same).

We now avoid a query during reindex and transaction processing that was very slow on Sql Server.

When batch 2 jobs with Reduction steps fail in the final part of the reduction step, this would often leave the job stuck in the FINALIZE state. This has been fixed; the job will now FAIL.

Fixed a bug where an NPE was being thrown when trying to serialize a FHIR fragment (e.g., backboneElement , compound datatype) to a string representation if the fragment contains a Reference.

Previously, it was possible to execute the $delete-expunge operation on a resource even if the user did not have delete permissions for the given resource type. This has been fixed.

Moving the Hibernate.Search annotation for text indexing from the lob column to the column added as part of the PostgreSql LOB migration.

Fixed a bug where 'List' would be incorrectly shown as 'ListResource' in the error response for a GET for an invalid resource.

Previously, updating a tokenParam with a value greater than 200 characters would raise a SQLException. This issue has been fixed.

Previously, either updating links on, or deleting one of two patients with non-numeric IDs linked to a golden patient would result in a HAPI-0389 if there were survivorship rules. This issue has been fixed for both the update links and delete cases.

Updated documentation on binary_security_interceptor to specify using STORAGE_PRE_INITIATE_BULK_EXPORT not STORAGE_INITIATE_BULK_EXPORT pointcut to change bulk export parameters.

Previously, hapi-fhir-cli: upload-terminology failed with a HAPI-0862 error when uploading LOINC. This has been fixed.

Previously, triggering a $meta via GET on a new patient with Megascale configured resulted in error HAPI-0389. This has been corrected This has been fixed.

Chained sort would exclude results that did not have resources matching the sort chain. These are now included, and sorted at the end.

Previously, in some edge case scenarios the Bulk Export Rule Applier could accidentally permit a Patient type level bulk export request, even if the calling user only had permissions to a subset of patients. This has been corrected.

Fix chained sorts on strings when using MS Sql

Previously, queries with chained would fail to sort correctly with lucene and full text searches enabled. This has been fixed.

Fixed a potential XSS vulnerability in the HAPI FHIR Testpage Overlay module.

Database migrations that add or drop an index no longer lock tables when running on Azure Sql Server.

Avoid lock contention by refreshing SearchParameter cache in a new transaction.

An incorrect migration script caused a failure when upgrading to HAPI FHIR 7.0.0 on PostgreSQL if the database was not in the public schema. Thanks to GitHub user @pano-smals for the contribution!

Fixed behaviour of the _language query parameter. Now it is picked up as search parameter in the resource provider and filters accordingly. Thanks to Jens Villadsen (@jkiddo) for the contribution!

Fixed a potential XSS vulnerability in the HAPI FHIR Testpage Overlay module.

The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• Jackson (Base): 2.15.3 -> 2.16.1
• SLF4j (Base): 2.0.3 -> 2.0.9
• Logback (Base): 1.4.7 -> 1.4.14
• Caffeine (Base): 3.1.1 -> 3.1.8
• Spring Framework (JPA): 5.3.27 -> 6.1.1
• Spring Boot (JPA-Starter): 5.3.27 -> 6.2.0
• Spring Data BOM (JPA): 2021.2.2 -> 2023.1.0
• Hibernate (JPA): 5.6.15.Final -> 6.4.1.Final
• Hibernate Validator (JPA): 6.1.5.Final -> 8.0.0.Final
• Hibernate Search (JPA): 6.1.6.Final -> 7.0.0.Final
• Commons-DBCP2 (JPA): 2.9.0 -> 2.11.0
• DataSource-Proxy (JPA): 1.9 -> 1.10
• Spring Boot (Boot+Starter): 2.7.12 -> 3.1.4
• Jetty (CLI): 10.0.14 -> 12.0.3
• Jansi (CLI): 2.4.0 -> 2.4.1
• Derby (CLI): 10.14.2.0 -> 10.17.1.0
• Commons-Lang3 (CLI): 3.12.0 -> 3.14.0
• Commons-CSV (CLI): 1.8 -> 1.10.0
• Phloc Schematron (Schematron Validator): 5.6.5 -> 7.1.2
• RestEasy (JAX-RS Server): 5.0.2.Final -> 6.2.5.Final

Added MDM support for FHIR R5.

A match result map field has been added to the $mdm-link-history operation. This new field shows the rules that evaluated true during matching and the corresponding initial match result when the link was created.

The error messages returned in an OperationOutcome when validating terminology codes as a part of resource profile validation have been improved. Machine processable location (line/col) information is now available through a pair of dedicated extensions, and error messages such as UCUM parsing issues are now returned to the client (previously they were swallowed and a generic error message was returned).

It is now possible to configure the strictness of concept display name validation using a new flag on the InMemoryTerminologyServerValidationSupport (for non-JPA validation) and JpaStorageSettings (for JPA validation). In addition, the error messages emitted by the validator when a concept display doesn't match have been improved to be much more useful.

Previously, it was only possible to clear a top-level field on a resource using TerserUtil. A new method has been added to TerserUtil to support clearing a value by FhirPath.

Added a new method getResourceByReferenceAndResourceType() in BundleUtil.java to find a specific Resource from Bundle using Reference.

Moving clinical reasoning to non-javax dependency for alignment with incoming hapi-fhir changes, bumping CR version and updating impacted classes and services

Added Terminology Troubleshooting Log to support troubleshooting terminology issues.

The ValidatorResourceFetcher will now resolve canonical URL references as well as simple local references.

There is a programmatic filter enabled which skips installation for certain resources based on their status in PackageInstallerSvcImpl. The filter can now be controlled via StorageSettings.

A new method on the IValidationSupport interface called lookupCode(LookupCodeRequest) has been added. This method will replace the existing lookupCode methods, which are now deprecated.

Updated $member-match operation signature to match the latest specification.

Updated Consent storage in $member-match operation.

Added support for id-only and empty payload content types for notifications triggered by R5, R4B, and R4 back-ported topic subscriptions.

It is now possible to mutate an HTTP response from the CLIENT_RESPONSE Pointcut, and pass this mutated response to downstream processing.

Added a map of additionalData to BulkExport job params. This will allow consumers of BulkExport to add additional data to be accessed at later steps by using various pointcuts in the system. Updated ConsentService so that BulkExport operations will call the willSeeResource method for each exported resource.

In code: Support lowercase for SQL columns and overridden column type/driver type SQL type string rules

An interceptor was added to block resource updates which would cause the resource to change Patient compartment. Please see JPA Server: Block Resource Updates Changing Patient Compartment for more information.

Upgrade Clinical Reasoning version for latest bug fixes and available operations. Add $evaluate library operation.

Added auto-version-references-at-path extension that allows to enable auto versioning references at specified paths of resource instances.

Hibernate SQL log filtering capability was added. See Hibernate SQL Log Filtering.

Subscription triggering via the $trigger-subscription operation is now multi-threaded, which significantly improves performance for large data sets.

Previously, resource body content went into one of 2 columns on the HFJ_RES_VER table: RES_TEXT if the size was above a configurable threshold, or RES_TEXT_VC if it was below that threshold. Performance testing has shown that the latter is always faster, and that on Postgres the use of the latter is particularly problematic since it maps to the largeobject table which isn't the recommended way of storing high frequency objects. The configurable threshold is now ignored, and the latter column is always used. Any legacy data in the former column will still be read however.

HAPI FHIR JPA now requires PostgreSQL 10+. Previously Postgres 9.4 was supported but this version has been dropped.

The reindexing and mdm-clear batch jobs now stream results internally for more reliable operation.

The resource dao interface now supports searching for IIdType or full resources.

Swapped from using javax.* to jakarta.* packages. This is a breaking change for a large majority of people who write custom code against HAPI-FHIR. Please see the migration guide for more information.

Previously, rule builder could not effectively handle Patient Type-Level Exports. It would over-permit requests in certain scenarios. This fix allows for accumulation of ids on a Patient Type-Level Bulk export to enable us to properly match the requested Patient IDs against the users permitted Patient IDs.

Fixed a bug where search Bundles with include entries from an _include query parameter might trigger a 'next' link to blank pages when no more results match results are available.

A regression was introduced in 2023.08.R01 which caused binary storage prefixes to not be applied to exported binary blobs. This has been fixed.

Updated documentation specifying the correct status (CANCELLED) to set the job status to if a job is cancelled.

Previously, when using revincludes and includes with iterate, while also using revincludes without iterate, the result omitted some resources that should have been included. This issue has now been fixed.

Added warnings when MDM candidate search parameters are not defined and candidate search limit is exceeded.

Previously, the $mdm-query-link operation would return values of field linkCreated and linkUpdated in scientific notation when the last digits are 0. This is now fixed and always returns in standard notation.

Previously, searching with parameter '_total' could influence chunked query resultsets and subsequently, paged results. This is now fixed.

Previously, expanding a ValueSet using hierarchical CodeSystem would fail in different scenarios with a constraint violation exception when codes (term concepts) were being persisted. This is now fixed.

Several fixes to the HAPI FHIR generated OpenAPI schema have been implemented. This means that the spec now validates cleanly. Thanks to Primož Delopst for the contribution!

The hapi-fhir-validation module inadvertently included a mandatory dependency on the Caffeine caching library instead of leaving it to implementors to choose which module to pick. This has been corrected.

Ensure batch2 jpa persistence always targets the default partition.

Previously, testing database migration with cli migrate-database command in dry-run mode would insert in the migration task table. The issue has been fixed.

Previously, it was impossible to find all resources from different partitions for $everything operation with partitioning.cross_partition_reference_mode=ALLOWED_UNQUALIFIED and dao_config.client_id_mode=ANY. It's fixed now

Ensure batch jobs target the default partition for non-gated steps.

The HFQL/SQL engine incorrectly parsed expressions containing a >= or <= comparator in a WHERE clause. This has been corrected. Additionally, the execution engine has been optimized to apply clauses against the meta.lastUpdated path more efficiently by using the equivalent search parameter automatically.

Previously, when creating an index as a part of a migration, if the index already existed with a different name on Oracle, the migration would fail. This has been fixed so that the create index migration task now recovers with a warning message if the index already exists with a different name.

Previously, CodeSystem $lookup with Remote Terminology Service enabled would throw NullPointerException when the CodeSystem included designations with no language value. Also, there was an inconsistency between input and output type string vs. code for property parameters. These issues have been fixed.

Previously, it was possible to store NPM Packages where the package name's case did not match the package ID's case, e.g. my-package was different than MY-PACKAGE. Names are now normalized to lower case before queries occur.

When using a chained SearchParameter to search within a Bundle as described here, if the Bundle.entry.fullUrl was fully qualified but the reference was not, the search did not work. This has been corrected.

Calling the method getOrCreateContentType in AttachmentUtil on an attachment with no content type would throw exception because contentType is a code not a string. This fixes the function to create an empty code as expected

A database migration added trailing spaces to server-assigned resource ids. This fix removes the bad migration, and adds another migration to fix the errors.

The addition of the indexes idx_sp_uri_hash_identity_pattern_ops and idx_sp_string_hash_nrm_pattern_ops could occasionally timeout during migration in Postgresql on large databases, leaving the migration table in a failed state, and Smile CDR unable to boot. Now existence of the index is checked before attempting to add it again.

Previously LIKE queries against resources would perform poorly on PostgreSQL if the database locale/collation was not 'C'. This has been resolved by checking hfj_spidx_string.sp_value_normalized and hfj_spidx_uri.sp_uri column collations during migration and if either or both are non C, create a new btree varchar_pattern_ops on the hash values. If both column collations are 'C', do not create any new indexes.

mdm-clear jobs are prone to failing because of deadlocks when running on SQL Server. Such job failures have been mitigated to some extent by increasing the retries on deadlocks.

Previously, certain mdm configuration could lead to duplicate eid identifier entries in golden resources. This has been corrected

Previously, FHIRPath expression evaluation when using the _fhirpath parameter would not work on chained use of 'resolve()'. This was most notable when using _fhirpath with FHIR Documents (i.e. 'Bundle' of type 'document' where 'entry[0]' is a 'Composition'). This has now been fixed.

When updating or reindexing a resource in the JPA server, if duplicate rows are present in the indexing tables, the duplicate rows may fail to be removed. Duplicates are not typically present in these tables, but can happen if an indexing job fails in some circumstances. The impact of these rows not being cleaned up is that resources may appear in search results that they should no longer appear in.

When encoding resources using the RDF parser, placeholder IDs (i.e. resource IDs starting with urn:) were not omitted as they are in the XML and JSON parsers. This has been corrected.

When encoding a Bundle, if resources in bundle entries had a value in Bundle.entry.fullUrl but no value in Bundle.entry.resource.id, the parser sometimes incorrectly moved these resources to be contained within other resources when serializing the bundle. This has been corrected.

$everything queries with MDM expansion were bypassing the partition boundary by return resources outside the partition. Also, POST (as opposed to GET) $everything queries with MDM expansion. The first issue was fixed by reversing the previous changes in 5493 and by filtering source and golden resources by partition ID. The second issue was fixed by correctly capturing the MDM expansion flag in POST $everything queries.

Previously, instance-level Patient Bulk Export batch job would fail if the _type parameter was empty. This is now fixed and _type will default to all resource types in the Patient Compartment plus Device.

Previously, the semantics of is-a were incorrect in Valueset Expansion. The implementation previously used the behaviour of descendent-of, which means that A is-a A was not being considered as true. This has been corrected. In addition, descendent-of is now supported, which compares for strict descendency, and does not include itself. Thanks to Ole Hedegaard (@ohetrifork) for the fix.

Fixed an issue where executing $trigger-subscription with a search URL criteria on a partitioned Subscription resource would result in the failure to deliver the affected resources. This issue has now been resolved.

Resource UserData RESOURCE_PARTITION_ID was incorrectly being set to null for the default partition. This has been corrected to use RequestPartitionId.defaultPartition()

Previously, when a transaction was posted with a resource that had placeholder references and auto versioning references enabled for that path, if the target resource was included in the Bundle but not modified, the reference was saved with a version number that didn't exist. This has been fixed.

Fixed a deadlock in resource conditional create.

Previously, searches that used more than one chained Bundle SearchParameter (i.e. Composition) were only adding one condition to the underlying SQL query which resulted in incorrect search results. This has been fixed.

Previously, an exception could be thrown by the container when executing a contextClosedEvent on the Scheduler Service. This issue has been fixed.

Previously bulk export operation was returning an empty response when no resources matched the request, which didn't comply with HL7 HAPI IG. This has been corrected.

Smile failed to save resources running on Oracle when installed from 2023-02 or earlier. This has been fixed.

Previously, expanding a 'ValueSet' with no concepts based on system urn:ietf:bcp:13 would fail with ExpansionCouldNotBeCompletedInternallyException. This has been fixed.

Previously, the number of threads allocated to the $expunge operation in certain cases could be more than configured, this would cause hundreds of threads to be created and all available database connections to be consumed. This has been fixed.

Clinical reasoning version bump to address reported 'null pointer' error that is encountered when running $evaluate-measure against a measure with an omitted measure.group.population.id

A non-superuser with correct permissions encounters HAPI-0339 when POSTING a transaction Bundle with a PATCH. This has been fixed.

Previously, searching for Bundle resources with read all Bundle resources permissions, returned an HTTP 403 Forbidden error. This was because the AuthorizationInterceptor applied permissions to the resources inside the Bundle, instead of the Bundle itself. This has been fixed and permissions are no longer applied to the resources inside a Bundle of type document, message, or collection for Bundle requests.

Change database upgrade script to avoid holding locks while adding indices.

Previously, conditional creates would fail with HAPI-0929 errors if there was no preceding '?'. This has been fixed.

Fixed a MeasureReport measureScoring bug impacting any measures currently using denominator-exception population will incorrectly calculate the score without following specification. This bug adds an extension to MeasureReport Groups to capture calculated denominator and numerator to bring transparency to the measureScore calculation and act as a dataSource of measureScore instead of behind the scenes calculations.

Previously, the Bulk Import ($import) job was ignoring the httpBasicCredentials section of the incoming parameters object, causing the job to fail with a 403 error. This has been corrected.

Previously, after registering built-in interceptor PatientIdPartitionInterceptor, the system bulk export (with no filters) operation would fail with a NullPointerException. This has been fixed.

Previously, after registering built-in interceptor PatientIdPartitionInterceptor, while performing an async system bulk export, the $poll-export-status operation would fail with a NullPointerException. This has been fixed.

Previously (this release cycle), when you call RemoteTerminologyServiceValidationSupport method lookupCode with a CodeSystem that has properties that are not string or Coding, the method would throw an exception. It should instead accept any type and convert any unsupported type to string. This has been fixed.

The legacy $lastn support has been removed, and only the redesigned 'advanced' version is now supported. The advanced version requires Elasticsearch.

Removed an incorrect implementation of $member-match.

In the JPA server, several database columns related to Batch2 jobs and searching have been reworked so that they no will longer use LOB datatypes going forward. This is a significant advantage on Postgresql databases as it removes a significant use of the inefficient pg_largeobject table, and should yield performance boosts for MSSQL as well.

Previously, testing database migration with cli migrate-database command in dry-run mode would insert in the migration task table. The issue has been fixed.

Previously, when creating an index as a part of a migration, if the index already existed with a different name on Oracle, the migration would fail. This has been fixed so that the create index migration task now recovers with a warning message if the index already exists with a different name.

Subscription triggering via the $trigger-subscription operation is now multi-threaded, which significantly improves performance for large data sets.

Previously, when using revincludes and includes with iterate, while also using revincludes without iterate, the result omitted some resources that should have been included. This issue has now been fixed.

The error messages returned in an OperationOutcome when validating terminology codes as a part of resource profile validation have been improved. Machine processable location (line/col) information is now available through a pair of dedicated extensions, and error messages such as UCUM parsing issues are now returned to the client (previously they were swallowed and a generic error message was returned).

It is now possible to configure the strictness of concept display name validation using a new flag on the InMemoryTerminologyServerValidationSupport (for non-JPA validation) and JpaStorageSettings (for JPA validation). In addition, the error messages emitted by the validator when a concept display doesn't match have been improved to be much more useful.

A regression was introduced in 2023.08.R01 which caused binary storage prefixes to not be applied to exported binary blobs. This has been fixed.

The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• Hibernate ORM (JPA): 5.6.12.Final -> 5.6.15.Final

The @Interceptor annotation can now be placed at the method level. This is used only as a marker, and does not change the behaviour or interceptors in any way. Thanks to Dominique Villard for the pull request!

The batch2 framework now supports warning messages.

A new interceptor called BalpAuditCaptureInterceptor has been added. This interceptor automatically generates AuditEvent resources in a HAPI FHIR server that are conformant with the IHE BasicAudit Log Patterns IG.

When invoking the instance level $reindex and $reindex-dryrun operations, the resulting status message and any warnings are now included in the response Parameters object as well as in the generated response HTML narrative.

When performing a resource reindex on a deleted resource, any search index rows will now be deleted. Deleting a resource should generally not leave any such rows behind, but they can be left if the resource is manually deleted using SQL directly against the database and in this case the reindex job will now clean up these unwanted rows.

Added a new SubscriptionTopicDispatcher service for use by java extensions that need to dispatch their own subscription topic notifications

Added support for Topic Subscription Filters. In R4B and R5, SubscriptionTopic notifications will interpret subscription filters as FHIR Search Parameters and perform an in-memory match of the focus resource against the search parameters. For R4 or user-generated event notifications, the caller can specify a custom implementation of ISubscriptionTopicFilterMatcher on the call to SubscriptionTopicDispatcher.dispatch() to provide custom filter matching logic.

Added support for Subscriptions R5 Backport to R4 as documented in http://build.fhir.org/ig/HL7/fhir-subscription-backport-ig/components.html.

The $delete-expunge operation has a new parameter cascade that can be used to request that resources with indexed references to other resources being deleted should also be deleted.

The SQL schema migrator now returns a status flag indicating whether the schema was initialized or not.

The IParser interface now has an additional pair of methods called encodeToString(IBase) and encodeToWriter(IBase, Writer) that can be used to encode fragments of resources, such as datatypes or infrastructure elements.

Previously, all revincludes would always be evaluated before all includes. Now the system checks if any revincludes are recursive (i.e. are modified :iterate) and if so, it evaluates all the includes first.

There was a bug with automatic partition ID selection where it was not bound to positive integers and could return a value that was out of range. This has been fixed.

It is no longer mandatory to provide a partition ID when executing the $partition-management-create-partition operation. If you omit the partition ID, one will be randomly selected from the available pool of values. Note that this may incur a small performance cost when creating partitions.

A new configuration option, JpaStorageSettings#setOnlyAllowInMemorySubscriptions(boolean) can now prevent the creation of Subscriptions that would need to be matched in the database. If enabled, only Subscriptions which can be evaluated IN-MEMORY are permitted to be created.

A new interceptor called BinarySecurityInterceptor has been added. This interceptor can be used to enforce access to Binary resources by using values in the Binary.securityContext element.

Added infrastructure to allow consumers to define MDM block rules based on fhir path and specific values. To utilize this feature, an IBlockListRuleProvider must be wired up with the required rules json.

Added a utility method to the TerserUtil class to facilitate the creation of new instances of BackboneElement classes.

Previously, calls to the Fulltext Search service (Lucene/Elasticsearch) did not invoker the JPA_PERFTRACE_INFO. Now, they invoke this pointcut with information of which query went out to the fulltext system.

Extended the existing MDM similarity algorithms to numeric values such that the input is normalized by removing all non-numeric characters from the string before the similarity algorithm is applied. This can be useful when wanting to measure similarity between identifying numbers or phone numbers where dashes or other special separating characters may be used.

A new SQL-like evaluator called the HAPI FHIR Query Language (HFQL) has been added.

The IFhirPath evaluator interface now has an additional overload of the evaluate method which takes in a parsed expression returned by the parse method. This can be used to improve performance in cases where the same expression is being used repeatedly.

Added support for :above, :below, :contains and :missing _source search parameter modifiers.

The Generic/Fluent client can now handle arbitrary (ie. non-FHIR) responses from $operation invocation by specifying a response resource type of Binary.

A new experimental SQL-like query syntax called HFQL (HAPI FHIR Query Language) has been added.

OpenAPI definitions were not working for R5 JPA servers. This has been corrected.

Added support for Subscription matching of ':above', ':below', ':contains' and ':missing' '_source' search parameter modifiers.

Conditional deletes that delete multiple resources at once have been optimized to perform fewer SQL select statements, which should improve performance on large deletes.

Introduce IBaseResource.isDeleted() method and convert code to use it. Add subscription_topic_troubleshooting log. No longer rely on ResourceGoneException to detect deleted subscription. Instead use the new isDeleted() method. Demote unexpected exceptions in HapiTransactionService from error to debug since these exceptions are expected e.g. when checking if a resource has been deleted by catching a ResourceGoneException

Searching for token parameters has been extended to systems and values of any length. Sorting by tokens is still limited to the first 200 characters each of the system and value. valid request.

Bulk export fetch operations now properly account for the partition being searched, which should improve performance in some cases on partitioned systems.

A new interceptor pointcut called STORAGE_BULK_EXPORT_RESOURCE_INCLUSION has been added. This pointcut is called for every resource that will be included in a bulk export data file. Hook methods may modify the resources or signal that they should be filtered out.

The class BulkDataExportOptions has been removed and replaced with an equivalent class called BulkExportJobParameters. This change is a breaking change unfortunately, but should be easily remedied as the replacement class has a very similar signature. This change reduces a large amount of duplication in the bulk export codebase and should make for more maintainable code in the future.

Previously, when running with mdm, the server automatically changed storage settings to enable message subscriptions. This has been changed to require the user to explicitly enable message subscriptions when running with mdm. If mdm is enabled and message subscriptions are not enabled, the server will now throw an exception on startup.

When performing a search using a plain server without a cache, pagination links could contain invalid offsets in the prev link. Thanks to Aleksej Parovysnik for the pull request!

When parsing a FHIR resource which has multiple value[x] choice fields, the parser does not call the IParserErrorHandler#unexpectedRepeatingElement method on the configured parser error handler. Thanks to Max Bureck for the pull request!

FHIR Patch operations on choice types in nested expressions did not work correctly. Thanks to Zach Smith for the pull request!

HashMapResourceProvider invoked the STORAGE_PRESTORAGE_xxx and STORAGE_PRECOMMIT_xxx pointcuts after storing the resource into memory. This means that if any interceptors threw an exception in an attempt to abort the transaction, this did not actually result in the resource storage being aborted. This has been corrected.

There is an issue where searching on paramName '_profile' with modifier 'missing' will not provide the correct search result. This has been fixed.

Under heavy concurrency, a bug resulted in identical tag definitions being rejected with a NonUniqueResultException some of the time. This has been corrected.

When performing a FHIR transaction containing both a conditional delete as well as a conditional create/update for the same resource, the resource was left in an inconsistent state. This has been corrected. Thanks to Laxman Singh for raising this issue.

When serializing resources in JSON, if resource came from an XML-parsed resource that contained comments, the JSON serializer incorrectly created an empty object. This has been corrected.

Two failures in the $delete-expunge operation were fixed:

• Jobs could fail if hibernate search was loaded but not enabled.
• Jobs could fail if the criteria included a _lastUpdated=lt[date] clause

When reindexing resources, deleted resources could incorrectly fail validation rules and cause the reindex job to not complete correctly. This has been corrected.

The BALP AsyncMemoryQueueBackedFhirClientBalpSink incorrectly used a non-blocking method to add events to the blocking queue, resulting in race conditions on a heavily loaded server.

There was a bug with the TerserUtil, where it would not overwrite non-empty values with empty values from a source resource. This has been corrected. Thanks to @nigtrifork for the fix!

Batch jobs occasionaly reported zero (0) record processed counts. This has been corrected.

When Index Contained References is enabled in the JPA server, Bundle resources could not be stored or indexed due to an incompatibility with the default Bundle search parameters. This has been corrected.

Previously, it was possible to create composite SP with any types of SP as components. This has been fixed by limiting the component SP types to String, Token, Date, or Quantity.

The _lastUpdated query parameter is no longer applied to _include or _revinclude search results.

Matching algorithms have been refactored to allow greater flexibility in setting and defining nicknames as well as allowing bean injection into matcher classes.

R5 Subscription.filterBy.resourceType failed to deserialize because the deserializer skipped all elements named 'resourceType'. This has been changed so that only toplevel resourceType elements are skipped in the deserialization process.

Previously, installing or uninstalling an implementation guide does not refresh the validation cache. This problem has been fixed.

Previously, logging level on specific services was set too high. This has been modified.

Previously, the resource server Id strategy was not honoured when doing a create with a conditional update operation on a resource where the ID was not assigned. This is now fixed.

Previously, performing a $validate-code operation with a remote terminology service on an invalid CodeSystem or ValueSet returned 500. This problem has been fixed.

Previously, set synchronous search size to a low value will make MDM startup validation fail. This is now fixed.

Running a $delete-expunge MSSQL or Oracle with over 10,000 resources results in a error and a stack trace, even though the job ends in status COMPLETE. This has been fixed.

Previously, when performing a $mdm-clear followed by an update to an existing resource without doing the required $mdm-submit first, a non-descriptive NullPointerException was thrown. To fix this, an exception that includes a more descriptive/suggestive message will be thrown when this condition is detected.

Previously, expanding a valueSet including a codeSystem that is unknown to the server would return an internal error(500) to the caller. This problem has been fixed.

Added a test to verify that all Foreign Key constraints are explicitly indexed as well. Added indexes to a number of tables that declared foreign keys that weren't explicitly indexed. This should not only make many of these queries (including $mdm-clear operations) much faster, it will prevent deadlocks in Oracle and other databases that require foreign keys to be indexed.

$mdm-clear has been fixed to respect batch-size parameter (unless it exceeds 500). It has also been made slightly more efficient.

Search bundle entry parts were being created as INCLUDE unless explicitly indicated as MATCH. This has been fixed. Now they are created as MATCH unless explicitly indicated as INCLUDE.

Bulk export with patient ID specific user rules/permissions will fail with Parameters that include that patient ID in a _typeFilter. This has been fixed.

Previously, the JPA_PERFTRACE_RAW_SQL was not firing for raw sql that occurred during a revinclude or include. This has been corrected.

Previously when requesting previous or prev link using BundleUtil.getLinkUrlOfType, only the literal requested link was returned. Now when requesting either previous or prev previous or prev link is returned. If both exist, equality is validated and exception is thrown on failure.

Nickname Service Config has been moved out of search parameters as an optional config to add. Nickname service now has an interface to allow flexibility in implementation.

Previously, the GenericClient would only support the HTTP GET method for paging requests. This has been corrected by adding the possibility for paging with an HTTP POST.

Previously, if a partition was created with an ID that was already in use, it would overwrite the partition that was using that ID. Now attempting to overwrite will return a 400

Previously, running batch jobs with postgresql as the db will result in SQL error. This has now been fixed.

Jpa migration task used awssdk StringUtils rather than apache commons string utils. Thanks to @Thopap for the contribution

Fixed a condition by which a NPE was sometimes thrown when performing gateway pagination with targets with property useHttpPostForAllSearches configured to true.

Updated the query that is supposed to help detect foreign key constraints that also do not have indexes to be a) simpler (smaller) and b) find more results. Added an additional entry to the whitelist as a result.

Previously, when the last source resource with a MATCH link was deleted, the golden resource remained in the database, leaving it orphaned. This has now been fixed such that when there are no more MATCH links left associated with a golden resource, the golden resource will automatically be deleted.

Fixed an issue where reindex batch jobs failed to start and were stuck in QUEUED status after a SearchParameter update.

Delete expunge (and possibly other batch 2 jobs) will fail when given more than 2100 resources with identical timestamps when using an mssql database. This has been fixed.

Fixed recently introduced regression that when following bundle next links through to the end, in certain circumstances a next link would appear in the final bundle when in fact it was the last page.

The system was failing to recognize custom resources during transaction processing. This has been fixed.

Fixed FindCandidateByExampleSvc.findMatchGoldenResourceCandidates to not return duplicate golden resource candidates.

Fixed a bug in FindCandidateByExampleSvc that resulted in class cast exceptions for IResourcePersistentIds that are not based on Long value ids.

Consider user-assigned IDs, instead of system assigned IDs, when sorting MDM links history.

Remove + from the list of characters that are escaped automatically since + is already an escaped character (representing a space) in the query part of URLs.

Returns better diagnostics message to expunge requests that failed due to unfinished batch deletion

Previously, all MDM field scores, including NO_MATCHes, were included in the final total MDM score. This has now been fixed so that only MATCHed fields are included in the total MDM score.

Previously, when the consent service would remove all resources to be returned, the response bundle would not provide the previous/next link(s). This has been corrected.

Previously, updating from Hapi-fhir 6.6.0 to 6.8.0 would cause migration error, it is now fixed.

When running a $delete-expunge with over 10,000 resources, only the first 10,000 resources were deleted. This is now fixed.

Previously, requesting an $expunge operation on CodeSystem resources while CS batch deletion is underway would return HTTP 500. This has been fixed to return HTTP 412 (precondition failed).

Previously, the reuse functionality did not operate correctly when dealing with POST and GET requests. This fix ensures that similar POST and GET export requests will be reused.

Fixed a dependency in the HSQL JDBC driver referencing a non-bundled class (javax.ServletOutputStream)

Fix gateway $everything operation to respect server configured default and maximum page sizes.

Added evaluation setting for hapi-fhir storage-cr module operations from outside. Updated provider loading from hapi-fhir instead of external server for caregaps and submitdata providers. Updated testing suite to depend on restful server for new provider loader

Previously, removing tags in a resource update with proper headers and versioning flag would not trigger a new subscription. This has been fixed.

The latest US Core IG includes two ValueSets with different contents, but the same FHIR Id and OID via two different included IGs (i.e. 2.16.840.1.113762.1.4.1010.9 via us.cdc.phinvads and us.nlm.vsac). Ingesting these duplicates in US Core failed with a 500 error. This has been resolved by logging the error and allowing the rest of the ingestion to proceed.

Previously, uploading a large vocabulary file (like Loinc) through the upload-external-code-system command would return an error. The issue has been fixed.

Jpa migration task used awssdk StringUtils rather than apache commons string utils. Thanks to @Thopap for the contribution

The HapiTransactionService (HAPI FHIR JPA Transaction Manager) has been improved and used in a number of new places (replacing the @Transactional annotation). This means that stack traces have much less transaction logic noise, and give a clear indication of transaction boundaries.

A new API has been added to the JPA server for storing resources which are actually placeholders for externally stored payloads. This API is not yet used for any published use cases but will be rolled out for future uses.

Added support for the $care-gaps operation defined by the DaVinci DEQM IG

When creating or modifying a SearchParameter in the JPA server, the local SearchParameter cache is now immediately flushed. This should help with situations such as tests where a SearchParameter is created and then used before the scheduled cache refresh typically occurs.

The IPackageInstallerSvc interface for managing NPM packages now exposes an uninstall method. Thanks to Jens Kristian Villadsen for the pull request!

Preliminary support for R5 and R4B SubscriptionTopic matching has been added. This is not yet complete, but the simplest use cases now work. Comments in the code with prefix 'WIP STR5' indicate areas that need to be extended.

Add support for MdmLink history in the data model. MdmLink history will be stored in the new mdm_link_aud table.

A new JPA feature called Uplifed Refchains has been added. This feature allows chained search expressions to be precalculated for much better performance when executing chained searches.

JPS server _sort expressions can now include a single chained search expression, such as a search for Encounter?_sort=patient.name.

Introduce configuration to enable the feature that stores the history of MdmLinks. Introduce the $mdm-link-history operation.

When performing a search using the Location:near search parameter, it is now possible to include this parameter in a _sort expression as well. This will result in locations being sorted by their proximity to the coordinates in the parameter value. Thanks to Jens Kristian Villadsen for the suggestion and algorithm design!

Add scaffolding for automated migration tests that use different database vendors.

Adds an Oracle embedded database to the automated migration test scaffolding. Also adds an initialization schema (version 5.1.0) with test data to the automated migration tests.

When executing FHIR transactions in the JPA server where the transaction contained large numbers of inline match URLs, the transaction processor will now prefetch the match URL targets in a single optimized batch. This avoids a potentially significant number of database round trips. In addition, the SQL query used for pre-resolving resource update targets in the transaction processor has been reworked to fully leverage available database indexes. This should result in significant performance improvements for certain large FHIR transactions.

Add an implementation of the Clinical Reasoning FHIR Repository to allow calling the available CR operations in hapi-fhir.

Added DSTU3 and R4 support for the DaVinci Documentation Templates and Rules (DTR) Questionnaire/$questionnaire-package operation. This operation allows a Questionnaire to be packaged as a Bundle with all the supporting resources that may be required for its use such as ValueSets and Libraries. This operation is used in context of prior authorization.

Added DSTU3 and R4 support for the FHIR Clinical Reasoning module operations ActivityDefinition/$apply and PlanDefinition/$apply. $apply allows for general workflow processing and is used in clinical decision support, prior authorization, quality reporting, and disease surveillance use cases.

Added R4 support for Questionnaire/$prepopulate, Questionnaire/$package and PlanDefinition/$package operations. These are operations are intended to support extended DaVinci DTR and SDC uses cases.

Added DSTU3 and R4 support for the DaVinci Structured Data Capture (SDC) operations Questionnaire/$populate operation and QuestionnaireResponse/$extract. These operations are used in data capture and exchange use cases, and are used by downstream specifications such as DaVinci Documentation Templates and Rules (DTR) for prior authorization.

Two new operations have been added to the JPA server: * Instance level $reindex performs a synchronous reindex of a single resource and returns a Parameters object containing all previous and new indexes * Instance level $reindex-dryrun simulates a reindex and shows the previous and new indexes generated

The ResponseHighlightingInterceptor (which renders a stylized HTML view of resources) will now include the resource narrative in the rendered view if one is present.

The FHIR R5 definitions have been upgraded to the final FHIR R5 release version.

It is now possible to perform a conditional create and a conditional patch on the same resource (i.e. the same conditional URL) within a FHIR Transaction bundle.

The JPA server in-memory resource matcher, which is used to improve the efficiency of subscription processing on eligible criteria, now has support for the _tag, _tag:not, _security, _security:not and _profile parameters.

The in-memory matcher used by the JPA server subscription processor has been optimized to reduce the number of FHIRPath expressions executed while processing in-memory matching.

The $reindex operation now supports several new optional parameters. 'optimizeStorage' can be used to migrate data to/from inline storage mode, and optimisticLock can be used to control whether to guard against concurrent data modification during reindexing. reindexSearchParameters can be used to contron whether search parameters are reindexed.

The JPA server REST-HOOK subscription delivery client had a hardcoded connection pool limit of 20 outgoing connections. This has been increased to 1000, since subscriptions themselves and their delivery channels already act as a rate-limiting step.

An unexpected failure to reindex a single resource during a reindex job will no longer cause the entire job to fail.

A new optional parameter called _typePostFetchFilterUrl has been added to the Bulk Export $export operation parameters. This parameter allows filters to be specified that will be applied in-memory to the resources after they have been initially fetched by the database. This can be used to allow complex filters which only remove small numbers of resources to be efficiently applied against large datasets.

Added mdm setting allowing for resources to be matched regardless of partition, also added mdm setting for the storage of all golden resources in a single designated partition

A new Pointcut called STORAGE_BINARY_ASSIGN_BLOB_ID_PREFIX has been added. This pointcut is called when a binary blob is about to be stored, and allows implementers to attach a prefix to the blob ID before it is stored.

Bulk Export now supports a new _exportId parameter. If provided, any Binary resources generated by this export will have an extension in their binary.meta field which identifies this export. This can be used to correlate exported resources with the export job that generated them. In addition, the binary.meta field of Bulk Export-generated binaries will also contain the job ID of the export job that generated them, as well as the resource type of the data contained within the binary.

Update the clinical reasoning module version to the latest release of 3.0.0-PRE2

Add documentation for $care-gaps operation

The InterceptorService now maintains an EnumSet of all registered interceptor Pointcuts, which should improve performance when testing for the existence of specific pointcuts.

A race condition in the Bulk Export module sometimes resulted in bulk export jobs producing completion reports that did not contain all generated output files. This has been corrected.

An inefficient query in the JPA Bulk Export module was optimized. This query caused exports for resources containing tags/security labels/profiles to perform a number of redundant database lookups, so this type of export should be much faster now.

The batch system now reads less data during the maintenance pass. This avoids slowdowns on large systems.

String and URI indexing has been improved in some multi-clause queries.

Postgres indexing settings have changed. Autovacuum will run more frequently, and hashed columns will track more most-frequent items.

The SQL query used to fetch pages of search results resulted in an inefficient use of the database in cases where resources have many different versions stored. Thanks to Primož Delopst for the pull request!

The SQL generated for the JPA server $trigger-subscription operation has been optimized in order to drastically reduce the number of database round trips for large triggering jobs. In addition, a bug prevented the subscription retriggering from fully executing in a multithreaded way. This has been corrected.

When performing FHIR resource searches, the resource ID fetch size has been reduced. This should improve the memory impact of performing searches which return very large numbers of resources.

The SQL column type used for inline resource storage mode in the JPA server has been changed in order to avoid abitrary database size limits. We now use text on PostgreSQL, long on Oracle, and varchar(MAX) on MSSQL. Previously varchar(4000) was used in all cases, requiring manual resizing in order to support longer values.

Chained and reverse-chained searches will be faster in some scenarios. All required columns are now included in the IDX_RL_TGT_v2 index.

Includes by canonical url now use an indexed query, and are much faster.

The $expunge operation has been slightly optimized and should issue fewer SQL statements to the database.

The settings beans for the JPA server have been renamed to better reflect their purpose. Specifically the ModelConfig bean has been renamed to StorageSettings and the DaoConfig bean has been renamed to JpaStorageSettings.

Transaction retry will now also apply to ObjectOptimisticLockingFailureException. This enables retry of $reindex work chunks when they collide with a DELETE operation.

Bulk export operations have been enhanced to be fully partition aware.

A new interface has been created for SearchParamWithInlineReferencesExtractor with a base implementation. This allows us to encapsulate logic for resolving and replacing inline resources to the same place is also cuts down on code duplication and allow for easier sharing of the same logic

Batch2 work-chunk processing now aligns transaction boundaries with event transitions.

The Resource $validate operation no longer returns Precondition Failed 412 when a resource fails validation. It now returns 200 irrespective of the validation outcome as required by the FHIR Spec for the Resource $validate operation.

When calling $everything on a Patient instance, the jpa server no longer traverses into other patients. Previously it was possible to pull in data from another patient for example, via a Provenance resource.

When calling $everything on a Patient instance, the jpa server no longer traverses into other patients. Previously it was possible to pull in data from another patient for example, via Provenance and Composition resources.

Bulk import operations have been enhanced to be fully partition aware.

Removed maven configuration hack for IntelliJ now that Jetbrains supports different java versions in test and main (IDEA-85478).

Previously, HAPI-FHIR converted R5 Subscriptions into R4 Subscriptions and triggered those subscriptions by resource changes in the same way R4 subscriptions are triggered. Now R5 Subscriptions are triggered based on the topic they subscribe to and the resource matching happens via the SubscriptionTopic resource. This also means that R5 Subscription endpoints are now delivered a subscription-notification Bundle as opposed to the resource as is the case with R4 Subscriptions.

Previously, conditional updates always followed the STU3 specifications. Now, conditional updates in an R4 server follow the R4 specifications.

With default configuration, Resource meta.tag properties: userSelected and version, were not stored in the database. This is now fixed.

When enabling both auto_create_placeholder_reference_targets and allow_inline_match_url_references, POSTing a bundle with a conditional create URL will fail with HAPI-0507. This has now been fixed.

The OverridePathBasedReferentialIntegrityForDeletesInterceptor now works on partitioned servers. Thanks to GitHub user @JorisHeadease for the pull request!

References to the patient/subject in IPS documents generated using the $summary operation were not replaced with the bundle-local UUID assigned to the patient. Also, some dangling references were left in the generated bundle. This has been corrected.

The HapifhirDal search method required use of TypedBundleProvider.getallresources to avoid null pointer issue on searches that are greater than the default querycount

The JPA server Patient:address SearchParameter did not index values in the element Address.district. This has been corrected.

Previously, the full set of meta tags is not present in the payload of the subscription message when UPDATE the resource. Now, this issue has been fixed.

Fix the BaseTransactionProcessor.UNQUALIFIED_MATCH_URL_START REGEX so that it correctly matches patterns that contain dashes in them.

Update IBaseCoding, Tag, and tinder Child to support new userSelected and version fields for resource tags.

Upgrade dependency on core to 5.6.97 including hapi-fhir code enhancements and unit test fixes.

Simultaneous DELETE and $reindex operations could corrupt the search index. This has been fixed.

Simultaneous conditional create or create-on-update operations no longer create duplicate matching resources.

The InteractionBlockingInterceptor did not have support for the interactions 'search' and 'history' despite them being declared in the code system. This has been fixed.

Previously, SPECIAL search parameters that were not nearness parameters, such as simple string Search Parameters, could not be used in chained queries. This has been resolved, and now queries like OrganizationAffiliation?location.some-special-param=abc will work as intended.

Previously, Quartz jobs were scheduled after the scheduler was started which resulted in a race condition where duplicate TRIGGER_ACCESS entries were being added to the QRTZ_LOCKS table. This has been fixed.

Default values are provided for the new UPDATE_TIME columns so batch jobs started before an upgrade can complete.

When executing FHIR transactions in the JPA server with automatic retry enabled, if automatic placeholder-reference creation is enabled the system could sometimes fail to automatically retry. This has been corrected.

There was a transaction boundary issue in the Batch2 storage layer which resulted in the framework needing more open database connections than necessary. This has been corrected.

The %now value for date search parameters did not properly set the time zone of the time being compared to. This has been corrected.

Batch job state transitions are are now transitionally safe.

When processing the Location:near Search Parameter, if a distance unit was supplied in the parameter value it was ignored and the distance was always assumed to be km. This has been corrected.

Fixing inconsistency between the method behavior and naming.

Removing previous bundle provider to allow for iterableBundle provider, which will allow for paging resources instead of full consumption of patient resources at once

Previously, the $export response Content-Location header would not use a fixed base URL. Now, the URLs that are returned will respect the Fixed Value for Endpoint Base URL.

Previously, searching with the '_source' parameter would return matching source for all resource types. This has been corrected.

Previously, when a _has query was performed with a non-existent reference field, the query succeed with empty results in H2 and failed with a 500 Server Error in Postgres. Now all database vendors will throw an InvalidRequestException if a non-existent reference field is provided for a _has query.

Previously, invoking $mdm-link-history on only the golden resource ID or source ID would fail due to faulty validation. This has been corrected.

When updating resource fields targeted by a Combo Non-Unique SearchParameter, previous indexes were not deleted meaning that old search values could still find the resource. This has been corrected.

The BundleBuilder utility class did not work with DSTU2 bundles. This has been corrected.

When querying $mdm-link-history with no inputs, the error message is mislaeading. Also, $mdm-link-history cannot handle comma-delimited inputs. Both issues are now fixed.

Previously, clicking the '$diff' button on FHIRweb module would direct the user to an empty page. Now it displays the correct page.

Make small changes to stabilize the clinical reasoning code base.

When there is an EID MDM MATCH, the MATCH link has no score. This issue has been fixed.

Fixed a regression introduced by 4624 that prevented subscriptions from loading into the in-memory subscription cache on system startup.

Processing multiple transaction bundles with similar entries on different threads would lead to Constraint Violations if the entries contained similar unique values. This is an error that can be fixed with Retry Handlers now.

Fixed an issue where a golden resource could not be chosen as no-match when another golden resource was already chosen as a match.

Previously, the Lenient error handler will accept invalid extension containing value and nested extensions by default. This will lead to errors when client further attempt to read, update or delete the resources. Now, this has been fixed and a DataFormatException will be thrown instead.

Previously, an mdm-submit operation with Prefer: response-async header resulted in a NullPointerException in the error logs and a batch job status of FAILED. This has been fixed.

Previously, if a binary resource was read via GET, and the binary content had been externalized and not reinflated, a NullPointerException would be thrown. This has been fixed.

Previously, performing $mdm-clear would soft delete Golden Record resources and keep them in the database. Now, $mdm-clear will expunge the resources from the database.

Previously, a FHIRPath patch delete operation where the path resolved to an element in a collection, the element was not always being removed from the collection. This has been fixed.

With default configuration, Resource meta.tag properties: userSelected and version, were not stored in the database. This is now fixed.

Previously, there was the possibility for a race condition to happen in the initialization of the email subscription processing component that would result in email not being sent out. This issue has been fixed.

Improved performance of mdm-clear operation by adding index and avoiding redundant deletion.

A recent regression prevented the SQL Migrator from running on Oracle. This has been corrected.

The tag being added on golden resources does not have a version, might as well add one.

Tags no longer provide a default false value for userSelected.

/Patient/{patientid}/$everything?_type={resource types} would omit resources that were not directly related to the Patient resource (even if those resources were specified in the _type list). This was in conflict with /Patient/{patientid}/$everything operation, which did return said resources. This has been fixed so both return all related resources, even if those resources are not directly related to the Patient resource.

Job maintenance service would throw an exception if a job definition is unknown, this would run maintenance on every job instance after it. Now the maintenance will skip over unknown job definitions and display a warning log message indication a job definition is missing.

Previously, when validating resources that contain a display in a Coding/CodeableConcept different from the display defined in the CodeSystem that is used, no errors are returned in the outcome. This is now fixed.

Running an $export that completes successfully results in a progress percentage of less than 100%. This has now been fixed.

Previously the SearchParameterCanonicalizer did not correctly convert DSTU2 and DSTU3 custom resources SearchParameters into RuntimeSearchParam. This is now fixed.

POSTing a Bundle with over 100 references to the same resource will fail with HAPI-2207 'Multiple resources match'. This has been fixed.

Previously, if the fhirId in ResourceTable happened to be set to an empty string, the resourceId would be missing when trying to generate the full ID string. This has now been fixed.

Previously, $binary-access-write operation didn't trigger STORAGE_BINARY_ASSIGN_BLOB_ID_PREFIX Pointcut. This has been fixed.

Requests to start an $export of Patient or Group will now fail with 404 ResourceNotFound when the target resources do not exist. Before, the system would start a bulk export background job which would then fail.

Initiating a bulk export with a _type filter would sometimes return resource types not specified in the filter. This has been fixed.

Update the IRuleBuilder to support Patient Export rules via the new patientExportOnPatient method on the IRuleBuilder. Previously, it was accidentally using Group Export rules.

Remove some references to all_constraints table in oracle database migration tasks which were causing errors for version 19c.

The InterceptorService no longer supports ThreadLocal interceptor registrations. This feature was deprecated in 6.2.0 due to lack of use and has never been enabled by default. Please let us know on the mailing list if this affects you.

The HAPI FHIR CLI run-server command has been removed. This command was used to launch a HAPI FHIR JPA Server instance. The hapi-fhir-jpaserver-starter project has long since duplicated and improved on the CLI version of the launcher, and maintaining both was been a maintenance burden. Users of the CLI command are recommended to migrate to the starter project.

Previously, the GenericClient would only support the HTTP GET method for paging requests. This has been corrected by adding the possibility for paging with an HTTP POST.

Fix for MSSQL migration failure related to Job instance UPDATE_TIME column default value not being set correctly

Under heavy concurrency, a bug resulted in identical tag definitions being rejected with a NonUniqueResultException some of the time. This has been corrected.

Simultaneous conditional create or create-on-update operations no longer create duplicate matching resources.

Fix for MSSQL migration failure related to Job instance UPDATE_TIME column default value not being set correctly

The batch system now reads less data during the maintenance pass. This avoids slowdowns on large systems.

Fix issue where end time is missing for bulk exports on completion

Default values are provided for the new UPDATE_TIME columns so batch jobs started before an upgrade can complete.

Fixed Oracle syntax problem with the fix that was previously provided via Pull Request 4630.

With default configuration, Resource meta.tag properties: userSelected and version, were not stored in the database. This is now fixed.

Simultaneous DELETE and $reindex operations could corrupt the search index. This has been fixed.

A race condition in the Bulk Export module sometimes resulted in bulk export jobs producing completion reports that did not contain all generated output files. This has been corrected.

An inefficient query in the JPA Bulk Export module was optimized. This query caused exports for resources containing tags/security labels/profiles to perform a number of redundant database lookups, so this type of export should be much faster now.

The HapifhirDal search method required use of TypedBundleProvider.getallresources to avoid null pointer issue on searches that are greater than the default querycount

Update IBaseCoding, Tag, and tinder Child to support new userSelected and version fields for resource tags.

Upgrade dependency on core to 5.6.97 including hapi-fhir code enhancements and unit test fixes.

A new DaoConfig configuration setting has been added called JobFastTrackingEnabled, default false. If this setting is enabled, then gated batch jobs that produce only one chunk will immediately trigger a batch maintenance job. This may be useful for testing, but is not recommended for production use. Prior to this change, fasttracking was always enabled which meant if the server was not busy, small batch jobs would be processed quickly. However this lead do instability on high-volume servers, so this feature is now disabled by default.

When Batch2 work notifications are received twice (e.g. because the notification engine double delivered) an unrecoverable failure could occur. This has been corrected.

Fixed a bug with batch2 which could cause previously completed chunks to be set back to in-progress. This could cause a batch job to never complete. Now, a safeguard to ensure a job can never return to in-progress once it has completed or failed.

When a Bulk Export job runs with a large amount of data, there is a chance the reduction step can be kicked off multiple times, resulting in data loss in the final report. Jobs will now be set to in-progress before processing to prevent multiple reduction steps from being started.

Fixed an edge case during a Read operation where hooks could be invoked with a null resource. This could cause a NullPointerException in some cases.

The $mdm-clear operation sometimes failed with a constraint error when running in a heavily multithreaded environment. This has been fixed.

When encoding resources using the RDF parser, placeholder IDs (i.e. resource IDs starting with urn:) were not omitted as they are in the XML and JSON parsers. This has been corrected.

When encoding a Bundle, if resources in bundle entries had a value in Bundle.entry.fullUrl but no value in Bundle.entry.resource.id, the parser sometimes incorrectly moved these resources to be contained within other resources when serializing the bundle. This has been corrected.

Previously, searches that used more than one chained Bundle SearchParameter (i.e. Composition) were only adding one condition to the underlying SQL query which resulted in incorrect search results. This has been fixed.

Fixed a deadlock in resource conditional create.

Previously, certain mdm configuration could lead to duplicate eid identifier entries in golden resources. This has been corrected

Previously, testing database migration with cli migrate-database command in dry-run mode would insert in the migration task table. The issue has been fixed.

Previously, when creating an index as a part of a migration, if the index already existed with a different name on Oracle, the migration would fail. This has been fixed so that the create index migration task now recovers with a warning message if the index already exists with a different name.

A database migration added trailing spaces to server-assigned resource ids. This fix removes the bad migration, and adds another migration to fix the errors.

The version of a few dependencies have been bumped to the latest versions (dependent HAPI modules listed in brackets):

• MSSQL JDBC (JPA): 9.4.1.jre8 -> 12.2.0.jre11
• Flexmark (All): 0.50.40 -> 0.64.8
• Logback (All): 1.4.4 -> 1.4.7
• H2 Database (JPA): 2.1.214 -> 2.2.220
• Thymeleaf (Testpage Overlay): 3.0.14.RELEASE -> 3.1.2.RELEASE
• xpp3 (All): 1.1.4c.0 -> 1.1.6
• HtmlUnit (All): 2.67.0 -> 2.70.0
• org.hl7.fhir.core (All): 6.0.22.2 -> 6.1.2.2

Updating storage-cr module to latest CQL 3.0.0, latest cql engine improvements, and Clinical Reasoning operations to leverage repository api pattern. This will remove several dependencies from within hapi-fhir to make future maintenance simpler and performance more robust.

Adding pointcuts for the following MDM Operations: MDM_CREATE_LINK, MDM_UPDATE_LINK, MDM_MERGE_GOLDEN_RESOURCES, MDM_LINK_HISTORY, MDM_NOT_DUPLICATE, MDM_CLEAR, MDM_SUBMIT.

Added a field that shows the total number of POSSIBLE_DUPLICATE links has been added to the $mdm-duplicate-golden-resources operation response.

Previously, updating an existing resource (resource A) to match a resource that it didn't match to before (resource B) would update only the already existing links on resource A. This behaviour has been changed. Now such an update will also add additional links, if necessary, from resource A to resource B, including adding a POSSIBLE_DUPLICATE between golden resource A and golden resource B.

Changes have been made to allow searching on multiple patient _ids when in a patient_id partitioned environment.

Previously, when updating an MDM link to NO_MATCH, the golden resource involved would maintain its previous values, as defined by survivorship service. This would result in out-of-date golden resources with data that might not be accurate anymore. Now, when a link is changed to NO_MATCH, golden resources will be rebuilt from the ground up using the MDM survivorship service, and the set of links/source resources available at the time of update.

Added an implementation of Clinical Reasoning CDS on FHIR to the CDS Hooks module that allows PlanDefinition worfklows to be processed as CDS Services using the $apply operation.

Combined the ExpandResources step and WriteBinary step in the new WriteBinary step v2 for bulk exports.

The error messages returned in an OperationOutcome when validating terminology codes as a part of resource profile validation have been improved. Machine processable location (line/col) information is now available through a pair of dedicated extensions, and error messages such as UCUM parsing issues are now returned to the client (previously they were swallowed and a generic error message was returned).

Added a service for generating metrics on mdm links and resources. This includes JPA queries and updated indices.

Added an API that allows to configure permission rules for operations with access to all resources. This permission is needed to allow a search across the entire patient's record in the scope of the $everything operation to access all resources that references input Patient, including resources outside of the patient's compartment.

Added storage property to prevent conditional updates from invalidating match criteria.

A new configuration option has been added to StorageSettings which enables support in the JPA server for the _language SearchParameter.

A new option has been added to the JPA server JpaStorageOptions which prevents the server from maintaining a version history. In this mode, when a new version of a resource is added, the previous version is automatically expunged.

It is now possible to configure the strictness of concept display name validation using a new flag on the InMemoryTerminologyServerValidationSupport (for non-JPA validation) and JpaStorageSettings (for JPA validation). In addition, the error messages emitted by the validator when a concept display doesn't match have been improved to be much more useful.

Added registries for CdsCrServices and CrDiscoveryServices in CDS Hooks to allow registration of custom services.

The generated OpenAPI documentation produced by OpenApiInterceptor will now include additional details in the individual resource type documentation, including the values of CapabilityStatement.rest.resource.documentation, CapabilityStatement.rest.resource.profile, and CapabilityStatement.rest.resource.supportedProfile.

The package installer overrides existing (built-in) SearchParameter with multiple base resources. This is happening when installing US Core package for Practitioner.given as an example. This change allows the existing SearchParameter to continue to exist with the remaining base resources.

Add settings for CDS Services using CDS on FHIR. Also removed the dependency on Spring Boot from the CR configs used by CDS Hooks.

Previously, when the payload of a subscription message exceeds the broker maximum message size, exception would be thrown and retry will be performed indefinitely until the maximum message size is adjusted. Now, the message will be successfully delivered for rest-hook and email subscriptions, while message subscriptions remains the same behavior as before.

Add support for non-standard _pid SearchParameter to the the JPA engine. This new SP provides an efficient tie-breaking sort key.

When executing an HFQL search with a FHIRPath filter on the id element, this will now be automatically converted into an _id search parameter match for better performance.

Subscription triggering via the $trigger-subscription operation is now multi-threaded, which significantly improves performance for large data sets.

Enable the search cache for some requests even when a consent interceptor is active. If no consent service uses canSeeResource (i.e. shouldProcessCanSeeResource() returns false); or startOperation() returns AUTHORIZED; then the search cache is enabled.

The background activity that clears stale search results now has higher throughput. Busy servers should no longer accumulate dead stale search results.

Sorting by _id now uses the FHIR_ID column on HFJ_RESOURCE and avoid joins.

Internal client-assigned ids are now resolved within the HFJ_RESOURCE table, avoiding a join to HFJ_FORCED_ID

Several enhancements were made to the IPS generator:

• Generated IPS documents will no longer include sections that contain no contents.
• NarrativeLink extensions now use the correct datatype (url instead of uri)
• Section profile URLs have been updated to no longer use an unknown URL
• Some resources added to the generated IPS did not have their FHIR server IDs replaced with a placeholder UUID.
• Immunization manufacturer was not fetched from the server

Previously, when using INLINE tag storage mode, a superfluous version of a resource would be created as a result of an update request which didn't have a real logical change to the resource but only changed the order of existing items in tag, security label or profile collections. This change prevents this behaviour. Also on resource retrieval, these meta collections are sorted alphabetically, based on (security, code) pair for tags and security labels.

Previously, when both resourceId and goldenResourceId are provided to the mdm link history operation, they will be treated as an OR. This is now changed to AND in order to comply with REST conventions.

Remote terminology operations that fetch ValueSets or CodeSystems now force _summary=false to allow local validation.

Previously, when executing an update on a resource that had to undergo MDM, a nullpointer could occur. This has been fixed.

Previously, the use of search parameter _lastUpdated as part of a reverse chaining search would return an error message to the client. This issue has been fixed

Fixed a bug where search Bundles with include entries from an _include query parameter might trigger a 'next' link to blank pages. Specifically, if _include'd resources + requested resources were greater than (or equal to) requested page size, a 'next' link would be generated, even though no additional resources are available.

Previously, type-level expunge was allowed even if expunge operation was turned off. This is now fixed.

Resolved an issue with type-everything search operations (eg, /Patient/$everything), where not all page results were being returned if _count was specified to be the same value as the maximum page size to fetch.

Previously, when a CDS hook was registered and called with a empty context, the server returned a 500. This behaviour has been fixed.

Reindex batch job threw an exception when no results matched the reindex request. This has been corrected.

Fixed a race condition in RestfulClientFactory that could cause validateInitialized() to deadlock. Fixed a race condition in FhirContext initialization that could produce a 'this.myNameToResourceType is null' NPE. Fixed a shutdown hook memory leak in BaseApp that happened when the command threw an exception; this memory leak only affects code that calls App.main repeatedly which is probably only in test code.

batch2 jobs on MS SQL Server were failing to transition to FAILED state after max retrials for the job are exhausted. This is now fixed.

Previously, the $mdm-link-history operation would result in a 404 response when it was executed after a $mdm-clear. This has now been fixed by removing link history on $mdm-clear.

Previously, $mdm-clear failed to expunge REDIRECTED golden resources which left them as orphans. This is now fixed.

Previously, when bulk export is enabled the resource list would be generated and would not be able to add a custom resource to that list. Now once a custom resource is added the list is rebuilt.

Added RequestDetails as part of the parameters when cleaning up possible matches to enable interceptors to access it.

Previously, executing a Group Bulk Export without defining the _type parameter would accidentally omit Patient and Organization types. This has been corrected.

Previously, the response status code set in a MethodOutcome of a Resource provider was not respected. This has been fixed.

Previously, when calling $everything operation on a Patient instance, it was possible to retrieve data related to another patient via a List or Group resources. This has been fixed.

Previously, GraphQL queries will error when using base resource search parameters such as '_id' after search parameter rebuild. This has been fixed.

A regression in the HAPI FHIR 6.6.0 JPA server meant that absolute resource references which also contained an identifier were rejected even if the server was configured to allow absolute references. This has been corrected.

Several fixes have been made to the IPS generator:

• The display names associated with several sections have been corrected to exactly match the LOINC definitions for their codes
• Immunizations will now be ordered from most recent to least recent
• IPS documents containing Consent resources for Advanced Directives could result in a crash
• IPS documents containing Procedure resources for History of Procedures with a performed date could result in a crash
• IPS documents containing AllergyIntolerance resources containing an occurrence but not a reaction date could result in a crash
• IPS documents containing AllergyIntolerance resources containing an onset value in string format could result in a crash
• IPS documents containing MedicationRequest resources with no associated Medication could result in a crash

Under some circumstances, a large $everything operation could enter an infinite loop and eventually timeout with a failure. This has been corrected.

A bug in DefaultProfileValidationSupport in R5 mode caused it to return duplicates in the lists returned by fetchAllStructureDefinitions(), fetchAllSearchParameters(), etc. This has been corrected.

Some search parameters include parenthetical expressions (e.g. (MedicationRequest.medication as Reference)). The leading ( was causing searches using parenthetical expressions to fail where the reference target was a contained resource. This has been corrected.

Update DSTU2 tags and security labels with support for userSelected and version elements. Also fix them on security labels in JPA storage.

Previously, issuing a reindex operation for resources on a specific partition would fail. This problem has been fixed.

Previously, performing a FHIR transaction containing both a conditional delete and a conditional update on the same resource would fail. This has been fixed.

Update DSTU3 validation resources to FHIR 3.0.2 instead of 3.0.1

Previously, the capability statement returned by the server would not declare conformance to IG when a bulk data export provider is registered with the server. This issue has been fixed.

Previously, the score field returned by $mdm-query-links operation would contain imprecise decimal values. This is now fixed and rounded to 4 decimal places.

A regression was introduced in 2023.08.R01 which caused binary storage prefixes to not be applied to exported binary blobs. This has been fixed.

Previously, on PostgreSQL, the $mdm-link-history operation would fail if all ids provided to a parameter are unknown, and the error will persist for all subsequent requests. This is now fixed.

Previously, Bulk Export will error when processing a resource if the patient compartment SearchParameter of that resource is not present. This has been fixed, the new behaviour is to ignore such resources.

Previously, issuing an expunge operation for resources on a specific partition would fail. This problem has been fixed.

Removed duplicate helperSvc bean in JpaConfig (also defined in imported MdmJpaConfig) to resolve BeanDefinitionOverrideException