Welcome to the winter release of HAPI FHIR! Support has been added for FHIR R4B (4.3.0). See the R4B Documentation for more information on what this means. Now onto the rest!

Breaking Changes

  • The ActionRequestDetails class has been dropped (it has been deprecated since HAPI FHIR 4.0.0). This class was used as a parameter to the SERVER_INCOMING_REQUEST_PRE_HANDLED interceptor pointcut, but can be replaced in any existing client code with RequestDetails. This change also removes an undocumented behaviour where the JPA server internally invoked the SERVER_INCOMING_REQUEST_PRE_HANDLED a second time from within various processing methods. This behaviour caused performance problems for some interceptors (e.g. SearchNarrowingInterceptor) and no longer offers any benefit so it is being removed.
  • Previously when ValueSets were pre-expanded after loinc terminology upload, expansion was failing with an exception for each ValueSet with more than 10,000 properties. This problem has been fixed. This fix changed some freetext mappings (definitions about how resources are freetext indexed) for terminology resources, which requires reindexing those resources. To do this use the reindex-terminology command."
  • Removed Flyway database migration engine. The migration table still tracks successful and failed migrations to determine which migrations need to be run at startup. Database migrations no longer need to run differently when using an older database version.
  • The interceptor system has now deprecated the concept of ThreadLocal interceptors. This feature was added for an anticipated use case, but has never seen any real use that we are aware of and removing it should provide a minor performance improvement to the interceptor registry.

Security Changes

  • Upon hitting a subscription delivery failure, we currently log the failing payload which could be considered PHI. Resource content is no longer written to logs on subscription failure.

General Client/Server/Parser Changes

  • Previously, Celsius and Fahrenheit temperature quantities were not normalized. This is now fixed. This change requires reindexing of resources containing Celsius or Fahrenheit temperature quantities.
  • Fixed bug where searching with a target resource parameter (Coverage:payor:Patient) as value to an _include parameter would fail with a 500 response.
  • Previously, DELETE request type is not supported for any operations. DELETE is now supported, and is enabled for operation $export-poll-status to allow cancellation of jobs
  • Previously, when a client would provide a requestId within the source uri of a Meta.source, the provided requestId would get discarded and replaced by an id generated by the system. This has been corrected
  • In the JPA server, when a resource is being updated, the response will now include any tags or security labels which were not present in the request but were carried forward from the previous version of the resource.
  • Previously, if the Endpoint Base URL is set to something different from the default value, the URL that export-poll-status returned is incorrect. After correcting the export-poll-status URL, the binary file URL returned is also incorrect. This error has also been fixed and the URLs that are returned from $export and $export-poll-status will not contain the extra path from 'Fixed Value for Endpoint Base URL'.
  • Previously, the :nickname qualifier only worked with the predefined name and given SearchParameters. This has been fixed and now the :nickname qualifier can be used with any string SearchParameters.
  • Previously, when executing a '[base]/_history' search, '_since' and '_at' shared the same behaviour. When a user searched for the date between the records' updated date with '_at', the record of '_at' time was not returned. This has been corrected. '_since' query parameter works as it previously did, and the '_at' query parameter returns the record of '_at' time.
  • Previously, creating a DSTU3 SearchParameter with an expression that does not start with a resource type would throw an error. This has been corrected.
  • There was a bug in content-type negotiation when reading Binary resources. Previously, when a client requested a Binary resource and with an Accept header that matched the contentType of the stored resource, the server would return an XML representation of the Binary resource. This has been fixed, and a request with a matching Accept header will receive the stored binary data directly as the requested content type.
  • A new built-in server interceptor called the InteractionBlockingInterceptor has been added. This interceptor allows individual operations to be included/excluded from a RestfulServer's exported capabilities.
  • The OpenApi generator now allows additional CSS customization for the Swagger UI page, as well as the option to disable resource type pages.
  • Modified BinaryAccessProvider to use a safer method of checking the contents of an input stream. Thanks to @ttntrifork for the fix!
  • Fixed issue where adding a sort parameter to a query would return an incomplete result set.
  • Added new attribute for the @Operation annotation to define the operation's canonical URL. This canonical URL value will populate the operation definition in the CapabilityStatement resource.
  • A new interceptor pointcut STORAGE_TRANSACTION_PROCESSING has been added. Hooks for this pointcut can examine and modify FHIR transaction bundles being processed by the JPA server before processing starts.
  • In the JPA server, when deleting a resource the associated tags were previously deleted even though the FHIR specification states that tags are independent of FHIR versioning. After this fix, resource tags and security labels will remain when a resource is deleted. They can be fetched using the $meta operation against the deleted resource, and will remain if the resource is brought back in a subsequent update.
  • Fixed a bug which caused a failure when combining a Consent Interceptor with version conversion via the Accept header.

Bulk Export

  • Previously, bulk export for Group type with _typeFilter did not apply the filter if it was for the patients, and returned all members of the group. This has now been fixed, and the filter will apply.
  • A regression was introduced in 6.1.0 which caused bulk export jobs to not default to the correct output format when the _outputFormat parameter was omitted. This behaviour has been fixed, and if omitted, will now default to the only legal value application/fhir+ndjson.
  • Fixed a bug in Group Bulk Export where the server would crash in oracle due to too many clauses.
  • Fixed a Group Bulk Export bug which was causing it to fail to return resources due to an incorrect search.
  • Fixed a Group Bulk Export bug in which the group members would not be expanded correctly.
  • Fixed a bug in Group Bulk Export: If a group member was part of multiple groups , it was causing other groups to be included during Group Bulk Export, if the Group resource type was specified. Now, when doing an export on a specific group, and you elect to return Group resources, only the called Group will be returned, regardless of cross-membership.
  • Previously, Patient Bulk Export only supported endpoint [fhir base]/Patient/$export, which exports all patients. Now, Patient Export can be done at the instance level, following this format: [fhir base]/Patient/[id]/$export, which will export only the records for one patient. Additionally, added support for the patient parameter in Patient Bulk Export, which is another way to get the records of only one patient.
  • Fixed the $poll-export-status endpoint so that when a job is complete, this endpoint now correctly includes the request and requiresAccessToken attributes.
  • Fixed a bug where /Patient/$export would fail if _type=Patient parameter was not included.
  • Previously, Group Bulk Export did not support the inclusion of resources referenced in the resources in the patient compartment. This is now supported.
  • A previous fix resulted in Bulk Export files containing mixed resource types, which is not allowed in the bulk data access IG. This has been corrected.
  • A previous fix resulted in Bulk Export files containing duplicate resources, which is not allowed in the bulk data access IG. This has been corrected.
  • Previously, the number of resources per binary file in bulk export was a static 1000. This is now configurable by a new JpaStorageSettings property called 'setBulkExportFileMaximumCapacity()', and the default value is 1000 resources per file.
  • By default, if the $export operation receives a request that is identical to one that has been recently processed, it will attempt to reuse the batch job from the former request. A new configuration parameter has been
  • introduced that disables this behavior and forces a new batch job on every call.
  • Bulk Group export was failing to export Patient resources when Client ID mode was set to: ANY. This has been fixed
  • Previously, Bulk Export jobs were always reused, even if completed. Now, jobs are only reused if an identical job is already running, and has not yet completed or failed.

Other Operations

  • Extend $member-match to validate matched patient against family name and birthdate
  • $mdm-submit can now be run as a batch job, which will return a job ID, and can be polled for status. This can be accomplished by sending a Prefer: respond-async header with the request.
  • Previously, if the $reindex operation failed with a ResourceVersionConflictException the related
    batch job would fail. This has been corrected by adding 10 retry attempts for transactions that have failed with a ResourceVersionConflictException during the $reindex operation. In addition, the ResourceIdListStep was submitting one more resource than expected (i.e. 1001 records processed during a $reindex operation if only 1000 Resources were in the database). This has been corrected.

CLI Tool changes:

  • Added a new optional parameter to the upload-terminology operation of the HAPI-FHIR CLI. you can pass the -s or --size parameter to specify the maximum size that will be transmitted to the server, before a local file reference is used. This parameter can be filled in using human-readable format, for example: upload-terminology -s \"1GB\" will permit zip files up to 1 gigabyte, and anything larger than that would default to using a local file reference.
  • Previously, using the import-csv-to-conceptmap command in the CLI successfully created ConceptMap resources without a ConceptMap.status element, which is against the FHIR specification. This has been fixed by adding a required option for status for the command.
  • Added support for -l parameter for providing a local validation profile in the HAPI FHIR CLI.
  • Previously, when the upload-terminology command was used to upload a terminology file with endpoint validation enabled, a validation error occurred due to a missing file content type. This has been fixed by specifying the file content type of the uploaded file.
  • For SNOMED CT, upload-terminology now supports both Canadian and International edition's file names for the SCT Description File
  • Documentation was added for reindex-terminology command.

JPA Server General Changes

  • Changed Minimum Size (bytes) in FHIR Binary Storage of the persistence module from an integer to a long. This will permit larger binaries.
  • Previously, if a FullTextSearchSvcImpl was defined, but was disabled via configuration, there could be data loss when reindexing due to transaction rollbacks. This has been corrected. Thanks to @dyoung-work for the fix!
  • Fixed a bug where the $everything operation on Patient instances and the Patient type was not correctly propagating the transactional semantics. This was causing callers to not be in a transactional context.
  • Previously when updating a phonetic search parameter, any existing resource will not have its search parameter String updated upon reindex if the normalized String is the same letter as under the old algorithm (ex JN to JAN). Searching on the new normalized String was failing to return results. This has been corrected.
  • Previously, when creating a DocumentReference with an Attachment containing a URL over 254 characters an error was thrown. This has been corrected and now an Attachment URL can be up to 500 characters.

JPA Server Performance Changes

  • Initial page loading has been optimized to reduce the number of prefetched resources. This should improve the speed of initial search queries in many cases.
  • Cascading deletes don't work correctly if multiple threads initiate a delete at the same time. Either the resource won't be found or there will be a collision on inserting the new version. This changes fixes the problem by better handling these conditions to either ignore an already deleted resource or to keep retrying in a new inner transaction.
  • When using SearchNarrowingInterceptor, FHIR batch operations with a large number of conditional create/update entries exhibited very slow performance due to an unnecessary nested loop. This has been corrected.
  • When using ForcedOffsetSearchModeInterceptor, any synchronous searches initiated programmatically (i.e. through the internal java API, not the REST API) will not be modified. This prevents issues when a java call requests a synchronous search larger than the default offset search page size
  • Previously, when using _offset, the queries will result in short pages, and repeats results on different pages. This has now been fixed.
  • Processing for _include and _revinclude parameters in the JPA server has been streamlined, which should improve performance on systems where includes are heavily used.

Database-specific Changes

  • Database migration steps were failing with Oracle 19C. This has been fixed by allowing the database engine to skip dropping non-existent indexes.

Terminology Server, Fulltext Search, and Validation Changes

  • With Elasticsearch configured, including terminology, an exception was raised while expanding a ValueSet with more than 10,000 concepts. This has now been fixed.
  • Previously when ValueSets were pre-expanded after loinc terminology upload, expansion was failing with an exception for each ValueSet with more than 10,000 properties. This problem has been fixed. This fix changed some freetext mappings (definitions about how resources are freetext indexed) for terminology resources, which requires reindexing those resources. To do this use the reindex-terminology command."
  • Added support for AWS OpenSearch to Fulltext Search. If an AWS Region is configured, HAPI-FHIR will assume you intend to connect to an AWS-managed OpenSearch instance, and will use Amazon's DefaultAwsCredentialsProviderChain to authenticate against it. If both username and password are provided, HAPI-FHIR will attempt to use them as a static credentials provider.
  • Search for strings with :text qualifier was not performing advanced search. This has been corrected.
  • LOINC terminology upload process was enhanced to consider 24 additional properties which were defined in loinc.csv file but not uploaded.
  • LOINC terminology upload process was enhanced by loading MAP_TO properties defined in MapTo.csv input file to TermConcept(s).

MDM (Master Data Management)

  • MDM messages were using the resource id as a message key when it should be using the EID as a partition hash key. This could lead to duplicate golden resources on systems using Kafka as a message broker.
  • $mdm-submit can now be run as a batch job, which will return a job ID, and can be polled for status. This can be accomplished by sending a Prefer: respond-async header with the request.

Batch Framework

  • Fast-tracking batch jobs that produced only one chunk has been rewritten to use Quartz triggerJob. This will ensure that at most one thread is updating job status at a time. Also jobs that had FAILED, ERRORED, or been CANCELLED could be accidentally set back to IN_PROGRESS; this has been corrected
  • All Spring Batch dependencies and services have been removed. Async processing has fully migrated to Batch 2.
  • In HAPI-FHIR 6.1.0, a regression was introduced into bulk export causing exports beyond the first one to fail in strange ways. This has been corrected.
  • A remove method has been added to the Batch2 job registry. This will allow for dynamic job registration in the future.
  • Batch2 jobs were incorrectly prevented from transitioning from ERRORED to COMPLETE status.

Package Registry

  • Provided the ability to have the NPM package installer skip installing a package if it is already installed and matches the version requested. This can be controlled by the reloadExisting attribute in PackageInstallationSpec. It defaults to true, which is the existing behaviour. Thanks to Craig McClendon (@XcrigX) for the contribution!

Welcome to a new major release of HAPI FHIR! Since there are many breaking changes, we are making this a major release.

Breaking Changes

  • Support for Java 8 has been dropped. A minimum of Java 11 is now required for HAPI FHIR. Java 17 is also supported.
  • Database indexes have been completely reworked for Search Index tables. This should result in significantly faster searches in most use cases.
  • A new batch operation framework for executing long running background jobs has been created. This new framework is called 'Batch2', and will eventually replace Spring Batch. This framework is intended to be much more resilient to failures as well as much more paralellized than Spring Batch job.

Security Changes

  • Previously, it was possible to update a resource with wrong tenantID. This issue has been fixed.
  • User was permitted to bulk export all groups/patients when they were unauthorized. This issue has been fixed.
  • The Spring Framework library was upgraded to version 5.3.18 in order to avoid depending on a version known to be vulnerable to CVE-2022-22965, known as Spring4Shell. HAPI FHIR is not believed to be vulnerable to this issue, but the library has been bumped as a precaution.

General Client/Server/Parser Changes

  • Added search parameter modifier :nickname that can be used with 'name' or 'given' search parameters.
  • Added a new pointcut: STORAGE_PRESTORAGE_CLIENT_ASSIGNED_ID which is invoked when a user attempts to create a resource with a client-assigned ID.
  • The XML and JSON Parsers are now encoding narratives of contained resources. Narratives were skipped before for contained resources.
  • Include property regex operation was not working when expanding ValueSet. This is now fixed
  • When performing a search with a _revinclude. the results sometimes incorrectly included resources that were reverse included by other search parameters with the same name. Thanks to GitHub user @vivektk84 for reporting and to Jean-Francois Briere for proposing a fix.
  • The HAPI FHIR server GraphQL endpoints now support GraphQL introspection, making them much easier to use with GraphQL-capable IDEs.
  • Support has been added to the JPA server for token :not-in queries.
  • SearchNarrowingInterceptor can now be used to automatically narrow searches to include a code:in or code:not-in expression, for mandating that results must be in a specified list of codes.
  • Support has now (finally!) been added for the FHIR Bulk Import ($import) operation.
  • A consent service implementation that enforces search narrowing rules specified by the SearchNarrowingInterceptor has been added.
  • GET resource with _total=accurate and _summary=count if consent service enabled should throw an InvalidRequestException. This issue has been fixed.
  • Reindexing jobs were not respecting the passed in date range in SearchParams. We now take date these date ranges into account when running re-indexing jobs.
  • The server now supports date searches with the NOT_EQUALS ( ne) prefix.
  • Previously the Fhir parser would only set the resource type on the resource ID for resources which implemented IDomainResource. This caused a few resource types to be missed. This has been corrected, and resource type is now set on the id element for all IBaseResource instances instead.

CLI Tool changes:

  • Previously there was no way to recreate freetext indexes for terminology entities. A new CLI operation, reindex-terminology now exists for this purpose.

JPA Server General Changes

  • _include now supports canonicals as well as standard references.
  • The resource JSON can now be stored and retrieved in the Lucene/Elasticsearch index. This enables some queries to provide results without using the database. This is enabled via JpaStorageSettings.setStoreResourceInLuceneIndex()
  • An occasional concurrency failure in AuthorizationInterceptor has been resolved. Thanks to Martin Visser for reporting and providing a reproducible test case!
  • When cross-partition reference Mode is used, the rest-hook subscriptions on a partition enabled server would cause a NPE. This has been resolved.
  • Group Bulk Export (e.g. Group/123/$export) now additionally supports Organization and Practitioner as valid _type parameters. This works internally by querying using a _has parameter

JPA Server Performance Changes

  • When deleting a large ValueSet operation was timing out. This issue has been fixed.
  • Performance for JPA Server ValueSet expansion has been significantly optimized in order to minimize database lookups, especially with large expansions.
  • When using JPA persistence with Hibernate Search (Lucene or Elasticsearch), simple FHIR queries that can be satisfied completely by Hibernate Search no longer query the database.
  • Added a new setting to BinaryStorageInterceptor which allows you to disable binary de-externalization during resource reads.

Database-specific Changes

  • When searching for date search parameters on Postgres, ordinals could sometimes be represented as strings, causing a search failure. This has been corrected.
  • A regression in HAPI FHIR 5.5.0 meant that very large transactions where the bundle contained over 1000 distinct client-assigned resource IDs could fail on MSSQL and Oracle due to SQL parameter count limitations.

Terminology Server and Validation Changes

  • ValueSet pre-expansion was failing when number of concepts was larger than configured BooleanQuery.maxClauseCount value (default is 1024). This is now fixed.
  • A regression in HAPI FHIR 5.7.0 meant that when UnknownCodeSystemWarningValidationSupport was configured for WARNING behaviour, validating a field with an implicit code system could incorrectly result in an error.
  • We now Provide a Remote Terminology Service implementation for the $translate operation.
  • The JPA server terminology service can now process IS-A filters in ValueSet expansion on servers with Hibernate Search disabled.
  • HAPI-FHIR no longer performs Repository Validation on resources that are implicitly created, e.g. placeholder resources.

Welcome to the winter-ish release of HAPI-FHIR 5.6.0!

HAPI FHIR 5.6.0 (Codename: Raccoon) brings a whole bunch of great new features, bugfixes, and more.

Highlights of this release are shown below. See the Changelog for a complete list. There will be a live Webinar (recording available on-demand afterward) on August 18 2021. Details available here: https://www.smilecdr.com/quarterly-product-release-webinar-reminder

Security Changes

  • Add new RuleBuilder options which allow you to specify additional resources and search parameters which match a given compartment. More explanations of the enhancements can be found in the documentation.
  • Previously, when a search query explicitly includes a search parameter that is for the same resource type but a different resource instance from the one(s) specified on the authorized list, the search narrowing interceptor would include both search parameters in the final query, resulting in an empty bundle being returned to the caller. Now, such a call will result in a 403 Forbidden error, making it more clear why no resources were returned.
  • Inline match URL searches (e.g. search URLs for conditional creates, conditional updates, etc.) are now subject to the same security and access control checks as other searches.

General Client/Server/Parser Changes

  • HAPI-FHIR will now index canonical references if the base url is set via property.
  • Added displayLanguage support for CodeSystem $lookup operation.
  • Loosened BCP47 validation to permit languages that are without region, e.g. nl instead of nl-DE or nl-NL.
  • Previously, % symbol was causing searches to fail to return results. This has been corrected.
  • Added an NDJSON parser.
  • Previously, the package registry would not work correctly when externalized binary storage was enabled. This has been corrected.

CLI Tool changes:

  • This PR eliminates the search coordinator threadpool, and executes searches synchronously on the HTTP client thread.

JPA Server General Changes

  • Improved ordinal date searches to handle lower granularity MONTH and YEAR searches.
  • Support for the _language search parameter has been dropped.
  • FHIR Batch transaction GET operations are now executed in parallel where applicable.
  • Fixed a regression which causes transactions with multiple identical ifNoneExist clauses to create duplicate data.
  • Added the ability for Hibernate Search to prefix all elasticsearch tables it creates, allowing you to have multiple HAPI-FHIR's using the same elasticsearch instance.
  • Support has been added for chained searches traversing contained resources much more effectively.
  • Experimental additions have been made to search parameter indexing in lucene. These can be enabled via advanced property.
  • Fixed a critical bug with Postgresql database which caused the VACUUMLO tool to delete data.
  • A new _id parameter has been added to the Patient/$everything type-level operation so you can narrow down a specific list of patient IDs to export.

JPA Server Partitioning Changes

  • Swagger UI is now supported in partitioned servers

Terminology Server and Validation Changes

  • Provided a Remote Terminology Service implementation for the $lookup Operation.
  • Support added for uploading non-current versions of LOINC ValueSets

JPA Server MDM Enhancements

  • _mdm parameter support has been added to the $everything operation.
  • Modified the MDM_AFTER_PERSISTED_RESOURCE_CHECKED pointcut to include additional information.
  • The $mdm-clear operation has been refactor to use spring batch.
  • Added $mdm-create-link operation.

Another quarter gone by, another HAPI-FHIR release.

HAPI FHIR 5.5.0 (Codename: Pangolin) brings a whole bunch of great new features, bugfixes, and more.

Highlights of this release are shown below. See the Changelog for a complete list. There will be a live Webinar (recording available on-demand afterward) on August 18 2021. Details available here: https://www.smilecdr.com/quarterly-product-release-webinar-reminder

Security Changes

  • Resolved multiple vulnerabilities by updating dependencies.

General Client/Server/Parser Changes

  • Added configuration to enable/disable various scheduled tasks.
  • Support for qualified * for _include and _revinclude has been added, e.g. _include=Observation:*.
  • Flyway migration used to enforce order by default. This has been changed so now the default behaviour is out of order migrations are permitted. Strict order can be enforced via the new strict-order flag if required.

CLI Tool changes:

  • Support for multiple header-passthrough option using -hp or --header-passthroughparameter has been added to hapi-fhir-cli commands: example-data-uploader, export-conceptmap-to-csv, import-csv-to-conceptmap and upload-terminology

JPA Server General Changes

  • DELETE _expunge=true has been converted to a Spring Batch job.
  • Added a new setting to JpaStorageSettings called Tag Versioning Mode, which determines how tags are maintained.
  • A new tag storage mode called Inline Tag Mode tas been added. In this mode, all tags are stored directly in the serialized resource body in the database, instead of using dedicated tables. This has significant performance advantages when storing resources with many distinct tags (i.e. many tags that are unique to each resource, as opposed to being reused across multiple resources).
  • A new interceptor has been addeed to the JPA server called ForceOffsetSearchModeInterceptor. This interceptor forces all searches to be offset searches, instead of relying on the query cache.
  • Added a new $reindex operation which creates a Spring Batch job to reindex selected resources.

JPA Server Performance Changes

  • A new setting has been added to the JpaStorageSettings that allows the maximum number of _include and _revinclude resources to be added to a single search page result. In addition, the include/revinclue processor have been redesigned to avoid accidentally overloading the server if an include/revinclude would return unexpected massive amounts of data.
  • When performing non-query cache JPA searches (i.e. searches with Cache-Control: no-store) the loading of _include and _revinclude will now factor the maximum include count."
  • Subscriptions will no longer be triggered on unversioned changes.
  • A new JpaStorageSettings setting called Mass Ingestion Mode has been added. This mode enables rapid data ingestion by skipping a number of unnecessary checks during backloading.
  • FHIR Transactions with writes in them will now aggressively pre-fetch as many entities as possible at the start of processing. This reduces the number of database roundtrips.
  • A new interceptor has been added for JPA servers that uses semaphores to avoid multiple concurrent FHIR transactions from trying to create/update the same resource at the same time. This can improve overall performance when writing many concurrent transactions since it avoids the need for retries.

JPA Server Partitioning Changes

  • PatientIdPartitionInterceptor now supports conditional creates of resources where the resource is in the patient compartment but the conditional URL does not contain a patietn reference.
  • The $evaluate-measure now works on a partitioned server.

Terminology Server and Validation Changes

  • Added support for ICD-10-CM.
  • A new interceptor ValidationMessageSuppressingInterceptor has been added. This interceptor can be used to selectively suppress specific vaLidation messages.
  • Support for LOINC 2.70 has been added.
  • Fixed a bug where ValueSet expansion did not correctly preserve order if multiple codes were included in a single inclusion block.
  • A new Validation Support Module has been added that can use NPM Packages to supply validation conformance artifacts programatically to the validator."

JPA Server MDM Enhancements

  • Too many MDM candidates matching could result in an OutOfMemoryError. Candidate matching is now limited to the value of IMdmSettings.getCandidateSearchLimit(), default 10000.
  • Searches for mdm-expanded references such as Observation?subject:mdm=123 were getting denied by access rules that did not recognize the :mdm suffix. This has been corrected.
  • $mdm-submit operation was only submitting 100 resources and then stopping. It now correctly submits all requested resources.

It's time for another release of HAPI FHIR.

HAPI FHIR 5.4.0 (Codename: Pangolin) brings a whole bunch of great new features, bugfixes, and more.

Highlights of this release are shown below. See the Changelog for a complete list. There will be a live Webinar (recording available on-demand afterward) on May 20 2021. Details available here: https://www.smilecdr.com/quarterly-product-release-webinar-reminder

Security Changes

  • A resource exhaustion vulnerability in the HAPI FHIR JPA server was corrected. Learn more about CVE-2021-32053 here. Thanks to Zachary Minneker at Security Innovation for reporting!

General Client/Server/Parser Changes

  • HAPI FHIR now supports OpenAPI (aka Swagger). See here for an example.
  • Normalization and Standardization interceptors have been added. These can be used to normalize selected data fields according to configurable rules prior to storage.
  • Contained resources can now reference to containing resources, as allowed in the FHIR Specification. Previously this direction was blocked and contained resources with no incoming reference from the containing resource were automatically stripped, as this style was not permitted in early versions of the FHIR specification. In addition, contained resource order will now be preserved during parsing round-trips.
  • New interceptors have been added that can automatically map terminology in response resources using HAPI FHIR terminology services, returning configurable canonical terminology in the response payload.
  • Support for the FHIR Prefer: handling=lenient header has been added via an optional interceptor.
  • The automatic CapabiityStatement generation has been completely rewritten for R4+ servers. CapabilityStatements now include many new data elements, such as supported profiles, revincludes, resource level operations, and more.
  • Token Search Parameters in GraphQL expressions are now correctly parsed.

JDK Changes

  • HAPI FHIR now supports JDK 16, and this version is used to execute our CI test suite in order to ensure continued compliance. The minimum Java version required in order to use HAPI FHIR remains JDK 8. This may be updated to JDK 11 in an upcoming release, as many of the libraries we use are now either contemplating or have already completed an upgrade to JDK 11 as a minimum requirement.

JPA Server General Changes

  • Support for the _list search parameter has been added to the JPA server
  • Support for the :contained modifier has been added, allowing searches to select from data in contained resources found within the resource being searches. Note that this feature is disabled by default and must be enabled if needed.
  • The JPA server now supports persisting FHIR extensions in Resource.meta
  • Bulk Export now supports Patient- and Group- based exports
  • Auto-created reference target placeholder resources now include an extension and an identifier if one is known
  • A profiling effort led to improvements in performance when processing large FHIR Transaction bundles
  • Resources imported into a repository via NPM Packages will now attempt to preserve the resource ID defined in the source package.

JPA Server Performance Changes

  • Searches with only a single search parameter now generate a more streamlined SQL expression (one unnecessary JOIN was removed), improving performance.
  • A new header X-Upsert-Extistence-Check (note there is a typo in the name, this will be addressed in the next release of HAPI FHIR! Please be aware if you are planning on using this feature) can be added which avoids existence checks when using client assigned IDs to create new records. This can speed up performance.

JPA Server Partitioning Changes

  • Resource Reindexing is now supported on partitioned servers.
  • FHIR Bulk Export is now supported on partitioned servers (note that this operation is run at the system level and includes data from all partitions. Future enhancements may allow for more nuanced exports on partitioned servers.)

Terminology Server and Validation Changes

  • ValueSet expansion can now optionally return codes in the same hierarchy that they are defined in their source CodeSystem.
  • Validation can now be configured to return only a warning when a code is found from a CodeSystem that is unknown/unavailable to the validator.

JPA Server MDM Enhancements

  • A new search mdm-expansion syntax has been added to FHIR searches on MDM-enabled servers. For example Observation?patient:mdm=Patient/123 can be used to search for Observation resources belonging to Patient/123 but also to other MDM-linked patient records.
  • MDM matching rules can now use FHIRPath expressions as selection criteria.
  • A new syntax has been added to Group Bulk Export that allows MDM matching to be used to include matches in the group to export.
  • MDM matching rules can now match on extensions, checking the URL and Value for equality.
  • A new NUMERIC matcher has been added, allowing matching using numeric values.

It's August, so it's time for our next quarterly relese: HAPI FHIR 5.2.0 (Codename: Numbat).

Security Notice:

  • Security Issue CVE-2020-24301: An XSS vulnerability has been fixed in the testpage overlay project. This issue affects only the testpage overlay module, but users of this module should upgrade immediately. Note that this issue is addressed in HAPI FHIR 5.1.0 (as well as 5.2.0+) so users who have already upgraded to HAPI FHIR 5.1.0 do not need to upgrade again to resolve this issue. It is listed here as we experienced a delay in obtaining a CVE number.

Major New Features:

  • The JPA SearchBuilder (which turns FHIR searches into SQL statements to be executed by the database) has been completely rewritten to not use Hibernate. This allows for much more efficient SQL to be generated in some cases. For some specific queries on a very large test repository running on Postgresql this new search builder performed 10x faster. Note that this new module is enabled by default in HAPI FHIR 5.2.0 but can be disabled via a JpaStorageSettings setting. It is disabled by default in Smile CDR 2020.11.R01 but will be enabled by default in the next major release.

  • Support for RDF Turtle encoding has been added, finally bringing native support for the 3rd official FHIR encoding to HAPI FHIR. This support was contributed by Josh Collins and Eric Prud'hommeaux of the company Janeiro Digital. We greatly appreciate the contribution! To see an example of the RDF encoding: hapi.fhir.org/baseR4/Patient?_format=rdf

Terminology Enhancements:

  • Integration with remote terminology services has been improved so that required bindings to closed valuesets are no longer delegated to the remote terminology server. This improves performance since there is no need for remote services in this case.

  • The CodeSystem/$validate-code operation has been implemented for R4+ JPA servers.

  • The JPA Terminology Server is now version aware, meaning that multiple versions of a single CodeSystem can now be stored in a single FHIR terminology server repository. ValueSet expansion, CodeSystem lookup, and ConceptMap translation are all now fully version aware. Note that implementing support for fully versioned terminology is mostly complete, but some validation operations may still not work. This should be completed by our next major release.

  • ValueSet expansion with filtering (e.g. using the filter parameter on the $expand operation) has now been implemented in such a way that it fully supports filtering on pre-expanded ValueSets, including using offsets and counts. This is a major improvement for people building picker UIs leveraging the $expand operation.

EMPI Improvements:

  • Identifier matchers have been added, providing native FHIR support for matching on resource identifiers

  • The $empi-clear operation performance has been greatly improved

Other Notable Improvements:

  • A new combined "delete+expunge" mode has been added to the DELETE operation in the JPA server. This mode deletes resources and expunges (physically deletes) them in a single fast operation. Note that with this mode must be enabled, and completely bypasses interceptor hooks notifying registered listeners that data is being deletes and expunged. It is several orders of magnitude faster when deleting large sets of data, and is generally intended for test scenarios.

  • The Package Server module now supports installing non-conformance resources from packages.

  • The _typeFilter parameter has been implemented for the $bulk-export module.

As always, see the changelog for a full list of changes.

Thanks to everyone who contributed to this release!

It's August, so it's time for our next quarterly relese: HAPI FHIR 5.2.0 (Codename: Manticore).

Notable changes in this release include:

  • An XSS vulnerability has been fixed in the testpage overlay project. This issue affects only the testpage overlay module, but users of this module should upgrade immediately. A CVE number for this issue has been requested and will be updated here when it is assigned.

  • Support for the new FHIR NPM Package spec has been added. Currently this support is limited to JPA servers, and support should be added to plain servers in the next release. Packages can be imported on startup, either by supplying NPM files locally or by downloading them automatically from an NPM server such as packages.fhir.org. Package contents (the StructureDefinition, CodeSystem, ValueSet, etc. resources in the package) can be installed into the repository, or can be stored in a dedicated set of tables and made available to the validator without actually being installed in the repository.

  • Support for the Observation/$lastn operation has been implemented thanks to a partnership with LHNCBC/NIH. This operation uses ElasticSearch to support querying for recent Observations over a set of test codes for one or more patients in a very efficient way.

  • The FHIR PATCH operation now supports FHIRPatch in addition to the already supported XML and JSON Patch specs. FHIRPatch is a very expressive mechanism for creating patches and can be used to supply very precise patches.

  • A new operatiion called $diff has been added. Diff can be used too generate a FHIRPatch diff between two resrouces, or between two versions of the same resource. For example: http://hapi.fhir.org/baseR4/Patient/example/$diff

  • Several performance problems and occasional failures in the resource expunge operation have been corrected

  • The memory use for Subscription delivery queues has been reduced

  • Snapshot generaton now uses a single snapshot generator codebase for generating snapshots across all versions of FHIR. This makes ongoing maintenance much easier and fixes a number of version specific bugs.

  • The maximum cascade depth for cascading deletes is now configurable.

  • AuthorizationInterceptor can now fully authorize GraphQL calls, including allowing/blocking individual resources returned by the graph.

  • GraphQL now supports POST form (thanks to Kholilul Islam!)

  • The LOINC uploader now supports LOINC 2.68

  • A new batch job framework has been introduced, leveraging the Spring Batch library. Initial jobs to use this new framework are the Bulk Export and EMPI modules, but eventually all long processes will be adapted to use this new framework.

  • TThe HAPI FHIR built-in Terminology Server now includes support for validating UCUM (units of measure), BCP-13 (mimetypes), ISO 4217 (currencies), ISO 3166 (countries), and USPS State Codes.

  • It is now possible to disable referential integrity for delete operations for speciific reference paths.

  • A regression has been fixed that significantly degraded validation performance in the JPA server for validation of large numbers of resources.

  • Unit tests have been migrated to JUnit 5. This change has no user visible impacts, but will help us continue to improve ongoing maintenance of our test suites.

As always, see the changelog for a full list of changes.

Thanks to everyone who contributed to this release!

A second point release of the HAPI FHIR 5.0 (Labrador) release cycle has been pushed to Maven Central.

This release corrects only two issues:

  • A snapshot dependency was accidentally left into the POM for HAPI FHIR 5.0.1. This has been corrected.

  • The default setting for the new partitioning feature "Add Partition to Search Indexes" was incorrectly set to enabled (true). It has been set to false, which was the intended default for this setting.

It's time for another release of HAPI FHIR!

This release brings some good stuff, including:

  • A new feature called Partitioning has been added to the JPA server. This can be used to implement multitenancy, as well as other partitioned/segregated/sharded use cases.

  • The IValidationSupport interface has been completely redesigned for better flexibility, extensibility and to enable future use cases. Any existing implementations of this interface will need to be adjusted.

  • Many improvements to performance have been implemented

  • FHIR R5 draft definitions have been updated to the latest FHIR 4.2.0 (Preview 2) definitions

  • The Gson JSON parser has been replaced with Jackson for better flexibility and performance

As always, see the changelog for a full list of changes.

Thanks to everyone who contributed to this release!

It's time for another release of HAPI FHIR!

This release brings some good stuff, including:

  • A new database migrator for the JPA server has been introduced, based on FlywayDB.

  • A major performance enhancement has been added to the parser, which decreases the parse time when parsing large Bundle resources by up to 50%.

  • Support for positional (near) search using geo-coordinates and positional distance has been added. This support currently uses a "bounding box" algorithm, and may be further enhanced to use a radius circle in the future.

  • Support for LOINC 2.67 has been added

As always, see the changelog for a full list of changes.

Thanks to everyone who contributed to this release!

It's time for another release of HAPI FHIR!

This release brings some good stuff, including:

  • Structures JARs have been updated to incorporate the latest technical corrections. DSTU3 structures are upgraded to FHIR 3.0.2, R4 structures are upgraded to FHIR 4.0.1, and R5 draft structures are upgraded to the October 2019 draft revision.

  • ValueSets are now automatically pre-expanded by the JPA server into a dedicated set of database tables. This "precalculated expansion" is used to provide much better performance for validation and expanion operations, and introduced the ability to successfully expand very large ValueSets such as the LOINC implicit (all codes) valueset.

  • Support for the FHIR Bulk Export specification has been added. We are now working on adding support for Bulk Import!

  • First-order support for ElasticSearch as a full-text and terminology service backend implementation. At this time, both raw Lucene and ElasticSearch are supported (this may change in the future but we do not have any current plans to deprecate Lucene).

  • Live Terminology Service operations for terminology file maintenance based on delta files has been added.

  • Binary resources and Media/DocumentReference instances with binary attachments stored in the FHIR repository can now take advantage of externalized binary storage for the binary content when that feature is enabled. This allows much better scalability of repositories containing large amounts of binary content (e.g. document repositories).

As always, see the changelog for a full list of changes.

Thanks to everyone who contributed to this release!

Also, as a reminder, if you have not already filled out our annual user survey, please take a moment to do so. Access the survey here: http://bit.ly/33HO4cs (note that this URL was originally posted incorrectly. It is now fixed)

The next release of HAPI has now been uploaded to the Maven repos and GitHub's releases section.

This release features a number of significant performance improvements, and has some notable changes:

  • A new consent framework called ConsentInterceptor that can be used to apply local consent directives and policies, and potentially filter or mask data has been added.

  • Initial support for draft FHIR R5 resources has been added.

  • Support for GraphQL and the _filter search parameter has been added.

  • The ability to perform cascading deletes has been added.

As always, see the changelog for a full list of changes.

Thanks to everyone who contributed to this release!

One of the things we often talk about in the FHIR standards development community is where FHIR currently sits on Gartner's Hype Cycle. The hype cycle is a coarse measure of the trajectory of new technologies on a journey from being "new and exciting silver bullets" to eventually being "boring useful technologies".

When you are a proponent of a new technology (as I certainly am with FHIR), probably the most important aspect to remember about the hype cycle is that you really only ever know where you are at any given time long after that time has passed. In other words, it's fun to ask yourself "have we passed the Peak of Inflated Expectations yet?" but you really won't know until much later.

Speculating is perhaps a fool's errand. I probably shouldn't try but I can't help but wonder if we have passed the peak yet.

The trajectory of HAPI FHIR's growth is interesting. FHIR has been growing over the last few years by all kinds of metrics. The connectathons keep getting bigger, the number of vendors participating keeps on getting bigger, and FHIR DevDays keeps on getting bigger.

If I look at our website in Google Analytics, I am curious about the trajectory.

While HAPI FHIR has seen pretty steady growth over the last few years, that growth has been either tapering or at least very unstable over the last 8 months.

Certainly I don't think HAPI FHIR has stopped growing. The number of messages on the support forum and the number of people with big production implementations these days certainly doesn't suggest that; however, things have certainly been weird the last 8 months.

Let's look at interest in FHIR overall. The next thing to look at is the FHIR Google Trends graph, which measures the number of people searching for terms on Google (a pretty decent indicator of general interest). The following graph shows the last 4 years for FHIR.

It would seem that FHIR itself saw a crazy explosion of interest back in May, too. That makes sense since FHIR R3 was released right before that peak.

Let's compare that with the graph for IHE. I don't think anyone would disagree that IHE sits firmly atop the Plateau of Productivity. Most people in the world of health informatics know what can be accomplished with IHE's profiles, and certainly I've worked with many organizations who use them to accomplish good things.

The FHIR and IHE Graph shows interest in FHIR in BLUE and IHE in RED.

So what can we take from this? I think the right side of the graph is quite interesting. FHIR itself has kind of levelled off recently and has hit similar metrics to those of a very productive organization.

I probably shouldn't attach too much meaning to these graphs, but I can't help but wonder...

HAPI FHIR's JPA Module lets you quickly set up a FHIR server, complete with a database for whatever purpose you might have.

One of the most requested features in the last year has been for support of custom search parameters on that server. Out of the box, the JPA server has always supported the default/built-in search parameters that are defined in the FHIR specification.

This means that if you store a Patient resource in the database, the Patient.gender field will be indexed with a search parameter called gender, the Patient.birthDate field will be indexed with a search parameter called birthdate, etc.

To see a list of the default search parameters for a given resource, you can see a table near the bottom of any resource definition. For example, here are the Patient search parameters.

The Need for Custom Parameters

The built-in parameters are great for lots of situations but if you're building a real application backend then you are probably going to come up with a need that the FHIR specification developers didn't anticipate (or one that doesn't meet FHIR's 80% rule).

The solution for this is to introduce a custom search parameter. Search parameters are defined using a resource that is – unsurprisingly – called SearchParameter. The idea is that you create one of these SearchParameter resources and give it a code (the name of the URL parameter), a type (the search parameter type), and an expression (the FHIRPath expression which will actually be indexed).

Custom Parameters in HAPI FHIR JPA

In HAPI FHIR's JPA server, custom search parameters are indexed just like any other search parameter. A new mechanism has been introduced in HAPI FHIR 2.3 (to be released soon) that parses the expression, adds any new or updated search parameters to an internal registry of indexed paths, and marks any existing resources that are potential candidates for this new search parameter as requiring reindexing.

This means that any newly added search parameters will cover resources added after the search parameter was added, and it will also cover older resources after the server has had a chance to reindex them.

This also means that you definitely want to make sure you have properly secured the /SearchParameter endpoint since it can potentially cause your server to do a lot of extra work if there are a lot of resources present.

Taking it for a Spin!

To show how this works, here is an example of a search parameter on an extension. We'll suppose that in our system we've defined an extension for patients' eye colour. Patient resources stored in our database will have the eye colour extension set, and we want to be able to search on this extension, too.

1. Create the Search Parameter

First, define a search parameter and upload it to your server. In Java, this looks as follows:

// Create a search parameter definition
SearchParameter eyeColourSp = new SearchParameter();
eyeColourSp.addBase("Patient");
eyeColourSp.setCode("eyecolour");
eyeColourSp.setType(org.hl7.fhir.dstu3.model.Enumerations.SearchParamType.TOKEN);
eyeColourSp.setTitle("Eye Colour");
eyeColourSp.setExpression("Patient.extension('http://acme.org/eyecolour')");
eyeColourSp.setXpathUsage(org.hl7.fhir.dstu3.model.SearchParameter.XPathUsageType.NORMAL);
eyeColourSp.setStatus(org.hl7.fhir.dstu3.model.Enumerations.PublicationStatus.ACTIVE);
// Upload it to the server
client
	.create()
	.resource(eyeColourSp)
	.execute();

The resulting SearchParameter resource looks as follows:

{
	"resourceType": "SearchParameter",
	"title": "Eye Colour",
	"base": [ "Patient" ],
	"status": "active",
	"code": "eyecolour",
	"type": "token",
	"expression": "Patient.extension('http://acme.org/eyecolour')",
	"xpathUsage": "normal"
}

2. Upload Some Resources

Let's upload two Patient resources with different eye colours.

Patient p1 = new Patient();
p1.setActive(true);
p1.addExtension().setUrl("http://acme.org/eyecolour").setValue(new CodeType("blue"));
client
	.create()
	.resource(p1)
	.execute();
Patient p2 = new Patient();
p2.setActive(true);
p2.addExtension().setUrl("http://acme.org/eyecolour").setValue(new CodeType("green"));
client
	.create()
	.resource(p2)
	.execute();

Here's how one of these resources will look when encoded.

{
  "resourceType": "Patient",
  "extension": [
    {
      "url": "http://acme.org/eyecolour",
      "valueCode": "blue"
    }
  ],
  "active": true
}

3. Search!

Finally, let's try searching:

Bundle bundle = ourClient
	.search()
	.forResource(Patient.class)
	.where(new TokenClientParam("eyecolour").exactly().code("blue"))
	.returnBundle(Bundle.class)
	.execute();
System.out.println(myFhirCtx.newJsonParser().setPrettyPrint(true).encodeResourceToString(bundle));

This produces a search result that contains only the matching resource:

{
  "resourceType": "Bundle",
  "id": "bc89e883-b9f7-4745-8c2f-24bf9277664d",
  "meta": {
    "lastUpdated": "2017-02-07T20:30:05.445-05:00"
  },
  "type": "searchset",
  "total": 1,
  "link": [
    {
      "relation": "self",
      "url": "http://localhost:45481/fhir/context/Patient?eyecolour=blue"
    }
  ],
  "entry": [
    {
      "fullUrl": "http://localhost:45481/fhir/context/Patient/2",
      "resource": {
        "resourceType": "Patient",
        "id": "2",
        "meta": {
          "versionId": "1",
          "lastUpdated": "2017-02-07T20:30:05.317-05:00"
        },
        "text": {
          "status": "generated",
          "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><table class=\"hapiPropertyTable\"><tbody/></table></div>"
        },
        "extension": [
          {
            "url": "http://acme.org/eyecolour",
            "valueCode": "blue"
          }
        ],
        "active": true
      },
      "search": {
        "mode": "match"
      }
    }
  ]
}

Custom Search Parameters in Smile CDR

Naturally, this feature will soon be available in Smile CDR. Previous versions of Smile CDR had a less elegant solution to this problem; however, now that we have a nice elegant approach to custom parameters that is based on FHIR's own way of handling this, Smile CDR users will see the benefits quickly.

I love GitLab. Let's get that out of the way.

Back when I first joined the HAPI project, we were using CVS for version control, hosted on SourceForge. Sourceforge was at that point a pretty cool system. You got free project hosting for your open source project, a free website, and shell access to a server so you could run scripts, edit your raw website, and whatever else you needed to do. That last part has always amazed me; I've always wondered what lengths SourceForge must have had to go to in order to keep that system from being abused.

Naturally, we eventually discovered GitHub and happily moved over there – and HAPI FHIR remains a happy resident of GitHub. We're now in the progress of migrating the HAPI Hl7v2.x codebase over to a new home on GitHub, too.

Along comes GitLab

The Smile CDR team discovered GitLab about a year ago. We quickly fell in love: easy self-hosting, a UI that feels familiar to a GitHub user yet somehow slightly more powerful in each part you touch, and a compelling set of features in the enterprise edition as well once you are ready for them.

On Tuesday afternoon, Diederik noticed that GitLab was behaving slowly. I was curious about it since GitLab's @gitlabstatus Twitter mentioned unknown issues affecting the site. As it turned out, their issues went from bad, to better, and then to much worse. Ultimately, they wound up being unavailable for all of last night and part of this morning.

A terrible day for them!

GitLab's issues were slightly hilarious but also totally relatable to anyone building and deploying big systems for any length of time. TechCrunch has a nice writeup of the incident if you want the gory details. Let's just say they had slowness problems caused by a user abusing the system, and in trying to recover from that a sysadmin accidentally deleted a large amount of production data. Ultimately, he thought he was in a shell on one (bad) node and just removing a useless empty directory but he was actually in a shell on the (good) master node.

I read a few meltdowns about this on reddit today, calling the sysadmin inexperienced, inept, or worse, but I also saw a few people saying something that resonated with me much more: if you've never made a mistake on a big complicated production system, you've probably never worked on a big complicated production system.

These things happen. The trick is being able to recover from whatever has gone wrong, no matter how bad things have gotten.

An exercise in good incident management

This is where GitLab really won me over. Check their Twitter for yourself. There was no attempt to mince words. GitLab engineers were candid about what had happened from the second things went south.

GitLab opened a publicly readable Google Doc where all of the notes of their investigation could be read by anyone wanting to follow along. When it became clear that the recovery effort was going to be long and complicated, they opened a YouTube live stream of a conference bridge with their engineers chipping away at the recovery.

They even opened a live chat with the stream so you could comment on their efforts. Watching it was great. I've been in their position many times in my life: tired from being up all night trying to fix something, and sitting on an endless bridge where I'm fixing one piece, waiting for others to fix theirs, and trying to keep morale up as best I can. GitLab's engineers did this, and they did it with cameras running.

So this is the thing: I bet GitLab will be doing a lot of soul-searching in the next few days, and hopefully their tired engineers will get some rest soon. In the end, the inconvenience of this outage will be forgotten but I'm sure this won't be the last time I'll point to the way they handled a critical incident with complete transparency, and set my mind at ease that things were under control.

It's January again, which of course means it's time for the January HL7 Working Group Meeting. As always, the first two days of the HL7 meeting brings FHIR Connectathon, and this was Connectathon 14.

I feel like every time I visit one of these meetings, the scale of the meeting astounds me and I can't imagine it being any bigger... and then that happens again the next time. The final tally at the September 2016 (Baltimore) Connectathon was 170 people. The final tally here in San Antonio was 209 so we continue to beat expectations.

I think we are finally passing a point where it's feasible to fit everyone in a half-size hotel ballroom. We may well have some hard decisions about whether the format still works or whether we need to turn people away in September.

Also amazing to me was the number of new faces. On the first day, Ewout Kramer asked the room for anyone who was a first-time attendee to a FHIR Connectathon to raise their hand. It looked like about half the room raised their hand so we're really expanding the pool of interested people right now. Exciting days for FHIR!

Monday night brought our usual HAPI & .NET Users Group. We discussed a proposal we're working on for a template-based approach to automatic resource narrative generation. There will be more on that in a future post.