News: The HAPI FHIR Blog


Custom Search Parameters in HAPI FHIR

Published: 08 Feb 2017 at 01:35
By: James
 

HAPI FHIR's JPA Module lets you quickly set up a FHIR server, complete with a database for whatever purpose you might have.

One of the most requested features in the last year has been for support of custom search parameters on that server. Out of the box, the JPA server has always supported the default/built-in search parameters that are defined in the FHIR specification.

This means that if you store a Patient resource in the database, the Patient.gender field will be indexed with a search parameter called gender, the Patient.birthDate field will be indexed with a search parameter called birthdate, etc.

To see a list of the default search parameters for a given resource, you can see a table near the bottom of any resource definition. For example, here are the Patient search parameters.

The Need for Custom Parameters

The built-in parameters are great for lots of situations but if you're building a real application backend then you are probably going to come up with a need that the FHIR specification developers didn't anticipate (or one that doesn't meet FHIR's 80% rule).

The solution for this is to introduce a custom search parameter. Search parameters are defined using a resource that is – unsurprisingly – called SearchParameter. The idea is that you create one of these SearchParameter resources and give it a code (the name of the URL parameter), a type (the search parameter type), and an expression (the FHIRPath expression which will actually be indexed).

Custom Parameters in HAPI FHIR JPA

In HAPI FHIR's JPA server, custom search parameters are indexed just like any other search parameter. A new mechanism has been introduced in HAPI FHIR 2.3 (to be released soon) that parses the expression, adds any new or updated search parameters to an internal registry of indexed paths, and marks any existing resources that are potential candidates for this new search parameter as requiring reindexing.

This means that any newly added search parameters will cover resources added after the search parameter was added, and it will also cover older resources after the server has had a chance to reindex them.

This also means that you definitely want to make sure you have properly secured the /SearchParameter endpoint since it can potentially cause your server to do a lot of extra work if there are a lot of resources present.

Taking it for a Spin!

To show how this works, here is an example of a search parameter on an extension. We'll suppose that in our system we've defined an extension for patients' eye colour. Patient resources stored in our database will have the eye colour extension set, and we want to be able to search on this extension, too.

1. Create the Search Parameter

First, define a search parameter and upload it to your server. In Java, this looks as follows:

// Create a search parameter definition
SearchParameter eyeColourSp = new SearchParameter();
eyeColourSp.addBase("Patient");
eyeColourSp.setCode("eyecolour");
eyeColourSp.setType(org.hl7.fhir.dstu3.model.Enumerations.SearchParamType.TOKEN);
eyeColourSp.setTitle("Eye Colour");
eyeColourSp.setExpression("Patient.extension('http://acme.org/eyecolour')");
eyeColourSp.setXpathUsage(org.hl7.fhir.dstu3.model.SearchParameter.XPathUsageType.NORMAL);
eyeColourSp.setStatus(org.hl7.fhir.dstu3.model.Enumerations.PublicationStatus.ACTIVE);
// Upload it to the server
client
	.create()
	.resource(eyeColourSp)
	.execute();

The resulting SearchParameter resource looks as follows:

{
	"resourceType": "SearchParameter",
	"title": "Eye Colour",
	"base": [ "Patient" ],
	"status": "active",
	"code": "eyecolour",
	"type": "token",
	"expression": "Patient.extension('http://acme.org/eyecolour')",
	"xpathUsage": "normal"
}

2. Upload Some Resources

Let's upload two Patient resources with different eye colours.

Patient p1 = new Patient();
p1.setActive(true);
p1.addExtension().setUrl("http://acme.org/eyecolour").setValue(new CodeType("blue"));
client
	.create()
	.resource(p1)
	.execute();
Patient p2 = new Patient();
p2.setActive(true);
p2.addExtension().setUrl("http://acme.org/eyecolour").setValue(new CodeType("green"));
client
	.create()
	.resource(p2)
	.execute();

Here's how one of these resources will look when encoded.

{
  "resourceType": "Patient",
  "extension": [
    {
      "url": "http://acme.org/eyecolour",
      "valueCode": "blue"
    }
  ],
  "active": true
}

3. Search!

Finally, let's try searching:

Bundle bundle = ourClient
	.search()
	.forResource(Patient.class)
	.where(new TokenClientParam("eyecolour").exactly().code("blue"))
	.returnBundle(Bundle.class)
	.execute();
System.out.println(myFhirCtx.newJsonParser().setPrettyPrint(true).encodeResourceToString(bundle));

This produces a search result that contains only the matching resource:

{
  "resourceType": "Bundle",
  "id": "bc89e883-b9f7-4745-8c2f-24bf9277664d",
  "meta": {
    "lastUpdated": "2017-02-07T20:30:05.445-05:00"
  },
  "type": "searchset",
  "total": 1,
  "link": [
    {
      "relation": "self",
      "url": "http://localhost:45481/fhir/context/Patient?eyecolour=blue"
    }
  ],
  "entry": [
    {
      "fullUrl": "http://localhost:45481/fhir/context/Patient/2",
      "resource": {
        "resourceType": "Patient",
        "id": "2",
        "meta": {
          "versionId": "1",
          "lastUpdated": "2017-02-07T20:30:05.317-05:00"
        },
        "text": {
          "status": "generated",
          "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><table class=\"hapiPropertyTable\"><tbody/></table></div>"
        },
        "extension": [
          {
            "url": "http://acme.org/eyecolour",
            "valueCode": "blue"
          }
        ],
        "active": true
      },
      "search": {
        "mode": "match"
      }
    }
  ]
}

Custom Search Parameters in Smile CDR

Naturally, this feature will soon be available in Smile CDR. Previous versions of Smile CDR had a less elegant solution to this problem; however, now that we have a nice elegant approach to custom parameters that is based on FHIR's own way of handling this, Smile CDR users will see the benefits quickly.

Tags: #SearchParameters, #HAPI, #Development, #FHIR

GitLab show us exactly how to handle an outage

Published: 02 Feb 2017 at 01:37
By: James
 

I love GitLab. Let's get that out of the way.

Back when I first joined the HAPI project, we were using CVS for version control, hosted on SourceForge. Sourceforge was at that point a pretty cool system. You got free project hosting for your open source project, a free website, and shell access to a server so you could run scripts, edit your raw website, and whatever else you needed to do. That last part has always amazed me; I've always wondered what lengths SourceForge must have had to go to in order to keep that system from being abused.

Naturally, we eventually discovered GitHub and happily moved over there – and HAPI FHIR remains a happy resident of GitHub. We're now in the progress of migrating the HAPI Hl7v2.x codebase over to a new home on GitHub, too.

Along comes GitLab

The Smile CDR team discovered GitLab about a year ago. We quickly fell in love: easy self-hosting, a UI that feels familiar to a GitHub user yet somehow slightly more powerful in each part you touch, and a compelling set of features in the enterprise edition as well once you are ready for them.

On Tuesday afternoon, Diederik noticed that GitLab was behaving slowly. I was curious about it since GitLab's @gitlabstatus Twitter mentioned unknown issues affecting the site. As it turned out, their issues went from bad, to better, and then to much worse. Ultimately, they wound up being unavailable for all of last night and part of this morning.

A terrible day for them!

GitLab's issues were slightly hilarious but also totally relatable to anyone building and deploying big systems for any length of time. TechCrunch has a nice writeup of the incident if you want the gory details. Let's just say they had slowness problems caused by a user abusing the system, and in trying to recover from that a sysadmin accidentally deleted a large amount of production data. Ultimately, he thought he was in a shell on one (bad) node and just removing a useless empty directory but he was actually in a shell on the (good) master node.

I read a few meltdowns about this on reddit today, calling the sysadmin inexperienced, inept, or worse, but I also saw a few people saying something that resonated with me much more: if you've never made a mistake on a big complicated production system, you've probably never worked on a big complicated production system.

These things happen. The trick is being able to recover from whatever has gone wrong, no matter how bad things have gotten.

An exercise in good incident management

This is where GitLab really won me over. Check their Twitter for yourself. There was no attempt to mince words. GitLab engineers were candid about what had happened from the second things went south.

GitLab opened a publicly readable Google Doc where all of the notes of their investigation could be read by anyone wanting to follow along. When it became clear that the recovery effort was going to be long and complicated, they opened a YouTube live stream of a conference bridge with their engineers chipping away at the recovery.

They even opened a live chat with the stream so you could comment on their efforts. Watching it was great. I've been in their position many times in my life: tired from being up all night trying to fix something, and sitting on an endless bridge where I'm fixing one piece, waiting for others to fix theirs, and trying to keep morale up as best I can. GitLab's engineers did this, and they did it with cameras running.

So this is the thing: I bet GitLab will be doing a lot of soul-searching in the next few days, and hopefully their tired engineers will get some rest soon. In the end, the inconvenience of this outage will be forgotten but I'm sure this won't be the last time I'll point to the way they handled a critical incident with complete transparency, and set my mind at ease that things were under control.

Tags: #Outages, #DevOps, #Git

January 2017 HL7 Connectathon in San Antonio

Published: 17 Jan 2017 at 23:00
By: James
 

It's January again, which of course means it's time for the January HL7 Working Group Meeting. As always, the first two days of the HL7 meeting brings FHIR Connectathon, and this was Connectathon 14.

I feel like every time I visit one of these meetings, the scale of the meeting astounds me and I can't imagine it being any bigger... and then that happens again the next time. The final tally at the September 2016 (Baltimore) Connectathon was 170 people. The final tally here in San Antonio was 209 so we continue to beat expectations.

I think we are finally passing a point where it's feasible to fit everyone in a half-size hotel ballroom. We may well have some hard decisions about whether the format still works or whether we need to turn people away in September.

Also amazing to me was the number of new faces. On the first day, Ewout Kramer asked the room for anyone who was a first-time attendee to a FHIR Connectathon to raise their hand. It looked like about half the room raised their hand so we're really expanding the pool of interested people right now. Exciting days for FHIR!

Monday night brought our usual HAPI & .NET Users Group. We discussed a proposal we're working on for a template-based approach to automatic resource narrative generation. There will be more on that in a future post.

Tags: #HL7, #Meetings, #Connectathon, #FHIR, #WGM
Newer Entries