Canary Video Transcript
4/17/2020

Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Error! Hyperlink reference not valid.Welcome! This is a video demonstration of the Canary Testing Framework.

The video is broken up into several chapters.  You can watch the video from beginning to end, or you can jump to the section most relevant to your needs by clicking on YouTube's blue time codes in the Table of Contents in the video description below.

This video demonstrates the Canary project, which is a set of tools for testing systems that implements standards-based exchange of mortality data.

It helps those who are developing or integrating systems verify the FHIR VRDR death record format. 

FHIR is HL7’s international standard and stands for Fast Healthcare Interoperability Resources. FHIR’s death record format is specified in the Vital Records Death Reporting (VRDR) FHIR Implementation Guide.

The Canary project is sponsored by the National Center for Health Statistics (NCHS), whose mission is to provide statistical information to guide actions and policies to improve the health of the American public.

Its Division of Vital Statistics (DVS) collects and disseminates the most complete data on births and deaths in the US, achieving that goal through contracts between NCHS and the states
and jurisdictional Vital Records Offices that are legally responsible for registration of vital events.

To achieve that goal technically, NCHS supports the e-Vital Standards Initiative, which supports the development of vital records standards, with a focus on using HL7’s FHIR industry  standard.

This enables interoperability amongst the Electronic Health Record (EHR), the jurisdictions’ vital records systems, the U.S. vital records systems, and potentially other public information systems for birth and death events.

This chart shows the flow of mortality data through some key systems.  The current system uses technologies shown in red, and the design for the new system based on the e-Vital Standards Initiative, is shown in blue.

Mortality data, which describes the death of a decedent,   is collected and recorded by individuals in multiple roles, including funeral directors, coroners, medical examiners, and certifying physicians.

That data is sent to the vital records office within the jurisdiction where the death occurred.

The EDRS then sends the data to NCHS, which parses it for multiple internal systems,  including the cause of death and demographic information.

As part of the e-Vital Standards Initiative, NCHS will assign an International Classification of Disease (ICD-10) coded cause of death, and send that back to the originating jurisdiction.

The jurisdiction vital records office may share the data with other systems inside the jurisdiction, including surveillance systems such as cancer registries.

The jurisdiction also shares the data with other jurisdictions (such as the decedent’s home state) through a system called the State and Territorial Exchange of Vital Events (or STEVE).

The data itself is transmitted from system to system using a combination of web forms, an existing standard called Inter-Jurisdictional Exchange Mortality Layout (IJE Mortality), the CDC’s Public Health Information Network secure messaging system (PHIN-MS), and ad hoc solutions, as noted in red text.

The intended goal of NCHS through the e-Vital  Standards Initiative is to modernize the exchange of data and embrace communication using the FHIR Standard, noted in blue, providing a single, consistent data and communication standard across the whole ecosystem.

Using FHIR will reduce the burden on stakeholders while improving timeliness and data quality, 
which in turn will promote better public health. Canary’s role in this landscape is to help the many jurisdictions, vendors, and system developers that are involved in this modernization effort, by minimizing point-to-point interdependence, and test implementations as they are built and integrated, using a common interpretation of the FHIR VRDR IG standard for mortality data.

In other words, using Canary, developers and integrators can verify that their individual system will fit into the larger modernized landscape.

Now that we’ve talked about the mortality ecosystem and the different roles within that ecosystem, let’s talk about the Canary app.

Canary is part of the Open Source project ‘Nightingale’. 

Its source repository is located at http://github.com/nightingaleproject/canary.

You should consider this page Canary’s project home, and the information here to be the most up-to-date.

There are 2 ways to run Canary. 

The simpler way is to use Docker, which is a technology that allows apps to run the same way on Windows, MacOS, and Linux.

You can find more information about Docker and installing it at www.docker.com.

With docker installed on your system, simply follow the instructions on Canary’s home page to run the Canary docker container.

Go to http://github.com/nightingaleproject/canary.

Copy the command line from the instructions and run it in a terminal console.

Here,  I’m running it in MacOS, but it is identical in either a Windows or Linux command line window.

Docker will download and run the container, and after about 30 seconds.

You can open a browser to http://localhost:8080, and you’ll see this page.

The other way to run Canary is most useful for developers or if for some reason you cannot run Docker on your system.

Again, go to the Canary home page at http://github.com/nightingaleproject/canary

Canary is built on .NET Core so you’ll first need to install that. Follow the full set of instructions at Canary’s project page.

We will now demo each tool individually.

This is the main page of Canary which we call the Dashboard. It presents a list of all the tools available in Canary. 

Canary’s tools can be divided into 2 parts: The first part is a testing framework. This is a set of tools that tests how well your system consumes and produces FHIR VRDR records.

The second part is a ‘swiss army knife’ collection of tools that lets you, for example, generate synthetic records.

A synthetic record is a record for testing purposes, using made-up but realistic data to properly test a system.

This tool collection also lets you validate records, convert records betweem FHIR and IJE Mortality standard formats, view records more easily using inspectors, and finally, generate a FHIR VRDR record using an easy to use form.

Together, these tools help implementers, integrators, and stakeholders understand, implement, and test data interoperability amongst systems in their jurisdictions and with NCHS.

We will now demo each of these tools in detail.

Generate Synthetic Death Record

This tool generates realistic synthetic FHIR VRDR death records.

A synthetic record is a sample record populated with realistic but made-up data for testing purposes.

Once you have a synthetic record, it can then be loaded into other Canary tools or another system to test interoperability.

This tool generates a VRDR record in  FHIR XML format, FHIR JSON format, as well as the corresponding IJE Mortality format.

You can click back and forth to see how the same data is represented in each of the formats.

On the right, there are 3 buttons that lets you download, paste, and POST the generated record.

To download the generated record in the format of your choice, first, click on the format you want, for example, to save it as a FHIR XML file, click on XML.

Then click the download icon. It will save it to your default downloads directory.

To save it as a FHIR JSON file, click on JSON, then click the download icon.

To save it as an IJE Mortality file, click on IJE, then click the download icon.

As you can see, all of these files are shown in the downloads directory.

It also lets you copy the record so you can paste it in another application or use elsewhere in Canary.

It will even let you RESTfully send the FHIR XML or FHIR JSON record to a FHIR endpoint or other REST consumer endpoint, right here in Canary.

In all cases, please make sure the system is a test system since these are synthetic records.

Each time you click on the ‘New Record’ button, it will generate a new synthetic record.

You can also click on these controls to customize some categories.

For example, you can specify Male or Female subjects.

You can specify the cause of death, which will change the associated data elements to reflect a natural death or injury or poison.

And you can specify the state where the death occurred.

The cities, counties, and states that are generated reflect real cities and counties in the selected state.

However, the street level data is synthetic.

This is helpful in certain implementations that requires real cities, counties, and state.

Validate FHIR VRDR Record

This tool validates a FHIR VRDR record for syntax and structural conformance.

This is useful to make sure that your implementation conforms to HL7’s Vital Records Death Reporting (VRDR) FHIR standard

before sending it to a FHIR server.

If a record does not conform, the tool will print out detailed information to help you modify the software that produced the record for conformance to the standard.

It does not, however, check the validity of the data or that the correct value set coding is being used.

It only checks for structural and syntactic correctness.

There are 3 ways to import a FHIR VRDR record.

You can Paste a record (for example, copied from the synthetic generator in the previous chapter).

You can load a local file.

You can use a REST tool to post to the specified endpoint.

We’ll demonstrate the tool’s functionality by uploading a record previously saved in the synthetic generator tool.

This is a valid FHIR XML record.

However, if I delete the string in the identifier, and click Submit, you will see an error message.

This is because FHIR does not allow the value of an identifier to be an empty string, it will generate an error, complete with line number where the error occurred.

This allows you to understand the error and fix it in the software that produced the record.

Now, I’ll paste back the ID that I deleted earlier.

Click submit. And now it is conformant again.

Death Record Format Converter

This tool will take a FHIR XML, FHIR JSON or IJE Mortality record, and convert it to one of the other formats.

This is useful when your implementation requires one of the formats, or if you prefer working with a specific format.

We’ll demonstrate the tool’s functionality by Importing a record saved from the synthetic generator tool earlier, then click Submit.

By default, Canary converts that record to a FHIR XML format.

To convert to a different format, click on JSON or IJE.

FHIR Death Record Creator

This tool provides a form to fill in the data for a FHIR VRDR record.

This is useful to create a FHIR VRDR record using your own data.

The tool starts with a blank form, and you can fill out each item.

And when you are done, you can click the Generate Record button to generate a FHIR VRDR record.

The generated record appears below the form, and once again, you can view the record as FHIR XML, FHIR JSON, or IJE, as well as download, copy, or POST to an endpoint.

IJE Mortality Record Inspector

This tool will read an IJE record, and print a human readable report of the data in tabular form.

This is useful to more easily read an IJE formatted record.

We’ll demonstrate the tool’s functionality by uploading an IJE record generated from the 
synthetic generator tool.

And it generates this report in tabular form, showing you the name of the field, position offset in the IJE record, data length in bytes, a description of the field, and the value of the field, and you can sort the details by clicking on the column headers, or use the browser’s search function to search for a specific term or data.

In this way, this tool allows you to very easily see at a glance, all the data that is stored in an IJE record.

We now will demonstrate the other part of Canary, the Testing section.

There are 3 categories of tools in the testing section: producing, consuming, and test cases.

Producing tools tests the ability of your system to produce (or send) conformant FHIR VRDR records.

Consuming tools tests the ability of your system to consume (or receive) FHIR VRDR records.
 
Finally, there are test FHIR VRDR records that you can use. Many of these records have   been used in Connectathons as test cases for system interoperability among vendors, jurisdictions, and NCHS.

All of the consuming tools follow a 3-step process.

Step 1: Automatically generate a FHIR VRDR record for you to import into your system.

Step 2:  Prompt you to fill in the form with data as they appear in your system.

Step 3:  Calculate FHIR conformance.

All of the producing tools also  follow a 3-step process.

Step 1:  Present fields that needs to be entered into your system.

Step 2:  Export the record from your system and import it to Canary.

Step 3: Calculate FHIR conformance.

The main difference between the producing and consuming tools in each category are in the amount of manual data entry required and automation available in the tools.

Producing FHIR Death Records

This tool lets you manually enter the data from a synthetic record into your system, then export it from your system, and test the output of your system for FHIR conformance.

This is useful to make sure your system outputs proper FHIR records for interoperability to other systems and jurisdictions.

Note that every time you open this tool, it will produce a new synthetic record, so you should do all the data entry without refreshing the page.

Here, you see a generated synthetic record.

Using the data in this form, enter it into your system.

At the end, it will prompt you to output the record from your system, and import it into Canary using the Paste/Upload/POST functions.

For this demonstration, I’ll upload a valid FHIR XML file to show what the tool will display when the test is successful.

Then click on Calculate, and Canary will check the uploaded record from your system for FHIR conformance.

In this case, the record is conformant.

And to demonstrate a non-conformant record, I’ll upload an invalid FHIR XML file to show what the tool will display when the tool finds errors.

Click on Calculate, and in this case, the record is non-conformant.

The tool will scroll to the top, and display the total number of properties checked, the correct properties, and the incorrect properties.

Now when you scroll down to each of the incorrect properties, which are rendered in pink, you will see the value it found, and the expected value.

And if you click to reveal the XML or JSON accordion panels, it will show you that this is what Canary found in the uploaded document, and how Canary interpreted the data.

This allows you to go through all the errors and correct them in your system, until Canary displays a conformant record.

Now if you want to keep the evaluation or if you want to email it to someone, you can click on the Generate Downloadable Report button, and it will generate an easy to email HTML file 
containing the evaluation findings.

Consuming Death FHIR Records

This tool lets you manually enter the data from a synthetic record into your system, then export it from your system, and test the output of your system for FHIR conformance.

This is useful to make sure your system outputs proper FHIR records for interoperability to other systems and jurisdictions.

Note that every time you open this tool, it will produce a new synthetic record.

Step 1 is to take the generated record and import that into your system using the standard Paste/Download/POST buttons.

Step 2 is to enter the data as it appears in the UI of your system into the Canary form. For the sake of time, we will keep all of the data blank.

Then click on Calculate, and Canary will verify your system’s parsing of the synthetic record against what you typed in from the UI.

Canary generates a report similar to the previous demo. Since we didn’t enter any data, Canary will report a lot of errors.

And just like the previous demo, you can use the report to help fine-tune your system.

Death Records Roundtrip (Consuming)

Similar to the first tool, this tool generates a synthetic FHIR VRDR record. But instead of letting you manually enter the data from your system, it lets you import an IJE record from your system, greatly simplifying the process. Once the IJE record is in Canary, it tests the output of your system for FHIR conformance.

This is useful to make sure your system properly imports FHIR records from other systems and jurisdictions.

Note that every time you open this tool, it will produce a new synthetic record.

Step 1 is to take the generated record and import that into your system using the standard Paste/Download/POST buttons.

Once the record is in your system, Step 2 is to export the IJE record from your system into Canary.

You can do that using the standard Paste/Upload/POST buttons here.

And Step 3 is to click Calculate. Canary will verify your system’s parsing of the synthetic record against your system’s exported IJE record.

Canary will again generate a report similar to the previous demo.

And just like the previous demo, you can use the report to help fine-tune your system.

Death Records Roundtrip (Producing)

This ‘round trip’ tool generates a synthetic IJE record for you to import into your system.

Once the IJE record is in, you export the record as a FHIR VRDR record and import that into Canary. Canary then tests the output of your system for FHIR conformance.

This is useful to make sure your system outputs proper FHIR records for interoperability to other systems and jurisdictions.

Note that every time you open this tool, it will produce a new synthetic record.

Step 1 is to take the generated record and import that into your system using the standard Paste/Download/POST buttons.

Once the record is in your system, Step 2 is to export the IJE record from your system into Canary.

You can do that using the standard Paste/Upload/POST buttons here.

And Step 3 is to click Calculate. Canary will verify your system’s parsing of the synthetic record against your system’s exported IJE record.

Canary will again generate a report similar to the previous demo.

And just like the previous demo, you can use the report to help fine-tune your system.

Connectathon FHIR Death Records (Producing)

This is a collection of test records that have been used at Connectathons as official test scenarios. They are similar to the synthetic records produced by other Canary tools, but this set is static and covers specific selected scenarios.

This collection of records is useful because they are constant, and are useful for testing your system during implementation and integration, as well as for regression testing when updating your system.

This list may grow as additional Connectathons take place.

This is an example of a cancer patient.

You can see the fields that you need to enter into your system.

At the end, it will prompt you to output the record from your system, and import it into Canary using the Paste/Upload/POST functions.

Then click on Calculate, and Canary will test the uploaded record from 
your system for FHIR conformance.

This concludes the Canary demo. We hope you find the tool useful.