User Group Monthly Meeting January 10, 2018 2:00 PM ET Agenda - PowerPoint PPT Presentation

HL7 Immunization User Group Monthly Meeting January 10, 2018 2:00 PM ET

Agenda ▪ Welcome ▪ Poll: Which perspective do you primarily identify yourself with? ▪ Functional Guide vol. 2: CDC Endorsed Data Elements, community review in January ▪ Updates ▪ Poll results ▪ SISC ▪ Todays Topic: Data Quality and ACK Messages

HL7 User Group Poll Topics Nathan Bunker, AIRA

User Group Topics for 2019 Area Topic Yes Maybe No Score VXU/ACK Messages Proper acknowledgement of message by IIS 21 7 0 49 HL7 Processing Support Interaction and support for lot inventory tracking 20 5 0 45 VXU/ACK Messages Appropriate triggers for when message should be sent 18 8 1 42 VXU/ACK Messages Proper processing of message by IIS 15 12 0 42 HL7 Processing Support Integration of vaccination matching with VXU and QBP 19 6 1 42 VXU/ACK Messages Proper construction of message by EHR 16 10 2 38 HL7 Processing Support Integration of patient matching with VXU and QBP 19 4 2 38 HL7 Processing Support Supporting data quality activities and reporting 17 4 0 38 Special Focus Areas Vaccine NDC 18 1 0 37 HL7 Processing Support Interaction and support for Vaccines for Children program 16 7 2 35 Special Focus Areas Updating and deleting vaccinations 17 1 1 33 Special Focus Areas Vaccine eligibility and vaccine funding source 16 3 1 33

SISC Update Craig Newman, Altarum

Acknowledgements to Identify Data Quality Issues Nathan Bunker, AIRA

Resources on AIRA Repository: • Guidance for HL7 ACK Messages to Support Interoperability Keyword: ACK messages • National Set of Error Codes Keyword: error codes

Resources, cont. • Data Validation Guide for the IIS Onboarding Process • IIS Data Quality Practices: Monitoring and Evaluating Data Submissions

ACK Guidance • Created after HL7 v2.5.1 r1.5 completed • AIRA Measurement and Testing • Unable to understand the meaning of many ACKs received • Was the patient and vaccination history accepted by the IIS or not? • Source of the problem • Many ACKs were not properly following r1.5 • Even if they were, the meaning was ambiguous

ACK Guidance • ACK Guidance was created to clarify implementation of ACK • Just 6 pages including the examples • Adds additional constraints on r1.5, but still compatible • Defines valid combinations of: • MSA-1 (Acknowledgement Code) • ERR-4 (Severity) • Better defines the concept of “error”

ACK Guidance • When the HL7 v2.5.1 r1.5 discusses an error, it could refer to: • General concept of an error as in something unexpected, unwanted, or wrong • Error (ERR) segment • Severity code of Error (E) • Acknowledgement Code of Application Error (AE) • ACK Guidance • Something unexpected, unwanted, or wrong

ACK Guidance • MSA-1 can be derived from ERR-4 fields • ERR-4 documents how serious the error is • Decision of severity is left to the determination of each IIS • However the distinction between E, W, and I must be preserved • The expectation on the sender must be consistent across IIS

ACK Guidance • Transaction successful / Transaction was not successful • Misleading language in original description • A warning (W) doesn’t indicate success if there also an error (E)

ACK Guidance • After a lot of discussion with the community this simple table was created:

ACK Guidance • What about MSA-1 (Acknowledgement Code)? • Must be consistent with the values in all the ERR-4 fields

ACK Guidance

ACK Guidance • What about correction and resubmission? • IIS might not process part of a message • Not stated how much of a message must be resubmitted • ACK is not well suited to indicate which parts were accepted • Senders should fix issue and resubmit all data • IIS will deduplicate information, if needed

ACK Guidance – Specific Scenarios Scenario (possible values an IIS could send) ERR-4 MSA-1 NDC for administered vaccination is not valid, vaccination can’t be E AE read by IIS, sender needs to fix and resend Expiration date for administered vaccination is missing W AE Does the sender need to fix this? I AA Patient phone number is missing W AE How important is this to fix? I AA no ERR AA Observation is sent, it’s valid in the guide, but the IIS doesn’t need I AA the information or can’t process it yet no ERR AA Refusal is sent correctly, but the IIS doesn’t accept refusals I AA no ERR AA

ACK Guidance – Specific Scenarios Scenario (possible values an IIS could send) ERR-4 MSA-1 A required HL7 field is empty, but the IIS doesn’t need this E AE information and can still process the message without problems W AE I AA

ACK Guidance • Take home messages for IIS • Upgrade ACK messages to meet guidance • Use E, W, and I correctly, must match your IIS business rules • Communicate early changes to ACK messages with submitters • Aligning with this standard will • Reduce communication issues • Allow submitters to fix problems without asking more questions • Allow EHR vendors to better automate and support IIS submissions

Data Quality in AART Discovery Reports • AART Discovery Reports • Exploratory testing of IIS functionality • All tests have to have a “right” answer to check results against • Section was created to test ability of IIS to identify data quality problems and report them back in ACK • Problems we had to solve: • Which data quality issues should an IIS be sensitive to? • Which really should be reported back and which should be silent? • What error severity level should be expected?

Data Quality in AART Discovery Reports • Original solution • List of issues from an older project to test a data quality tool • Sorted into three broad priority errors: High, Medium, Low • Submit message with no data quality issues • Submit message with the data quality issue • Expect at least one additional ERR segment for data quality issue • Not looking at any values in ERR segment, just presence of one more ERR

Data Quality in AART Discovery Reports • Current solution • Continue with same set of issues in the same categories • Send message with one data quality issue • Expect IIS to return an ERR segment indicating: • There is an issue in the field where we put the data quality issue • Do not look at the value in ERR-4 • No other expectations for any of the other fields • Current problems: • Still don’t know which data quality issues an IIS must be able to message back

Data Quality in AART Discovery Reports • Potential future solutions: • Continue in Discovery testing • Could look for specific errors codes to be returned • Could test for support of proposed National Error Codes • Would need to get community decisions on some type of priority and categorization of which error checks need to be supported • Would probably never look at what value is actually being placed in ERR-4 • Other functional testing handles this area, for example if an IIS says they accepted a message with MSA-1 of AA and no error segments then the functional test might expect to query and retrieve the submitted information

National Error Codes • Next step after ACK guidance, what codes should be sent in • ERR-3 (HL7 Error Code) • ERR-5 (Application Error Code) • Recommendations are still within the r1.5 standard • NIST testing software should allow these to be used • Not required for IIS to support, maybe a requirement for future version • For now, EHRs are probably not looking at these fields much

National Error Codes • ERR-3 (HL7 Error Codes) • We can’t change these in our guide, they are fixed by HL7 • 1 – Illogical Date Error • 2 – Invalid Date • 3 – Illogical Value Error • 4 – Invalid Value • 5 – Table Value Not Found • 6 – Required Observation Missing • 7 – Required Data Missing • Not sufficient to convey the variety of errors IIS might send back

National Error Codes • A set of general classes of errors was defined: • Existing (error codes documented in Release 1.5) • Conflicting Data (data within a single message is internally inconsistent) • Inappropriate Date • Invalid Data • Lookup (an expected record cannot be found based on data in the message) • Message Construction (structural issues based on local business rules) • Missing Data • Processing Error • Data Sharing

National Error Codes

National Error Codes • IIS should not create or use their own local codes • IIS are not expected to support every code defined • Implementation of these codes recommended when: • Upgrade their ACK/RSP messaging capabilities • Implement new data quality checks and validations (i.e. recognize new error conditions) • Implement expanded error reporting • Contact AIRA if you can’t find a national code that meets a specific scenario

National Error Codes • Expectations on receiving systems: • Must gracefully handle values in ERR- 5 that they don’t recognize • Must be able to handle ERR-5 being empty • The long term vision is that ERR- 5 will be useful to EHR’s as they improve their handling of errors • IIS must not redefine national codes, so that ERH’s can have confidence to take specific action for specific codes