First published as a KLAS blog after Leeds teaching Hospitals NHS Trust received the analysis of the Arch Collaborative. If anyone wants access to the full Arch Collaborative results from Leeds then feel free to get in touch, happy to share.
The Arch Collaborative exists to ensure that we understand what the users of our systems really think of them!
The technology industry is one of only two industries[1] that describe their customers as users. Launching the Arch Collaborative locally ensures that each healthcare system that takes part can move further and further away from that ill-gotten term, user to a new paradigm where we have valued customers with opinions that matter, perceptions we should act upon, and innovations that we would be foolish not to consider.
The first time that my organisation, NHS Leeds Teaching Hospitals Trust (LTHT), considered the Arch Collaborative was in early 2018. We regularly share ideas and concepts with two NHS Trusts: The University Hospital Southampton and the Salford Royal Foundation. Those Trusts had taken part in the survey and were clear that it was a great way to really understand the clinical views of the Electronic Health Record (EHR) and the way it is implemented.
My organisation has risen to the EHR challenge in a different way than many. 15 years ago, our organisation decided to begin building its own EHR. We released new functionality in subsequent years until it became clear in 2017 that the organization had evolved the solution to the point where it was a clinically developed EHR.
Taking on the Arch Collaborative survey felt like the next step in understanding the direction we should head. It could be the basis for a strategic road map.
Standing up on such a public stage was a big decision for an organisation that has invested so personally in the creation of an EHR. For us, this wouldn’t be a comment on the vendor implementation or the partner development of the training materials; this would be a comment on what we had built, what we had prioritised and what we had invested our time in.
There were no gimmicks, backing tracks, or staged production; the Arch Collaborative just asked for an evaluation of our raw digital ability.
By the time we agreed to get involved in the Arch Collaborative, there had been a number of departmental changes in our organisation. We brought together individual teams and elements in the hope that we could form a super group.
We were so nervous about what would happen next that a colleague compared this process to an audition for the a cappella singing team at university, but we were resolute to know how our voice fit into the digital health mix.
The Arch Collaborative involves getting the broadest clinical input possible to a series of questions about the functionality and implementation of the EHR solution within the organisation. The survey is quite in-depth and requires time and energy to work through. We asked one of our Chief Clinical Information Officers (CCIO) to take on the project. The CCIO worked with our digital engagement team to ensure that the survey terminology was anglicized and then to widely promote the survey. In the first week, over 400 members of the hospital team had completed the survey; by the time we closed the survey, over 980 members of the workforce[2] had completed it.
We were so proud that so many clinicians had come to our gig. We were not playing to an empty stadium—they had come to join in and sing about the EHR they used every day.
Our organisation uses the EHR for point-of-care delivery; over 19,000 unique users accessed the system in September of 2018. In the same month, there were over 74 million interactions with the system. An average nurse is now collecting over 100,000 data items a year!
When we consider the size of the audience that the Arch Collaborative response will reach, the throughput of the system feels huge. That comparison to the a cappella sing-off is more like the national sing-offs at the Kennedy Centre in the film Pitch Perfect.
When organisations and senior staff members look at the success of EHR implementations in the NHS, it is easy to focus on the traditional project management triumvirate of cost, time, and quality. That is understandable—these are important aspects of a large-scale procurement project.
But a lesson hard learned and seemingly relearned many times over in digital healthcare is that an EHR project is not just a procurement project. The Arch Collaborative was the perfect way for us to test the pitch and tone of our EHR.
We believe we have an approach worthy of blueprinting for reuse but not a specific system, although that is possible. We are more keen to consider the approach we have taken—an approach that includes the following: open standards; the concept of the geography as a platform for care rather than separate healthcare systems trying to interact and integrate; and the clinical focus we have placed in the prioritization of developments.
Each of these methods has been a major part of how we developed the #LeedsDigitalWay, and we believe it is worthy of blueprinting and digital implementation in healthcare across the world.
Ultimately the Arch Collaborative at LTHT would be a comment on the concept of the #LeedsDigitalWay as much as it would be about the actual EHR.
In discussions with KLAS about the decision to take part in the Arch Collaborative, they stressed that our taking part showed humility, a strong word that meant a great deal to us. Around the same time, a tweet from Damian Hughes (@LiquidThinker) resonated particularly with our reaction to the Arch Collaborative results:
Ego is often a roadblock to your development. Humility is a key to a new pathway.
Taking the ego out of delivery means that we can adapt and learn more quickly and ensure that the silos that so easily spring up between clinicians and digital leaders can be avoided.
The results from the Arch Collaborative are not for the faint of heart. They deliver a complex, true, and statistically sound message that will shine a very powerful light on the weaknesses of the work that you have done and specifically highlight the areas that you can change to improve your “Net EMR Experience score” with minimal effort. Unlike a HIMSS score, the Arch Collaborative is based not on what is in the “box” but how the box is used and the success of its functionality.
The headline score for Leeds Teaching Hospitals Trust was a 41% Net EMR Experience score. This is the macro score that sits front and centre on the report. The score ranges from -100% to +100% and is built up from the entire survey. We were pleased with our score.
60% of our staff members described themselves as “pleased with the experience” that the EHR offers, while 19% are frustrated daily. The detail of the Arch Collaborative report allows you to investigate how to improve each evaluated area as well as the headline figures.
By offering just four hours of training every year to every staff member that uses the EHR (that’s over 19,000 people, remember), we could improve our Net EMR Experience score by a further 10%.
That final statistic makes a digital leader in the NHS pause for thought; the cost-to-impact revenue on that 10% Net EMR Experience change is not insignificant, and the debate about where the cost sits would be a long one to resolve. Is it the digital team’s job to continue to deliver business changes? If the digital solution has been embedded in everyday life, should it be a cost of ownership?
35% of our team members that use the EHR daily would describe themselves as proficient in the use of the solution. That seemed immediately positive. However, 8% of our staff members indicated that they struggle every day.
The Arch Collaborative shines a light on what you need to do and the evolution that you need to inspire. Being on the receiving end of a complex statistical readout of your digital agenda enables you as a digital leader to take a breath, look around you, and consider how you move to the next stage.
Computer Weekly refers to the CIO role and its responsibility for the transformation of a system for driving business outcomes. It suggests that the CIO role is the “third leg of the stool” of modern “business” evolution, the other legs being marketing and sales. In healthcare, we are also in a modern evolution, made clear in the following quadruple aims:
- Purpose
- Productivity
- Efficiency
- Better health and better care
The Arch Collaborative provides an opportunity to focus on the aims of the quadruple claims, but it doesn’t provide the means.
The Collaborative is grounded in the quadruple aims by accident rather than by design, but it does expose how necessary EHR capabilities needs to be implemented with the aims in mind. The Collaborative does not pass judgement on the EHR, though—it offers the statistical vision of how to improve.
If we consider the Virginia Mason Institute improvement method that was based on the Toyota Production System management methodology, we can understand how to innovate and improve using the Arch Collaborative as a baseline measure and the evolutionary plans as the rapid-improvement plans.
To ensure that what we deliver is received better, we need to find a way to offer 19,000 extremely busy people a way to not do what they do for four hours of the year! (I picked those words carefully.)
We can offer the opportunity to do the training and learning relatively easily. It is a great deal more challenging to find four spare hours for each professional who needs to use the EHR to devote to the digital agenda. It feels like a budget issue at first; who is going to pay for this? However, it soon becomes clear that it is an organisational culture issue.
The statistics from the Arch Collaborative allow you to dive into perceptions from different parts of the clinical team. The definitions need a little work to map with NHS language, but they work well at a rough-order view. For LTHT, the Collaborative highlighted a difference that we already knew, but the existence of the analysis reinforces where to focus. Clinical roles placed the LTHT EHR in different percentiles of approval, and they map as follows:
- All clinicians – 72nd percentile
- Physicians – 41st percentile
- Nurses – 63rd percentile
- Allied Health Professionals – 59th percentile
The results also include sophisticated symptom analysis to distil some key phrases for us to work with:
- Enhance initial EHR training and follow-up education to focus on supporting efficient clinician/speciality-specific workflows using personalization tools.
- EHR personalisation tools for shortcuts, filters, and report views have the highest impact on satisfaction. The use of these tools should be a focus in initial training and follow-up education.
- Timely, helpful support for clinicians’ EHR requests will improve the clinicians’ trust in leadership and the wider digital agenda. Trust will be further built as clinicians are solicited for their input into workflow designs and personalisation tools and as those enhancements are delivered as communicated.
- Engaging clinicians in ongoing enhancement of the EHR will support a culture of teamwork at Leeds that will ensure that the group coalesces around the digital agenda.
This kind of commentary was very powerful for LTHT for two key reasons: first, it wasn’t particular commentary on missing functionality—it concentrated on additional ways for system use; second, it refocused on the engagement piece as an area for improvement. We took these statements as suggestions for how we can do better.
The distance we have to travel on the journey of improvement is not to be underestimated. KLAS and the Arch Collaborative may have hit on something important. If the rest of the NHS spent the time to consider their suggestions, the wealth of comparison data that would become available could bring about the change in attitude and aptitude that digital healthcare needs so badly. Southampton has completed their Arch Collaborative research, too.
The CIO in Southampton, Adrian Byrne, commented, “I think it’s hard to come up with a set of measures to get a good evaluation report. We want to have some things we change and refine and some things we keep the same, so we can measure improvement. I like the Arch Collaborative’s ability to measure across peers. That is its main benefit. We can measure improvement ourselves, but it’s all arbitrary. KLAS has a great record in research and tends to provide real insight.”
That is the key. The Arch Collaborative today, in its full glory, enables LTHT to build its strategic direction for the continued evolution of the EHR. As more NHS organisations take part in the survey, more souls are bared, and more agreement is reached for sharing the report’s details, then we will build a platform that can inspire the next phase in the NHS digital revolution where the stars align. We will ensure that digital healthcare is about collaboration between CIOs and digital leaders who lean into the challenge together.
My last comment is from many years ago:
Forgetfulness is in the learners’ souls because they will not use their memories… they will be hearers of many things and will have learned nothing; they will appear to be omniscient and will generally know nothing; they will be tiresome company, having the show of wisdom without the reality.
Socrates (5th Century BC)
Let us prove the genius wrong. Let us learn from each other by remembering the past and noting the opportunities of the future with a humility that allows us to continuously learn and collaborate. As David Amerland says;
Collaboration is the new competition!
[1] The illegal drug trade has used the term since the 1960s and yet the technology industry has remained the only other business to maintain this reference.
[2] 530 clinicians, 147 Advance practice clinicians, 153 nurses, 154 allied health professionals