{'fraction_non_alphanumeric': 0.024001320567844174, 'fraction_numerical': 0.013271706833938593, 'mean_word_length': 5.259764414135152, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Improving patient understanding on discharge from the short stay unit: an integrated human factors and quality improvement approach

This intervention used a systems approach to improve patient understanding on discharge from the adult acute medicine short stay unit (SSU). Patient understanding was assessed across five domains: diagnosis, medication changes, follow-up care, return instructions and knowing who their consultant was. The aim of this approach was that at least 90% of patients achieved near-complete understanding (score >4) on questionnaire across all five discharge domains by the end of April 2021. Pre-intervention most patients received verbal instructions and only a minority received written information. Through staff interviews, we identified the electronic discharge document (EDD) as a practical source of written information. However, testing with patients showed that the format required substantial redesign to be written in patient-friendly language, using signposting, spacing information out and avoiding jargon. The effect of this intervention was assessed with a structured telephone questionnaire, which included both a patient self-rated score and a comparative understanding score to assess true patient understanding of the revised EDD. Pre-intervention 29 discharged patients were interviewed across 10 days and post-intervention 10 patients were interviewed in 7 days. Patients consistently over-rated their understanding of discharge information. Only one patient achieved the aim of comparative understanding >4 across all domains post-intervention. Understanding improved across all but one of the domains, the exception being medication changes. An important unanticipated consequence was that interviews identified inconsistencies in EDD information and gaps in patient understanding, which required escalation to the SSU team. In summary, this intervention improved patient understanding across four of the five domains. However, further work is required on process reliability for the redesigned EDD and on improving understanding of medication changes. Furthermore, the interviews revealed clinically important inconsistencies in EDD information and gaps in patient understanding.


ABSTRACT
This intervention used a systems approach to improve patient understanding on discharge from the adult acute medicine short stay unit (SSU). Patient understanding was assessed across five domains: diagnosis, medication changes, follow-up care, return instructions and knowing who their consultant was. The aim of this approach was that at least 90% of patients achieved near-complete understanding (score >4) on questionnaire across all five discharge domains by the end of April 2021. Preintervention most patients received verbal instructions and only a minority received written information. Through staff interviews, we identified the electronic discharge document (EDD) as a practical source of written information. However, testing with patients showed that the format required substantial redesign to be written in patient-friendly language, using signposting, spacing information out and avoiding jargon. The effect of this intervention was assessed with a structured telephone questionnaire, which included both a patient self-rated score and a comparative understanding score to assess true patient understanding of the revised EDD. Preintervention 29 discharged patients were interviewed across 10 days and post-intervention 10 patients were interviewed in 7 days. Patients consistently over-rated their understanding of discharge information. Only one patient achieved the aim of comparative understanding >4 across all domains post-intervention. Understanding improved across all but one of the domains, the exception being medication changes. An important unanticipated consequence was that interviews identified inconsistencies in EDD information and gaps in patient understanding, which required escalation to the SSU team. In summary, this intervention improved patient understanding across four of the five domains. However, further work is required on process reliability for the redesigned EDD and on improving understanding of medication changes. Furthermore, the interviews revealed clinically important inconsistencies in EDD information and gaps in patient understanding.

PROBLEM
Our SMART (specific, measurable, attainable, relevant, time-bound) objective was to improve patient understanding of the discharge domains of diagnosis, medication changes, follow-up care, return instructions and patients knowing who their consultant was in the short stay unit (SSU) by attaining a score of at least 4 (near-complete understanding) on all questionnaire domains in 90% of the patients by April 2021. Previous studies have shown that patient understanding on discharge from hospital is limited [1][2][3][4][5] and that this is associated with poor outcomes including increased risk of readmission to hospital. [5][6][7][8] The SSU at Ninewells Hospital in Dundee, Scotland, receives adult patients from the Acute Medical Admissions Unit who require an additional 24-72 hours care. 9 After this period, they are either discharged home or transferred to a specialty ward. The team involves acute care consultants, doctors in training, nursing staff, healthcare support workers, a pharmacist and administrative staff.
The SSU team had identified improving patient information as an important organisational aim, 10  Open access extent of the problem in the SSU. The project focused on the improvement aims of being patient-centred and efficient. 11 The aim of this approach was to improve understanding on the discharge domains of diagnosis, medication changes, follow-up care, return instructions and patients knowing who their consultant was in the SSU. During pre-intervention data collection it was shown that the patients had a poor understanding of their discharge domains. One out of five patients had no comparative understanding of their diagnosis with 31% responding neither agree nor disagree or lower for understanding their diagnosis, 17% did not know symptoms that would prompt them to revisit a doctor, 69% did not know who their consultant was and 42% of the patients had only partial concordance or lower for understanding instructions to take their medications.
In addition, average patient understanding varied greatly based on day of discharge, which could suggest different clinical teams were more effective at conveying patient understanding than others due to time constraints or other external environment factors. Furthermore, the gap between patient self-rated understanding and comparative understanding increased with age displaying that the older patients are the more likely they are to overestimate their understanding.
During the study and planning period, there were changes to standard practice for the discharge of patients. In the planning phase, discharged patients had only verbal instructions and a copy of their electronic discharge document (EDD) was sent to their general practitioner (GP). Occasionally on patient request, their EDD would be printed out and given to them. A week prior to preintervention data collection, a new EDD format was introduced with no prior training to staff. Then, after 1 week of pre-intervention data collection, hospital policy changed so that nursing staff were requested to give a copy of the EDD to all discharged patients. Hence, pre-intervention data collection includes 1 week of previous standard practice of solely verbal instructions and 1 week of new standard practice of giving patients a copy of their EDD. The intervention included adaptation of the EDD format in response to feedback from patients.

BACKGROUND
Previous studies have shown that patient understanding is limited on discharge. 1-4 12-14 Potential causes include: time constraints, 2 5 13 15 16 limited capacity for staff to convey complicated important information, 1 7 17 poor patient health literacy 13 16 18 19 and patient culture of not asking or clarifying questions. 2 5 The result is that the patients and their families are frequently unable to properly understand key medical information, for example, knowing their diagnosis, 7 any changes to their medications, 7 20 plans for follow-up and instructions for return. 7 20 21 To make improvements in a system you must understand it well 'Without understanding, there could be a focus on changes that do not lead to improvements and might even make things worse'. 22 We applied the SEIPS (Systems Engineering in Patient Safety) V.2.0 human factors model to understand the work system for discharge from the SSU and to identify opportunities for collaboration and communication between clinicians and the patient. 23 The SSU is fast paced to meet high flow of incoming patients, with the potential to cause the discharge process to be rushed in response to external environment pressures. 7 13 15-17 Patient understanding about discharge plans is dependent on the people in the working system, who are the patients as well as the clinicians. 23 Challenges for the clinicians include pitching information at the patient education level or health literacy, 3 7 18 avoiding medical jargon 4 6 20 and managing the large amount of information that must be given at once. 1 7 Moreover, patients are likely to be fatigued and stressed at the time of discharge. 7 8 17 24 These are all barriers to understanding, especially if information is only conveyed verbally.
The EDD provides an opportunity to give written information to patients. This is a summary document sent to patient's GP on their hospital stay. However, this is likely to be written in medical language that is typically above patient's reading age, 6 20 which in the UK has been estimated at 9 years old. 25 Our intervention involved collaborative work between staff and patients to improve the format of written information from the EDD. In addition to collecting information about self-reported understanding we assessed patients' knowledge, because previous studies have shown that they may overestimate their understanding. 3 4 14 Some studies report that only 20% of the patients are aware of their comprehension deficit. 5 26 MEASUREMENT Process measures: 1. Percentage of patients receiving an EDD on discharge from hospital 2. Percentage of patients receiving an EDD in patientfriendly language on discharge from hospital. Outcome measures: 1. Primary: Patient understanding of discharge domains was measured by telephone questionnaire. This assessed both self-rated understanding and actual understanding in comparison with the EDD. A dual assessment was undertaken as literature highlighted self-rated understanding was greater than actual. [3][4][5] The questionnaire was adapted from two validated tools from the Picker Institute (the Picker Patient Experience Questionnaire) 27 and Emergency Department Consumer Assessment of Healthcare Providers and Systems (ED CAHPS). 28 2. Secondary: Patient confidence in managing their condition with information received while in hospital; measured on a Likert scale at the end of the telephone questionnaire. To assess whether improved information is giving increased confidence. Balancing measures: 1. Staff survey about time pressures and satisfaction with communication assessed by a paper questionnaire given on the ward, with both Likert responses and a free text box. This was to help the sustainability of the change by monitoring any negative staff impact.

Operational definition of primary outcome
Comparative score: A score of 1-5 based on comparison of patient verbal responses to the patient notes (EDD) and scoring using predefined tabular criterion. Score done on all patients interviewed pre-intervention and post-intervention.

DESIGN
A 4-week pre-interventional and post-interventional study in the form of telephone questionnaires evaluated the effect of giving a patient a copy of their adapted EDD on the understanding of five discharge domains: diagnosis, medication changes, follow-up plans, return instructions and who their consultant was. The design of the intervention was informed by previous interventions (see Background). Improving patient experience and information is a priority for NHS Tayside.
The improvement team included acute care consultants, senior charge nurse, foundation year doctor, ward admin and office admin staff.
Patients were first involved by asking what their preference for receiving written information was which then informed the design of the patient-friendly discharge document. Many Plan-Do-Study-Act (PDSA) cycles were then conducted (table 1) on structuring questions and documenting responses by telephone questionnaire. The outcome measures were chosen to assess how to best help patients understand more about their hospital stay.
We anticipated that recruitment of representative patients would be challenging and would require clear, prespecified inclusion and exclusion criteria informed by other studies. Inclusion criteria: Patients discharged from the SSU, >18 years, fluent in English, cognitively capable, being discharged back to their home and not a secondary care facility and gave informed consent to participate in the study. 5 8 29-32 In addition, post-intervention we only included patients who had received a copy of their EDD. 5 8 29-31 Exclusion criteria included receiving treatment for alcohol misuse, 2 5 in isolation on ward for COVID-19 or had a standing warning for violence. As patients had to be consented before accessing their notes, some were excluded after consented.
The intervention was based on available literature, adapted to changing COVID-19 restrictions and informed by healthcare team and patient opinion collected through multiple PDSA cycles (table 1).
The healthcare team's main concern with change ideas were time taken to complete them, hence the change idea was integrated into tasks already undertaken. The change idea was to restructure the 'Correspondence to the GP' section of the EDD to be written in simple terms to the patient. This is a summary document sent to patient's GP on their hospital stay. The patient-friendly structure was informed by patient preference using signposting, spacing information out and avoiding jargon, as tested in a PDSA cycle. Prior to implementation, a PowerPoint teaching session was delivered to doctor staff 5 explaining how to structure this document along with a summary document (online supplemental appendix 1). The suggested template for the patient-friendly document was laminated and put in the doctor's room where EDDs were written. Previous success in similar settings was achieved with printed materials 3 33 34 for patients and this change idea coincided with staff preference for timeeffectiveness, hence was expected to succeed. In addition, as this task was integrated into existing systems the change was thought to be sustainable.
Foundation doctors were encouraged to signpost verbal information to the patient to increase the likelihood of patient understanding 34 and to use the teachback method in a nurturing way. 5 This combination of verbal information bridges the gap of health literacy in patients and is preferred by patients. 5 34 35 The period was chosen to coincide with deadlines and likelihood of yielding a substantial sample size as informed by observations. Telephone questionnaires/ interviews were a common data collecting technique used in patient understanding studies 3 5 12 26 32 33 36-38 and with COVID-19, the safest way to collect data. The questionnaire was administered verbally to not disadvantage 39 40 patients with a low literacy. Likewise, a written questionnaire was likely to be incomplete and disadvantage those with poor literacy or vision.
The likely problem to encounter was yielding a suitable sample due to poor telephone questionnaire respondence, this was attempted to be overcome by consenting all possible patients. Likewise, another problem to overcome was habit, attempting to get staff to undertake a task they had previously done differently. This was attempted to be overcome by reminding staff each morning/afternoon of the change and putting up posters on the ward as reminders.

STRATEGY
The method uses the model for improvement 41 which includes addressing the following questions: What are we trying to accomplish?, How will we know a change is improvement? and What changes can we make that will result in an improvement? In addition to using PDSA cycles which were undertaken to inform the structure of the project which demonstrate fidelity to the theory.

Open access
These small tests of change were either adopted, adapted or abandoned. 42 Prior to the use of PDSA cycles an observation period was conducted to see the current working system.
Four-week observations were undertaken including shadowing various healthcare staff and creation of system process maps, hierarchal task analysis, [43][44][45] cause and effect diagram and driver diagrams (online supplemental appendix 1) to understand the system and generate change ideas.
We completed a total of 17 PDSA cycles (table 1), which were grouped around three categories surrounding the project aim: measurements (n=9), developing a change idea (n=3) and documentation of results (n=5).
The measurements PDSA cycles revolved around the development of the patient and staff questionnaires, Open access developing a change idea cycles to assess the feasibility of the change idea and to ensure it worked and documentation pertaining to the development of consent and information forms and results from the data collection. The measurement cycles (1.1-1.9) developed both the patient and staff questionnaires. Both were adapted many times due to investigation of: Appropriate content of the patient questionnaires, rewording of questions, removal of questions, changing of the structure of questionnaire and creation of an understanding scoring system.
The patient questionnaires derived from a merger of two pre-existing non-specific to the SSU or discharge domain questionnaires one of which was UK based 27 and the other USA. 28 The staff questionnaire was comprised solely of a few selected questions from the NHS National Staff Survey. 46 The change idea cycles (2.1-2.3) were developed from initial observations in the Acute Medical Unit then the SSU. Feedback gathered from the driver diagram (online supplemental appendix 1) facilitated potential change ideas. Facilitators to patient understanding were identified as staff willingness to help patients, repetition of information to patients and the use of a discharge summary. Barriers to potential change ideas were time constraints, patients disinterest in self-management, staff confidence conveying medical information and over-estimation of perceived patient understanding: 'Teach back is helpful but we can usually gauge understanding of patient so don't feel like we need to do this'-Registrar on SSU.
Educational materials were designed and given to doctors including an interactive PowerPoint, poster for the ward and a summary document (online supplemental appendix 1).
The structure of the adapted EDD was informed by literature and patient preference: using spaced out information, signposting and avoiding jargon.
The documentation cycles (3.1-3.5) developed patient consent forms, information sheets and results of patient questionnaires. The information sheets maintained mainly original content. A spreadsheet was developed to easily document patient questionnaire responses from the telephone interviews.

RESULTS
There were 56 discharges pre-intervention and 67 discharges post-intervention from the SSU. The number of patients who gave consent, were contactable and met all the inclusion criteria was 29 pre-intervention and 10 post-intervention (figure 1).
There were 25 post-intervention patients who were contactable and did not have clinical reasons for exclusion. However, 8 (32%) had not received any EDD and 7 (28%) did not receive an EDD in the redesigned format so only 10 (40%) of these patients were contacted for telephone questionnaire.
Process measures showed that pre-intervention only 9 (31%) of 29 eligible patients received an EDD. Post-intervention this improved to 17 (68%) of 25 eligible patients receiving a copy of their EDD, of whom 10 (40%) received the EDD in patient-friendly language.
The run chart (figure 2) showed that patients consistently overestimated their understanding with mean selfrated understanding higher than the comparative score for 15 (88%) of 17 data points. The aim of achieving a score of at least 4 (near-complete understanding) across Open access all five domains was only achieved by one patient postintervention and no patients pre-intervention. However, the mean score was >4 for 5 (71%) of 7 post-intervention data points versus 3 (30%) of 10 pre-intervention (figure 2).
Intervention effect could be assessed in all patients for two of the five domains and understanding increased postintervention in both: identity of consultant from 9 (31%) of 29, to 6 (60%) of 10; diagnosis from 8 (62%) of 29, to 9 (90%) of 10. There were no follow-up arrangements for 8 patients' pre-intervention; of the remaining 21, 13 (62%) understood the plan compared with 9 (90%) of 10 post-intervention. Information about who to contact if worried was only received by 2 patients pre-intervention compared with 8 patients' post-intervention, of whom 7 (88%) had near-complete understanding. The most difficult domain to assess was changes to medicines because of variation in the information patients had received as well as the number and complexity of changes. Understanding was only assessed in 18 patients pre-assessment and 8 patients post-assessment, in whom understanding was near complete in 56% pre-intervention versus 63% post-intervention.
The communication and ability of patients to comprehend their discharge domains was further explored by Q20 'Do you have any additional comments on the communication you received by staff during this visit?'. Pre-intervention a patient stated: "Just um the doctors, I get that they are extremely busy, but they need to explain things a bit better. Doctors are like a whirlwind then you don't have enough time to process it and ask any ques-tions…"-Patient 6.
Post-intervention from the patient perspective "…and I thought how am I going to get this? And even when I wasn't sure, the fact I had everything written down made it very easy…"-Patient 42, the improvement was seen as beneficial.
Unfortunately, the balancing measures of staff perception of time-constraints was shown to increase in both the nursing and doctor groups; displaying an increase in 'unrealistic time pressure' with negligible change in 'ability to meet the conflicting demands' on their time at work. There was a slight increase in satisfaction with information given to patients in the nursing group but a drop in satisfaction with information given in the doctor group.
At the end of the questionnaire there was a box for 'Any additional comments': IDLs take longer this way but is good for patient knowledge. We will get quicker at doing them.-Doctor staff respondent 21 Furthermore, two of the doctor respondents highlighted a concern in the staff questionnaire with conveying clinical information using patient-friendly language: I am concerned about the difficult of communicating complex medical concepts with patient-centred letter, ie, non-ca125+ teratoma vs non-malignant growth and the reduced ability for hospital medics trying to understand post admissions via past EDD.-Staff respondent 19 This was an unintended consequence of the improvement; it was anticipated that it would take increased time to complete but this would dissipate with time as stated with doctor staff respondent 21. However, the increased difficulty in communicating between professionals with non-medical language was unanticipated.
An important unanticipated consequence was that the interviews picked up inconsistencies within EDDs and identified concerning patient comprehension deficits. Initially, if a deficit was identified patients were referred to contact their GP or pharmacist for further information. However, escalation in hospital was required for a few scenarios: a medication mix-up, patient unaware of a prescription, patient unaware of a face-to-face follow-up appointment, unawareness of an important side effect of medication and a mix of patient details in the EDD. These inconsistencies would have been corrected within the system without input due to safety netting features within department, however the identification within the project allowed them to be solved faster.

Lessons and limitations
The project highlighted an important issue in the SSU that was previously unexplored and showed that the use of a patient-friendly discharge document can aid in patient understanding. As anticipated from previous studies 2 5 13 15 16 acute medicine unit is a busy fast-paced environment that can cause compromise in communication. Our systems approach enabled staff to identify the EDD as a practical source of written information and to work with patients to redesign the EDD to improve comprehension.
Our results confirmed that patients often overestimate their understanding. Some current patient experience questionnaires 27 28 only assess self-reported understanding Open access so further work is required on sustainable methods for measuring understanding. An important unanticipated finding of this project was that the telephone interviews were a safety net for clinically important inconsistencies in the EDD and for concerning patient comprehension deficits.
The main limitation was 15 patients had to be excluded post-intervention because 8 were not given a copy of their EDD by nursing staff and 7 were given an EDD that was not written in patient-friendly language by the doctors. Despite both groups being reminded each day of the project, the post-intervention process reliability for patients receiving a redesigned EDD was only 40% (10 of 25). Continuing the intervention will require more time to be spent on understanding resistance to change and improving the process reliability.
Another important limitation is that the outcome measure assumed all five discharge domains had equal importance and masked the possibility that improvement might only occur in some domains. Our results suggested that understanding of medication changes is the most challenging domain. 47 Selection bias may have occurred due to differing patient access to and capabilities of using telephones. We attempted to minimise this by differing the times of calls 5 and calling three times.
The results of this work could be generalised to other environments due to similarities in working environments and similar discharging structures; however, would not be directly transferable due to differing working and patient populations.

CONCLUSION
A patient-friendly discharge document can aid in patient understanding, with most improvement surrounding the domains of diagnosis and patients knowing who their consultant was. There was not a substantial change in medication changes, but this domain is very complex, so a blanket solution was unlikely to improve it. Patients liked having information written down as it allowed them to share the information with their family who were unable to visit them while in hospital due to COVID-19 safety measures.
This project confirmed that staff and patients were able to work together to redesign existing documents to improve comprehension of discharge information in a way that was time-effective for staff. However, there is still work to do to ensure that all patients reliably receive the redesigned EDD on discharge.
The potential financial saving from this project is the reduction of readmissions 5 7 21 and reduction of patients seeking their GP to explain their hospital visit. However, measurement of these outcomes was not realistic for this short-term improvement project.
The project is sustainable due to utilisation of preexisting systems and processes which makes it feasible to continue. The next step is to establish a reliable process in the SSU with the intention to then trial in other wards within the hospital.
Contributors JLEC is the corresponding author. LU is the guarantor for the overall content, accepts full responsibility for the finished work and the conduct of the study, has access to the data collected and controlled the decision to publish. All authors made substantial contributions to the design, acquisition, analysis and interpretation of data and have drafted or substantially revised the work.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Patient consent for publication Not applicable.
Ethics approval Project was approved by local clinical governance. Participants gave informed consent to participate in the study before taking part.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available upon reasonable request. Data was anonymised and stored on a spreadsheet for analysis.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.