This chapter addresses objectives 2, 5 and 7 and gives an overview of the three case study sites and their staff data, describing how IR has been implemented, developed and supported at each site. This chapter also investigates trends in patient outcomes (retrieved from routinely collected NHS ward data) in the context of the introduction of IR and other care improvement initiatives that have been introduced. An analysis of the costs and benefits of IR is also provided.
Ward profile data method
In phase 3, in-depth case studies were conducted to explore the extent to which the CMO configurations of IR identified in the literature and from the stakeholder consultation event were compatible with or relevant to modern health service delivery and the experiences of health-care staff and those of patients and their carers. The results from the survey, alongside NHS trust characteristics such as location and number of beds, were used to select the case study sites to be as broadly representative as possible, while maintaining the diversity necessary for qualitative data. The co-investigators discussed and agreed the selection of trusts together to identify sites where IR had been implemented differently (e.g. hourly versus every 2 or 4 hours), sites with different ward layouts (e.g. predominantly single rooms vs. Nightingale layouts) and sites of different sizes. Two trusts were approached and declined to participate in the study before the three case study sites were finalised.
Two wards were identified by senior trust staff in each hospital (one acute and one care of older people, to reflect the areas of concern in the Francis inquiry1). Trusts were able to choose their own wards and were not given any other criteria for how to do so by the study researchers. The research team spent 2–3 weeks based on each ward to undertake data collection.
To map the organisational and service delivery context in which IR was implemented, individual interviews were undertaken with directors of nursing/senior nurse managers and ward managers at each of the case study sites and a matron at site 3. A topic guide (see Appendix 4) was designed for this purpose, to collect information about the implementation and development of IR in each trust, details about how IR was undertaken, monitored and evaluated and any future development plans for the intervention. A total of 17 individual interviews were undertaken with trust and ward managers and a matron (Table 8). Audio files were transcribed and examined to produce a detailed description of the organisational context in which IR was introduced at each site.
| Case study site | Staff interviewed (n) | ||
|---|---|---|---|
| Senior nurse manager (Director of Nursing) | Ward manager | Matron | |
| 1 | 3 | 2 | 0 |
| 2 | 3 | 2 | 0 |
| 3 | 4 | 2 | 1 |
Ward and trust managers at each case study site were also asked to provide any documents that they could that contained information on their team or service, including nursing shift patterns, staffing information, sickness levels and vacancy rates. Researchers prepared ward layout maps to highlight the ecology of each case study ward, including the layout of patient beds. Detailed ward profiles including the ward layouts can be seen in Appendix 5 and a summary is provided in Tables 9–12. Reproduced examples of the IR documents for each site are provided in Appendices 6–9.
| Trust characteristic | Site 1 | Site 2 | Site 3 |
|---|---|---|---|
| Size | Large (> 800 beds) | Large (> 800 beds) | Very Large (> 1000 beds) |
| Location | Urban | Urban with rural catchment area | Urban |
| Bed occupancy (%) | > 90 | > 90 | > 90 |
| Ward characteristic | Site 1 | Site 2 | Site 3 | |||
|---|---|---|---|---|---|---|
| Ward a | Ward b | Ward a | Ward b | Ward a | Ward b | |
| Specialty | Health care for older people | Acute medicine, endocrinology | Acute trauma orthopaedic | Health care for older people | Acute medicine, cardiac and respiratory | Health care for older people |
| Number of beds | 24 | 24 | 32 | 32 | 26 | 18 |
| Predominant ward layout | 3–7 bed bays | 3–7 bed bays | Single, en suite rooms | Single, en-suite rooms | 4 bed bays | Nightingale |
| Nursing organisation | NS | Three teams | NS | Four teams | NS | NS |
| Shift pattern | 12-hour shifts | 12-hour shifts | Combination of 12-hour shifts and early (07.00–13.30) and late (13.00–19.30) shifts | Combination of 12-hour shifts and early (07.00–13.30) and late (13.00–19.30) shifts | 12-hour shifts | 12-hour shifts |
| Nursing team | Site 1 | Site 2 | Site 3 | |||
|---|---|---|---|---|---|---|
| February 2017 | January 2017 | April 2017 | May 2017 | July 2017 | July 2017 | |
| Nursing staff establishment at the time of data collection | ||||||
| Band 7 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| Band 6 | 2.0 | 2.0 | 3.6 | 3.6 | 4.9 | 2.1 |
| Band 5 | 15.19 | 17.29 | 17.2 | 20.9 | 21.5 | 12.7 |
| Band 4 | 9 | 0 | 2.6 | 6.6 | 0 | 0 |
| Band 3 | 0 | 0 | 6.5 | 5.8 | 0 | 0 |
| Band 2 | 15.09 | 11.09 | 19.6 | 7.1 | 9.5 | 12.4 |
| Total | 33.28 | 31.38 | 50.5 | 45.0 | 36.9 | 28.2 |
| Vacancy rate | 4 (FTE) | 4 (FTE) | 4.4 (FTE) | 8.1 (FTE) | 19.78% | 9.69% |
| Agency/bank use | 127 shifts covered by temporary staff | 82 shifts covered by temporary staff | 6.7 (FTE)2.0 RN4.7 HCA | 11.4 (FTE)4.0 RN7.4 HCA | 8.06 (FTE) | 4.57 (FTE) |
| Sickness | RN: 19.74% HCA: 0% | RN: 11.56% HCA: 45.08% | 1.1 (FTE); 3.0% | 2.2 (FTE); 5.4% | 2.54 (FTE) | 0.38 (FTE) |
| Implementation of IR | Site 1 | Site 2 | Site 3 |
|---|---|---|---|
| When introduced | Some discrepancy; some time between 2009 and 2012, most likely 2011 | Some discrepancy, some time between 2014 and 2016, most likely 2014 | Because of staff changes, the exact date is unclear; it was around 2012/13 |
| Circumstances | Part of a strategy to reduce patient harm and increase care quality | Part of the strategy to reduce increased falls risk due to a predominantly single-room environment and develop compassionate care | Ward managers able to decide whether to implement IR or not. The majority of wards were thought to be implementing it at the time of the study |
| Staff involvement | Not initially, but recognised that this was ill-judged and staff were involved to redesign the IR process and documentation, which was piloted on some wards before roll-out to all wards | Managers reported a period of testing and piloting, although they think that IR was probably implemented too quickly across the trust | Owing to staff changes, it is unclear the degree to which staff were involved in implementation |
| Documentation | Four-page A4 booklet. Has been frequently revised according to perceived need. Includes 4Ps questions and the ‘Is there anything else I can do for you?’ question | Two-sided form. Includes 4Ps questions and the ‘Is there anything else I can do for you?’ question | Two versions of the IR form depending on a patient’s pressure sore risk score (Waterlow score of < 10 or ≥ 10). Both are part of a 49-page nursing documentation bundle IR form for those with a Waterlow score of < 10 includes 4Ps questions but not the ‘Is there anything else I can do for you?’ question IR form for those with a Waterlow score of ≥ 10 includes assessment of surface, skin, position, incontinence and nutrition but not the 4 Ps questions or the ‘Is there anything else I can do for you?’ question |
| Trust IR policy | Detailed trust policy | Detailed trust standard operating procedure | No formal IR policy |
| Frequency of IR | Hourly between 8.00 and 22.00, every 2 hours between 22.00 and 8.00 hours. Time intervals pre-written onto form | Frequency could vary according to risk assessment as long as rationale was recorded. Minimum frequency of every 4 hours. Time of IR was not specified, so specific time was entered by staff | IR form for patients with a Waterlow score of < 10 is every 2 hours, with time intervals written onto form IR form for patients with a Waterlow score of ≥ 10 asks staff to ‘continuously’ complete the form and staff are required to write in the time the patient is seen |
| Who does IR | Both HCAs and RNs. RN required to complete IR at 2.00, 8.00, 12.00, 16.00, 20.00 hours | Any member of clinical staff who had read the IR standard operating procedure and had received training in SKIN and Falls bundles | Both RN and HCA staff |
| Adaptation of IR beyond Studer format | IR documentation included questions about mobility, bed rail position, special mattress, body map to record skin integrity and presence of medical devices | IR documentation included questions offering drinks/snacks, falls prevention, body map to record skin integrity and presence of medical devices. Space available to document any actions resulting from IR | IR form for patients with a Waterlow score of ≥ 10 included assessing skin inspection, nutrition and special mattress needs |
The method of retrieval of routinely collected outcome data
Routinely collected outcome data from the NHS Safety Thermometer were retrieved for each of the case study wards. The NHS Safety Thermometer (www.safetythermometer.nhs.uk; accessed 13 September 2017) is a measurement tool to monitor commonly occurring patient harms in the NHS. It is used locally to support improvement in the delivery of harm-free care. Safety thermometer data were requested from case study trusts. The data received were analysed separately for each trust because of inconsistencies in the way in which they were recorded and aggregated. The main goal of this part of the analysis was to triangulate with the observational data. There were limitations because of the design of the study itself. Many things had changed at the same time in the health service and it would be hard to say that any pattern over time was due to IR, unless one could compare hospitals doing IR with others not doing IR. However, there were no controls (i.e. hospitals that did not undertake IR at all). It was hoped that some hospitals might have adopted early and others might have suspended IR for various reasons, giving us enough of a mixture to do a ‘synthetic controls’ analysis, which is increasingly used in economics, but this was limited as there were only six case study wards. Furthermore, safety thermometer data were collected on only 24 patients per month; analysis was, therefore, exploratory. Each of the various harms were summarised as line charts showing the percentage over time, with 95% binomial confidence intervals superimposed. Autocorrelation charts were generated to check for any persisting patterns over time, when one month seemed to influence the next. All data handling and analysis was done in Stata version 14.
The goal of quantitative analysis in this project was to fit within the realist evaluation framework. IR has not been implemented as a universal intervention that could be expected to show a homogeneous effect in all settings and times, like the introduction of a vaccine. Therefore, these quantitative analyses attempted neither causal inference nor estimation of the effect of IR. The purpose was to explore and understand patterns in the survey, Care Quality Commission (CQC) ratings and safety thermometer data, and to provide stimulus for qualitative data collection and interpretation.63
Safety thermometer data
Safety thermometer data were obtained from the three case study sites. Each hospital aggregated, labelled and shared the data in a different way, which limited analysis. Ideally, the data would have been combined into one table and then trends over time that might relate to IR would have been explored, for example a decrease in harms such as pressure sores at the time of IR being initiated in each hospital. However, the site differences meant that each hospital had to be analysed separately. There were also the following additional limitations:
- The data from different sites covered different time periods.
- IR was implemented before the safety thermometer data that could be accessed (there may be older, similar audit indicators, but not in a consistent format that could be compared across sites and months).
- Two sites gave aggregate data only, not at patient level.
- Case mix could not be considered with the safety thermometer data as there were no variables to give context for each patient, such as demographics, prescriptions and diagnoses.
- Sample size was small, at about 24 per month.
- An assumption that there were no inclusion biases was required.
Charts of the various harms were drawn as percentages, changing over time, for each ward. On these, the 95% CI is shown: this is the range of percentages that could contain a true rate of harms, given that sometimes the safety thermometer data will, by chance, pick up too many harms and sometimes too few. If a horizontal line can be drawn across the chart that does not leave that green area, then that means that the data obtained were not inconsistent with there having been no trend over time at all. However, even this basic form of statistical inference needs a further caveat: examining several wards over many months on several indicators is likely to produce an apparently interesting pattern just by chance. For this reason, these charts were used simply to feed into discussions in interviews and to contextualize observational data. Figure 3 shows the percentages of any new harms for each ward (anonymised) that provided data. In each case, IR had been initiated before the safety thermometer data began.
FIGURE 3 Percentages of any new harms for each case study ward. (Figure not reproduced here; see the original article.)
One case study site (site 2) identified wards as having different identities before and after a change of building location; the data were analysed on this basis. There are, therefore, eight wards highlighted in Figure 3 instead of six.
Method of cost analysis
A bottom-up approach to costing of IR activity was intended, using resource data collected in the staff interviews, non-participant observation and shadowing, and detailed information about ward context. However, this was not possible because of the difficulty in observing IR as a discrete activity and a lack of organisational memory of how IR was implemented. Therefore, the aim of this analysis was to calculate the incremental cost to the NHS of conducting IR in the wards included in the study, instead of not conducting IR (see Appendix 10). Three main types of cost were included in the analysis to estimate the total incremental costs:
- direct costs – the cost of any consumable resource directly used for the implementation of IR
- opportunity cost – the value of an alternative service that was given up as a result of performing IR
- indirect costs – cost of start-up, development and training not directly attributable to IR.
Costs and benefits
The incremental costs amounted to £8.27 per patient-day for hourly IR. Incremental costs dropped to almost half (£4.47 per patient-day) if IR was conducted every 2 hours. This difference should be expected, as the opportunity cost of nursing time drives ≈90% of the total incremental costs of IR. The dominance of nursing time costs makes a formal sensitive analysis irrelevant. Total incremental costs of IR are directly correlated with the frequency of IR and the hourly wage of the nurses dedicated to it. Total incremental cost and frequency of IR seem to be linearly correlated: by reducing the frequency by 50%, the total incremental costs dropped by a similar percentage. Total incremental costs of IR are less sensitive to the hourly pay of nursing staff, as the assumption used in the base case already allocates 70% of IR to HCAs (bands 2 and 3). By allocating 100% of IR to HCAs, total incremental costs would drop to £7.19 for hourly IR and to £3.96 for IR every 2 hours.
The results of this incremental cost analysis lead to a relevant conclusion: IR rounding imposed on the NHS an increase in the daily cost of hospitalisation in the range of £4.47–8.27 per patient, depending on the frequency of IR. For a ward of similar size to the ones included in this study, this would represent an annual increase in hospitalisation costs, estimated to be in the range of £38,164–70,609. Because NHS costs are calculated on full absorption basis, which includes the allocation of all overhead costs to health services produced, the annual incremental costs of IR could exceed £100,000 per hospital ward in the case of hourly IR.
This cost analysis shows that total incremental costs imposed by IR on the NHS budget would require a full economic evaluation of its value for money.