Pilot trial outcomes

The focus of this chapter on reporting the core outcomes of the pilot CRT is intended to inform the design of a future definitive evaluation, including the measurement of CLECC economic benefits. This chapter focuses on a comparison between experimental groups on the core outcomes for the study: quality of staff–patient interactions, patient evaluation of emotional care and nursing self-reported empathy. Intracluster correlation is also reported for each outcome. These data will enable sample size calculation and inform outcome measure selection and use in a future trial.

Quality of staff–patient interactions

The observed quality of staff–patient interactions, assessed using QuIS scores, between experimental groups at follow-up is shown in Table 31 . The distribution of interaction scores across the five available categories is broadly similar to baseline distribution, with most interactions rated as positive care and fewest interactions rated as negative protective for each experimental group.

There are more positive (social plus care) and fewer negative (protective plus restrictive) scores for intervention wards than for control wards at follow up (78% vs. 74%, 8% vs. 11%). Chi-squared testing of these results suggested a significant difference between experimental groups (p = 0.017), but the results shown in Table 32 indicate that, once other variables are taken into account in the analysis, the odds of a negative interaction are not significantly reduced because of the effect of the CLECC intervention. Results are in the direction of an effect favourable to CLECC, that is, there were fewer negative interactions on intervention wards, but this was not a statistically significant difference.

The proportion of negative interactions per patient was calculated on all of the QuIS data (n = 140 patients at follow-up). The results in Table 31 indicate that, although some patients had no negative interactions, others had up to 67% of their interactions rated negatively. On average, 10% of interactions per patient at follow-up were rated as negative.

In total, 92% of patients (i.e. 129 out of 140) at follow-up were observed for a full 2-hour period. For patients observed for the full 2-hour period, the number with one or more negative interactions during that 2-hour period was calculated (Table 31). Findings did not differ significantly by experimental group (chi-squared p = 0.744).

Table 33 illustrates the total negative interactions by individual ward. At the individual ward level, some wards (n = 3) appear to have improved their QuIS ratings and some appeared to have deteriorated (n = 3). The proportion of patients experiencing at least one negative interaction in a 2-hour period varies between individual wards from 25% to 64%. Two of the intervention wards showed an improvement between baseline and follow-up (i.e. the proportion of patients with negative interactions decreased), two showed a deterioration and the two control wards also showed deterioration.

Patient evaluation of emotional care

Patient evaluations of emotional care using the PEECH tool, administered through written patient survey, did not differ significantly by experimental group (Mann–Whitney U-test, p > 0.05), although small non-significant differences in total score and three out of four subscales favoured CLECC (Table 34).

Fewer patients in the CLECC group than the control group had low scores in the connection subscale (63% vs. 79%) (Table 35), but this was not adjusted for any potential differences at baseline or in patient characteristics or ward effects.

The results shown in Table 36 indicate that, once other variables are taken into account in the analysis, the odds of low connection scores are lower on the intervention wards, but not significantly so. Model 2 has been adjusted for ward, Model 3 for ward and baseline and Model 4 for ward, baseline and patient characteristics (age, gender, ethnicity and education level).

The results are in the direction of an effect favourable to CLECC, that is, CLECC may be associated with a reduction in the odds of having a low score in the connection subscale of PEECH. However, this association is no longer significant when we adjust for baseline and patient characteristics.

Patient evaluations of emotional care in each ward at follow-up are shown in Table 37 . Higher scores indicate better patient-reported experiences. As at baseline, connection consistently scores lower than the other subscales. Total mean PEECH scores range from 47.6 on ward C to 53.8 on ward B (out of possible total of 66). There was a small improvement in total mean PEECH score summed for all the wards from baseline to follow-up [48.9 (SD 11.7) to 49.9 (SD 10.8)]. There are variations in the direction of change over time between individual wards, with ward E showing deterioration across all subscales and total PEECH score, and wards B, D and C showing improvement in all subscales and total PEECH score.

Nursing staff self-reported empathy

Levels of self-reported empathy using the JSE scores from the nursing staff survey (RNs and HCAs) varied across the individual wards at baseline and at follow-up. Higher scores indicate greater empathy.

Empathy scores by experimental group at follow-up are shown in Table 38 . Mean and median empathy scores were similar across experimental groups at follow-up. A Mann–Whitney U-test confirmed no significant difference between groups (p = 0.800).

There was a small reduction in mean empathy score for all wards from baseline to follow-up (113 vs. 112). Mean empathy scores decreased on four wards from baseline to follow-up and increased on two wards (Table 39).

Intracluster correlation

At ward level, the ICCs for QuIS, PEECH (subscales and total) and JSE were low (< 0.1). ICC was high at the observation session level for QuIS (Table 40).

Economic evaluation

The implications of the findings reported in this chapter are briefly explored in relation to an economic evaluation. None of the outcomes reported here seems appropriate for use in cost-effectiveness analysis. Such analysis typically takes the form of cost per unit of a particular outcome, usually the primary outcome. The primary outcome here, QuIS, is not expressed as a single number, which rules out any use of cost per QuIS unit. The same broadly applies to PEECH. Although nurse empathy is expressed as a single score, interpretation of a cost per unit of nurse empathy would be difficult. Given these difficulties, the most productive way forward would involve an impact inventory of the sort recommended in a recent authoritative review.123 Besides being good practice for all economic evaluations, this would ensure that relevant data were presented on the range of costs and benefits involved.

Chapter summary

This chapter has focused on a comparison between intervention and control wards on the core outcomes at follow-up for the study, namely quality of staff–patient interaction, patient evaluation of emotional care and nursing self-reported empathy. The results suggest that CLECC may have a favourable effect in reducing negative interactions between staff and patients, and in reducing patients’ experiences of lack of emotional connection with staff, but, as expected, we did not detect significant differences once other variables were accounted for. We found no evidence that nursing staff empathy improved because of CLECC, but these results in particular have to be viewed in the context of the low response rate to nursing surveys. Improving staff survey response rates in a future evaluation will improve confidence that bias is not skewing the results.

We reported between-ward differences, but differences at this level and on this small scale are as likely to be explained by random variation as by any other cause.

This chapter also reported on intracluster correlation for the three core outcome measures. All measures showed low variance at the ward level. However, there was a clear design effect apparent with QuIS at an observation session level, and this will need to be accounted for in the design of a future trial.

These results reported here will enable sample size calculation and inform outcome measure selection and use in a future trial, and are followed up further in Chapter 11 .