Work package 2: CITRINE

This chapter reports findings from the health economic component of the PERCEIVE programme. It consists of (1) systematic review of studies that have investigated staff activity on psychiatric inpatient wards and particularly the time staff spend with patients, (2) systematic review of economic evaluations of therapeutic interventions delivered in inpatient settings, (3) demonstration of simulation modelling methods to provide such evidence, (4) development of a patient-based questionnaire to measure staff contacts and activities on inpatient wards and (5) results from this questionnaire in terms of staff and activity costs and predictors of these costs.

Systematic review of studies measuring staff–patient contact time on psychiatric inpatient wards

Greater staff–patient interaction and greater patient activity (i.e. participation in therapeutic or social activities rather than being socially disengaged or spending time alone) in inpatient wards improves clinical outcomes for patients with mental illness.107 The changing composition of UK mental health-care service provision in recent years has put a strain on acute mental health resources, which has affected how nurses interact with and treat patients. This is mirrored in most western health-care systems. The greater emphasis and spending on community-based mental health care has also reduced the portion of the mental health-care budget allocated to inpatient psychiatric care, leading to a reduction in the number of acute care and long-stay beds, although the amount spent on inpatient care remains substantial.

The average daily number of available psychiatric beds in England dropped from 67,122 in 1997–8 to 26,929 in 2007–8.108 Admitted patients are more likely to have psychotic disorders and in 2006/7 the median length of stay for these patients was 42 days. At a cost of £268 per bed-day of acute inpatient psychiatric care,109 this average length of stay would cost £11,256.

Given the reduction in acute beds, priority naturally goes to those patients who are more severely mentally ill, particularly for those who have been involuntary admitted or have major social problems.110 Nurses report that they feel pressure to discharge patients who may not have fully recovered in order to free up beds.13 Owing to the severity of the patient mix, staff members find it difficult to provide interaction and therapeutic activities as much of their time is spent looking after very ill patients.

In a 2004 survey by the Sainsbury Centre for Mental Health,111 it was found that although 64% of wards routinely offer social and leisure activities (e.g. going to the gym, bingo) and 73% offered practical therapeutic activities (e.g. money management, cooking skills), psychological therapies were not routinely offered on the majority of acute psychiatric wards. Art therapy was the most commonly available therapy, available in 49% of wards, compared with 35% for psychosocial interventions, and less than 20% for cognitive behavioural therapy, solution-focused behavioural therapy and family therapy. Besides the difficulty of staff having the time to offer activities because they are occupied with taking care of very ill patients, the reports suggest that the lack of activities and talking therapies may be partially explained by the use of agency staff, the culture of the psychiatric wards and staff shortages, especially of occupational therapists (OTs) and psychologists.

Assessing both the extent of interaction between patients and staff, and patients’ participation in activities, may be important for both clinical and economic reasons. Therefore, it is helpful to determine the extent of this input. This might help in planning for the costs of specific units by determining the appropriate amount and grade of staff members. Nursing and other professional time comes with a cost and while most studies and payment systems currently focus on the average cost of an inpatient day, there are reasonable grounds for deriving per-patient inpatient costs. These grounds include the requirement in economic evaluations that costs reflect the actual resources used. A further, linked, reason for using actual per patient costs is that any form of prospective payment (such as ‘Payment by Results’ in the UK) should link prices/tariffs as closely as possible to care received.

Most of the high costs of inpatient care are attributable to personnel costs. Beecham et al.112 reported that staff costs account for 70% of the costs of child and adolescent psychiatric inpatient units. Similarly, McKechnie et al.113 found that the percentage of costs attributable to staffing varied by the type of unit, but ranged from 74% to 91% of total care and treatment costs of psychiatric wards.

Collins et al.107 have suggested that clinical outcomes are associated with patient activity and social interaction. If this is the case, then devoting more staff time to direct (and hopefully therapeutic) patient activities and care could have a positive effect on patient care and may be cost-effective. Indeed, a study by Dodds and Bowles114 found that changing the allocation of nursing time from formal observation of inpatients to structured, individualised activities, which often took place on a one-to-one basis, improved patient-reported quality of care while the number of staff sicknesses and staffing costs decreased. The authors estimated that in a best-case scenario, an inpatient ward could save up to £44,878 on staffing costs over a year if they adopted this change in nursing time allocation. Furthermore, research indicates that, for patients, the most important aspect of quality inpatient care is that staff members take their time with, and care for, patients.115 Of course it should be stressed that staff time is not synonymous with therapeutic input. Merely spending more time with patients might not improve outcomes. However, given the above findings it does seem logical to expect improvements to be linked with what staff and patients do on wards and as such recording staff and patient activities seems a reasonable exercise.

The aims of this review are to (1) identify studies that have measured nursing and patient activity and interaction on psychiatric inpatient wards, (2) compare methodologies used in these studies and (3) examine how much time is typically spent in staff–patient interaction. Finally, we make suggestions for future work based on the findings.

Methods

Reprinted from International Journal of Nursing Studies, Vol. 47 Issue 7 Sharac J, McCrone P, Sabes-Figuera R, Csipke E, Wood A, Wykes T, Nurse and patient activities and interaction on psychiatric inpatients wards: a literature review, pp. 909–17 Copyright (2010), with permission from Elsevier.15

We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines by setting up a search strategy as well as independently checking papers for inclusion and searching reference lists. Initially, the electronic databases EMBASE (1980–2008), MEDLINE (1950–2008) and PsycINFO (1806–2008) were searched with the following keywords: ‘inpatient’, ‘mental’, ‘psychiatric’, ‘hospitaliz(s)ed’, hospitaliz(s)ation’, ‘time’ and ‘activity’. A final check of the literature was made on 30 November 2009. Included studies were limited to those published after 1970 that were written in English and reported activity on psychiatric inpatient wards. Abstracts were reviewed by JS and PM to assess eligibility for inclusion. The reference lists of included studies were checked by PM for further relevant literature. The criteria for inclusion were that (1) the study was based in an inpatient setting and (2) that it measured or observed and recorded time spent by patients and/or staff on different activities. Given that studies differed substantially in their methods and contexts, we did not attempt to combine the results in a formal way. In addition, given the study heterogeneity we did not have predetermined measures of quality but we do discuss study characteristics in the text.

Results

The search strategy identified 86 papers of potential relevance. After reviewing abstracts, two studies were initially included. A further 11 studies were included following examination of reference lists. Study characteristics are summarised in Table 8.

Four of the studies were performed in England, three in Australia, two in Scotland and one each in Finland, Canada, the USA and Northern Ireland. Before the findings from the studies are described, we will summarise the methods used and make some inference about the quality of the studies.

Studies used data collected through interviews, questionnaires, audits of attendance registers, and observations of staff and patients on wards at varying time intervals, usually recorded with an observational instrument and coded according to specific categories.

Interviews or questionnaires

Using this methodology, staff members are surveyed or interviewed about the time spent at work. In the study by Ryrie et al.,110 staff members recorded their activity every 15 minutes over a 7-day period according to a set of pre-designated categories. The authors suggest that the 7-day period may not have been sufficiently representative and that some changes to staffing levels took place after the study. They also felt that 15-minute intervals may have been too long to detect changes in activity. In another study,116 staff members were interviewed on a daily basis and reported all activities, but not the time spent, undertaken in the past hour as well as an estimate of the number of minutes spent in direct patient contact. This study covered more time periods than Ryrie et al.,110 and so may have picked up more fluctuation in activity levels. The authors considered the number of nurses surveyed (n = 40) to be a limitation but this is in fact a relatively large sample compared with other studies (Ryrie et al.110 used a sample of n = 8). Neither of these studies included night shifts. The method of self-report of time spent has the advantage that it requires relatively small amounts of researcher time to complete. However, a disadvantage of this method is that, given the hectic pace on the inpatient ward, the staff members may have little time to complete the surveys or interviews and so may under or over-report certain activities. There is also the danger of response bias, in which the staff members may self-report more patient contact than actually occurs.

Audits of attendance registers

Audits of attendance registers have been used to assess patient participation in group activities. This can be used as a verification tool for patient self-report of activities or in conjunction with an observational design as was done by Radcliffe and Smith.119 This method is helpful in measuring group activity without the time and effort required for observational studies and to double-check patients’ reports, which may be inaccurate owing to confusion caused by symptoms or medication. The disadvantage is that it can be difficult to verify group attendance as some patients have difficulty concentrating and may drift in and out of group activities or attend for, say, 10 minutes of an hour-long activity. Radcliffe and Smith119 also point out that it is difficult to establish the accuracy with which activity leaders record attendance.

Observations of patients and staff

This has been the most commonly used method in studies reviewed. Researchers observe patients, staff or both for specified periods of time and record the activities they take part in, often using an observational instrument in which activities are designated into categories and coded according to category.

Altschul123 conducted one of the seminal observational studies of nursing activity. She observed 40 nurses in four wards of a Scottish hospital during 4-hour periods over 3- to 4-week time periods. Interactions lasting more than or equal to 3 minutes were included and timed and the nature of the interaction recorded. Another Scottish study (supervised by Altschul) observed 14 charge nurses for 12 hours each.122 Activities were recorded and categorised. Although these were early studies, they were comparable in observation period and number of nurses included to later work.

A Canadian study observed patients and staff in three wards over a 1-week period.124 Each ward was observed 60 times. It is likely that observations did reflect ward activity accurately during that period but, as with other studies, it is unclear whether 1 week is sufficient to draw conclusions. In an early study from the USA, patient and staff activity on four inpatient wards was observed.121 Six 150-minute sessions of observation were conducted for each ward and one person was observed during that time. Inter-rater reliability was tested and found to be reasonable, although variability was quite high for observation of social activities.

In Australia, Sanson-Fisher et al.120 made observations of patients and staff on a single ward. Two 6-day periods of observation were used and during this time observers moved through the ward twice every hour and recorded activities taking place. Although limited to one ward, it was an advantage that two periods (separated by 6 weeks) were used and also that a reliability test was conducted for 10% of observations. The percentage agreement between observers was between 83% and 100%, depending on the activity type. This study was followed by a comparison of patient activity on two different wards by the same authors.12 Ten patients were selected from each ward and observed six times for 5 minutes. This does represent a relatively short period of total observation compared with other studies. In another Australian study, Sandford et al.118 observed nursing activity when nursing numbers on wards differed. Initially observations were carried out on one ward over a 20-day period. Nurse numbers varied by day with either five, six or seven nurses being on duty. The impact of this on interactions was measured. Subsequently, nurse activity was observed in three wards over 12 weeks, allowing for natural fluctuations in staff numbers. In each case, nurses were observed every 5 minutes for 5 hours.

An in-depth analysis of the activities of seven inpatients in Finland was conducted by Lepola and Vanhanen125 as part of a more qualitative study. The patients were deliberately selected and the observation period was relatively short and as such the findings would not be generalisable. Elsewhere, Higgins et al.13 observed all nurses and five patients in each of 11 sites. Activities were recorded every 15 minutes during all shifts over a 1-week period. As with other studies, this period of time might be too short to ensure representativeness. Whittington and McLaughlin117 observed 20 nurses one at a time for one shift. This led the authors to call into question the representativeness of the data. Another limitation noted by the authors was that a convenience sample was used. However, this does appear to be the norm in the activity studies reviewed and may not be as serious a problem as the short observation period. Indeed, a convenience sample may be the most practical (and least disruptive) way of conducting such studies.

The aforementioned study by Radcliffe and Smith119 observed activity in 16 wards and six hospitals. This was an advantage in terms of representing a wide spread of wards, but the observation period was relatively short (1 week) and only three 10-minute observations per day were made.

The observational method is popular and does appears to be the most accurate way of measuring activity as activities are recorded as they occur, so are not subject to recall bias of retrospective reports or response bias. However, the disadvantage of this method is that it is subject to the observer effect, in that staff members would know they were being watched and their actions recorded, so they may alter their usual schedules of patient care. It is especially common for observational studies to measure activity according to specific categories. Observation does, though, entail a relatively high level of input on the part of researchers.

Some studies that have assessed the use of nursing time have found reasonably similar amounts spent with patients: 48%,116 43%117 and 50%.110 Other studies have though demonstrated that higher grade nurses spend less of their time directly with patients: < 30%,13 27%122 and 29%.116 (The former figure is for G-grade nurses. The authors reported that F-grade nurses spent around 45% of their time with patients.13 Conversely they spend disproportionately more time on administrative tasks.) In one study it has also been shown that time spent with patients has fallen over time for all grades of staff.13 Elsewhere, Sandford et al.118 reported that only 15–18% of staff time was spent with patients and 31–34% with other staff members (with the remainder being spent alone), while Sanson-Fisher et al.120 found that 24% of staff time was spent interacting with patients.

Although up to a half of staff time might be spent directly with patients, relatively little of this time appears to be spent on providing specific therapeutic interventions: 4%,116 17%117 and 13–20%.110

In her early work, Altschul123 found that nurse–patient interaction accounted for only 1% of observed time on wards and only 30% of patients spent more than 1% in interactions with nurses. Willer et al.124 later reported more positive results with patients spending only 39% of their time in ‘isolated passive behaviour’.

Another observational study121 found that inpatients in four US psychiatric wards spent approximately 50% of their time alone and only about one-third of their time engaged in social activity. Time spent alone by patients was positively related to deviant behaviours and inversely related to social activity. Staff members spent more time than inpatients on social activity (average of 46% across the four wards) and spent significantly more time with other staff members than with patients, for example they were observed near patients less than 20% of the time but spent about twice as much time near other staff members. Before this, Willer et al.124 had also demonstrated that staff spent twice as much time with other staff compared with time spent with patients. Interestingly, Sandford et al.118 found that increasing staff numbers does not necessarily help as staff in their study then spent more time with other staff.

High rates of inpatient solitary behaviour were also reported in two Australian studies about 30 years ago. Sanson-Fisher et al.120 observed that patients spent 49% of their time alone. In a subsequent study by the authors,12 it was found that most of the patients’ time was spent in solitary behaviour in both the mental health hospital ward (83%) and the general hospital psychiatric ward (62%), and little time was allocated to staff–patient interaction on both wards (9% and 13%, respectively).

Higgins et al.13 found that patients spent only 4% of their time with staff, while they spent 28% of their time talking with other patients, watching TV or doing nothing, compared with 17% of their time in therapy. Slightly more patient time (15%) appeared to be spent in organised activities in the small study by Lepola and Vanhanen125 but this was broadly defined as ‘activities related to the patient being together with other patients and the nurses’.

Radcliffe and Smith,119 in their observational study of 16 UK inpatient wards, observed that patients spent the majority (84%) of their time socially disengaged, that is not interacting with staff members or other patients. Only 4% of patients’ time was spent in organised activities such as talking groups or art therapy.

Discussion

This review identified studies that have examined how staff and patients on acute inpatient psychiatric wards spend their time and how much time is spent in staff–patient interaction. Studies published over a 35-year period from seven countries revealed quite consistent and clear findings: (1) on average, around 50% of staff time is spent in contact with patients, (2) the more senior the staff member, the smaller amount of time they spend with patients, (3) the amount of time spent delivering ‘therapy’ is probably in the region of 4–20% and (4) relatively little patient time is spent in contact with staff and much is spent in isolation. There is also evidence to suggest that staff time with patients is reducing over time13 and that increasing staff numbers may not result in more time spent with patients.118 The finding that staff spend relatively little time in direct contact with patients and that patients spend little time with staff do not go hand-in-hand as this depends on the staff-to-patient ratio.

The review also examined the methods used in the studies, most of which observed staff and/or patients over a period of time. A number of these studies were relatively small, including only limited numbers of staff. However, others were more representative in that they covered activity across different wards and sometimes across different hospitals. The majority of studies observed activity for only a limited period of time (sometimes just 1 week) and, as such, may not reflect typical activity patterns, but this is less of a concern in the multiward/site studies. This was recognised as a problem and it is perhaps inevitable given the limited research time that they would have been faced with. Most studies also focused only on daytime shifts. A further recognised problem with the studies is that observers might influence the very activity that they are measuring.

Other methods to measure activity are self-report and collection of data from audits/administrative systems. Self-report by nurses may result in recall bias and an incentive to ‘inflate’ time spent in activities considered more ‘worthwhile’. Self-report and observational methods have been compared (on non-psychiatric wards) and differences found.126 Nurses reported significantly more time spent on patient care (40% compared with 33% observed) and ward-related activities (7% compared with 3% observed). However, although significant, these differences are not particularly large. The authors also report that observational methods were more acceptable to staff than self-report, which perhaps reflects the busy workload on inpatient wards.

Limitations

There are a number of limitations to the review. First, the number of studies (n = 13) identified was relatively low. This is surprising given the broad search strategy and the long inclusion period. This suggests that there is little interest in how staff and patients spend their time on psychiatric inpatient wards, or that research resources have not been made available to address this or that studies have not been published in academic journals. Second, they related to a long period of time and specific settings; as such, it is challenging to draw firm messages from the findings. Third, we restricted ourselves to papers published since 1970 in English. Inpatient services have, of course, existed for a far longer period and it is likely that important work will have been published prior to 1970 and in non-English language journals. Unfortunately, research time did not allow for a search of these. Fourth, ‘grey literature’ (e.g. unpublished reports) was not reviewed. Finally, only two papers were initially obtained through the electronic search. This was surprising, but searches of the reference lists did reveal substantially more literature.

Implications

It has been known for many years that what takes place on wards is important for patient outcomes. A landmark study by Wing and Brown11 compared three hospitals in the UK from 1960 to 1968 to determine if the ward atmosphere can affect patients’ clinical outcomes. The findings suggest that poor social environments in wards, in which patients spend much of their time not engaged in any activity, exacerbate mental illness. Although patients were not considered to differ in illness severity between the wards, they exhibited fewer negative symptoms in the ward with the best social environment. There were also changes exhibited over the years in all of the wards, for as the social environments improved so did the patients’ conditions. Conversely, when two of the wards’ social environments deteriorated over time after initial improvements, the patients’ clinical conditions worsened and they became more socially withdrawn.

In a more recent study,127 researchers changed the structure of an inner-city acute care ward by introducing treatment planning meetings, group therapy and community meetings. Two months after implementation, patients on the programme ward reported significantly higher scores of involvement (energy in treatment and patient activity), support between staff and patients, and practical orientation (providing patients with practical skills in preparation for discharge).

The lack of structured activities and patient contact indicated in this literature review is thought to be attributable to a variety of issues. These issues include a more severe patient mix so that staff members spend more time ‘containing’ than offering activities to patients, staff shortages of psychologists and OTs, and a trend over time of nurses spending less of their time interacting with patients and more of their time on non-patient activities such as administrative duties.

Some options to improve social engagement and participation in group activities are that (1) activities could be offered that are able to be led by unqualified staff or unqualified staff could be trained to lead some groups, (2) efforts could be strengthened to recruit volunteers onto the wards to lead activities and (3) the introduction of information technology onto the wards might be used to streamline administrative duties and paperwork so that nurses are freed up to devote more time to caring for patients on a one-to-one basis.

Systematic review of economic evaluations of therapeutic activities on psychiatric inpatient wards

Chapter 4 will report on the evaluation of an intervention to increase access to therapeutic activities on psychiatric inpatient wards. Such interventions require resources that could be used to provide other forms of care to people with mental health problems (or indeed patients in other clinical areas) and, as such, these interventions need to be evaluated in terms of cost-effectiveness, and Chapter 4 does include such an economic evaluation. As a precursor, we conducted a systematic review of economic evaluations of therapeutic activities on wards.

Methods

Initially, the electronic databases EMBASE (1947–2014 week 4), MEDLINE (1946–2014 week 4), and PsycINFO (1806–2014 week 3) were searched with combinations of ‘inpatient’ or ‘admission’ or ‘admitted’ or ‘hospital$’ in the title plus ‘healthcare util$’ or ‘health care util$’ or ‘healthcare cost$’ or ‘health care cost$’ or ‘cost$’ or ‘econ$’ or ‘financ$’ in the title plus ‘mental’ or ‘psychiatric’ in the title. Abstracts were reviewed by PM to assess eligibility for inclusion and checked by LK. The criteria for inclusion were that (1) the study was based in an inpatient setting and (2) that it reported an economic evaluation of a specific intervention rather than a model of care. As in the previous review, we did not attempt to combine the results in a formal way.

Results

The search strategy identified 236 papers. None of these met the search criteria.

Discussion

Inpatient care constitutes a substantial proportion of mental health-care expenditure and yet there appear to have been no economic evaluations of interventions that take place on inpatient wards. Although this is worrying, it was not an unexpected finding given that there is little clinical evidence regarding inpatient interventions and it is rare for economic findings to be produced in the absence of clinical results.

Modelling the cost-effectiveness of inpatient interventions: social cognition training case study

Decision modelling is a way of synthesising both economic and clinical evidence from a variety of sources in order to assist decision-makers in evaluating health technology. Thus, it can aid in understanding the trade-offs between costs and benefits of an intervention by explicitly and systematically laying out information about the health-care processes. Decision models can, for instance, be helpful if clinical trials are unable to capture the scope of an intervention (e.g. because its time horizon is beyond the typical length of a randomised trial) or if economic evaluation alongside a clinical trial is not feasible (e.g. because the intervention is already in place, other ethical issues or prohibitively high costs).

The literature review presented earlier revealed that there have been very few economic evaluations of inpatient-based interventions for people with mental health problems. The aim of this section is to demonstrate how modelling can be used to generate such information using a case study. It provides preliminary evidence regarding the cost-effectiveness of a psychological intervention for schizophrenia that is not routinely provided. Owing to the limitations of medication treatment, psychological interventions over the past few decades have gained increasing recognition in the treatment of schizophrenia.128 However, many of these have not been empirically validated and their cost-effectiveness has not been established.128 This case study uses social cognition training (SCT), which was provided as part of the DOORWAYS intervention in form of social cognition and interaction training (SCIT)129 to illustrate the considerations relevant in constructing a decision model for health economic evaluations.

Methods

The target population of the decision model were inpatients with schizophrenia. SCT was compared with treatment as usual (TAU). The economic value of SCT was assessed from a health-care perspective and a time horizon of eight months was assumed so that no discounting of costs or benefits was required. There were only limited data on care pathways for this intervention and the outcomes experienced by patients. Therefore, a simple decision tree structure was chosen (Figure 1).

FIGURE 1 Decision tree structure. (Figure not reproduced here; see the original article.)

Evidence was collected on the risk of relapse for each trial arm over the time horizon of the model.

For the purposes of this exploratory model, the general psychopathology subscale of the Positive And Negative Symptom Scale (PANSS) was chosen as an outcome measure. PANSS is a commonly used, disorder-specific, non-preference-based instrument. It measures symptoms in patients with psychosis rather than quality of life. This means that cost-effectiveness estimates based on this measure are not easily comparable to alternative interventions in other disease areas. However, our model uses an equation estimated by Siani et al.,130 which predicts EuroQol-5 Dimensions (EQ-5D) utility values based on PANSS scores. The EQ-5D utility values were then used to calculate quality-adjusted life-years (QALYs). QALYs combine the health-related quality-of-life values and the length of time that this quality is enjoyed into a single summary metric.

Relevant clinical trial evidence was identified based on two recent systematic reviews of the SCIT literature and studies citing these.131,132 The evidence on the reduction in the PANSS subscale was synthesised using a fixed-effects model for continuous measures assuming that there was no correlation between baseline and follow-up values within each trial arm. The meta-analysis was conducted in Stata^® (version 11; StataCorp LP, College Station, TX, USA).133 To date, there are only a few studies investigating the durability of treatment effects in SCT.131 For this reason, the treatment effect was assumed to decrease in a linear fashion up to the end of the time horizon of the model.

We identified only one small observational study by Hodel et al.134 (n = 19) investigating the effect of SCT on relapse rates. It suggested that this intervention may reduce rehospitalisation at 8-month follow-up. Conservatively, the reduction in relapse rate was assumed to lead only to cost saving but not improvements in quality of life. Based on results from elsewhere in the programme, the average length of admission was assumed to be approximately 45 days. To determine the cost of inpatient stay, data were derived from a nationally recognised source.134 The length of SCT in the literature was variable, ranging from 6 to 24 weeks.135 Based on the studies used to inform the model, treatment duration of 8 weeks with two 45-minute sessions per week led by a psychotherapist was assumed. The cost of a clinical psychologist was used as a proxy for the cost of a psychotherapist.136

In addition to calculating the differences in costs and benefits of the intervention, the cost-effectiveness of the interventions was compared using a net monetary benefit approach.137 If λ is defined as the value that the decision-makers assign to increasing QALYs by one unit and E and C are the incremental costs and differences between the two treatments, the net benefit of the intervention can be defined as NB = λE – C. Based on this framework, a probabilistic sensitivity analysis was conducted for a range of λ values from £0 to £40,000 per QALY (i.e. the probability that SCT was cost-effective was calculated based on random draws from the distribution of the input parameters). The results of this analysis were displayed on a cost-effectiveness plane as well as a cost-effectiveness acceptability curve. In addition, a threshold analysis was conducted by examining at which point the net benefit of SCT would be zero assuming that the reduction in relapse rates in the observation study was biased upwards.

Results

Three relevant studies were identified in the literature review.138–140 The weighted mean reduction in PANSS score attributable to SCT compared with TAU was estimated to be –3.49 (95% CI –8.01 to 1.04) (Figure 2). The study by Hodel et al.134 suggested that SCT reduced the risk of relapse by 26% (95% CI –70% to 17%) percentage points. The cost of the intervention was calculated to be £1632 and the savings for an averted hospitalisation were £15,615. In the base-case scenario, SCT was estimated to both increase QALYs by 0.009 (95% CI –0.007 to 0.018) and decrease the costs by £2532 (95% CI –£3282 to £8110); Figures 3 and 4 show the joint distribution of costs and benefits. In other words, the model indicated that it was the dominant treatment option. In addition, the probability that SCT was the most cost-effective was found to be high (> 75%) at all levels of willingness to pay for a QALY improvement. The threshold analysis suggested that in order to be at least as equally cost-effective as TAU, SCT needed to reduce the risk of hospitalisation, on average, by 8.5% to 10.5% depending on the willingness to pay for a QALY improvement.

FIGURE 2 Forest plot of meta-analysis: reduction in general symptoms PANSS. (Figure not reproduced here; see the original article.)

FIGURE 3 Cost-effectiveness plane: mean costs and effects of social cognition relative to TAU with 95% confidence region. (Figure not reproduced here; see the original article.)

FIGURE 4 Cost-effectiveness acceptability curve. (Figure not reproduced here; see the original article.)

Discussion

One should be cautious not to overstate the results of this exploratory model because some strong assumptions were used to be able to construct a coherent framework based on relatively sparse data. For example, the mapping or ‘cross-walk’ approach employed in this study has been described as a ‘second best’ solution and introduces additional uncertainty and errors into estimates141 Although we showed that the reduction in risk of hospitalisation of less than half of the base-case scenario may be sufficient for SCT to be cost-effective, it is highly uncertain to what extent the findings by Hodel et al.134 are generalisable. As noted by Henderson,135 research on SCT ‘remains very much in the developmental stage’. Based on current evidence, there is little robust evidence regarding its effectiveness but the results of this model suggest that it might have the potential to be a cost-effective intervention. However, before making the transfer from a developmental stage to routine clinical practice, further knowledge regarding the effects of SCT on quality of life and in the long term is required.

The objective of this section has been to demonstrate the feasibility of using a modelling approach to assess the cost-effectiveness of inpatient-based therapeutic interventions. As will have been apparent, this approach is dependent on the quality of the data available. Although it can circumvent the need to collect primary data from patients, it still requires robust estimates to be made of model parameters and to explore any uncertainty around these estimates in an appropriate manner. Having said that, the modelling approach is generally feasible and, as long as it is conducted with transparency, it represents an accepted way of generating information on cost-effectiveness.

Development of a measure of service use

The aim of this section is to describe the development and assess the reliability of a tool (the CITRINE) to achieve the objectives of collecting data on the care contacts and therapeutic activities of patients while on a psychiatric ward. This would potentially enable a more precise costing of inpatient psychiatric services.

Methods

Reproduced from Sabes-Figuera R, McCrone P, Sharac J, Csipke E, Craig TK, Rose DS, et al. Developing a tool for collecting and costing activity data on psychiatric inpatient wards. Epidemiol Psychiatr Sci 2012;21:393–9.142 © Cambridge University Press 2012.

In developing the tool we conducted interviews with inpatient staff to discuss its content and structure. The professionals interviewed included two ward managers, three OTs, one nurse and one therapy and benefits co-ordinator. All were working at psychiatric inpatient units in the South London and Maudsley NHS Foundation Trust. The interviews lasted for approximately 30 minutes and the main objective was to identify the group activities that take place on the wards for which professionals were most likely to have contacts with inpatients. Staff sometimes provided detailed schedules of patient activity on their wards. The first draft of the tool was then circulated for comments and suggestions to a group of experts including psychiatrists, psychologists and nurses.

Twenty-five service users from inpatient psychiatric wards were then interviewed using the questionnaire. The objective was to determine the acceptability of the tool, service users’ opinion about difficulties in answering the questions and to evaluate how practical it was to collect data in this way. Two focus groups were conducted in parallel with this feasibility component. The first included eight OTs, with seven nurses from an inpatient ward comprising the second. Participants were asked for their opinions about the tool and for any suggestions for improvements.

The process of consultation with staff resulted in the development of a tool that allows activities specific to an inpatient ward to be measured. The list of group activities was amended to include only the activities that had taken place in the previous week. Interviewers list the availability of activities (based on the ward calendar for the relevant week) and, therefore, each version of the tool is unique to that particular ward and time period. Initial questions on contacts with members of staff were shortened. The staff contacts section was also revised so that service users could indicate the name of professionals they have met even if they did not know or remember their job title. The tool is shown in Appendix 9.

This next study assessed the level of congruence between the information provided by service users using the final version of the tool and information that could be obtained from other sources. It took place in two stages. Initially, information on the activities attended by 41 service users over the preceding 7 days was obtained from the OT of each ward and the number of contacts with nursing staff members was collected from the patient’s records for the same period. This information was also collected from the same patients and period using the tool.

Information on the number and duration of staff contacts and activities attended by 22 additional service users was collected over a 14-hour direct observation period (08.00–22.00). An adapted 1-day version of the CITRINE questionnaire was used to collect the relevant information from the same service users over this period. (This stage was added later and so the patients are different from the 41 previously included.)

Contacts with staff members were combined with unit costs measured in UK pounds for the financial year 2007/8 from an established source.109 In the first phase (n = 41) we assumed an average nurse contact of 15 minutes. Activity costs were calculated specifically for the study based on session duration, preparation required, staff involved and materials for each activity provided on each ward. These data were combined with unit costs of staff time109 and with information on the average number of service users attending each activity to obtain an estimated individual cost for each group activity. Activities were subsequently classified in categories according to their type and cost. (A list of all unit costs is available from the authors.) An average of these activity costs was used in the analyses presented here. The significance of differences in reported and observed/recorded mean resource use was assessed using a paired t-test and the agreement was further assessed using the concordance correlation coefficient.

Results

The CITRINE tool typically took between 5 and 10 minutes to complete and most of the service users who participated in the feasibility study found it easy or very easy to answer the questions. Table 9 reports 7-day data from the CITRINE and from information covering the same period from OTs and case notes for 41 service users. The mean age was 37 years and 19 of the participants were women. Most participants (68%) had a primary diagnostic of psychosis or bipolar disorder and the average length of stay at assessment was 43 days. However, there was great variation in this represented by a SD of 66 days.

Service users report attending more activities than is contained in case notes (see Table 9). Attaching an average unit cost of activities suggests a cost difference of £10 per person. Case notes also report more one-to-one nursing contacts, resulting in a cost difference of £4 per person. Although neither difference is large, the discrepancy in the number of activities reported is statistically significant.

The comparison of data obtained from the 1-day observational study and the 1-day version of the tool shows good congruence in terms of activities attended and psychiatrist contacts. However, the congruence in terms of contact with nurses and other staff is less good. The differences in the 1-day version of the tool were significant or of borderline significance. The concordance correlation coefficient for total costs from this part of the reliability study was 0.79. The cost implications for the difference in nursing contact are important given that the observational data suggest that this accounts for one-third of the total cost.

Discussion

The process of developing the CITRINE tool has involved input from a wide range of staff involved in providing care to those receiving psychiatric inpatient services (nursing staff, OTs, ward managers and psychologists). Furthermore, key input to the process has been provided by the recipients of this care. The tool collects data directly from service users. From a pragmatic perspective, this is appropriate as it is the most practical way of obtaining the breadth of activity data that we require for economic studies. In addition, it emphasises the service user’s perspective in reporting activity. Of course this could mean that activities that are entirely unmemorable will not be recalled or costed (e.g. conversations with nurses that could be very similar every day such as ‘you have a telephone call’, ‘do you want to go for a walk?’). However, this type of very brief contact that is not remembered by service users is not the focus of the tool.

The result is a tool that has the objective of collecting information on the therapeutically relevant activities and staff contacts that take place on psychiatric wards and at the same time is acceptable to service users and relies on them as the main source of the relevant information. There have been other attempts to describe interventions and services provided in inpatient mental health settings, such as the International Classification of Mental Health Care.143 However, this instrument is a tool to classify services according to type of care, rather than one that provides information on the specific services that patients receive.

The main function of the tool will be its use in economic studies, combined with information on relevant unit cost of the services and staff time. Therefore, its level of accuracy in reflecting what is actually taking place within psychiatric wards is paramount. The reliability study carried out offers some reflections on this issue. In particular, the questionnaire (1) is a good source of information on the number of activities attended by service users and on contacts with psychiatrists, (2) may provide acceptable information on interactions between service users and non-nursing care staff (OTs, psychologists, social workers, etc.) and (3) may be more limited in recording time spent in service user–nursing staff interaction.

In relation to nursing staff contacts, there are some aspects that should be considered. First, this type of contact is common and, consequently, creates difficulties for accurate recall. Second, service users might report only contacts that they think are significant or meaningful for them. This classification is subjective and can result in some contacts being labelled as ‘non-contacts’ by service users. For example, a member of staff may have spent some time asking how a patient was and would regard this as a contact, but it may not have been recognised as such by a patient if it was very brief or unwanted. This may have been the key reason for the discrepancy between the observed nurse–patient contacts and the patient-reported contacts. The mental health status of some of the service users may affect their ability to provide accurate information, although data on this are lacking. Although time spent with nursing staff needs to be measured correctly, the difficulty seems to apply only to one-to-one contacts. Contacts as part of organised activities are more readily measured and, therefore, the disagreement over total cost is limited.

The alternative sources of information on inpatient psychiatric activities and staff contacts are not free of problems. Registers and electronic databases are designed to support clinical care and not to record activities within wards. Furthermore, there might be intra- and interward variability on the level of completion of these and on the accuracy of the information recorded. Observational data are an alternative but require too many resources to be a realistic when studies involve large numbers of individuals/wards with data collected at several time points.

There are limitations with this work. First, we did not conduct the full range of psychometric tests, although we did assess validity and reliability. It should be stressed that we do not regard this as a clinical instrument but rather as a simple recording schedule to ascertain more accurately the use of care on inpatient wards. We anticipate that the tool being adapted for different circumstances and extensive tests on one particular version would have limited usefulness. Second, and related, we did not assess inter-rater reliability. The content of the tool, though, is such that ambiguity of responses should be limited and, hence, inter-rater reliability is not as important as with a tool for which interviewer interpretation is required. Third, the activities relate to wards in one particular hospital. It may be that in another setting a more complex array of activities would present more of a challenge. However, the tool does require the listing of specific activities at the start and so this problem is reduced. Fourth, the relatively poor concordance on contacts with nurses is a cause for some concern. This has been discussed, but the wording of the question may not be clear. Greater definition of what constitutes a contact may be required. Fifth, service users may not always have been clear about the profession of the person they had contact with. However, we have dismissed this because no service users mistook a psychiatrist for nurse and this is likely to be the category of error that would most skew the costs data. Self-report data are associated with such problems. Sixth, to rely on self-report data might be a limitation of the tool although a number of studies have suggested that patient recall of service use is acceptable.144–146 Asking for the duration of contacts may be difficult but these durations are meant to be approximations.

Analysis of baseline costs

This section has the following objectives: (1) to describe the care contacts, therapeutic activities and related costs for patients on psychiatric wards in a large inner-city hospital and (2) to identify patient predictors of these costs.

Methods

Measures

The measures were (1) sociodemographic data: gender, age, ethnicity, level of education, marital status, and living and employment situation, (2) clinical information: diagnosis, whether or not the current admission to hospital was under the Mental Health Act72 (i.e. involuntary), at what age the service user had their first contact with mental health services and time since their first psychiatric admission, (3) functioning: Global Assessment of Functioning (GAF)147 scores were obtained for each service user. In addition, in order to explore the impact of service users’ perceptions on service use, the VOICE was included. Data on use of services were collected using the CITRINE.

Service costs

The service use data obtained with the questionnaire were combined with appropriate unit costs for 2008 obtained from national sources for care professionals.109 Similar data were not available for the unit cost of therapeutic activities and these were therefore calculated specifically for the study. Staff members responsible for organising activities were asked to provide details on the session duration, preparation required, staff involved and materials for each activity provided on each ward. These data were combined with unit costs for these resources and with information on the average number of service users attending each activity to obtain an estimated individual cost for each group activity. Activities were subsequently classified in categories according to their type and cost.

Statistical analysis

Multiple regression models were constructed to identify factors that explained variations in costs. Dependent variables were the cost of one-to-one care contacts and costs of therapeutic activities. A simple regression model for each potential independent variable was run and those variables with statistical significance for any of the two dependent variables were included in the final model. All selected variables were entered in a single block in a fixed-effects regression to take account of data being obtained from service users on 17 different wards. Cost data are often positively skewed and this may result in regression residuals that are similarly skewed, which is a violation of the assumptions underlying the linear regression model. Consequently, non-parametric bootstrapping methods were used.148 Bootstrapping involves resampling with replacement from the original data a sufficiently large number of times so that the population from which the sample is drawn can be approximated. Here, 5000 samples were automatically generated and bootstrapped 95% CIs were generated.

Results

Of those service users who fulfilled the inclusion criteria, 62% agreed to participate in the study. Service use data were available for 402 service users; however, some values for sociodemographic and clinical variables were missing for some participants or their length of stay was < 7 days. It was decided to include in the analysis only participants with complete data and the final sample size was 334. It was found that those with incomplete data had lower GAF functioning scores (35.6) than those included (39.8) (p < 0.01).

Characteristics of the sample are shown in two tables (see Tables 10 and 11). Two-thirds had a diagnosis of schizophrenia or bipolar disorder and a similar proportion had been admitted under the Mental Health Act. The average length of stay at the point of assessment was almost 7 weeks, but with a wide range. The GAF scores indicate a high level of disability. As stated above, the VOICE was designed so higher scores reflected lower quality of care. Therefore, those service users with a score of 38 or lower felt that the quality of care received was good. Alternatively, those with scores of 95 and higher, perceived that inpatient care was of low quality. In our sample, 21% had a VOICE score of 38 or lower and 5% had a score of 95 and higher. Therefore, most of the sample felt that the quality of care was neither good nor bad.

Table 12 summarises the data collected with the CITRINE instrument, providing information on the number of therapeutic activities attended and number and type of one-to-one care contacts reported by participants during the period of 7 days. Almost 10% of the sample stated that they had not had any one-to-one contact and more than one-fifth reported that they had not attended any activities. The average number of one-to-one care contacts for the whole sample was less than one per day.

More than 40% of participants reported no contact with a member of nursing staff in the past week and for those who did have contact, their frequency was slightly higher than one contact every 2 days (Table 13). Participants with no nursing contacts were not more likely to have contact with other staff members (and so this does not indicate a problem of distinguishing between care staff). The number of participants who reported contacts with the other types of care staff was low, except for psychiatrists, with whom almost three-quarters of the sample had a weekly average of 1.6 contacts.

The service use data were combined with the unit costs for 2008 to obtain the costs of therapeutic group activities and one-to-one care contacts during the previous week. There was significant variation within the different group activities in the cost per participant attendance. This variation is the result of three factors: first, the different qualifications and number of professionals involved; second, the duration of the activities and the time necessary for their preparation; and, finally, the number of participants who attended each activity. This number varied from 2 to 14.

Table 14 shows the average cost of group activities and one-to-one care contacts over the past week for the 334 participants. Costs were highest for time with a psychiatrist and accounted for more than half of the cost of care contacts. The cost of group activities represented a small proportion of the total cost, as did contacts with nursing staff. If the total figure, cost of group activities and one-to-one care contacts for the 1-week period is translated to a cost per day, the value is slightly more than £30 per day. There is substantial variation between individuals, as reflected by the SDs.

The regression analysis of the variation in costs of one-to-one contacts (Table 15) showed that older participants had lower costs. A higher score on the VOICE instrument was related to lower cost, indicating that participants with a worse perception of inpatient care also use fewer inpatient resources. For instance, a decrease in the VOICE score of 10 points would imply, assuming other variables do not change, an increase in costs of £11. There were significant cost differences between inpatient wards. Regarding the activities model, the VOICE score was, again, significantly associated with cost (see Table 15). Service users with a higher level of education had higher activity costs. (Subsequent analyses revealed this to be caused by more engagement with activities rather than use of higher-cost activities.) The results of the two models also show the positive relationship between the two types of cost.

Discussion

It was striking that 40% did not report contact with nursing staff. The results for therapeutic activities attended do not indicate that the low level of contacts with staff was compensated by more attendances. We have estimated that the cost of care contacts and therapeutic activities is around £30 per day, which is just slightly higher than 10% of the average cost per bed-day of £288 for 2008/9.149 This figure of £288 includes the costs of other resources not considered in the present study such as cleaning, maintenance, food, laundry, drugs and other treatments, etc. Crucially, the figures here do not include staff time that is not spent in contact with patients. In relation specifically to nursing staff costs, Bowers and Flood150 analysed expenditure data (i.e. a ‘top-down’ approach including non-contact time) for 136 wards and they found a cost per bed-day equal to £90, while in the current study we found that the daily cost of nursing staff contacts was £4.20. The difference could be partially caused by an under-reporting by service users of any contacts and, specifically, nursing staff contacts. However, our review of studies that measured nursing and patient activity and interaction on psychiatric inpatient wards found proportions of staff time spent in direct contact with patients (24–48%) and/or providing specific therapeutic interventions (4–20%).

High scores on the VOICE questionnaire, representing low satisfaction, were related to low costs of staff contacts and activities; however, the direction of the association is not clear. It is possible that attending fewer activities and having fewer contacts with staff is a result of a worse perception of care received, or low levels of care received may lead to poor perceptions of care. In relation to sociodemographic characteristics of the patients, age was inversely related to staff contact costs, while service users with ≥ 12 years of education had activity costs 23% higher than those with fewer years of education. Nevertheless, it is interesting that McCrone et al.151 found that older people and those with lower levels of education had higher costs of community mental care in a sample of individuals with severe mental illness. The age effect is the opposite of what was found here and may be related to other factors (e.g. physical health in the elderly and/or chronicity relating to poor function). In addition, it should be recognised that these costs may be influenced by the structure of services; staff are present on inpatient wards (i.e. they are available), whereas in the community it may be more complex to arrange a contact.

These findings need to be considered while taking account of the limitations of the study. First, a relevant proportion of inpatients, almost 40%, refused to participate in the study. It was not possible to collect information on demographic or clinical characteristics for this group of patients, therefore preventing a comparison with those who agreed to participate. However, subsequent data collection as part of WP4 indicates that those who consented were representative of people admitted to the wards. There are also some limitations of the CITRINE questionnaire which were discussed earlier.

The low costs found reflect low levels of reported interaction between care professionals and inpatients. This is concerning because it has been established that in inpatient care, outcomes are associated with the level of attention patients received from staff.152 The relationship found in this study between costs and service users’ views of the quality of care, measured by the VOICE instrument, emphasises these previous findings.

Conclusion

This chapter has summarised the existing evidence from studies exploring staff–patient interactions on inpatient wards. The key findings of this review are that despite evidence of the benefits of and official guidance for therapeutic interaction, reports of low activity and social engagement for patients have remained stable for 35 years and that limited nursing time is spent in direct contact with patients. It provokes the need for better exploration of how psychiatric inpatient staff members allocate their time on the ward within the constraints of limited acute care resources and how their time could be spent in more effective ways. This is especially important given that relatively few studies have addressed this topic which is likely to be of key interest to mental health professionals and patients.

Increasing therapeutic activities is a core part of this programme and it is of interest that no existing economic evaluations of such interventions have been identified. Increasing the evidence base is necessary and one approach is to use modelling methods, and we have provided a demonstration of these techniques. The main alternative to modelling is to collect primary data and we have developed a measure of staff contacts and activities to do this. The CITRINE is a tool that, despite some limitations, provides adequate information on the activities that take place within psychiatric wards. Therefore, its use is recommended, alone or in combination with other sources, in economic analyses of inpatient care. This chapter provides analyses of costs data based on this measure. This is the first study, to our knowledge, to present a detailed analysis of the cost of care contacts and activities on adult psychiatric inpatient wards. The results show that service users report low levels of quality contacts with staff members.