Methods

Introduction

This report is based on data collected in a two-phase mixed-methods evaluation of Rounds. This chapter provides an overview of the research approach and questions. Further specific details are presented in Chapters 4–7.

Our approach has been broadly informed by ‘realist’ principles, and our aim is to understand how, in which contexts and for whom does participation in Rounds affect staff well-being at work, social support for staff, changes in relationships between staff and patients, ‘ripple’ effects in the wider teams and organisation in terms of compassionate care and patient care improvements; and to make recommendations regarding the role of Rounds in health-care provider staff support in order to inform future practice.4–6

Overview of study

A mixed-methods evaluation of Rounds was conducted in two sequential phases.

• Phase 1 (scoping reviews, review of relevant theories, interviews with Rounds architects, and national mapping study):
  • scoping reviews of Schwartz Rounds literature and alternative interventions to define Rounds, compare Rounds with alternatives and summarise the evidence base
  • exploring relevant theories and conducting interviews with original Rounds architects to identify programme theory and understand how Rounds might work.
• Phase 2 (survey and nine organisational case studies):
  • a survey of health-care staff who attend Rounds and those who do not (within case controls) in 10 sites to determine if Rounds have an impact on staff engagement and well-being
  • realist informed in-depth organisational case studies to understand how Rounds ‘work’ (context and outcomes), the mechanisms by which well-being and social support might be influenced, and staff experiences of attending, presenting and facilitating Rounds.4

Our mixed-methods approach enabled us to address our varied aims and objectives (Table 1), and to seek elaboration, enhancement and clarification of results from one method with results from another, providing more than the sum of their parts. Some methods were sequential (e.g. mapping interviews were used to further understand data provided in mapping survey) and others were concurrent (phase 2 surveys and case studies). Data were mixed at numerous points (e.g. the mapping survey informed our sampling for phase 2). The quantitative and qualitative phases were equally weighted, and the findings were integrated and discussed after first being analysed and discussed separately. The overall study design is illustrated in Figure 2 and the relationship to the study objectives and data collected is detailed in Table 1.

FIGURE 2 Overview of Schwartz evaluation components and processes. a, Six out of nine case studies (interviews and observation) and six out of 10 sites (survey) were the same. (Figure not reproduced here; see the original article.)

Phase 1

We undertook two scoping reviews of the literature on Rounds to define Rounds and determine the evidence base, and of other interventions similar to Rounds in order to compare them.7 We also explored the theoretical literature on reflection, group work and disclosure/discussion of emotional/challenging events to try to identify the mechanisms by which Rounds may work, which helped develop our survey for phase 2 (see Chapter 3), and, together with our interviews with the original architects of Rounds, informed our initial programme theory (see Chapter 8). The methods and findings from the scoping review are reported in detail in Chapter 3 and Taylor et al.7

We conducted an online cross-sectional survey of all Rounds providers in England that had signed a contract with the PoCF by the start of the project in September 2014 when the mapping phase began (n = 77). We then undertook telephone interviews with Rounds champions (usually facilitators or clinical leads) in the same sample to explore their reasons for adoption and experiences of implementation further. For the collation and analysis of secondary data to map and profile providers running Rounds, the sample comprised all organisations implementing Rounds in the UK by July 2015 (n = 115). We wanted to map the profiles of current UK Rounds provider organisations to determine the reasons for implementation of and, when adopted, attendance at Rounds, and estimate the resource implications of establishing and sustaining Rounds. These data also informed sampling of the case study sites in phase 2. The methods and findings of the survey and telephone interviews are reported in detail in Chapter 4.

Phase 2

Thirteen providers were purposively sampled from phase 1 [to maximise the variation of organisations in terms of size, location and type of provider (acute trusts, hospice, community/mental health care providers)] and length of time running Rounds to provide 10 sites for the longitudinal survey, and nine organisations for the organisational case studies. Six sites participated in both survey and organisational case study components (Table 2). A long list of potential case study sites was drawn up against the sample quota in our protocol while phase 1 mapping data (survey and interviews) were being gathered. Approximately 20 providers were on this list, and in-depth discussions (including presenting to steering groups) in 16 providers ensued; one declined to take part, one was not suitable as Rounds rotated in four sites, and we decided that one was not suitable for a variety of reasons. Rounds were not sampled as a result of phase 2 survey responses, as survey and in-depth field work were often concurrent.

Following a pilot survey, we conducted a two-wave survey with 833 Rounds attenders at baseline (follow-up, n = 484) and 2680 Rounds non-attenders at baseline (follow-up, n = 578), to provide within-case controls. To maximise new attenders, seven sites were selected in which Rounds began during the study (see Chapter 5). Surveys comprised eight validated measures and additional demographic and Rounds-specific information, with a total of 47 items (see Report Supplementary Material 1 for details of the survey items). These items were selected to measure hypothesised mechanisms by which Rounds might be expected to work (see Appendix 1). Surveys were longitudinal and were administered before Rounds attendance (for attenders) and after 8 months (see Appendix 2 for the baseline and follow-up surveys). They were designed to evaluate whether or not regular attendance at Rounds has an impact on health-care staff well-being, social support for staff and behavioural change towards patients and colleagues, compared with staff who do not attend Rounds. The detailed methods and findings of the survey are reported in Chapter 5.

Realist evaluation methods were used in nine case study sites.4–6,8 These sites were identified, using purposive sampling, to allow more in-depth work to be undertaken through interviews with Rounds attenders, panellists, facilitators, clinical leads, steering group members, board members and non-attenders, and non-participant observation of Rounds, steering group meetings and panel preparation. The case study sites were sampled to provide maximum variation of information-rich cases9,10 that included aspects of Rounds implementation theorised to have an important effect on implementation and Rounds impact (size of institution and percentage of total staff who attend; established and new Rounds providers11,12 and early and late adopters),13 and also a range of providers (see Table 2). Data from case study sites were collected to examine staff experiences of attending, presenting and facilitating Rounds and how Rounds are conceptualised and implemented, and the mechanisms by which well-being and social support might be influenced (or not). This included, for example, reasons for variance in attendance and attendees’ experiences, any influence on team hierarchy/teamwork and on coping with stress, and any reported behaviour change towards patients and colleagues, including wider ‘ripple’ effects felt in day-to-day practice. It had been our intention (as per protocol) to shadow staff in practice to observe ‘ripple’ effects, and we developed observation protocols to observe staff with patients with support from our patient and public involvement (PPI) group members. However, practically, it was very difficult to identify specific ‘ripple’ effects and follow staff in practice at the right times to observe these effects. We therefore omitted this aspect of data collection and informed the National Institute for Health Research of this change to the protocol. ‘Ripple’ effects were gathered through reporting in interviews.

Case study interviewees were identified through purposive and snowball sampling and then invited to take part. Informed written consent was taken and participants were given a unique identification (ID) number to ensure anonymity and confidentiality. The selection of panel preparation, steering groups and Rounds for observation was negotiated with the facilitator/lead site contact with the steering group and with individuals involved (panellists in panel preparation). In practice, researchers tried to observe every Round during their fieldwork period in each case study (5–6 months) and as many steering groups and panel preparation meetings as possible. Consent and confidentiality procedures were more challenging in Rounds. However, every effort was made to inform potential attenders about the presence of the researchers before the Round took place (e.g. on posters and in e-mails), thus giving attenders sufficient time to make a decision about attendance based on their knowledge of our presence. On arrival, attenders were asked to tick a box on a modified sign-in sheet, saying that they were happy for the King’s College London research team to observe the Round. As there was a possibility that attenders would not be aware of our presence prior to the Round, at the start of each Round either the researchers or the facilitators announced that researchers were present and were taking field notes, and explained that any participant could approach the researcher (who was identified to attenders) and ask for any information they may have provided during the Round to be withdrawn and not used in the research. We also undertook two focus group interviews in November 2016 with Rounds experts (mentors from within and outside our case study sites) to test our realist findings. Further details of the methods and findings from the in-depth qualitative case studies are reported in Chapters 6–8.

Mixed-methods analysis

Following the completion of all data analyses, we carefully examined the data from our separate phases/methods;14 for example, three members of the team (JM, CT and JD) interrogated our phase 2 survey data for the six case studies for which we also had qualitative data, and vice versa, and cross-referenced these in our findings where applicable. We reviewed the survey data by case study site and examined the descriptive statistics. Although the small numbers of regular attenders in each case study site prevented us from analysing these data any further, we have explored our divergent and similar results (see Table 28) and present the limitations of each method in Chapter 9. Our phase 1 mapping interviews were also analysed to inform our realist questions in phase 2.

Methodological frameworks

This mixed-methods study is significantly informed by realist evaluation as follows:4,6

• phase 1: literature review – although we did not undertake a realist synthesis, we did use the literature to identify our initial programme theory and identify proposed mechanisms by which Rounds might work
• phase 1: mapping survey – not realist informed
• phase 1: mapping interviews – realist informed regarding identifying mechanisms of action
• phase 1: visit to SCCH in Boston – informed initial realist programme theory
• phase 2: survey – measures informed by mechanisms identified in literature
• phase 2: case study data collection and analysis – realist evaluation.

The realist evaluation approach acknowledges that intervention programmes do not necessarily work for everyone, as people are different and are embedded in different contexts,4 and so it helps researchers to go beyond the simple question ‘do Schwartz Rounds work?’ It seeks to understand better what works, and for whom: which staff, which organisations, in what circumstances. Realist evaluation is concerned with understanding causal mechanisms and the conditions under which they are activated to produce specific outcomes, which is highly relevant for this study. It recognises the interwoven variables that operate at different levels in organisations and thus suits complex social interventions. The realist approach to data collection and analysis is driven by retroduction, which starts with the empirical and explains outcomes and events through identifying the underlying mechanisms that are capable of producing them.15 (See Chapters 7 and 8 for more details of realist evaluation methodology and analysis, and see Chapters 4–6 for the mixed methods and analysis used in the mapping survey and experience interviews.)

Research ethics

Three different ethics approvals were necessary for this study. The pilot survey and mapping study were not subject to NHS ethics or other NHS research governance processes as they involved only NHS staff, not patients.

1. Ethics approval for the mapping study in phase 1 was granted by the ethics committee at King’s College London (King’s College Psychiatry, Nursing and Midwifery Research Ethics Review Sub-Committee reference PNM/13/14–159). Research and development (R&D) approval and letters of access [CSP (Coordinated System for gaining NHS Permissions) number 148528] were also required for all sites before invitations to participate could be sent. These were issued by individual trusts/hospices (see Data collection access and Figure 3).
2. Ethics approval for the phase 2a pilot survey was obtained from University of Sheffield Research Ethics Committee (REC) (application number 001869).
3. Ethics approval for phase 2 survey and case study interviews and observation was granted by the National Research Ethics Service Committee London-South East (REC reference 15/LO/0053).

Following receipt of the favourable opinion from the REC, we obtained R&D permissions from our study sponsors and then local R&D permissions from each trust with a view to starting data collection. Research passports were obtained for the study researchers who conducted on-site data collection. Once each study site checked and approved the application, letters of access were issued.

Patient and public involvement in the study

Our approach to PPI was based on the principles that such involvement should be meaningful, respectful, relevant and collaborative. Patient representatives were involved from the project’s inception. Patients understood the challenges staff faced and believed, post Francis, that interventions to support staff, which had the potential to have an impact on patients, were very important and required studying. We took the view at the grant application stage, and thereafter, that those whom this study would benefit were both staff (the proximal outcome target group for our intervention) and patients (the distal outcome target group). We therefore involved both throughout the study. We actively involved patients through membership of the Project Steering Group (PSG) in order to improve the quality of the research and relevance of the study to NHS users and to support our dissemination activities. Members of the PSG included two PPI representatives who had previously provided input at the proposal stage, as well as a Schwartz Rounds staff representative. The PSG provided oversight of all aspects of the study, and, alongside other group members, our PPI representatives and Rounds staff members advised on data collection, commented on the findings emerging from the research and supported dissemination through links with their local networks. In phase 2, the PPI representatives were closely involved in drafting the staff and patient information leaflets explaining how we intended to observe and shadow staff in their everyday practice to observe the ‘ripple’ effects of attending Rounds (see above regarding the non-completion of this aspect of data collection). Collectively, their contribution has been highly valued in relation to highlighting the benefits to patients of Rounds. Details of the advisory and steering groups are provided in Acknowledgements.

Study management

The project was led and managed by the principal investigator (PI) at King’s College London. At the University of Sheffield and The King’s Fund there was a lead investigator. To co-ordinate work across the centres, weekly or fortnightly teleconferences were held involving the PI, all of the co-applicants and the staff employed on the grant. During the study period, three project advisory group meetings and five steering group meetings were held (see Acknowledgements for details of the steering group and advisory group membership).

Summary

This chapter reports the methods used in our mixed-methods study, which were significantly informed by realist methods. The study was undertaken in two phases: (1) scoping the literature and mapping of Rounds in the UK and (2) a survey for Rounds attenders and non-attenders in 10 sites and interviews and observation of Rounds in nine case studies. PPI was included throughout the study. Governance arrangements included advisory and steering group meetings. The backdrop to the study was a rapidly changing landscape in terms of uptake of Rounds in the UK following the publication of the Francis report3 and financial support from the Department of Health and Social Care, LETBs and Macmillan.