Methods for the overarching study

This overarching study was necessarily dynamic in nature as our BRACE team portfolio of evaluations took shape over 5 years. This is reflected in our account here of the research methods used by the team to interrogate, inform and synthesise the portfolio of studies. The methods are set out in five phases that reflect the evolution of the study over time:

• Phase 1: development and scoping of overarching study
• Phase 2: initial work on cross-cutting themes
• Phase 3: cross-analysis of data from individual evaluations and identifying knowledge gaps
• Phase 4: targeted research to fill knowledge gaps
• Phase 5: overall analysis and synthesis.

Discussions among the research team, reflecting on findings to date and planning subsequent steps in the light of them, were an important element of all five phases.

Phase 1: development and scoping of overarching study

The initial driver for this study to focus on multiple long-term conditions was a prioritisation process undertaken by the BRACE team in 2018 using James Lind Alliance priority setting principles and methods to determine a longlist and then a shortlist of five candidate topics for evaluation. This process was guided and facilitated by our James Lind Alliance co-applicant in the BRACE team (Katherine Cowan) and commenced with an e-mail – sent to all BRACE Steering Group and Health and Care Panel members – asking for topic suggestions. The BRACE Health and Care Panel has over 40 members including patients and the public, system leaders, managers, clinicians and other practitioners, voluntary sector organisations, and health service researchers. Over 100 suggestions were received and reviewed using a Delphi process.29 A final shortlist of 20 topics was brought to a one-day Prioritisation Workshop in July 2018, chaired by Katherine Cowan and attended by 25 members of the Health and Care Panel including PPI representatives, health and social care professionals and managers, health services researchers, and people working for national healthcare representative organisations. Working first in small groups and then in plenary, attendees participated in three rounds of facilitated discussion and prioritisation. The workshop culminated in a prioritised list of topics, as shown in Box 1, with BRACE committing to taking the top five topics forwards into discussions with the NIHR Health and Social Care Delivery Research (HSDR) Programme.

It is of note that in addition to the number one priority topic, several others reflect dimensions of the health and care needs of people with multiple long-term conditions, including the use of technology to support chronic disease management, supporting people to self-manage and live well, and patient involvement in service design and innovation. This confirms one of the overriding insights from this overarching study, namely that living with long-term conditions is what very many people do, including more than three-quarters of adults in England aged over 60,14 and hence this emerges swiftly as a key priority for both user and professional stakeholders.

Following discussion with the HSDR Programme Director and secretariat, it was agreed that the needs of people with multiple long-term conditions should form the focus of the BRACE overarching study.

The BRACE team next undertook a literature review to identify and explore existing frameworks and theories for conceptualising services for people of all ages who are living with multiple long-term conditions. The review of frameworks and theories searched the Health Management Information Consortium via Ovid up to July 2019 (https://ospguides.ovid.com/OSPguides/hmicdb.htm). The search on frameworks yielded 78 results. The review was written up in an internal BRACE working paper (see Report Supplementary Material 1) and was used as the basis for the BRACE team to hold a project workshop in October 2019 to determine which frameworks and theories would be most appropriate to inform the overarching analysis of our portfolio of evaluations. The selected frameworks – House of Care30 and SELFIE (Sustainable intEgrated chronic care modeLs for multimorbidity: delivery, FInancing and performancE)31 – are described in Chapter 4.

A full protocol was developed for the overarching study in November 2019–January 2020, working in close partnership with our PPI Panel, Steering Group and the full BRACE research team. The protocol was revised in Autumn 2020 in response to NIHR HSDR feedback. The scoping of the overarching study, development of research questions and writing of the protocol was led by three members of the BRACE team, including the Centre director and RAND/Cambridge BRACE lead.

Phase 2: initial work on cross-cutting themes

Building on the initial prioritisation work and literature review of frameworks and theories for studying multiple long-term conditions, we undertook the following activities:

• Initial thematic analysis of BRACE evaluations in 2021. Informed by team workshops and our PPI activity, we examined project data from the BRACE evaluations that were completed or in progress at that point, and identified four initial themes of potential interest, which we followed up on in interviews with key informants (e.g. academics outside BRACE who work on rurality and health care). The themes we decided to explore at this initial stage were:
  • rurality and its impact on experiences of people living with multiple long-term conditions;
  • workforce models and the ways in which these are adapted (or not) to meet the needs of people with multiple long-term conditions;
  • the role of new care models and
  • approaches to information-sharing within and between organisations providing services to people with multiple long-term conditions.

The picture that began to emerge at this stage was of apparently widespread awareness in the NHS and research communities of multiple long-term conditions as an important issue for health care, but few examples of this being reflected in services currently provided, or in service innovations being researched.

• Work with our PPI co-applicant National Voices. We interviewed the Chief Executive of National Voices and met with approximately 20 members of the Heads of Policy group of National Voices’ member organisations at a workshop in April 2021. Those organisations represent health service users and carers. We tested our emerging insights from initial work for the overarching study and explored these questions:
  • What do people with multiple long-term conditions most need as we recover from the COVID-19 pandemic?
  • Are any of the cross-cutting themes in this study (e.g. rurality, workforce models, care models, information sharing) especially important for understanding the experience or delivery of care for people with multiple long-term conditions?
  • Are there particular service issues faced by people with multiple long-term conditions from rural areas?
  • From the perspective of people with multiple long-term conditions, are there any concerns about how changes in the healthcare workforce might affect their experience of care?

The National Voices Heads of Policy workshop revealed an overriding concern for better co-ordination between different parts of health services when presented with patients with multiple long-term conditions. This was a general concern and not specific to rurality, or workforce changes, or particular care models or approaches to information sharing. The lack of co-ordination was described as particularly stark for people with both mental and physical health conditions – they reported little co-ordination between mental health services and other health services.

• Building aspects of multiple long-term conditions into the design of individual evaluations. Members of the research team for the overarching study on multiple long-term conditions worked closely with the principal investigator and team of each BRACE evaluation. For example, this resulted in Phase 2 of the hospitals managing general practices evaluation9 focusing on the impact of this service model on people living with multiple long-term conditions. In a similar vein, the evaluation of the impact of telephone-first general practice4 assessed the extent of any inequalities in access for people living with multiple long-term conditions, and this study of digital-first primary care8 examined the experience of health professionals in providing care for people with multiple long-term conditions. The BRACE overarching study team provided input to the design of research tools for these studies, as they did to others where the focus on multiple long-term conditions was less explicit.

Phase 3: cross-analysis of data from individual evaluations and identifying knowledge gaps

In team discussions, the researchers reflected on what had been learnt in Phase 2 and agreed that it indicated a need to focus the remainder of the overarching study on understanding the apparent divergence between seemingly widespread awareness of, but rather less attention to addressing, the particular needs of patients living with multiple long-term conditions. The next phase of activity entailed cross-analysis of the data amassed through individual BRACE evaluations, using this to develop a set of three further questions to inform the final analysis and synthesis of the evaluation portfolio. Thus, Phase 3 was:

• Synthesis of findings from the final reports of 10 BRACE rapid evaluations, to determine what they revealed collectively about how service innovations work (or not) for people with multiple long-term conditions. We used the House of Care model30 and the SELFIE framework31 to underpin and structure the synthesis of findings. Our approach took a cross-case analysis approach, starting from the research questions in the protocol. The findings from the first seven completed evaluations were analysed in July–August 2022 and from the remaining three evaluations in January 2023.

•   Synthesis was undertaken through desk research, with two researchers reviewing all outputs from evaluations and bringing them together in a first draft of a matrix (which ultimately developed into the version provided in Report Supplementary Material 2), summarising what evidence can be drawn from each evaluation in relation to the overarching study research question. The research team discussed and agreed the draft matrix. A synthesis workshop was then held in August 2022 with eight members of the wider BRACE research team, including at least one researcher from the research team of each of the seven evaluations that were then complete, to discuss and revise the matrix and identify themes emerging from the cross-case analysis. One of the two researchers who had conducted the initial review of evaluation outputs then, in January 2023, reviewed the outputs from the final three BRACE evaluations, added them to the cross-analysis matrix, and discussed them with researchers from those three evaluation teams. The resulting complete matrix was then reviewed by the other members of the research team for the overarching study.
•   We concluded following the synthesis workshop in August 2022 that, referring to the SELFIE framework (Leijten et al.),31 our evidence was most plentiful with respect to the dimension ‘service delivery’ (most notably in the studies of hospitals managing general practice,9 Primary Care Networks,1 digital-first primary care8 and telephone-first general practice4) and to the dimension ‘technologies and medical products’ [which was prominent in the BRACE evaluations of pulse oximetry6 and artificial intelligence (AI) technology in social care7]. With reference to the House of Care framework,30 BRACE evaluations had predominantly concerned the ‘organisational and supporting processes’ aspect.

Phase 4: targeted research to fill knowledge gaps

In view of this emerging overall picture, and following discussion with the HSDR secretariat in October 2022, we developed in a research team meeting the following questions and methods to fill knowledge gaps identified by the overarching study team and to guide the remaining phase of this study:

• What might good remote monitoring for people with multiple long-term conditions look like? The BRACE evaluations of oximetry at home6 and AI in social care7 had both touched on the role of remote monitoring outside healthcare settings but found little specifically about people with multiple long-term conditions. Telecare technologies such as those are, however, potentially highly relevant to people living with multiple long-term conditions. By focusing on one specific type of service (remote monitoring), we sought to explore what evidence already exists of whether and how people with multiple long-term conditions have needs or wishes (for remote monitoring) that differ from those of other people; the outcomes; and how far such different needs are being met.

• Method: A focused, rapid review of published evidence on remote monitoring for people with multiple long-term conditions (October–November 2022), followed by a research team workshop with PPI input to reflect on that evidence (December 2022).

• To what extent does it appear that the needs of people with multiple long-term conditions are driving the approach to integration being taken by Integrated Care Systems and the organisations within them? We had found from BRACE evaluations of Primary Care Networks1 and vertical integration of hospitals and general practices9 that the needs of people with multiple long-term conditions had not had much of a driving role in the places we studied. With Integrated Care Boards taking over local commissioning responsibilities in the NHS in England after 1 July 2022 alongside Integrated Care Systems, we decided to return to some previous interviewees to seek their views on whether this recent organisational change might be expected to affect how they could meet the needs of serving people with multiple long-term conditions in driving efforts towards integration.

• Method: Interviews with stakeholders at local and regional level in Integrated Care Systems covering those local areas where BRACE evaluations have been undertaken (late October to mid-December 2022).

• What might a system of service innovation that prioritised people with multiple long-term conditions look like, and how could this be achieved? Cross-analysis of BRACE evaluations in 2022 revealed little evidence of the service innovations concerned having prioritised people with multiple long-term conditions. Interviews with a sample of national and regional stakeholders were planned to seek insights into how a system of service innovation that does prioritise people with multiple long-term conditions would differ from the current approach.

•   Method: Interviews with stakeholders at national level with policy-makers, innovation/research funders and organisations representing service providers (December 2022–January 2023).

For the rapid review, a search strategy was developed by the study team and searches were conducted in Scope and PubMed. Both searches targeted English-language papers published from 1 January 2017 onwards and were conducted on 4 and 5 October 2022, respectively. The search was limited to the UK, European Economic Area (EEA) countries, Switzerland, Australia, Canada, New Zealand and the USA. The search process found 1411 papers in total. A pilot screening was conducted where 30 articles were dual-screened for eligibility based on title and abstract, after which the inclusion and exclusion criteria were further refined (see Report Supplementary Material 3, Table 3.3). A total of 17 papers met the inclusion criteria at screening, with a further two identified via snowballing in the citations of included papers during the extraction stage, resulting in a total of 19 papers – 2 reviews and 17 primary studies – included in the review. The 19 papers were extracted by a single reviewer using a template, which was analysed by the research team to identify main themes using narrative analysis.

For the interviews, a semistructured interview guide was developed and covered the topic of service innovations for people with multiple long-term conditions, with specific questions related to local Integrated Care System work, along with questions about identification of good practice, integration with mental health, and barriers to better meeting the needs of people with multiple long-term conditions.

The sampling approach consisted of both convenience and snowball sampling. An initial list of potential participants was general by the research team (n=21). Participants were also asked during recruitment and interviews to identify additional individuals that the research team should approach. A total of 18 interviews (with 19 individuals) were carried out over the period November 2022–January 2023 by three researchers. Participants held senior positions within Integrated Care Systems or Integrated Care Boards, acute trusts, Primary Care Networks and other provider organisations. We also interviewed academics and individuals with a policy-oriented background working for or representing coalitions of health and care charities (see Report Supplementary Material 4, Table 4.1 for the roles of the interviewees).

The interview approach received ethical approval from the University of Birmingham Arts and Humanities Research Ethics Committee. Interviews were held online via videoconferencing software (Microsoft Teams^®, Microsoft Corporation, Redmond, WA, USA) and were recorded using the built-in recording and transcription software. All individuals who agreed to participate were given a participant information sheet, had an opportunity to ask questions, and were asked to review and sign the consent form. All but two interviews were recorded using the recording and transcription software within Microsoft Teams, and these were used to summarise responses to questions in the interview topic guide for each participant. For the two interviews that were not recorded (due to failure of the technology), the interviewer took extensive notes during the interview and used those to summarise in line with the other interviews. The summaries of responses under each question were collated and analysed across both national and subnational levels and key themes were distilled into a working paper.

Phase 5: overall analysis and synthesis

The research team held two internal synthesis workshops (in January and February 2023) to consider together the findings of the evidence review on remote monitoring, the thematic analysis of national and regional stakeholder interviews, and the cross-project analysis. This led to the collective drafting of a framework for this report.

On 6 March 2023, a 90-minute, online workshop was held to sense-check the implications for policy, research and practice developed by the study team and explored in this report. The 10 workshop participants included representation from academia, an NHS Academic Health Science Network, a general practice super-partnership, NICE, NHS England, health/social care charities and two individuals with lived experience. During the workshop we shared the emerging themes from the synthesis of our work to date and sought discussion and input about the implications for our work for practice, policy and research.

In the next section of this synopsis report, we explain how we framed the overarching study of service innovations for people with multiple long-term conditions and how the portfolio of BRACE evaluations was interrogated for this purpose. Later sections set out our findings, organised according to the six themes identified, followed by implications for practice, policy and research.