Introduction
In line with realist methodology, there is an understanding that using different data sources is likely to be most valuable to explore different elements of context, mechanism and outcome. For example, data about context may be obtained from observations, about mechanisms from interviews and about outcomes from quantitative data collection. Bringing together of all these data sources can support a comprehensive understanding of the way complex phenomena work and explain why things might work differently in different settings and contexts.
Multiple forms of triangulation were built into the Phase 2 qualitative data collection (see Chapter 4) and analysis including triangulation of data collection methods and formal theories. In the final stage of our analysis, theories derived from the Phase 2 qualitative results were examined using the data generated from the quantitative analyses.
Methods
We initially planned a detailed exploration of which elements of context, mechanism and outcome of each theory could be tested using each data source and considered using matrices for identifying areas of agreement and disagreement for each piece of evidence collected across the different data sets as described elsewhere.154 However, we were limited by the fact that our qualitative data and theories are largely based around individual reasoning (clinicians and patients), whereas only whole department-level data were available for quantitative analysis, meaning there were no ‘matching’ pieces of evidence in many areas for tabulation. We therefore elected to adopt a ‘following the thread’ approach.155 Each data set was analysed separately using appropriate methods (as outlined in Chapters 3–6). Key themes and questions arising following these analyses were then identified, and following the identification of questions in one data set, others were examined to provide a multifaceted explanation. We first identified questions raised through the qualitative data analysis explored through quantitative data analyses (see Approach 1), then identified noteworthy findings from the statistical analysis and cross-checked with both the qualitative data set and the marker conditions analysis (see Approach 2).
Approach 1 – identifying questions raised by the qualitative data
The team involved in the Phase 2 qualitative analysis met to review the theories developed, discussing each element of identified context, mechanism and outcome. We considered what we needed to know from the quantitative data to support, refute or refine the theories developed and identified key questions to address using the quantitative data and analyses.
Theory area 1 – streaming and patient flow
Our three theories about streaming and patient flow identified characteristics of individuals (experience, confidence, understanding of the GP-ED model) and systems (streaming guidance, staffing levels, ‘exit block’, oversight of the streaming process) which resulted in effective or less effective streaming. These were in turn understood to impact on patient flow in terms of patients being appropriately streamed to a GP, being seen more quickly, spending less time in the ED and ED doctors being able to more efficiently focus their time on ED patients. We characterised sites as having more or less robust streaming systems based on these factors using our qualitative data, grouping them into three categories (those with most, some or few features of an effective streaming system) and used the routine data and the marker condition analyses to explore whether those with systems which appeared most effective resulted in shorter waiting times when compared to other sites with less of the identified features. The four sites categorised as having most features of effective streaming were Integrated-1, Parallel-2, Outside-1 and Outside-3.
Routinely collected data
As discussed in Chapter 5, time series analysis was conducted using routine data from all the case sites including three control sites over 8 years.
Data on waiting time to treatment were not available and were generally poorly and inconsistently recorded. However, we did have data on the length of time patients who visited the ED stayed in the department. These data include all patients who visited the department (including those who were seen by ED clinicians or primary care clinicians). The results of the analysis showed that there was an upward trend in length of time in the department at most study sites (including the three control sites) over the study period both pre and post intervention.
Two sites showed a statistically significant decrease post intervention, both of which (Parallel-2, Outside-3) we had identified as having more features of effective streaming (e.g. senior trained nurses streaming at the front door, good oversight of the streaming system). Outside-3 also had a separate unit for geriatric emergency care which may have influenced LOS in the ED. The other sites where we categorised streaming as appearing most effective (Integrated-1, Outside-1) showed a trend towards increase in the length of time all patients spent in the department. Another site (Parallel-3) which had also established a system for redirecting patients to community primary care services also showed a non-significant increase in time in the ED after the GP model was introduced.
Marker conditions
Time spent in the department for the less severely unwell patients (those requiring lowest levels of investigation and treatment – see Chapter 6) streamed to GPs was less than those patients seen by ED clinicians across all marker conditions: back pain (a difference of 46 minutes, p < 0.001), chest pain (a difference of 64 minutes, p < 0.001), cough/breathlessness (a difference of 52 minutes, p < 0.001); child with a fever (a difference of 23 minutes, p < 0.001); abdominal pain (a difference of 1 hour 10 minutes, p < 0.001); headache (a difference of 1 hour 1 minute, p < 0.001).
Summary
The marker condition analysis did indicate the potential for GPs to improve the flow of the least unwell patients through the ED. However, at the whole ED level examined by the routine data, despite some sites demonstrating established and robust streaming systems and ED staff perceptions that streaming patients to a primary care clinician helped improve flow and decrease waiting times in the ED, the analysis demonstrated increased time in the ED for patients at most sites. Therefore, our theories related to streaming and patient flow have limited support from quantitative data and remain mostly based on qualitative evidence reported by ED staff and patients.
Theory area 2 – the general practitioner role
We were interested in the extent to which the roles GPs took on related to the models they were working within. Our qualitative data suggested individual reasoning such as decision-making approaches and attitudes to risk played a large part, possibly more so than model type. However, we also recognised that model type could influence and constrain individuals’ behaviours (e.g. when investigations were not available). We had identified that a ‘GP approach’ might result in use of fewer investigations, less ‘observation time’ in the ED and possibly fewer acute hospital admissions while a more ‘EM approach’ might be associated with GPs using more hospital investigations and acute admissions. We identified two sites (Parallel-4 and Outside-1) to test this theory with no access to acute investigations, clear guidance on the role of the GP, strong leadership supporting a GP role and governance processes which supported a GP approach.
Routine data
Time in the department
As described above, there was a positive trend for increasing time in the department over time for all three control sites, and most GP-ED model sites. Time in the department showed a trend towards increasing both pre and post intervention at the two sites of interest where the GP role appeared well supported (Parallel-4 and Outside-1) (no statistically significant change post intervention).
Investigations
The use of overall investigations within the ED was analysed. Data were non-stationary suggesting poor quality and that rates were influenced by coding/recording practices. Findings were a mixed picture of the use of investigations across control sites and all GP-ED models. There were three sites where the GPs had no access to acute investigations (Parallel-4, Outside-1, Outside-2) where we expected the ‘usual GP approach’ to be enabled, one site (Parallel-4) showed a statistically significant change in trend from increasing investigations (pre intervention) to decreasing investigations (post intervention). Data quality at Outside-2 precluded analysis.
Admissions
For acute hospital admissions, there were increasing admissions over time for all the control sites and a mixed picture across the intervention sites. Changes in trend were largely towards fewer admissions. Two changes reached statistical significance. At Parallel-4 (identified as an example of a site facilitating ‘the GP role’), there was a reversal in trend from increasing to decreasing admissions post intervention. At Integrated-1, where a more EM role would be expected, there was an increase on the background upward trend in admissions post intervention.
Reattendances
Reattendances from routine data were measured at 28 days in contrast to the marker conditions data where they were measured at 7 days. There was a trend demonstrating increased reattendances for most control sites and intervention sites. All four parallel sites demonstrated a downward trend pre intervention, with a subsequent increase in reattendances post intervention (three of these reached statistical significance including Parallel-4, a site of particular interest for facilitating the GP role). Data quality at Outside-1 prevented analysis.
Marker conditions
As described in Chapter 6, the marker conditions data were analysed from a single case study site (Parallel-2), where GPs and ANPs saw patients streamed to the primary care service and were encouraged to use a ‘usual GP approach’.
Across all conditions, the GPs saw fewer patients in total than the ED clinicians and the patients seen required lower levels of investigation and treatment. As described above, time in the department for the less severely unwell patients (defined as those requiring lowest levels of investigation and treatment) streamed to GPs was less than for those patients seen by ED clinicians across all marker conditions, due to a shorter period between seeing a clinician and discharge. Due to data quality, it was not possible to compare use of investigations between GPs and ED clinicians.
For patients who were less severely unwell, for three of the marker conditions (those with breathlessness, children with a fever and abdominal pain), GPs made statistically significantly fewer acute hospital admissions than ED clinicians.
For all marker conditions, there were no significant differences in the proportion of reattendances to the ED within 7 days. Overall, there were 24,834 initial attendances across all marker conditions; 83% (n = 20,658) of these were seen by ED staff, and 17% (n = 4176) were seen by GPs. Of these, 1586 (6%) reattended within the first 7 days after discharge. Of those who were seen by ED staff, 1319 (6%) reattended, while of those that were seen by a GP, 267 (6%) reattended [χ² (1) = 0.0004, p > 0.05]. Using reattendance within 7 days as a marker of safety, these data support the theory that GPs managing less severely unwell patients in an ED setting does not lead to safety issues, compared with ED clinicians.
Summary
Marker conditions data from a single parallel case site (where GPs were encouraged to maintain a GP approach seeing less severely unwell patients streamed to a GP service), showed that patients were likely to spend less time in the department and less likely to be admitted with no difference in reattendance rates at 7 days. Data on investigation use were of poor quality and not included in the analysis.
Routine data for all intervention sites and three control sites, also showed poor quality of investigation data creating unreliable findings. General trends for time in the department and reattendance rates at 28 days were increasing across most intervention and control sites (though not always reaching statistical significance). Any influence on admission rates and time in the department receiving care described in the marker condition analysis was not seen consistently in the wider routine data set.
While the marker conditions analysis supports the theory that a GP approach in an ED setting can be different to an ED clinician approach, it remains unclear whether this is due to individual clinicians’ management behaviours or service-level differences in ways of working. The quantitative data did not help us to identify particular models which were associated with favourable outcomes that could be attributed to facilitating the ‘GP role’.
Theory area 3 – teamwork and communication
Our theory about teamwork described that where there is effective teamwork, safer care will be delivered. We therefore classified sites according to the presence of features identified in our earlier theory building in relation to strong teamworking (ED leadership/culture with a positive view of GP contribution, compatible computer systems, physical proximity/informal face-to-face communication opportunities with ED team, clear formal system for allocating patients, evidence of interprofessional learning/supporting junior staff, strong leadership of the GP team, employing regular GPs and strong communication within the primary care team). Three sites with particularly strong teamwork and communication were identified (Integrated-1, Integrated-3, Parallel-2).
Routine data
The routine data showed a mixed picture in the pre-intervention phases for the reattendance outcome at 28 days, with a clearer picture post intervention where eight of the nine sites demonstrated a trend towards an increase. At the ‘strong’ teamwork and communication sites (one showed a statistically significant trend for increased reattendances post intervention (Integrated-1), one showed a non-statistically significant increase (Parallel-2) and the third had inadequate data for calculation (Integrated-3).
Marker conditions
The site for marker conditions analysis was one of the sites we identified as demonstrating strong teamwork and communication (Parallel-2). The marker conditions analysis did not identify any difference in reattendance rates between patients consulting with GPs or ED clinicians at 7 days at this site.
Summary
The routine data indicated increasing rates of reattendance across most intervention sites and the control sites. This may be due to changes in levels of morbidity among the population, public behaviours or service configurations (including availability of services elsewhere, notably in-hours primary care) rather than necessarily representing a change in the quality of the care provided at the ED. The marker conditions data suggested that GP care in the ED appeared to be as safe as ED clinician care using reattendances as a proxy. We have not identified data that support or further develop our theory related to the strength of communication and teamworking within the ED.
Theory area 4 – patient experience
Two theories were developed from the qualitative data around patient experience. These related to the relationship between patients’ expectations of ED care and subsequent satisfaction with the service provided, and how patients experienced being redirected away from the ED.
None of the quantitative data collected related to redirection, so this theory could not be further refined.
Our qualitative findings indicated that some patients access the ED because they expect to be seen by a doctor on that day (even though they expect a long wait) and found it acceptable to be streamed to a GP. If it also meant that their time in the ED was shorter than they expected, then they seem to be satisfied with their care. However, these findings were based on a small number of patients and there were insufficient qualitative data available on satisfaction to draw meaningful conclusions about differences in satisfaction across the different GP-ED models.
Routine data
Our routine data did not identify any model which was consistently associated with shorter stays in the ED; in fact, overall time in the department increased at most sites over the study period. We did not have any data quantifying patient satisfaction across the different sites to test our theory of a relationship between waiting times and satisfaction.
Marker conditions
Our marker conditions analysis did suggest that for patients requiring the lowest level of investigation and treatment, being seen by a GP was quicker than being seen by an ED doctor.
Summary
While the marker conditions analysis indicated that patients attending ED who required the fewest investigations and treatments might be more satisfied by being seen by a GP due to a shorter ED stay, we could not identify any particular GP-ED model that appeared most likely to consistently decrease LOS.
Theory area 5 – economic implications: provider-induced demand
Theories developed on the economic implications of the GP-ED models centred around three themes – that GPs would arrange fewer investigations and admissions (as discussed in Theory area 2), that GPs would free up ED staff to care for sicker patients, so improving patient flow through the department (as discussed in Theory area 1) and that GP-ED models might create ‘provider-induced demand’.150
Our qualitative data (see Chapter 4) indicated that in some EDs, Parallel-2 being the strongest example, the staff perceived that when the primary care service started, more people started attending the ED for primary care because the service was visible and easily accessible. There was also qualitative evidence from patients that they may choose to attend the primary care service in the ED because it may offer them more convenient and quicker access to a GP than in their usual GP service.
To inform review of the routine data, we classified our intervention sites based on features that increased their visibility to the public (signage over a separate entrance, patients able to self-present, 111 redirecting to the GP-ED service, patients being informed they are being streamed to GP, media/press coverage of unit/local awareness). We identified five sites which we classified as ‘highly visible’ (Parallel 2, 4 and Outside 1, 2 and 3).
Routine data
Attendance rates showed a mixed picture across control sites, across all intervention sites and within GP-ED models, but with a general picture of attendances increasing over time. At one of the ‘highly visible’ sites of interest (Outside-3), this reached statistical significance, with statistically significant increases post intervention also seen at two other sites (Integrated-1, Parallel-1). However, the quantitative data measured all ED attendances (with no focus on primary care attendances) and the qualitative data described staff perceptions based on their experiences of individual patients attending with primary care needs.
Marker conditions
The marker conditions analysis did not set out to address questions around provider-induced demand.
Summary
Our theory about provider-induced demand in distinct and visible services is not supported by the data we have on ED attendances over the study period, but our qualitative findings suggest that there is a perceived additional demand for primary care at an ED in services where the primary care service is visible, easily accessible and more known about within the local area.
Approach 2 – identifying questions raised by the quantitative data
Exploring key insights from the statistical analysis
Noteworthy findings from the statistical analysis were then identified and cross-checked with both the qualitative data set and the marker conditions analysis (Table 20) in order to attempt to explain any signals in the data that may not have been explored in depth as a result of the theory-driven analysis above.
| Outcome measured | Key insights from the statistical analysis | Possible explanation from the qualitative data | Fit with marker condition data |
|---|---|---|---|
| ED attendances over time (2010–8) | No evidence of reduction in attendances, with increased attendances in all intervention sites where change could be assessed with confidence (some statistically significant). | Perception at some sites that increases in primary care demand have been triggered by the visibility, accessibility and local awareness of the GP-ED model. | Not assessed. |
| ED reattendances within 28 days | Controls all show increases over time. At intervention sites, all that could be calculated show increased reattendances post intervention except for Integrated-2 (statistically significant decrease). | Integrated-2 was a very small ED and saw a limited range of patients with more unwell patients being taken to an alternative hospital. (After the study period, this ED was downgraded to an urgent care centre.) | Consistent – no difference in reattendance rates between patients seen by GP or ED clinician was identified. |
| Average time in the ED | Increased at all three control sites during study period and most intervention sites. Two intervention sites (Outside-3, Parallel-2) identified with evidence of reversal in upward trends in average time in the ED. | There was a new frailty unit at Outside-3 introduced at the same time as the GP-ED model. Note this site also showed a negative trend for admissions. | Consistent. Parallel-2 showed a reversal of the pre-intervention upward trend for time spent in the ED in the post-intervention period. The number of patients seen by the GP-ED model will determine the impact on department level averages. |
| Investigation use | Mixed picture of investigation use from control sites (flat or upward trends). Also, a mixed picture within each model. Two sites of interest showing statistically significant decreases in investigations post intervention (Parallel-3, Parallel-4). | At Parallel-4, GPs had no access to investigations.At Parallel-3, there was a structured pathway for redirecting patients to community primary care. GP-ED model saw only small proportion of overall ED attendances limiting potential impact. | Our marker condition analysis used investigation use to define the severity of the condition. This makes it challenging to also use the analysis to interpret the appropriateness of investigation use across different patient and clinician groups. |
| Admissions | All three control sites have upward trends, mixed picture across intervention sites.Statistically significant post-intervention changes identified at two sites.Parallel-4 showed a reversal of direction of trend from increasing to decreasing admissions.Integrated-1 showed increased admissions post intervention. | At Parallel-4, ‘the GP role’ was supported and staff gave examples of GPs managing paediatric patients without the need for admission.At Integrated-1, staff perceived that demand was increasing due to new housing developments in the area. | In the marker condition analysis for five of the six marker conditions, in the lowest severity group, GPs made fewer admissions than ED staff. There was a trend for decreasing admissions in the routine data for this site, but this was not statistically significant. |
Moving towards a refined programme theory
During our qualitative analysis, we described the theories through the core mechanisms identified along the patient journey through a GP-ED model, focusing on understanding how the models worked. However, when considering the presentation of our refined theory, we felt that it would be helpful to put more emphasis on the differences observed in ‘what was working where’ and identifying the contexts which facilitated or inhibited the mechanisms we had identified. We adopted this approach because it was felt most likely to help address the key question of most interest to commissioners and practitioners, that is, if you were going to implement a GP-ED model, which one should you choose (where and why)?
The refined programme theory diagram below is shown in three parts. Each diagram shows the same central pathway of the patient journey through the ED: being selected to see a GP; being provided with care; and leaving the ED having received a certain level of investigation and treatment; resulting in a level of satisfaction with the experience for the individual, and patient flow through the department.
We provide three different versions of the diagram to highlight similarities and differences between the three GP-ED models we studied. We recognise that each site visited had some unique characteristics, so these diagrams represent a high-level summary of key features for a given model, rather than an exhaustive description of every possible variation.
The patient journey through the ED is depicted with grey arrows. Context is labelled in blue, mechanisms within the green arrows and outcomes in red circles. We are most confident in the data supporting the links towards the left side of the diagram(s) which explore individual level decision-making amongst streaming staff and GPs. The links between these processes and patient- or departmental-level outcomes shown on the right of the diagrams remain unproven by the available quantitative data.
Compared to the other models, the streaming process is less influential at inside–integrated sites (Figure 19). The ‘invisibility’ of the GP service means its impact on patient expectations is also more limited. Unlike in other models, inside–integrated models allow for GPs to take on supervision of junior ED doctors which was felt to be a particular asset at some sites.
FIGURE 19 Programme theory to describe contexts, mechanisms and outcomes in inside–integrated GP-ED models. (Figure not reproduced here; see the original article.)
At outside–onsite models (Figure 20), the process of ensuring that the right patient got to see the GP was more complex, and so more open to potential problems. Their high visibility and accessibility were likely to have a greater impact on patients’ expectations and experiences. At these sites, GPs sometimes took on a supervisory role for a wider primary care team including other health professionals.
FIGURE 20 Programme theory to describe contexts, mechanisms and outcomes in outside–onsite GP-ED models. (Figure not reproduced here; see the original article.)
Inside–parallel models (Figure 21) showed the most variation in the way the services were set up, some were more similar to inside–integrated models, others more distinct as inside–parallel models. The lack of clarity around the breadth of the GP role could be a particular concern in these models though this might be overcome in settings with strong clinical leadership.
FIGURE 21 Programme theory to describe contexts, mechanisms and outcomes in inside–parallel GP-ED models. (Figure not reproduced here; see the original article.)
Conclusions
We took a structured approach to triangulating our different data sets, using both inductive and deductive approaches to maximise the insights generated.
We identified uncertainty in the links between context, mechanism and outcome identified in the theories our qualitative work generated as well as some conflicting findings between the routine data analysis and the local marker conditions analysis (both for the marker condition site and in general). Our qualitative data did offer some explanation for some patterns observed in the routine data analysis. We focused our refined programme theory on highlighting the differences in behaviours noted (largely at an individual level) between the three GP-ED models studied. This may be helpful when needing to consider how individuals may respond to a certain situation/model. We have focused our theories on explaining the areas where we are most confident in the data (links between context and mechanism) and identify where uncertainty remains (particularly around confirming links between context and outcomes and mechanisms and outcomes). We can be less confident in predicting outcome patterns at a department level for factors such as patient flow through department and investigation use, with some of this uncertainty being driven by the quality of data available for analysis.