Routine data analysis to examine clinical effectiveness and cost consequences of different general practitioner–emergency department models

Introduction

This chapter summarises the analysis of routinely collected data available for the different GP-ED models.

Study aim; chapter aims and objectives

The overall aim of the study is to determine clinical effectiveness and the resources, costs and consequences of different GP-ED models compared to others (including controls) with respect to key outcomes, and to further understand the ways in which service design and setting (context) generate variations in outcomes. The study is based on 13 purposively selected sites representing a range of GP-EDs models (3 inside–integrated, 4 inside–parallel, 3 outside–onsite) including Control (3 with no GP-ED model). A combined statistical and health economic analysis plan was produced prior to analysis and available from the study authors (see Report Supplementary Material 2).

Clinical effectiveness analysis

Methods

Patient-level routinely collected data relating to ED attendances and subsequent hospital admissions were obtained from HES AE and HES APC data sets (via NHS Digital) for study sites located in England; and from EDDS and PEDW data sets (via SAIL) for study sites located in Wales. The attendance-level data are summarised as time series (per site, aggregating data for each study fortnight) for the following variables:

• counts of ED attendances
• reattendance at same ED
• ED attendance leading to a hospital admission (defined by patient record appearing in the APC data set)
• investigations undertaken during an ED attendance
• treatments delivered during an ED attendance
• average time (minutes) of an ED attendance
• length of stay (LOS) (days) of hospital admission.

Table 5 shows study fortnights for which routine data were provisioned at study sites with the first study fortnight with a GP model (intervention) in place; these define the maximum number of aggregated values for variables, before and after introduction of GP model.

Data from 1 October 2010 to 30 September 2018 were eligible for inclusion. This study window was intended to allow acquisition of data covering at least 1 year before and after the GP-ED model was implemented at a study site. Including both start and end days, this study window comprises 2922 days, including two ‘extra days’ – 29 February – in 2012 and 2016. For the purposes of data aggregation, the study window divides into 208 complete fortnightly periods (starting with 1 October 2010–14 October 2010, 15 October 2010–28 October 2010 and ending with 7 September 2018–20 September 2018) and a single 10-day period (21 September 2018–30 September 2018).

The date of introduction of GP models varied between 1 October 2005 (Outside-2) and 1 April 2018 (Parallel-4); the study window did not include 1 year’s data both before and after introduction of the GP model at these two study sites or at Integrated-3. Emergency department data were available for the full study window for 11 sites (Parallel-2 and Outside-2 were the exceptions); in contrast, Inpatient data (i.e. HES APC) were only available for the full study window at one site (Control-3).

Data processing

We first cleaned the routine data provided, linked ED and inpatient data, and assigned eligible attendance-level data to study sites; summarising the quantity of data per site, and documenting issues noted. Each attendance was further categorised as (1) a reattendance; (2) an attendance leading to hospital admission; (3) an attendance at which a medical investigation was undertaken and (4) an attendance at which a medical treatment was delivered. We also recorded the time in ED (in minutes), and, where an admission was recorded in that hospital’s inpatient records, the LOS (in days). We then aggregated data using study fortnights, and documented adjustments made in producing aggregated data – for example, to accommodate missing data or incomplete study fortnights.

We are aware that various factors may affect the quality of data assigned to study sites and produce variations in data assigned to a study site over time. These include, inter alia, site-specific factors such as staffing levels and local practices, which then affect one or more outcome measures. We first assessed data quality, checking for completeness and the presence of unexpected features (e.g. ‘spikes’) or changes in trends. These explorations were presented and interpreted in the context of each site’s known history over the study window, which may provide reasons for such data ‘spikes’, omissions or unexpected variations over time attributable to local circumstances (e.g. NHS reconfigurations) or complexities.

We initially summarised the quality of the base variable (count of ED attendances per study fortnight), categorised as A1 (good); A2 (intermediate) or A3 (poor). Data regarded as A1 are included in formal analyses. Where data are categorised as A2, we either (1) noted subsets of data deemed to be anomalous and exclude these from analysis, or (2) noted limitations within the data (typically associated with coding in underlying routine data), included data in analyses and interpreted results in the light of these limitations. Data categorised as A3 are regarded as too poor in quality or quantity to be included in analyses.

We then assessed the quality of other variables in turn, documenting cases where data issues were deemed to require further data exclusions.

Variable definition

Key variables (counts of ED attendances; reattendances at the same ED) are generated directly from an ED attendance recorded in routine data; this high-level information was generally recorded to reasonable quality. Reattendances (further attendance at the same ED within 28 days) were expressed as proportions of all ED attendances per study fortnight.

Other variables depend on further information recorded at sites (e.g. investigations; treatments; disposition from ED) and the recording of this information was expected to be of variable quality, both across time; and across sites. We recorded the proportion of ED attendances leading to hospital admission; we derived one measure using ED Disposal codes (with better completeness over time), and a second measure based on hospital inpatient records (requiring linkage between ED and inpatient data).

We recorded the proportion of ED attendances for which any medical investigation was undertaken during the ED attendance, and the proportion of ED attendances during which any medical treatments were delivered. For these variables, variation in recording practices across time and sites precluded using records on specific subgroups of investigations or treatments.

We recorded an average time (in minutes) of ED attendances per study fortnight in ED, and an average LOS (in days) of hospital admission, with the convention that admission and discharge on the same day is recorded as a stay of 0.5-day duration.

Analysis

For all variables, we used standard time series analysis methods [time plots, correlograms of autocorrelation functions (ACFs), trend lines] to assess the nature and extent of linear trends and seasonality in data; (3). We fit ordinary least squares (OLS) linear trends to capture trends, and hence the change in trend before and after an intervention point at intervention sites, rather than time series methods (such as differencing) which accounts for a wider range of non-stationarity in data but lacks a useful parametric basis for capturing change in trend.

Since a seasonal component may be present in some variables, we defined a seasonality value for each study fortnight as follows. We first used a sine function with a period of 365.25 days to calculate a daily seasonality value, with the maximum value +1 in mid-July, the minimum value −1 in mid-January and zeroes around the spring and autumn equinoxes. For each fortnightly period, we then defined the seasonality factor to be the average of the 14 values for the days in that fortnight (with appropriate adjustment at the end of the study window).

Where the quality and quantity of available data allowed, we summarised the structure in a time series at an intervention study site by: (1) a pre-intervention trend coefficient, denoted by b₀, (2) a post-intervention trend coefficient, denoted by b₁, (3) the change (post–pre) in trend, denoted by D and (4) a seasonality coefficient, denoted by b_s. For control study sites, these summaries comprised: (1) a single trend coefficient, denoted by b₀ and (2) a seasonality coefficient (b_s). We noted where change in trends align with dates of known NHS reconfigurations (and hence a change in data recording scope and/or protocols), which may also have been confounded with the introduction of a GP-ED model. To facilitate interpretation of results, we analysed proportions as percentages.

We describe trends as upwards (associated with a positive coefficient) or downwards (negative coefficient), and where possible, focus on change in trend, which can itself be negative or positive. Some care, therefore, is needed with interpretation, since a positive coefficient, and hence an upward trend, are not necessarily associated with good or desirable healthcare outcomes. We emphasise changes in trend sufficiently large enough to be deemed statistically significant but note that the assessment of statistical significance here depends inter alia on numbers of study fortnights in pre- and post-intervention phases. Although assessment of seasonal variation is secondary here, similar remarks apply to interpretation of the seasonal coefficient; a positive value for this coefficient implies generally higher values of variable of interest in the summer (over winter), and a negative coefficient the reverse.

In all sites, we determined the extent of the underlying temporal dependency within a time series (after allowing for trend and seasonality) by assessing and summarising (via the first three values, denoted r₁, r₂, r₃) its ACF, which indicates the strength of any remaining temporal dependence or structure within de-trended and de-seasonalised data. Details on this short-term correlation structure are reported in Report Supplementary Material 3; this structure allows an initial assessment of the appropriateness or otherwise of the assumption of a simple linear trend and may also be used to inform further formal analyses.

Results

We assessed the quality of the base variables (ED attendance) to be A1 in seven study sites (spread across all four subgroups); A2 in five study sites and A3 in one study site (Outside-2), where the general unavailability in much of the study window was compounded by a lack of clarity linking ED attendances to that ED. There were generally good-quality data at study control sites and across Integrated sites, and a small reduction in data quality in Parallel sites; there is, however, a further reduction in data quality across the Outside sites, which hampers discussion of this subgroup. Due to the poor quality of data at Outside-2, this site has been largely excluded from descriptive analyses presented below. Further issues and limitations associated with specific variables are noted in the relevant Extended Tables (see Report Supplementary Material 3).

The quality of data on other variables was generally consistent with the categorisation of the main variable (ED attendance), although further issues were noted for selected combinations of variables, sites and time points. Such issues were particularly noticeable in the two variables which rely on records of activities undertaken during an ED attendance (medical investigations undertaken; medical treatments received), where some time plots indicate relatively large and sudden changes in variables’ values. These changes, consistent with a change in recording protocols or staff, are sometimes prolonged, but sometimes of only a few fortnights’ duration, while some appear to be temporary (so followed by reversion to previous levels), while others are seemingly more permanent.

The remainder of this section examines the variables in turn; in each case, we provide examples of the data at one intervention study site and one control study site. Using the data quality assessments above, we use either Control-1 or Control-2 and focus on Integrated or Parallel intervention sites. We also provide a table summarising results for all study sites, with a brief commentary on these results. With data at study control sites providing some overall context, we outline the extent to which patterns or themes emerge from an assessment of the quantitative data for study sites with any GP-ED model; and then look at the consistency, similarities and differences within subgroups of study intervention sites.

Emergency department attendances

Figures 5 and 6 illustrate ED attendance data at two example study sites: Integrated-1 and Control-3.

FIGURE 5 Emergency department attendances per study fortnight and OLS trend lines for Integrated-1. Note: Vertical orange line indicates approximate fortnight of GP model introduction at this site. (Figure not reproduced here; see the original article.)

FIGURE 6 Emergency department attendances per study fortnight and OLS trend line for Control-3. (Figure not reproduced here; see the original article.)

For Integrated-1 (see Figure 5), we observe a strong upward trend and regular seasonality, with evidence of an increased trend post intervention. This interpretation is consistent with the coefficient values b₀ = 2.988, b₁ = 10.759, D = 7.771 and b_s = 118.4; see Table 5. We note that the de-trended/de-seasonalised data show only moderate short-term correlation; this supports the use of linear trendlines to capture the structural temporal variation in this variable.

For Control-3 (see Figure 6), we observed a consistent downward trend over the study window, and clear evidence of seasonality; this is consistent with the coefficient values b₀ = –1.236 and b_s = 145.9; see Table 6. We again note that the de-trended/de-seasonalised data show moderate short-term correlation; this supports the use of a linear trendline to capture the structural temporal variation in this variable.

Table 6 summarises this outcome with a mixed picture across study control sites, with two upward trends and one downwards. This mixed picture also extends to study intervention sites and subgroups of intervention sites, but the general picture is of attendances showing increasingly upwards trends over time, as exemplified by three statistically significant positive changes in trend, at Integrated-1, Parallel-1 and Outside-3. Four other intervention sites show changes in trend that are not statistically significant; and there are insufficient data to make an assessment at the other remaining three sites.

Extended Table 6a (see Report Supplementary Material 2) provides further details on data quality categorisations A2 and A3.

Emergency department reattendances

Figures 7 and 8 illustrate reattendance data at two example study sites: Parallel-3 and Control-1.

FIGURE 7 Reattendance per study fortnight and OLS trend lines for Parallel-3. Note: Vertical orange line indicates approximate fortnight of GP model introduction at this site. (Figure not reproduced here; see the original article.)

FIGURE 8 Reattendances per study fortnight and OLS trends for Control-1. (Figure not reproduced here; see the original article.)

For Parallel-3 (see Figure 7), we observe a downward trend in the pre-intervention phase, followed an upward trend in the post-intervention phase; no distinct seasonality is present. This interpretation is consistent with the coefficient values b₀ = −0.015, b₁ = 0.006, D = 0.021 and b_s = 0.035; see Table 7. We note that the de-trended/de-seasonalised data show only moderate short-term correlation; this supports the use of linear trendlines to capture the structural temporal variation in this variable.

For Control-1 (see Figure 8), we observed a consistent upward trend over the study window, but no clear evidence of seasonality; this is consistent with the coefficient values b₀ = 0.009 and b_s = 0.043; see Table 7. We again note that the de-trended/de-seasonalised data show only moderate short-term correlation; this supports the use of a linear trendline to capture the structural temporal variation.

Report Supplementary Material 3 (Extended Table7a) provides details on further data issues.

Table 7 shows a mixed picture across study control sites and pre-intervention phases in study intervention sites, with roughly equal numbers of upward and downward trends. A clearer picture emerges in the post-intervention phases in study intervention sites; where eight of nine sites show an upward trend, and, in six of these cases, this trend has increased over that in the pre-intervention phase. Data quality issues make interpretation difficult for the Outside sites, while the Integrated sites include the one site with a change from upward to downward trend. However, a more uniform pattern is observed across Parallel sites, where a downward trend pre intervention is followed by an upward trend post intervention, with changes found statistically significant in three of the four sites.

We note that the de-trended/de-seasonalised data show reasonably moderate short-term correlation; this provides some support for the use of linear trendlines to capture the structural temporal variation.

Hospital admissions

Given the extended coverage ED data for this variable, we present here data based on emergency department disposition (EDD) codes. Figures 9 and 10 illustrate hospital admissions (EDD) data at two example study sites: Parallel-1 and Control-1.

FIGURE 9 Hospital admissions per study fortnight and OLS trend lines for Parallel-1. Note: Vertical orange line indicates approximate fortnight of GP model introduction at this site. (Figure not reproduced here; see the original article.)

FIGURE 10 Hospital admissions per study fortnight and OLS trend lines for Control-1. (Figure not reproduced here; see the original article.)

For Parallel-1 (see Figure 9), we observe an upward trend in the pre-intervention phase, followed an essentially flat trend line in the post-intervention phase; there is some evidence of seasonality. This interpretation is consistent with the coefficient values b₀ = 0.009, b₁ = –0.001, D = 0.010 and b_s = –1.613; see Table 8. We note that the de-trended/de-seasonalised data show reasonably moderate short-term correlation; this provides some support for the use of linear trendlines to capture the structural temporal variation.

For Control-1 (see Figure 10), we observed a consistent upward trend over the study window, but no clear evidence of seasonality; this is consistent with the coefficient values b₀ = 0.013 and b_s = –0.156; see Table 8. Here, the de-trended/de-seasonalised data show evidence of more persistent short-term correlation; a linear trend is best regarded only as a first step in capturing the structural temporal variation.

Report Supplementary Material 3 (Extended Table 8a) provides details on further data issues.

Table 8 indicates upward trends in all three control study sites; with a more mixed picture across the intervention study sites, where there are examples of upward and downward trends in both pre-intervention and post-intervention phases. Changes in trend are largely downward (as observed in five of seven sites), and with reversal of trend direction from upward to downward observed at in two sites (Parallel-1 and Parallel-4), and from downward to upward (Parallel-3). Changes in trend are deemed to be statistically significant at two sites; Parallel-4 (as discussed above) and Integrated-1, with an increase in the upward trend. The mixed picture seen within the Parallel subgroup of sites also extends to the Integrated subgroup; as above, data issues affect the interpretation of results in the Outside subgroup.

Medical investigations undertaken

Figures 11 and 12 illustrate data on medical investigations undertaken at two example study sites: Parallel-1 and Control-3.

FIGURE 11 The proportion of ED attendances where a medical investigation was undertaken per study fortnight for Parallel-1. Note: Vertical orange line indicates approximate fortnight of GP model introduction at this site. (Figure not reproduced here; see the original article.)

FIGURE 12 The proportion of ED attendances where a medical investigation was undertaken per study fortnight for Control-3. (Figure not reproduced here; see the original article.)

For Parallel-1 (see Figure 11), we observe a downward trend in the pre-intervention phase, which broadly continues, with a slight further decrease, in the post-intervention phase; there is little obvious evidence of seasonality. This interpretation is consistent with the coefficient values b₀ = −0.007, b₁ = –0.012, D = –0.005 and b_s = –0.620; see Table 9. We note that the de-trended/de-seasonalised data show reasonably moderate short-term correlation; this provides some support for the use of linear trendlines to capture the structural temporal variation.

For Control-3 (see Figure 12), we observe no simple trend pattern over the study window, or clear evidence of seasonality; interpretation of the corresponding coefficient values b₀ = 0.089 and b_s = –0.477 (see Table 9) therefore requires some care. Here, the de-trended/de-seasonalised data show evidence of persistent short-term correlation; this is consistent with a more complex form of non-stationarity in the time plot, emphasising that a linear trend is best regarded only as a first step in capturing this structural temporal variation.

Table 9 shows flat or upward overall trends in the three control sites, but with some strong evidence that linear trends do not fully capture the more complex patterns of structural temporal variation. In the intervention sites overall and within subgroups of these sites, the picture is generally more mixed in terms of upwards and downwards trends pre and post intervention; there is some indication of reversal in the initial trends at three sites. Statistically significant changes in trend were indicated at three sites: Integrated-1, with reversal of a slight downward trend to an upward trend; Parallel-4, with another reversal, but this time from upward to downward; Parallel-3, with an initial downward trend followed by an even stronger downward trend. However, interpretation of these findings also needs to reflect the strong evidence that linear trendlines do not fully capture the more complex patterns of structural temporal variation observed.

Medical treatment delivered

Figures 13 and 14 illustrate data on medical treatments delivered at two example study sites: Outside-3 and Control-1.

FIGURE 13 The proportion of ED attendances where a medical treatment was delivered per study fortnight for Outside-3. Note: Vertical orange line indicates approximate fortnight of GP model introduction at this site. (Figure not reproduced here; see the original article.)

FIGURE 14 The proportion of ED attendances where a medical treatment was delivered per study fortnight for Control-1. (Figure not reproduced here; see the original article.)

For Outside-3 (see Figure 13), we observe an upward trend in the pre-intervention phase, which is then reversed in the post-intervention phase; there is little obvious evidence of seasonality. This interpretation is consistent with the coefficient values b₀ = 0.015, b₁ = –0.615, D = −0.630 and b_s = 0.737; see Table 10. We note that the de-trended/de-seasonalised data show strong short-term correlation; this shows that the use of linear trendlines may not fully capture the structural temporal variation.

For Control-1 (see Figure 14), we observe a complex pattern over the study window, with no clear evidence of seasonality; interpretation of the corresponding coefficient values b₀ = 0.170 and b_s = –0.709 (see Table 10) therefore requires some care. Here, the de-trended/de-seasonalised data show evidence of persistent short-term correlation; this is consistent with a more complex form of non-stationarity in the time plot, emphasising that a linear trend is best regarded only as a first step in capturing this structural temporal variation.

Table 10 shows flat or upward overall trends in the three control sites, but with some strong evidence that linear trends do not fully explain the more complex patterns of structural temporal variation observed in time plots. In the intervention sites overall, there is a consistent pattern of upward trends in pre-intervention phases followed by downward trends in post-intervention phases, with changes at three sites (Integrated-1; Parallel-1; Parallel-3) sufficiently large to be flagged as statistically significant. However, interpretation of these results also needs to reflect strong and widespread evidence that linear trendlines do not fully capture the more complex patterns of structural temporal variation observed in this variable at study sites.

Average time in emergency department

Figures 15 and 16 illustrate data on average time in ED at two example study sites: Parallel-1 and Control-3.

FIGURE 15 The average time in ED per study fortnight for Parallel-1. Note: Vertical orange line indicates approximate fortnight of GP model introduction at this site. (Figure not reproduced here; see the original article.)

FIGURE 16 The average time in ED per study fortnight for Control-3. (Figure not reproduced here; see the original article.)

For Parallel-1 (see Figure 15), we observe an upward trend in the pre-intervention phase, which continues, with a further increase, in the post-intervention phase; there is little obvious evidence of seasonality. This interpretation is consistent with the coefficient values b₀ = 0.139, b₁ = 0.324, D = 0.185 and b_s = −3.02; see Table 11. We note that the de-trended/de-seasonalised data show only moderate short-term correlation; this provides support for the use of linear trendlines to capture the structural temporal variation.

For Control-1 (see Figure 16), we observed a consistent upward trend over the study window, and some evidence of seasonality; this is consistent with the coefficient values b₀ = 0.126 and b_s = −12.90; see Table 11. Here, the de-trended/de-seasonalised data show evidence of moderate short-term correlation; this supports a linear trend as a first step in capturing the structural temporal variation.

Table 11 shows upward overall trends in the three control sites, and with evidence that linear trends capture the structural temporal variation observed in time plots. In the intervention sites overall, there is generally a consistent pattern of upward trends in both pre- and post-intervention phases, balanced by evidence of reversals in upwards trends at two sites (Parallel-2; Outside-3). However, interpretation of some of these results also needs to reflect evidence that linear trendlines sometimes do not fully capture the more complex patterns of structural temporal variation observed at some study sites (Integrated-1; Control-2), although doing so to a reasonable extent at the others.

Average hospital length of stay

Figures 17 and 18 illustrate data on average hospital LOS (in days) for admissions following an ED attendance at two example study sites: Outside-1 and Control-3.

FIGURE 17 The average hospital LOS per study fortnight for Outside-1. Note: Vertical orange line indicates approximate fortnight of GP model introduction at this site. (Figure not reproduced here; see the original article.)

FIGURE 18 The average hospital LOS per study fortnight for Control-3. (Figure not reproduced here; see the original article.)

For Outside-1 (see Figure 17), we observe an upward trend in the pre-intervention phase, followed by a downward trend in the post-intervention phase; there is little obvious evidence of seasonality. This interpretation is consistent with the coefficient values b₀ = 0.002, b₁ = –0.008, D = –0.010 and b_s = –0.004; see Table 12. We note that the de-trended/de-seasonalised data show some moderate short-term correlation; this supports for the use of linear trendlines to capture the structural temporal variation.

For Control-3 (see Figure 18), we observed a consistent downward trend over the study window, and limited evidence of seasonality; this is consistent with the coefficient values b₀ = –0.002 and b_s = –0.171; see Table 12. Here, the de-trended/de-seasonalised data show evidence of weak short-term correlation; this supports a linear trend as a first step in capturing the structural temporal variation.

Table 12 shows a mixed picture in the control sites, with two downward trends and one upwards; there is some evidence that linear trends capture the structural temporal variation observed in time plots. There is also a mixed picture in intervention sites overall and in some subgroups, with some trendlines essentially flat (e.g. Integrated-1; Integrated-2; Outside-3). Change in trend is deemed statistically significant at three sites: Parallel-1, where an initial downward trend is reversed; Outside-1, where an initial upward trend is reversed (see Figure 17); and Parallel-4, where an initial downward trend is followed by a greater downward trend. As Figures 17 and 18 illustrate, this variable also displays a greater degree of random noise than is observed in other variables.

Summary of routine data analysis

Data quality

Data quality across sites was variable, tending to decrease as the level of integration between GP-ED model and the ED decreased. It also varied across the variables studied, with more inconsistency in those variables defined via details coded in routine data. Limitations in the routine data provided effectively precluded the inclusion of one study site (Outside-2) in analyses. We are more confident in interpretations on analyses for some outcomes (attendances, reattendances, admissions, LOS) than others (investigations, treatments, average time in ED) due to data quality issues.

We note that some variables initially under consideration (e.g. time to treatment in the ED; receipt of particular treatments) were excluded from analyses due to further significant issues within routine data. We also note the possibility of extending the list of variables to be investigated (e.g. by refining the reattendance period, or by considering further summary measures for some outcomes).

Summary of findings

The routine data indicated that, in general, at the GP-ED model sites, attendances increased over time, with accelerated increases at some sites in the post-intervention phase; see Table 6. Reattendances within 28 days in the post-intervention period increased over that in the pre-intervention period in 8 (of 9) sites assessed. Overall, intervention sites also demonstrated a consistent trend of increasing average time in the ED across both the pre- and post-intervention phases, with (non-significant) accelerations at 6 (of 8) sites. Data on investigation use at intervention sites demonstrated a mixed picture, difficult to interpret due to the complex patterns of structural temporal variation. This complexity was also observed in data relating to treatments delivered: overall, there was a pattern of increasing treatments delivered pre intervention, followed by decreasing treatments delivered post intervention – the significance of the identified trends is uncertain due to the non-linear nature of trends in data. A mixed picture was seen in relation to hospital admissions in the Integrated and Parallel sites, with data quality issues at the Outside sites, with examples of both increases and decreases both pre and post intervention. Similarly, average length of hospital stay showed a mixed picture both over all intervention sites and within models with some trendlines remaining essentially flat.

Assessment of trend methodology

The adoption of linear trendlines to assess temporal trends and changes in those trends was effective for some variables (including key variables related to ED attendances and hence reattendances) but less so for others, including those which depended on details coded in routine data. Possible reasons for this include changes in recording policies and protocols over time at sites; time plots include examples of non-linear non-stationarity for the two variables summarising medical investigations undertaken and medical treatments received. Findings and limitations for specific variables are summarised in the relevant Tables, and then briefly discussed.

Implications for advanced times series modelling

Prospects for more advanced time series models for data from subgroups of study sites (and, therefore, a single model for data from all study sites) appear to be somewhat limited. The main objection to fitting models which include a single subgroup intervention parameter is the mixed picture in trends and changes in trend observed across sites for most variables. Thus, we would intuitively expect a mixed picture across sites to yield an overall estimate in which (for example) a positive trend at one site largely cancelled out a negative trend at another site, leading to an overall estimate which failed to capture behaviour observed at either site, with a poorer fit than that obtained from two separate analyses.

Cost–consequences analysis

A CCA is an economic evaluation where the costs and outcomes (consequences) are reported without aggregation or weighting, presenting them separately, allowing the reader to form their own opinion and apply their own preferences and respective importance to their decision-making processes. The CCA approach is often recommended for complex interventions with a broad range of effects, which are difficult to measure in a single common unit. A CCA uses a balance sheet to help give a comprehensive but uncomplicated set of information on costs and outcomes for decision-makers.

Method

Costing the HES AE, HES APC, EDDS and PEDW data sets

The study variables were derived from the linked data sets with Healthcare Resource Group (HRG) codes already coded for each individual activity record. Healthcare Resource Group codes group patient events that use similar resource to each other for the purposes of hospital payments; for most events, these HRG codes include staff costs, investigations and treatments bundled into one, with certain procedures (e.g. rehabilitation) and high-cost drugs excluded.

These data were then matched on the HRG codes to the relevant annual NHS Reference costs (e.g. 2017–8). Emergency department data (HES AE and EDDS) were costed using the accident and emergency (A&E) lookup data based on the HRG code and whether the patient was admitted to hospital or discharged; inpatients (HES APC and PEDW) were costed using the EL (Elective Inpatient), NEL (Non-Elective Inpatient), NES (Non-Elective Short Stay) and DC (Day Case) lookup data based on the patient’s admission route (emergency or elective) and their LOS.

Where costs could not be identified, we imputed costs were using average costs for the year of that attendance/admission. Costs were valued in Great British pounds at 2019 prices, inflating previous year’s costs at a rate of 3.5% per annum to reflect the study time horizon of over 12 months. This methodological approach enables the presentation of resource use and the related costs, based on the recorded routine data for individual patients.

Intervention costs

Intervention costs (the costs of the GP-ED models) were obtained from published resources and directly from one of the participating hospitals (also used in Chapter 6).

Perspective

The costs of GP-ED models were considered from a healthcare provider (UK NHS) perspective over a 9-year period between 1 April 2010 and 31 March 2019.

Cost–consequences analysis

Descriptive statistics were reported for ED attendances and inpatient admissions. Using the dates of when each participating site implemented the intervention, patient attendances were classified into pre and post intervention; costs were reported for both periods for each of the model types. As control sites had no intervention, costs were reported over the whole study period. By subtracting mean resource use, per patient, for each of the model types from the control site, incremental differences were calculated and reported as mean cost (£)/time (minutes) saving per episode (ED attendance and inpatient admission).

Results

The results of the CCA are presented below. The healthcare costs (Table 13) are described for each model type within the study, with incremental differences reported in the cost–consequence balance sheets (Tables 14 and 15).

Analysis of healthcare resource use and costs

The mean ED and other secondary healthcare resource use for each model type are shown in Table 13. There were 7,447,626 ED attendances in total across the different sites over the study period. Mean post-intervention ED visit estimated costs for the intervention sites were £163.59 [standard deviation (SD) £73.51] with mean costs associated with inpatient admissions were £2130 (SD £1542) while for the control site mean costs were £162.89 (SD £55.02) and £2202 (SD £1815) for ED visits and inpatient admission, respectively.

Mean post-intervention ED costs ranged from £154.16 (SD £66.09) in the inside–integrated model to £179.19 (SD £76.77) in outside–onsite, while mean inpatient costs ranged from £1987 (SD £2007) in the outside–onsite model to £2177 (SD £1552) in the inside–parallel model.

Negligible incremental ED attendance cost differences were observed between the model types and the control (see Table 14) with an extremely small increased cost (£0.70) observed when looking at all models combined compared with control sites, though a saving of £72 when looking at inpatient admissions. When looking at ED visits by individual models, the inside–integrated model saw a small cost saving (£8.73), but small cost increases were seen in the inside–parallel (£9.51) and outside–onsite model (£16.30). With respect to inpatient admissions, all models saw small cost savings (£25–215) with the outside–onsite model seeing the largest saving (£215). In addition to the negligible cost differences in the ED, there were also very small time savings (range 3.5–23.5 minutes) compared to the control.

Resource use data were costed individually; it is therefore possible for costs to vary depending on what the ED visit or hospital admission entailed, that is, more complicated visits incur a greater cost than a less complicated visit of the same duration; in this situation, it is possible for one model type to have less, or equal, resource use than the other model type, but still have higher costs.

Most of the costs incurred came from hospital admissions. Not only were individual costs higher than ED visit costs, the total costs are greater for inpatient admissions also. The proportion of admissions across the model types varied. Admission costs increase as a patient’s stay in hospital increases, especially when the admission is an emergency admission via ED, and therefore reducing emergency admissions and LOS, where possible, results in cost savings for the hospital. Each of the different GP-ED models saw small savings in LOS (1.1–1.2 days), which is possibly where the cost savings occurred. While each of the GP-ED model types had reduced admission LOS and costs compared to the control, it is an assumption that the case mix for each of the model types was the same (but possible that the type of patients seen and admitted are different, thus resulting in different lengths of stay and costs).

In Chapter 7, we explore specific aspects of the quantitative data with further context provided by known events and initiatives at case sites to refine our theories. The five themes explored there are:

• effectiveness of streaming
• GP role
• effective teamwork and communication
• patient experience
• provider-induced demand.