Workstream 2: a clinical quality of care study

Addressing differences that may impact on neonatal outcomes, independent of unit designation as NICU and LNU.

Introduction

In Chapter 4 we explored whether unit designation, that is NICU compared with LNU, impacted on outcomes for very preterm babies. In this chapter, we explore whether unit designation matters at all. We investigate whether differences in the quality of clinical care provided influence outcome, independent of unit designation as LNU or NICU.

Aim

To explore the relationship between quality of early neonatal care and neonatal outcomes, for babies born at 27–31 weeks gestation, in England.

Objectives

1. to choose the most appropriate measures of quality of care to study, and
2. to explore a relationship between these measures of quality of care and neonatal outcomes, for preterm babies born at 27–31 weeks gestation.

Method

Study design

We conducted a retrospective analysis exploring the relationship between quality of care provided in neonatal units (independent of unit designation) and neonatal outcomes for babies born at 27–31 weeks gestation in England. We explored quality of care by studying the neonatal unit adherence structural and care process measures of quality of care.62 The measures chosen were based on the hierarchy of scientific evidence available and/or nationally considered best practice. Our hypothesis was that units that paid attention to this would have better outcomes.

The study design was selected following an initial unsuccessful attempt to engage NICU and LNU in a questionnaire-based survey on unit practices. This survey was conducted between July and November 2018; only seven of 20 units (10 NICU and 10 LNU) responded. As a result of the poor response, we focused on other measures of quality of care, as described below. Details of the survey questionnaire can be obtained from the chief investigator, on request.

Choosing the measures of quality of care

1. We reviewed
   1. measures within the National Neonatal Audit Programme (NNAP) (prespecified targets or benchmarking exercises set for 2018)96 (see Report Supplementary Material 5) and
   2. evidence-based and national consensus measures in early preterm care, from data captured in the OPTI-PREM neonatal data set (hereafter called ‘OPTI-PREM data set measures’).97–103

We categorised units by assessing how well they adhered to these measures, and the degree of data completion for NNAP measures. We considered units that adhered best, and had the most complete data sets, to be ‘higher performing’, and those that did not, to be ‘lower performing’. We utilised quartiles within the data set to categorise these.

1. We compared two defined outcomes captured in the OPTI-PREM data set, in units that were high or lower performing, to explore a relationship between differences in measures of quality of care between these units, and neonatal outcomes.

Rationale for selection of the measures

Rationale for using NNAP measures: the NNAP96 is a national clinical audit. It is run on behalf of the NHS, by The RCPCH, and is commissioned by the Healthcare Quality Improvement Partnership. It sets audit measures which are either prespecified standards or benchmarking exercises. These are based on published evidence or consensus for best practice in neonatology. A publicly available annual report provides comparison charts for each neonatal unit’s adherence with each measure. Part of the purpose of the NNAP is to identify outliers, which can result in regulatory action from the Care Quality Commission, if adherence to a locally produced action plan is poor. As a result, NNAP audit measures have become standards defining good quality of neonatal health care. Adherence to national guidance in the form of NNAP audit measures reflects organisational culture to improve outcomes and deliver good patient care. This involves fulfilling the requirements of each measure, and sufficient data completion to allow accurate monitoring. Because these measures were to be used as surrogate markers for neonatal unit quality of care, the data did not need to be specific to babies born at 27–31 weeks gestation.

Rationale for using non NNAP measures: these were measures extracted from the OPTI-PREM neonatal data set. We selected variables relating to early preterm care, due to increasing recognition of the importance of these in optimising outcomes for preterm babies.97–103

Data source

1. For NNAP measures we downloaded publicly available NNAP data for 2018 from the RCPCH website: https://nnap.rcpch.ac.uk/annual-reports.aspx.
2. OPTI-PREM data set measures and outcomes were extracted from the OPTI-PREM data set described in workstream 1. We used 2018 data. We limited this to 1 year, to minimise the effect of variation in structure and care process measures through change of leadership, protocols, staff, etc.

Data variables

1. NNAP: We utilised the audit measures that were captured by NNAP in 2018 (see Report Supplementary Material 5).
2. OPTI-PREM data set measures: We selected six variables (measures) relating to early preterm care to study.
   1. receipt of any antenatal steroids
   2. normothermia within an hour of neonatal admission
   3. percentage receiving non-invasive ventilation (NIV), of all babies receiving respiratory support) on day 1 of life
   4. percentage requiring intensive care support on day 1 of life, receiving 1 : 1 nursing support
   5. receipt of mother’s breast milk on day 1 of life
   6. delayed cord clamping at birth.

Data completion for National Neonatal Audit Programme measures

We excluded measures which had no missing data (minimising inappropriate separation of mother and term, and late to moderate preterm babies, nurse staffing). We also excluded measures where the percentage of missing data was < 10% (antenatal steroids, BPD, breastmilk feeding at discharge home, promoting normal temperature on admission for very preterm babies), and for the remaining included measures, we only counted units as having missing data if their percentage of missing data was ≥ 10%. Failure to exclude these measures/units would have increased the volatility of any analysis, where missing data for a very small number of patients (even one) could impact whether units are in the category of having missing data versus not having missing data. Bloodstream infection and central line associated bloodstream infection were excluded because information on degree of missing data by neonatal unit was not available.

This left a final list of NNAP audit measures to be used to categorise units according to data completion:

1. Antenatal magnesium sulphate.
2. Parent consultation within 24 hours of admission.
3. Parental presence at consultant ward rounds.
4. On-time screening for ROP.
5. NEC.
6. Follow-up at 2 years of age.

To define unit adherence through data completion, and since there were no standards for acceptable levels of data completion, we adopted the following approach:

We selected those above the uppermost quartile for missing data versus those below the uppermost quartiles against which to assess outcomes. We excluded measures for those variables where there was mainly 100% data completion as a ‘top quartile’ for data completion was not possible to define here.

Figure 3 outlines the cohort for WS2. A schematic showing demographic parameters chosen for inclusion in workstream 2 analysis is outlined in Appendix 4, Figure 11.

FIGURE 3 Flow diagram outlining the cohort for workstream 2. (Figure not reproduced here; see the original article.)

Inclusion and exclusion criteria

For NNAP data, we selected only units classified as NICU and LNU. We excluded:

1. units with data for < 10 patients
2. units who had requested the NNAP not to make their data publicly available
3. variables related to outcome, when assessing adherence to NNAP audit measures99
4. variables with > 10% levels of missing data, when assessing adherence to NNAP audit measures
5. variables with unknown levels of missing data
6. variables with all units having > 90% data completion when assessing data completion for NNAP audit measures.

For OPTI-PREM data set measures, we excluded:

1. babies not born in a NICU or LNU
2. babies discharged in 2014–7
3. babies with data other than from their first episode of care
4. babies without a recorded gestational age
5. babies born in units with < 10 babies born at 27–31 weeks gestation and outlier units for data completion
6. babies with significant congenital anomalies (see Appendix 1, Table 31) (for multivariate analysis only)
7. variables with > 10% missing data.

Statistical methods

After categorising units by measures of quality of care, and defining quartiles, we compared the demographic neonatal and unit profiles of our comparator groups. This was to identify significant differences that could introduce bias or confounding when conducting analyses on associations with outcomes. To assess whether there were significant differences between the patient populations, chi-squared, Fisher’s exact tests and weighted two sample t-tests were utilised as appropriate.

Multivariate analyses were conducted to explore associations between adherence with OPTI-PREM data set measures or NNAP audit measures, and outcomes of mortality and LOS while in neonatal care. These analyses were conducted for the cohort (27–31 weeks gestation) and not by individual weeks gestation. To adjust for known, measured confounders the following variables were included in the multivariate analysis: gestational age, birthweight, gender, multiplicity of pregnancy, IMD score, resuscitation at birth requiring adrenaline or cardiac massage and place of birth. In the analyses involving the binary outcome variable, mortality, logistic regression was used, and for the continuous outcome variable, LOS, linear regression was used. For these regression analyses, the clustering of babies was not considered in the analysis.

This workstream was supported by a clinical researcher undertaking a PhD with the Department of Health Sciences at the University of Leicester. As a result of the COVID-19 pandemic, and his return to support clinical care, the anticipated analyses on other key outcomes, as undertaken in workstream 1 (ROP, SBI, NEC, BPD and BMF) could not be addressed. For the same reason, multivariate analysis was undertaken for unit adherence, and univariate analysis only, for data completion.

Choosing outcome measures

We chose two outcome measures (both of which had 100% data completion):

• Mortality (pre-discharge)
• Length of stay (LOS)

We had initially planned to also interrogate major morbidity outcomes, but due to the impact of the COVID pandemic, this was not possible within the time frame of the study for this workstream.

Results

The cohort

Workstream 2 cohort of OPTI-PREM comprised 4986 eligible babies.

Description of cohort and selection of measures

Of the 185 neonatal units in England, 65 SCBU were excluded, leaving 119 [44 NICU (37%) and 75 LNU (63%)] units and data for 4986 babies available for study. The distribution of gestational age, birthweight, multiplicity and IMD score mirrored that of workstream 1.

In our workstream 2 cohort an average of 64 babies/unit, born at 27–31 weeks gestation, were born in NICU compared to 30 babies/unit in LNU. This difference was largely due to a higher proportion of babies born at 27 and 28 weeks gestation being born in NICU (70.3% and 59.1% born in NICU vs. 29.7% and 40.9% born in LNU, respectively, p ≤ 0.01). Babies born in NICU weighed 75 g less than those born in LNU (mean birthweight for NICU was 1279 g compared with for LNU, 1354 g; p ≤ 0.01). The IMD score was higher for NICU than LNU (p ≤ 0.01).

Two avenues for assessing adherence to measures were explored, based on unit culture and adherence to evidence-based or consensus-based practice:62

1. unit adherence to NNAP measures [units meeting prespecified targets (yes/no) and units in the upper quartile for benchmarking exercises (yes/no)]. Within this we also explored data completion for NNAP measures selected
2. unit adherence to other early care measures supporting optimal preterm care, with variables extracted from the OPTI-PREM data set [units meeting prespecified targets (yes/no)].

The selection of these measures, and how we determined the upper quartiles, are described below.

Selecting measures of quality of care

Unit adherence to National Neonatal Audit Programme measures

After exclusions, our sample size for adherence to NNAP measures of quality of care was 98 units, and 4594 babies.

We identified the following NNAP measures for analysis:

1. Antenatal steroids (prespecified target 85%).
2. Antenatal magnesium sulphate (benchmarking measure).
3. Promoting normal temperature on admission for very preterm babies (prespecified target 90%).
4. Minimising inappropriate separation of mother and late to moderate preterm baby (benchmarking measure).
5. Minimising inappropriate separation of mother and term baby (benchmarking measure).
6. Parent consultation within 24 hours of admission (prespecified target 100%).
7. On-time screening for ROP (prespecified target 100%).
8. Receipt of BMF at time of discharge from neonatal care (benchmarking measure).
9. Nurse staffing: number of shifts with enough staff to meet ‘total’ nurse specifications required for shift (prespecified target 100%).

We ordered units that met the audit threshold for prespecified targets (yes/no) and benchmarking measures (% of babies that met the measure) from highest to lowest into a hierarchical list. Of the nine measures considered, no unit had adherence to all nine or eight of the nine measures (Table 8). Based on their distribution, above the upper quartile which we defined as ‘higher-performing’ units were those units meeting 4 or more of the above measures (i.e. ‘yes’ for meeting prespecified measure, or above the upper quartile for the benchmark, or both), and < 4 as ‘lower-performing’ units. Twenty-one neonatal units met the criteria for ‘higher performing’ and the remaining 77 were classified as ‘lower performing’ against adherence to overall NNAP audit measures.

Unit data completion for National Neonatal Audit Programme audit measures

For assessment on data completion, 100 neonatal units met the inclusion criteria. There was 100% data completion for the categories of minimising inappropriate separation of mother and term, and late to moderate preterm, babies and nurse staffing. There were < 10% missing data in the categories of antenatal steroids, BPD, breastmilk feeding at discharge home, and promoting normal temperature on admission for very preterm babies. We therefore excluded these measures from the analysis since missing data for a very small number of patients (even one) could impact categorisation of units. We also excluded bloodstream infection and central line associated bloodstream infection as information on degree of missing data by neonatal unit was not available. We identified the list of NNAP audit measures to categorise units according to data completion as:

1. Antenatal magnesium sulphate.
2. Parent consultation within 24 hours of admission.
3. Parental presence at consultant ward rounds.
4. On-time screening for ROP.
5. NEC.
6. Follow-up at 2 years of age.

We created two hierarchical lists, in which the units were ordered by data completion and split into two groups based on upper quartile, or  with any missing data for 2 or more audit measures (n = 17), and higher-performing units those with any missing data for < 2 audit measures (n = 83) (Table 9). Table 10 details the NNAP measures against those utilised in our study to assess quality of care.

Crude summary statistics on categorising units into higher and lower performing based on adherence to NNAP measures, and data completion are described in Tables 11 and 12. There was no difference between the cohorts, except for IMD.

Unit adherence (upper quartiles of performance) to other early care measures from the OPTI-PREM data set

Of the six variables chosen, two were excluded due to a high proportion of missing data. These were delayed cord clamping at birth, and receipt of mother’s milk on day 1 of life. The remaining four variables were used to define quartiles for: any dose of antenatal steroids given;102 normothermia temperature measured within 1 hour of admission to the neonatal unit;101 ratio/percentage of babies given NIV of all those requiring ventilatory support on day 1 of life;100 and ratio/percentage of babies requiring intensive care (IC) provided with 1 : 1 nursing care on day 1 of life.108 We categorised ‘higher-performing’ units as being in the top quartile for adherence for two or more measures, and ‘lower-performing’ units as those with adherence to less than two measures. Thirty-three units met the criteria for ‘higher-performing’, and 80 for ‘lower-performing’ units.

Crude unadjusted summary statistics of categorisation of units into higher- or lower-performing units based on other early care measures from OPTI-PREM are described in Table 13. There was a higher proportion of LNU, and lower deprivation by IMD, in the higher-performing group.

Distribution of overall outcomes for the cohort

Pre-discharge mortality

Overall pre-discharge from neonatal care unadjusted mortality was 3.3%. This ranged from 7.7% for babies born at 27 weeks gestation, to 1.5% in those born at 31 weeks gestation. When separating by gestational week, and designation of unit of birth (NICU vs. LNU), a consistent trend was seen for increased mortality across the gestational age range in NICU (see Table 14). This was true for surgical and non-surgical NICU.

Length of stay

The unadjusted overall length of neonatal unit stay (LOS) was 54 days; this ranged from a mean of 36 days for babies born at 31 weeks gestation, to 84 days for babies born at 27 weeks gestation. When separating LOS for babies by gestational week, and designation of unit of birth, a consistent trend was seen for increased LOS in NICU versus LNU across the gestational age range (Table 15).

Analysis of outcomes by adherence to quality of care measures

Pre-discharge mortality

1. Adherence to NNAP audit measures: on univariate analysis of pre-discharge mortality, comparing higher-performing units versus lower-performing units, we detected a difference (2.2% compared with 3.6%; p = 0.04) for the whole cohort of babies. We could not identify a gestation-specific difference. On multivariate analyses, this difference lost significance (aOR 1.22, 95% CI 0.43 to 1.35) (see Table 16).
2. Data completion for NNAP measures: we did not detect a significant difference either for the whole cohort (4.0% compared with 3.2%; p = 0.21), or at each gestational age in weeks at birth, for data completion of NNAP audit measures (see Table 16) on univariate analysis. Multivariate analysis was not conducted for data completion for NNAP measures due to restrictions of time during the PhD process as a result of the COVID pandemic.
3. Adherence to other early care measures from the OPTI-PREM data set: We did not detect any significant difference on adherence to OPTI-PREM data set measures, between higher-performing and lower-performing neonatal units (aOR 1.22, 95% CI 0.78 to 1.93) (Table 16).

Length of stay

1. Adherence to NNAP audit measures: we detected a significant difference between the two groups, on univariate analysis of LOS and adherence to NNAP audit measures. The number of hospital days in neonatal care was significantly lower in higher-performing neonatal units for the total cohort of babies born at 27–31 weeks (difference in weighted mean LOS 3.7 days, 95% CI 0.6 to 6.8; p = 0.02). On multivariate analysis using linear regression to explore associations between adherence with the NNAP audit measures and LOS, we found a difference of 1 day, that is shorter hospital stay in neonatal care for babies in the group of units that were higher-performing compared to lower-performing units (95% CI 1.029 to 1.081; p < 0.001) (see Table 17). The variables entered into the linear regression model explained 46.2% of the variation in LOS.
2. Data completion for NNAP measures: for data completion for NNAP audit measures, we did not detect differences across the whole cohort, or at each gestational age in weeks (see Table 17) on univariate analysis. Multivariate analysis was not conducted for data completion for NNAP measures due to restrictions of time during the PhD process as a result of the COVID pandemic.
3. Adherence to other early care measures from the OPTI-PREM data set: on analysis of LOS for the composite measures in the OPTI-PREM data set (steroids, temperature, ventilation, nursing), we found a significant difference. LOS was significantly shorter in those units that were higher-performing compared to those lower-performing units (difference in weighted mean LOS 3.1 days, 95% CI 0.4 to 5.8; p = 0.02). On multivariate analysis, the LOS for babies in neonatal units in the higher-performing group was 1 day shorter than for the low-performing group. This was statistically significant (95% CI 1.008 to 1.053; p = 0.007) (Table 17). The variables entered into the linear regression model explained 46.7% of the variation in LOS.

Exploring differences between higher-performing and lower-performing neonatal units

When analysing the cohorts to compare high- and lower-performing units, we noted that there was a significantly higher proportion of LNU in the higher-performing units (p = 0.01) for the analysis using the early care measures from the OPTI-PREM data set.

The increased ratio of LNU : NICU in ‘high’ performing units was only statistically significant for the ratio/percentage of babies requiring intensive care (IC) provided with 1 : 1 nursing care on day 1 of life. For two other measures relating to use of NIV in babies requiring ventilatory support, and normal temperature within 1 hour of admission, this relationship did not reach statistical significance. Overall, for the combination of measures (high performing being defined as in the top quartile for two or more measures out of the four), the ratio of LNU : NICU was increased and statistically significant.

Notably, using the early care measures from the OPTI-PREM data set there was a higher proportion of normothermia in the first hour of admission to neonatal unit, and use of non-invasive respiratory support on day 1 in babies needing respiratory support, where there was no active resuscitation with adrenaline/cardiac massage.

The proportion of babies in each quintile for the IMD_Q, within higher-performing compared with lower-performing units, was also significantly different for adherence to these measures. There was a significant trend towards a less deprived population of patients being in units that were ‘higher performing’ (p < 0.01) (Table 18).

Discussion

We explored quality of care within units, using measures of quality of care that were evidence-based, or considered best practice nationally. We explored the culture of units, through how much they aligned with these measures. We investigated whether these were associated with two clinical outcomes: mortality, and LOS while in neonatal care, for babies born at 27–31 weeks gestation in the OPTI-PREM cohort.

We identified higher-performing units for our defined measures of quality of care, by assessing their adherence to prespecified national audit targets and using a quartile structure for measures that were national benchmarking exercises. We found that higher-performing units had a shorter duration of hospital stay of at least 1 day, when compared to those graded as lower performing by our study definitions. When we looked at this in detail, we found that for the OPTI-PREM data set measures, the proportion of LNU : NICU was higher in higher-performing units, and that this was noted for the combined measures and 1 : 1 nursing alone. The higher-performing LNU were also in areas of less deprivation, by IMD scores.

Findings in context

Overall, our exploratory findings support our hypotheses. Units that are striving to comply with national guidance in the form of NNAP audit measures and practice evidence-based care would be expected to have better outcomes for their babies, and this could result in the small but significant difference in LOS. It could also be that units who work hard to comply with national audit, evidence-based and consensus best practice measures are likely to have differences in the structure and process of care that contribute to shorter LOS, that we were unable to measure. These could include for example, early implementation of BMF, more opportunities for parents to provide skin-to-skin care, discharge home on nasogastric tube feeding, and/or availability of community neonatal nurse outreach support.

For OPTI-PREM data set measures, the differences noted are plausible. LNU are smaller volume than NICU,11 and care for fewer ill babies overall compared to NICU.15 Therefore, we might expect that LNU would prioritise 1 : 1 nurse staffing for babies that require intensive care on day 1 of life. This association might explain why LNU are disproportionately represented in the top quartile for this measure when using OPTI-PREM data set measures. Our work highlights the importance of identifying the most appropriate measures of quality of care. What we infer from our work is that where babies are likely to be more ill, adherence to highest quality of care as judged by our study criteria, was less likely to be achieved.

The observation of a significant trend towards a less deprived population of patients in units in the top quartile is important. This is unlikely to be direct and unifactorial, and more likely to be related to systemic differences in structures and processes of care between units and the wider healthcare system in more affluent compared with deprived areas.109,110

The implications of these preliminary results, if validated, could be substantial. Assuming that the shortened day is a day of special care (with carer present), and that neonatal units are reimbursed ~£535 by NHS England to provide this care, this could represent a substantial cost saving to the NHS in England. It is likely that the effects we are seeing extend beyond the cohort of babies born at 27–31 weeks gestation, to all babies admitted into neonatal care.

Being able to discharge babies 1 day earlier would influence cot capacity and therefore movement within and between neonatal units. Our findings suggest that there is substantial value in promoting best quality of care in neonatology through adherence to evidence-based and consensus best practice measures in England. Enhancing this may optimise not just neonatal outcomes, but overall health service delivery and costs of care for sick and preterm babies in England.

Strengths and limitations

Our study uses a robust method of assessing quality of care and provides a model for further studies evaluating quality of neonatal service delivery using routinely collected operational data through the NNRD. Our results suggest that units that have better adherence with the NNAP audit measures, and/or practice more evidence-based medicine (according to our chosen measures), have a reduced LOS by an average of 1 day, for babies born at 27–31 weeks gestation. These results must be interpreted with caution. The multivariate analyses allowed for adjustment of important confounding variables, but less than half of the variance in outcomes was explained by the models. Several important, recognised confounders were excluded from consideration due to lack of data (e.g. condition of baby at birth, mother’s health status pre- and during pregnancy, ethnicity, etc.), and we have not adjusted for unknown confounding factors through use of an IV approach as in workstream 1. We did not find an association between mortality and quality of care as defined by our study. This may be due to small sample size, as the overall mortality was 3.3%. Therefore, while plausible, this exploratory workstream requires further validation with a larger data set and adjusting for measured and unmeasured confounders for degree of illness of baby.

Summary points from OPTI-PREM clinical quality of care study, workstream 2

What was already known prior to the study

• Quality of neonatal care, in relation to structure, care processes and clinical practices does influence neonatal outcomes.
• Whether this quality of neonatal care is independent of unit designation is unknown.
• There are no data on whether quality of care provided by neonatal units caring for very preterm babies born at 27–31 weeks gestation can influence outcomes, independent of unit designation.

What this study adds

• We developed a novel use of national targets and benchmarking exercises to stratify units according to high-performing compared with lower-performing units for quality of care.
• We found a statistically significant difference in LOS for high-performing units.
• Units in high areas of social deprivation and those with less staff were less likely to meet standards of best practice.
• Our work needs validation through larger studies and accounting for unmeasured confounders.