Synopsis

Background

Hospital-acquired and ventilator-associated pneumonias (HAP/VAP) are the most frequent reason for prescribing antibiotics in the intensive care unit (ICU). Antibiotic treatment for these patients must begin at clinical diagnosis, whereas culture and susceptibility of the causative bacteria take 48–72 hours. This leads to a hazard that patients are undertreated if they have an unusually resistant pathogen, not covered by the empirical antibiotics given. More often, patients with broadly susceptible bacteria are given broad-spectrum antibiotics, exerting selection pressure for resistance for no discernible gain.

Many assert that the key to improvement lies in swifter microbiological investigation, using molecular methods to seek pathogens and resistance genes in respiratory secretions without culture, thereby delivering results in hours rather than days. Thus far, however, there is scant real-world evidence that such methods cost-effectively deliver better stewardship or better clinical outcomes, let alone whether their results – which are not comprehensive – will readily be trusted by either ICU physicians or microbiologists. The INHALE programme aimed to address all these aspects, exploring the utility, performance and acceptability of molecular microbiology for HAP/VAP in practice.

Objectives

The original objectives of the programme are listed below.

1. To define, across four different ICUs, the accuracy of three molecular diagnostic systems claiming to rapidly identify pathogens and their resistance genes in respiratory specimens from patients with HAP and VAP.
2. Based on (1) above, to select the best-performing test and to conduct a randomised controlled trial (RCT), comparing clinical outcomes and antibiotic utilisation in patients whose treatment is guided by the test versus those managed conventionally with empirical treatment refined by microbiological culture.
3. To measure clinicians’ willingness to adopt molecular diagnostics and treat patients based on their results.
4. To determine whether the outcomes justify the cost.

The programme is structured into five work packages (WPs). In WP1, addressing objective 1, we compared the agreement of multiple molecular microbiological tests for HAP/VAP with conventional microbiology and each other. In WP2, we reviewed the current microbiology, antibiotic treatment and outcomes of HAP/VAP in order to inform the design of the RCT and aid selection of the molecular test, addressing objectives 1 and 2. In WP3, we took the best-performing test from WP1 and ran a RCT comparing stewardship and clinical outcomes for (a) patients whose treatment was managed using this diagnostic and (b) those managed conventionally, with empirical therapy adapted once laboratory results became available, addressing objective 2. In WP4, we explored the views of ICU physicians and microbiologists apropos molecular testing in general, and the preferred test in particular, before and during the RCT, addressing objective 3. Lastly, in WP5, we reviewed the cost-effectiveness of the tests employed in WPs 1 and 3, addressing objective 4.

The programme was divided into two phases, with phase I comprising WP1, WP2 and pre-trial components of WP4 and WP5. Phase II encompassed WP3 and during trial components of WP4 and WP5 (Figure 1).

FIGURE 1 INHALE programme structure. NIHR, National Institute for Health and Care Research; PCR, polymerase chain reaction; PPI, patient and public involvement. (Figure not reproduced here; see the original article.)

Changes to the programme

The programme’s core objectives have remained the same throughout it is lifetime. Some changes were made, either to overcome obstacles or to improve the programme’s design. The duration of the programme was also extended, from an initial 5–7.5 years. This was due to effects of the coronavirus disease discovered in 2019 (COVID-19) pandemic and some aspects of the research taking longer than anticipated.

Work package 1

• The original programme planned to conduct a head-to-head comparison of three molecular tests for pneumonia. However, the Novodiag platform was not ready on time.
• To replace Novodiag, we carried out an evaluation of nanopore metagenomic sequencing. This technology had a lot to offer, but its deployment to treat patients in real time would have been premature and hence it was not considered for final progression to the RCT.
• It became clear early in WP1 that recruitment was too slow from the original four planned sites. We therefore expanded the number of recruiting sites to 15. We also allowed inclusion of frozen samples.
• A more refined statistical analysis plan, incorporating Bayesian latent class (BLC) modelling and concordance analysis, was developed to more closely reflect the nature of the data. This allowed the sample size for WP1 to be reduced from the original 868 to between 500 and 700 samples.

Work package 2

• Data collection methodology was changed, and consent was sought from participants to allow more detailed patient-level data to be collected.

Work package 3

• Instruments were placed directly in participating ICUs at the point of care rather than the laboratory. Pre-trial evidence suggested this would lead to faster turnaround-time and better uptake.
• Given lower than expected recruitment in WP1, the number of sites for WP3 was increased.
• Disruption caused by the COVID-19 pandemic forced the trial to stop recruiting in March 2020. Sites were gradually reopened from July 2020 onwards, and formally included eligible COVID-19 patients after that date.
• COVID-19 patients had worse clinical cure rates than other participants and, as a result, it was necessary to increase the trial sample size from 466 to 552 (including 5% attrition).
• While recruitment to the main trial was paused, we launched an observational COVID-19 substudy at five sites during which the Pneumonia Panel was used to diagnose secondary infections and aid patient management.

Work package 4

• The planned pre-trial methodology was replaced with a vignette-based interview method. This allowed more detailed understanding of clinicians’ decision-making in relation to antibiotic prescribing.
• Methodology for task 4.4 was changed from short interviews to short ‘EMA’ questionnaires, supplemented with in-depth interviews, to achieve higher completion rates. During the COVID-19 pandemic, WP4 ran a substudy looking at prescriber attitudes to antimicrobial stewardship (AMS) and rapid diagnostic in COVID-19 patients.

Work package 5

No changes.

Work package 1: clinical laboratory evaluation

Some text in this section has been reproduced with permission from Enne et al.1 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) licence, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

Introduction

Bacteriology culture of sputum, bronchoalveolar lavage (BAL) or endotracheal tube aspirate is considered the gold standard for microbiological diagnosis of pneumonia. Culture is far from an ideal diagnostic method. It is slow, typically requiring at least 48–72 hours to be performed, often longer due to process delays. Meanwhile, patients are treated with empirical broad-spectrum antibiotics. Culture is an insensitive method and fails to identify a pneumonia pathogen in as many as 50% of cases.2,3 Several faster and more sensitive molecular tests have recently been developed to address this issue.

Aims of work package 1

To compare the performance of three different molecular tests for the microbiological diagnosis of HAP and VAP with routine microbiological culture. We prospectively evaluated the BioFire FilmArray Pneumonia Panel, Curetis Unvyero Pneumonia panel and Oxford Nanopore Technologies (ONT) MinION nanopore metagenomic sequencing. Nanopore sequencing was introduced mid-way through the WP, when withdrawal of the originally planned Novodiag test was confirmed. We evaluated analytical performance, ease of use, user acceptability and suitability for deployment into a routine setting.

Methods

Routine lower respiratory samples (sputum, tracheal aspirates, endotracheal tube exudates and BALs) were included if they had sufficient volume (> 400 µl) and were from patients within a participating ICU, hospitalised for ≥ 48 hours and about to receive a new or change of antibiotic to treat a suspected lower respiratory tract infection (LRTI). Samples were eligible if collected within 12 hours of antibiotic initiation. Sample and relevant patient details were collected.

Clinical samples were evaluated within 72 hours of collection using the BioFire FilmArray Pneumonia Panel and the Curetis Unyvero pneumonia panel (see Appendix 1, Table 19), following the manufacturers’ standard operating procedures (SOPs), or frozen within 72 hours and tested later. A proportion of specimens had MinION nanopore metagenomics sequencing performed within 7 days of collection using a bespoke processing protocol developed during a sub-project of this programme.4 The specimens were tested at two sample processing sites: University College London and the University of East Anglia; supplementary reference tests were also undertaken.

A bespoke database was developed in Research Electronic Data Capture (REDCap).

Analyses were carried out using Stata^® (v 15) (StataCorp LP, College Station, TX, USA) and R (v 3.5 or above) (The R Foundation for Statistical Computing, Vienna, Austria), and followed a predefined, detailed statistical plan. Diagnostic performance was estimated for each polymerase chain reaction (PCR) target, taking routine microbiology results as the gold standard. Estimates were also calculated using BLC models5,6 incorporating results from both PCR tests, and routine microbiology.

Results

Recruitment for INHALE WP1 took place between 14 September 2016 and 31 May 2018, with 652 eligible samples collected across 15 participating sites. The majority, 60.1%, were from VAP patients, while 39.9% came from those with HAP. Routine culture generated a result in a median time of 70.2 hours [interquartile range (IQR) 51.1–92.1 hours] and was positive for likely pathogens in 44.2% of samples. However, there was considerable site-to-site variation, with positivity for likely pathogens ranging from 29.5% to 85.7% across sites (Figure 2). This variation was much less pronounced by PCR. This demonstrates lack of standardisation in either (1) clinical diagnosis of HAP/VAP or (2) in the interpretation of respiratory culture across the UK, or both. It also highlights the unreliability of routine microbiology as a gold standard.

FIGURE 2 Sample pathogen positivity (%) by site using PCR vs. culture. (Figure not reproduced here; see the original article.)

Routine culture isolated expected pathogens, with the most common organisms isolated being Staphylococcus aureus, Pseudomonas aeruginosa and Enterobacterales (Figure 3). Rates of antimicrobial resistance (AMR) among these key pathogens were relatively high. Virology results were as expected but only 17.3% of samples had routine virology performed.

FIGURE 3 Frequency of bacterial pathogens isolated from WP1 samples by routine culture. Reproduced with permission from Enne et al.1 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) licence, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original text. (Figure not reproduced here; see the original article.)

There were 631 eligible test results for the Unyvero, and 632 for the FilmArray. The overall positivity rate was 60.4% for Unyvero and 74.2% for FilmArray, considerably higher than for routine microbiology (chi-squared test: p < 0.0001). Most samples had multiple organisms detected by PCR. The principal species detected by PCR and their relative prevalence were similar to routine microbiology (Figure 4). Among viruses detected by the FilmArray, rhinovirus was the most prominent (n = 55), followed by influenza A (n = 29) and B (n = 25).

FIGURE 4 Frequency of pathogens in WP1 samples detected by the PCR tests. Reproduced with permission from Enne et al.1 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) licence, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original text. (Figure not reproduced here; see the original article.)

Polymerase chain reaction assay sensitivity was > 95% for most targets, with negative predictive values (NPVs) > 98% (Table 1). Specificity and positive predictive values (PPVs) were lower, due to PCR tests detecting more organisms per sample and finding more positive samples than routine microbiology. Often, both PCRs found the same organism as each other when routine microbiology failed to record any organism, casting doubt on routine microbiology as a gold standard.3

We performed a range of analyses to accommodate for the observation that routine microbiology culture is an imperfect ‘gold standard’. A concordance analysis was carried out to measure agreement between culture and PCR tests. Most results were either fully or partially concordant. Major discordance, where PCR failed to detect a cultured pathogen, was rare at 4.6% for Unyvero and 1.8% for FilmArray.

To address the problem of culture representing a poor gold standard, performance parameters were also obtained using BLC analyses.5,6 This method accounts for all available results and makes no presumptions about one particular test being the reference. BLC analysis showed routine microbiology was the least sensitive technique, with sensitivity values for individual pathogens ranging from 27.1% to 68.7% (Table 2). In contrast, sensitivity values for the PCR tests remained high; FilmArray sensitivity ranged from 89.4% to 99.3% versus 83.9% to 96.9% (except Klebsiella aerogenes, 48.4%) for Unyvero. Specificity and PPV values for both PCR tests increased substantially compared with the values calculated using routine microbiology as a gold standard.3

Due to the withdrawal of the Novodiag PCR test, resources were diverted to developing and evaluating a different method – nanopore metagenomic sequencing.4 This method can generate a result in approximately 6 hours directly from a respiratory sample. It was used to evaluate a subset of 336 WP1 samples, 238 of which met quality control (QC) parameters. The distribution of organisms detected was similar to culture, with a sample positivity rate of 77%, slightly higher than that obtained by PCR. In terms of performance, overall sensitivity was 87.6% compared to culture, while specificity was 40.2%. We noted significant site-specific variation in sensitivity, ranging from 70% to 100%.

The final component of WP1 was the overall evaluation of the platforms and the choice of a suitable test for progression to WP3. We used criteria based on performance, user-friendliness and cost (see Appendix 2). Because all three molecular tests are much more sensitive than routine culture, it was further decided to add a concordance-based analysis alongside traditional sensitivity and specificity calculations of results compared with classical culture-based testing. Test platforms had to meet two essential criteria to be considered for progression for WP3. These were:

1. Implementability. The machine must be sufficiently developed to allow routine use in the NHS. The hands-on time required to process samples must be < 45 minutes and the machine must be usable by an operator with skill levels expected of a Biomedical Scientist Band 5 or lower. The test must have/or be close to obtaining CE-marking as an in vitro diagnostic (IVD) assay.
2. Concordance. Major discordance, that is failures by the test to find pathogen(s) detected by routine microbiology, must account for < 5% of all tests performed.

Tests that passed the 2 essential criteria where then scored against 10 ‘Desirable Criteria’, scoring a total of 150 (see Appendix 2, Table 20). In addition to diagnostic performance, we also considered factors important for implementation in NHS settings. Some criteria were assessed based on known characteristics of the machines while the ease of use and quality of customer service were assessed using user questionnaires. The study’s health economists calculated the cost-per-test, taking into account equipment purchase cost, delivery cost and staff costs. The scale was weighted towards accurate detection of pathogens, with implementation-based criteria given a lower weighting. A meeting was held to review the data and decide the platform to progress to WP3.

Nanopore metagenomic sequencing was judged to have failed against the essential criterion of implementability. The test has great potential due to the breadth of pathogens it can detect and the wealth of information generated. Nevertheless, the hands-on time and necessary skill required for processing made it unsuitable for an NHS diagnostic laboratory at the time, although improvements to the pipeline have since been made.

Both the FilmArray and Unyvero diagnostic platforms met the two essential criteria and were then scored against predefined and agreed ‘desirable criteria’ as outlined in Appendix 2. FilmArray scored 105 points versus 68 for Unyvero (Table 3). Unyvero was more concordant with routine microbiology, but FilmArray had better sensitivity; Unyvero had a broader target panel but more failed tests. FilmArray performed better on characteristics relating to implementation, ease of use, turnaround time and user experience. Accordingly, we progressed the FilmArray Pneumonia Panel for the INHALE WP3 RCT.1

Discussion and conclusions

Both PCR tests, the Unyvero and the FilmArray, were considerably faster than routine microbiology and detected more organisms, highlighting the known poor sensitivity of routine microbiology in pneumonia.2,3 Notably, PCR tests tended to detect the same additional organisms in a given sample, implying these additional detections were ‘real’. This finding was corroborated by the fact that we were able to culture additional organisms using a comprehensive culture method. A limitation is that, unlike the molecular tests, routine microbiology was decentralised, performed across 11 different hospital laboratories, receiving specimens from the 15 ICUs.

To analyse test performance, we initially took routine microbiology as a gold standard. Only half of PCR results were fully concordant with routine microbiology, with the remaining partial concordances and minor discordances mostly reflecting additional organisms detected by PCR. Per-pathogen sensitivity performance was consistently good. Sensitivity and specificity values are similar to those reported by others for the two PCR tests.7–10

We adopted BLC analysis to overcome the limitations of routine microbiology culture.5 Using BLC, (1) the sensitivity of routine microbiology was estimated to be extremely poor and (2) the specificity and PPV of the PCR tests were considerably higher than those calculated using routine microbiology as the ‘gold’ standard. This suggests that both PCR tests were superior to routine microbiology, and that the latter should perhaps not be considered a gold standard technique. A caveat is that it is perhaps predictable that two similar PCR tests (albeit with different primers and detection methods) should agree better with each other than with a dissimilar culture-based method.1 A potential limitation is that PCR may detect residual nucleic acids from dead organisms, although our comprehensive culture data refute this. It should be noted that all patients in this study were severely ill, clinically diagnosed with respiratory infection and received antibiotic treatment; it therefore seems more reasonable to consider an organism found by any one method as potentially significant. The run times of the machines are measured in hours rather than the days required for culture. Total turnaround time for PCR could not be measured here because the tests were run retrospectively. However, we established that the median transport time of samples from the ICU to the laboratory was 6 hours. If the advantages of speed are to be realised, the machine must be placed in, or near to, the ICU. The decision of whether to adopt a rapid diagnostic into routine clinical practice will depend not only on its performance but also on the practicalities. Here, we evaluated potential for implementation, finding the FilmArray faster, smaller and easier to use then the Unyvero. Accordingly, we have taken the FilmArray Pneumonia Panel forward into INHALE’s RCT described in WP3.

Work package 2: epidemiology of the units

Introduction

Intensive care unit patients are a complex and heterogeneous group of individuals in whom antimicrobial use is high. They are prone to healthcare-associated infections and can succumb to HAP or VAP, defined as pneumonia developing 48 hours after hospitalisation or ventilation, respectively. Clinical scoring systems such as sequential organ failure assessment (SOFA) for adults and paediatric logistic organ dysfunction (PELOD) for children can be used to measure their condition and prognosis. The rate of HAP and VAP among ICU patients is considerable, varying between 5% and 40% of patients, depending on the setting.11 The attributable mortality of VAP in ICU is high, at 13% on average but is considerably higher in certain subgroups such as surgical patients or those with high severity scores on admission.12 Treatment is generally with broad-spectrum antibiotics but with some variation depending on local resistance epidemiology and practises. The additional cost of HAP and VAP infections is considerable and has been estimated at around $40,000 per hospital stay.13

We conducted a detailed survey of HAP and VAP patients which enabled us to have better understanding of local bacterial epidemiology and prescribing practises.

Aims of work package 2

1. Define baseline HAP and VAP rates, and proportions of clinically diagnosed HAP/VAP patients with a pathogen identified by microbiology.
2. Determine and compare pathogen aetiology and resistance prevalence rates in HAP/VAP.
3. Review overall antibiotic resistance rates in each ICU. Review antimicrobial prescribing in the four ICUs.
4. Review clinical cure rates for HAP/VAP in relation to microbiological results (if any) and empirical antibiotics.
5. Therefore:
   1. Inform selection of the molecular test to progress to WP3.
6. Aid design of the step-up/step-down treatment algorithm for WP3. Collect patient-specific clinical data and scores of disease progression and severity; these will be used to assess any need for stratification of patients in WP3 and to formulate secondary outcomes. Key scores for which data will be collected are: SOFA score14 and PELOD-2 score.15
7. Inform the descriptive health-economic analysis (WP5).

The overarching aim of this WP is to gather data for protocol development, outcome definition and the power calculation for the randomised trial performed in WP3.

Methods

Work package 2 took place at BUPA Cromwell, Great Ormond Street Hospital (GOSH), Norfolk and Norwich University Hospital (NNUH) and University College London Hospital (UCLH).

A protocol was prepared to define data collection and consent procedures. Eligible participants were inpatients within one of the four participating ICUs and had to be included in WP1. Additionally, they needed to provide consent for data collection, assent was given by parents or guardians in the case of child participants and relatives or a consultee in the case of those lacking capacity to consent.

Data were collected for up to 21 days after enrolment. Data included reason for admission, type of pneumonia, demographics, prior hospital stays, clinical scores (SOFA and PELOD-2), ventilation need, mortality, cure of pneumonia and antibiotic prescriptions during the study period and 15 days before enrolment. Additionally, a phone call was made to participants at the end of the study to establish their current health, current need for antibiotics and general practitioner use. The EuroQol-5 Dimensions, five-level version (EQ-5D-5L) quality-of-life questionnaire was performed during the phone call.

A bespoke REDCap database was made to aid data collection, which was primarily performed by research nurses from medical records. Core study staff checked data and determined whether antibiotics prescribed were appropriate and proportionate against pathogens found by routine microbiology or PCR (Unyvero or FilmArray). Five stewardship categories were defined for both routine microbiology and PCR results: (1) active and proportionate therapy, (2) active and disproportionate therapy, (3) inactive therapy, (4) negative result, therapy compliant with hospital guidelines and (5) negative result, therapy not compliant with hospital guidelines. These classifications were further aggregated to overall appropriate (active and proportionate and guideline compliant) and inappropriate (active and disproportionate, inactive and guideline non-compliant).

Unit-level data on antimicrobial prescribing and AMR were collected from hospital information systems. Prescribing data were extracted from pharmacy records, whereas microbiology data were obtained from Laboratory Information Management systems. Rates of HAP/VAP were estimated from WP1 screening logs. Denominators to calculate incidence rates were extracted from hospital bed-management records. We also reviewed each unit’s empirical prescribing policies for HAP/HAP. Data were acquired for a period of 15 months from October 2016 to December 2017.

Data were analysed according to a predefined analysis plan using Stata (v 15) and R (v 3.5 or above). Descriptive summary statistics for participants were generated. Relationships between outcomes and appropriateness of treatment (described as two groups: appropriate or not) were investigated using multiple regression models adjusting for gender, age, site, antimicrobial therapy in the 15 days before enrolment and type of pneumonia and Pediatric Index of Mortality/Acute Physiologic Assessment and Chronic Health Evaluation score. Summary descriptions were generated for unit-level data.

Results

Patient-level data

One hundred and forty-two patients were recruited between 12 December 2016 and 29 May 2018. The numbers recruited per site were as follows: Cromwell (7), GOSH (35), NNUH (39) and UCLH (61).

Table 4 shows the baseline demographics of the WP2 population. The majority were male (65.5%); 75.4% were adults and the remainder were children. Adults had a median age of 65.1 years, while children had a median age of 1.0 years. The most common comorbidities were cardiovascular conditions, solid tumours and diabetes. The median length of hospitalisation prior to enrolment was 8.7 days, with a median time in ICU of 5.0 days; 52.1% had VAP and the remainder had HAP. Mean adult SOFA score at randomisation was 6.2, suggesting moderate severity of illness. Most participants had received antibiotics (for any condition) in the 15 days prior to enrolment (Table 5). The antimicrobials prescribed range from those commonly used as first-line agents such as amoxicillin–clavulanate, to those used in surgical prophylaxis (metronidazole) to broad-spectrum agents generally reserved for the seriously ill (piperacillin-tazobactam and meropenem).

Among the 142 patients, 73 (51.4%) received empirical treatment within the ambit of local guidelines, 46 (32.4%) received non-guideline empirical therapy and 23 received non-empirical therapy based on a susceptibility result. Commonly used regimens were piperacillin/tazobactam ± an aminoglycoside, coamoxiclav, ceftazidime or ciprofloxacin + teicoplanin.

Table 6 shows that when judged by either PCR result, more participants were considered to be receiving inappropriate treatment than when judged by routine microbiology. When considering routine microbiology results, 53.5% of therapy was inappropriate, compared to circa 60% for PCR. Furthermore, inactive therapy accounted for 14.1% of cases when judged by culture compared to 25–27% for PCR. This is explained by the finding that PCR tests found a pathogen more often than culture. This WP was observational and hence treating clinicians were not aware of the PCR results.

At day 21, 62.7% of patients were considered cured of pneumonia when using the definition in the WP2 protocol. This increased to 74.6% when the WP3 definition was used. The main difference between the two was that in WP2 any death was considered not cured, whereas in WP3 pneumonia was considered cured in those that died providing pneumonia did not contribute to the death. In the remaining analyses, the WP2 definition was used. Overall mortality at 21 days was 18%. Figure 5 shows mean SOFA score over 21 days. Overall mean scores reduced over the 21-day period for those at NNUH and UCLH from an average of 6 to close to 4. In contrast, at the Cromwell, the mean score remained around 6.5 after 21 days. The association between clinical cure and average daily clinical score was investigated. Mean SOFA score over 21 days was 4.86 for those who were cured, compared to 6.20 for those who were not. This association was significant when examined using a regression model adjusting for confounders [odds ratio (OR) 0.63, 95% confidence interval (CI) 0.45 to 0.85, p = 0.002, N = 99]. However, in an equivalent analysis, a significant association was not identified in the smaller sample of children: mean PELOD-2 6.1 for those who were cured compared to 6.19 in those who were not; OR 0.96, 95% CI 0.64 to 1.47, p = 0.85, N = 24.

FIGURE 5 Mean SOFA score among adult WP2 participants from enrolment to day 21 (n = 101). SE, standard error. (Figure not reproduced here; see the original article.)

Table 7 shows cure rates per stewardship category. Overall, all three diagnostic methods show a slightly higher cure rate when treatment was judged appropriate as opposed to inappropriate. For routine microbiology, cure rates are reasonably similar across all five stewardship categories. However, in the case of PCR, cure rates are noticeable lower than for culture when inactive treatment for a positive result is given, at just 48.6% for FilmArray and 54.1% for Unyvero. This was significant for FilmArray (χ² = 6.02, p < 0.05) but not Unyvero. A logistic regression analysis was used to investigate the relationship between clinical cure and appropriate versus inappropriate categories adjusting for confounders. This demonstrated that there was no statistically significant association between appropriateness of treatment and outcome for all three diagnostic methods (routine microbiology: OR 1.23, 95% CI 0.57 to 2.60; FilmArray: OR 1.51, 95% CI 0.69 to 3.38; Unyvero: OR 1.40, 95% CI 0.63 to 3.21).

Regression analyses were used to investigate the effect of appropriateness of antimicrobial therapy on mortality and mean SOFA or PELOD-2 score over the 21-day study period. No association between these clinical outcomes and appropriateness of therapy was identified by any diagnostic method.

Unit-level data

This WP collected unit-level prescribing and microbiology data from the participating ICUs. Obtaining data proved to be more difficult than anticipated as statistics on antimicrobial prescribing and microbiology results are not routinely collated by the hospitals.

Only two sites, UCLH and GOSH, had provided sufficient data to enable an estimate of the HAP/VAP incidence rate. At GOSH, the rate was 12.6 cases of HAP/VAP per 1000 bed-days, whereas, in the adult population at UCLH, it was almost double, 22.9 cases/1000 bed-days. UCLH ICU collected around 130 lower respiratory samples per quarter, compared to 180 at NNUH, a smaller unit. Cromwell collected 30, in keeping with its small bed numbers and data were unavailable for GOSH. There were discrepancies in the positivity rate of samples, with UCLH having a ~35% positivity rate compared to ~65% at Cromwell and ~80% at GOSH.

Figure 6 shows the distribution of organisms identified from positive respiratory samples. The NNUH and UCLH have the expected epidemiology for HAP/VAP, with sizeable proportions of S. aureus, Enterobacteriaceae and P. aeruginosa, while Acinetobacter baumannii accounted for < 2% of isolates. The Cromwell, with a patient population including a high proportion of previously hospitalised transfers from abroad, had a radically different epidemiology, dominated by P. aeruginosa and Enterobacteriaceae but with a substantial proportion (14%) of A. baumannii, and only tiny numbers of S. aureus. At GOSH, Enterobacteriaceae and S. aureus were common, but P. aeruginosa accounted for a much lower proportion of isolates than at other sites. In contrast, the traditionally community-acquired organisms, Haemophilus influenzae and Moraxella catarrhalis, were frequently isolated from LRTI samples at GOSH. These differences suggest a different aetiology of infections at GOSH, likely due at least in part to their paediatric patient population and a larger number of direct community admissions.

FIGURE 6 Pathogen epidemiology of ICU HAP and VAP at GOSH, NNUH, BUPA Cromwell and UCLH. Note: NNUH data relate to 6 months only, for winter 2016–7. (Figure not reproduced here; see the original article.)

Figure 7 shows unit-level antimicrobial prescribing for all conditions. Prescribing tracked with epidemiological differences: the NNUH substantially used piperacillin/tazobactam as well as penicillins and aminoglycosides, UCLH – now unusually – was a heavy user of cephalosporins, while the Cromwell used carbapenems and colistin heavily, reflecting (1) high rates of beta-lactam resistance in Enterobacteriaceae and (2) the considerable numbers of carbapenem-resistant non-fermenters (P. aeruginosa and A. baumannii). Many of the latter organisms are from long-stay patients receiving prolonged therapy; thus, a small number of patients accounted for much of the unusually large colistin usage. At GOSH (Figure 8), the most commonly used antimicrobials were aminoglycosides, piperacillin/tazobactam and penicillins with comparatively less cephalosporin and carbapenem use.

FIGURE 7 Adult antimicrobial prescribing in defined daily doses (DDDs)/1000 bed-days, ICU patients January–March 17 and October–December 17 (‘Winter’) and April–September 17 (‘Summer’). (Figure not reproduced here; see the original article.)

FIGURE 8 Children’s antimicrobial prescribing in days of therapy (DOTs)/1000 bed-days, ICU patients January–March 17 and October–December 17 (‘Winter’) and April–September 17 (‘Summer’). (Figure not reproduced here; see the original article.)

Discussion and conclusions

Individual patient-level data collected from 142 patients enabled a better understanding of HAP/VAP patients ahead of protocol design for WP3. It was also useful for trialling collection of relatively complex data and enabled teething problems to be resolved. Prior use of antimicrobials in this group was high, and concerningly many had already received broad-spectrum and last-resort antimicrobial therapy.

When the appropriateness of empirical HAP/VAP therapy was compared to pathogen detection, a number of concerning findings were made. Only 46.5% of participants were receiving appropriate empirical therapy when judged against culture and 14% were predicted to have received inactive therapy. These figures worsened when judged against PCR, finding only ~39% receiving appropriate therapy and 25–27% receiving inactive therapy. Furthermore, those receiving inactive treatment by PCR also had lower cure rates. This may be a reflection of the greater sensitivity of PCR methods and suggests additional organisms observed using these tests are clinically significant and should be treated. However, such findings were not significant in a regression analysis, with the small sample size a likely contributory factor.

Unit-level data revealed considerable heterogeneity among different UK ICUs. Antimicrobial prescribing practices differed markedly between the units included. Similarly, there were notable differences in pathogen epidemiology. Differences in positivity rates of respiratory cultures were also noted here and in WP1 and allude to different processing practices at participating laboratories. These observations highlight the need for multicentre studies to ensure this diversity is captured.

Learnings from WP2 were incorporated into WP3 protocols.

Work package 3: randomised controlled clinical trial with internal pilot

Some text in this section has been reproduced with permission from Enne et al.16 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) licence, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

Introduction

It is acknowledged that ICUs are the ‘incubators of antibiotic resistance’ – in that they concentrate those patients in need of antibiotics, who coincidentally also are at greatest risk of developing hospital-acquired infections. HAP and VAP are by far the most common infections in this context. Accordingly, the use of broad-spectrum antibiotics is particularly prevalent, prompting both the emergence of AMR and its spread in this cohort of acutely vulnerable patients. We leveraged learning from WP1 and WP2 to inform the design and execution of a RCT investigating the potential for rapid diagnostics to provide

1. superior antibiotic stewardship; and
2. at least equivalent clinical outcomes.

The trial was ‘pragmatic’ – defined by broad recruitment criteria to reflect the ‘real life’ elements of antimicrobial prescribing across UK ICUs. The FilmArray Pneumonia Panel diagnostic chosen in WP1 was placed at the point of care in participating ICUs to shorten time to result and improve clinical engagement. The diagnostic result was provided alongside an optional prescribing algorithm designed to promote good AMS.

Methods

Abbreviated methodology is described below and is extracted from the published protocol paper by High et al.17

Registration

The trial was registered with the following number: ISRCTN16483855 on 15 July 2019.

Design

A pragmatic, multicentre, open-label, parallel-group, RCT to investigate clinical, safety and cost-effectiveness of the FilmArray Pneumonia Panel test plus trial-based prescribing algorithm versus standard care, with the aim of showing non-inferiority in participant outcomes and superiority in AMS.

Inclusion and exclusion criteria

Patients were eligible if the following criteria were met:

1. About to receive an antimicrobial to treat a suspected lower respiratory infection (LRTI – including suspected HAP/VAP) for the first time, or a change in existing antimicrobial for LRTI because of deteriorating clinical condition. Inpatients in a participating ICU/critical care unit (CCU).
2. Hospitalised for > 48 hours.
3. Sufficient volume of airway specimen obtained for routine testing at site plus 200 μl for the FilmArray test.

Setting

The trial took place in 11 adult and 3 paediatric ICUs/CCUs across England. The group mainly consisted of NHS hospitals but included one private (adult) hospital.

Consent

Consent followed a retrospective process, due to the urgent need to treat ICU pneumonia promptly. Patients or their consultees were approached for consent as soon as possible after randomisation. For children (aged 15 years or under), the parent(s) or guardian were approached to give consent for their child to participate.

Randomisation and blinding

Participants were allocated in a 1 : 1 ratio to receive either standard care or the trial intervention, together with treatment as usual, using REDCap, a web-based randomisation system. Allocation was blocked (using blocks of randomly varying block length) and stratified by site.

Group assignment was concealed from all team members before randomisation, but subsequently the trial was open label to all except Stewardship Committee assessors.

Procedures

Patients in each group had a lower respiratory tract specimen collected before randomisation. For patients in the intervention group, part of the sample was run, as swiftly as possible, using the Pneumonia Panel performed in the ICU by a trained member of the clinical team. The ICU care team were provided with the results and a localised antimicrobial prescribing algorithm (see Appendix 3) translating the test’s results to prescribing advice. For control patients, a portion of sample was frozen at −20 °C or below within 24 hours, and batched frozen samples from control patients were shipped to one of two central laboratories and tested on the Pneumonia Panel, but results were not provided to clinical teams. All samples underwent standard-of-care culture microbiological culture.

Outcomes

Coprimary outcomes

1. Non-inferiority in clinical cure of pneumonia at 14 days post randomisation.

Cure of pneumonia defined as: absence of (1) death where the pneumonia was considered causative or at least contributory, (2) septic shock (except when associated with a documented non-respiratory infection) and (3) relapse of the pneumonia. Relapse is defined as an infectious pulmonary event, associated with clinical and radiological signs of HAP or VAP, or a worsening of two points of the baseline multiple organ dysfunction score (SOFA or PELOD-2) or (4) other evidence that the original pneumonia is not cured.

1. Improvement in AMS at 24 hours post randomisation. Defined as:

Participants on active and proportionate antimicrobial therapy within 24 hours of clinical diagnosis, where active therapy is defined as receiving an antimicrobial active against the organism(s) in vitro and proportionate as defined in the prescribing algorithm specific to that site.

Secondary outcomes

1. ICU/CCU length of stay – time from randomisation to discharge from ICU/CCU
2. Number of ventilator-free days over 21 days post randomisation (VAP participants only surviving 21 days post randomisation)
3. Mortality – death from any cause within 28 days of randomisation
4. Incidence of septic shock – within 21 days of randomisation
5. Change in SOFA (ΔSOFA) score from randomisation to 7 days post randomisation (adults)
6. Change in PELOD-2 (ΔPELOD-2) score from randomisation to 7 days post randomisation (children)
7. Change in paediatric SOFA (pSOFA) (ΔpSOFA) score from randomisation to 7 days post randomisation (children)
8. Proportion of participants on antibiotics active/inactive against the pathogen(s) found at 24 and 72 hours from randomisation
9. Proportion of participants on proportionate/disproportionate antibiotics in relation to pathogen(s) found at 72 hours from randomisation
10. Proportion of participants on narrow-spectrum antimicrobials at 24 and 72 hours from randomisation
11. Proportion of participants with specific adverse events associated with antibiotics within 21 days from randomisation
12. Proportion of participants that contract a secondary pneumonia within 21 days from randomisation
13. Total antibiotic usage in defined daily doses (DDDs) at 21 days post randomisation (all conditions)
14. In patient-stay-related costs.

For each patient, a Stewardship Committee reviewed whether treatment was active and proportionate at 24 and 72 hours post randomisation in the light of all microbiological data from culture and molecular testing. The terms of reference can be found in the appendix (see Appendix 4).

Sample size and statistical analysis

Data for the first 100 patients from WP2 provided estimates for the initial sample size calculation. For the non-inferiority outcome, the clinical cure rate observed in WP2 (using WP3 definition) was 72.4%, and initial power calculation assumed a cure rate of 70%. Since including COVID-19 patients in the trial, cure rate of pneumonia decreased. The sample size calculation was updated to reflect a new assumed cure rate of 55% and defining a non-inferiority limit of 13%, required at least 528 participants to achieve 91% power (chosen to maintain 90% power for the coprimary analyses) with a significance level still at 5%.

From WP2 data we found, under standard care, that 53% of patients received antibiotics that were both appropriate and proportionate at 24 hours. A clinically important improvement rate of 20% was agreed (to 73%). With the sample size of 528 participants required for the non-inferiority outcome, such a difference would be detected with 99% power at a 5% significance level. To allow for 5% attrition, the final sample size was agreed at 552 participants.

Analyses were based on ITT (all participants were analysed according to the group to which they were randomised, irrespective of the treatment received). and followed a predefined statistical analysis plan using Stata version 17 (StataCorp. Stata Statistical Software: Release 17. College Station, TX: StataCorp LLC; 2021). The detailed statistical analysis plan for WP3 is publicly available at https://norwichcrtu.uea.ac.uk/ctudocs_public/inhale/Analysis%20plan%20v1.0%20-%20INHALE%2022-02-2022.pdf

Protocol change: COVID-19

Due to the COVID-19 pandemic, the trial was halted in March 2020 and resumed in July 2020 with a protocol change to include participants known to have COVID-19, provided they met all other criteria. The updated sample size calculations were based on an assumption of around 30% of the final sample size being patients with COVID-19.

During the period the main trial was halted, an observational substudy, allowing use of the Pneumonia Panel in the clinical management of COVID-19 patients with suspected secondary bacterial infections was conducted at five trial sites. Data were collected around pathogens identified and antimicrobial prescribing.

Results

Recruitment

The first patient was randomised on 5 July 2019 and all sites were open by the end of the year. The COVID-19 pandemic forced the trial to halt recruitment on 16 March 2020, with 185 participants having been recruited. Recruitment resumed from July 2020, and was completed on 18 August 2021, some 6 weeks ahead of schedule.

Trial results

We recruited 554 patients, with primary outcomes calculable in 531, which included 92 children. Median age for adults was 61 years, and 7.5 months for children. At randomisation, 33.6% had COVID-19 infection (Table 8 and Figure 9).

FIGURE 9 Study profile. Shaded boxes indicate numbers still being followed up. a Four additional patients (one intervention, three control) were randomised but later found to be ineligible. These four are excluded from numbers in the figure. b These withdrawals represent nine participants who were initially randomised but subsequently withdrew consent to all aspects of the trial including use of any of their data. Data for these patients were removed prior to analysis. c Death does not impede 24-hour stewardship outcome from being determined unless the participant died before 24 hours. d Two further participants were excluded by stewardship committee due to ineligibility, one was clearly moribund at recruitment and the other was a case of neonatal sepsis with no signs of pneumonia. e Death at any time does not impede 14-day clinical cure outcome from being determined. Clinical cure was also available for some patients who were transferred to another hospital (n = 3 intervention, n = 3 control). (Figure not reproduced here; see the original article.)

Time to result post randomisation was 36 times quicker in the intervention arm, at 1.7 hours [standard deviation (SD) 0.8] compared to 110 hours (SD 116.2) in the control arm.

Intention-to-treat analysis for the coprimary superiority (stewardship) outcome at 24 hours after randomisation showed that 205/268 (76.5%) intervention arm participants were receiving active and proportionate antibiotics 24 hours after randomisation, as adjudged by the Stewardship Committee, versus 147/263 (55.9%) in the control arm [estimated difference after accounting for site 21%, 95% CI 13% to 28% (OR 2.57, 95% CI 1.77 to 3.73), p < 0.001].16 Importantly, 8.6% of patients in the intervention group were considered to be on inactive antibiotics at 24 hours, as opposed to 22.4% – a threefold difference (Table 9).

One hundred and fifty-two of 268 (56.7%) intervention arm participants had clinical cure of pneumonia at 14 days, versus 171/265 (64.5%) control patients. The estimated difference, after accounting for site, was −6% with 95% confidence limits of −15% to 2%. These values overlap the non-inferiority margin of 13%, meaning that non-inferiority was not established.16 Adjusting for age, baseline SOFA/pSOFA, COVID status and bloodstream infection in the 7 days preceding randomisation slightly reduced the estimated difference and CI [estimated difference −5% (95% CI −12% to 3%)], with the lower limit now falling just within the non-inferiority region (Table 10).

Analyses of secondary outcomes supported the primary stewardship results, with improvements consistently apparent for the intervention arm. More intervention arm participants had active and proportionate antibiotics at 72 hours and more received active antibiotics (irrespective of proportionality) at 24 and 72 hours, with differences relative to the control arm being statistically significant. Only 16.9% of patients received narrow-spectrum antibiotics at 24 hours, with no significant differences between arms. There were no differences in antibiotic consumption between the two arms as measured by DDD. There was a tendency for control arm participants to receive more broad-spectrum antimicrobials, principally aminoglycosides, carbapenems and piperacillin–tazobactam, whereas intervention arm participants received more narrow-spectrum drugs, although this did not reach significance. Antibiotic-associated diarrhoea was more frequent in the intervention arm, occurring in 26/263 (9.9%) patients, versus 14/257 (5.5%) in the control arm [estimated difference after accounting for site 6% (95% CI 4% to 9%)] (see Tables 9 and 10).

Twenty-eight-day mortality was 31.3% in the intervention arm and 28.2% in the standard-of-care arm. A Kaplan–Meier plot (Figure 10) shows higher risk of death for the intervention group, but this was not significant in a Cox regression analysis. No significant differences between arms were observed for ICU length of stay, ventilator-free days, incidence of septic shock or occurrence of secondary pneumonia. Trends in organ dysfunction over time were measured in both adults and children. We found no significant differences in the degree of reduction of these scores in either adults or children or between trial arms. No severe or unexpected adverse events were reported in either trial arm.

FIGURE 10 Kaplan–Meier plot for mortality. (Figure not reproduced here; see the original article.)

COVID substudy

During the closure of the main RCT during the first wave of the COVID-19 pandemic, a substudy was performed at five participating sites (UCLH, Royal Free Hospital, Watford, Aintree and Chelsea and Westminster) allowing the Pneumonia Panel to be used prospectively on ICU patients suspected of a secondary bacterial infection. Results were collected to help understand the microbiology of secondary respiratory infection in COVID-19 patients, as well as antimicrobial prescribing in COVID-19 patients and utility of the Pneumonia Panel.

A total of 157 tests were performed for 126 patients (31 patients were tested twice, 5 or more days apart) between March and May 2020. Also, 55.0% of tests had a bacterial pathogen identified, Klebsiella pneumoniae the most common organism. Pathogens found were those typically associated with HAP and VAP; community-associated bacteria and viruses were rare. Comparison with FilmArray results from an equivalent pre-COVID population in WP1 revealed Klebsiella spp. were more prevalent (p ≤ 0.008) while S. aureus, P. aeruginosa and Escherichia coli (p ≤ 0.001) were less common. Antibiotic data were available for 121 patients. Sixty-four per cent of patients were on antibiotics 24 hours before FilmArray testing; this figure rising to 74% 24 hours after testing (Table 11). Twenty-two out of 44 patients with positive results continued on the same antibiotic 24 hours after the test, 19 patients were changed following a positive result including 5 changes from inactive to active therapy. Among the remaining 23 patients who were off antibiotics prior to the test, 16 were commenced on antibiotics within 24 hours following a positive result. Antibiotics were stopped in 10/34 patients within 24 hours of a negative test result.

Discussion

INHALE WP3 was a pragmatic trial, including any critically ill adult or child with HAP or VAP for whom the treating clinician had decided to start or change antibiotics. Placing the diagnostic in the ICU and providing a prescribing algorithm, tailoring treatment to the pathogen(s) and AMR gene(s) detected, allowed for evidence-based therapy to be prescribed some 36 times faster than in the standard-of-care arm. The Pneumonia Panel led to a highly significant, 21% absolute improvement in antibiotic stewardship (relative change 37%), defined as the proportion of HAP and VAP patients on active and proportionate therapy 24 hours post randomisation. This advantage persisted at 72 hours. In addition, patients in the intervention arm were three times less likely to be on inactive antibiotic therapy 24 hours post randomisation. Despite these significant improvements, INHALE WP3 failed to confirm non-inferiority of clinical cure at 14 days.

Secondary clinical outcome measures – mortality and evolution of the SOFA score – also favoured the control arm but differences were not significant. Given the borderline results, there is uncertainty whether we have observed a small but meaningful effect in favour of the control arm or perhaps confounding ‘noise’, recognised to commonly affect ICU trials.

Many previous evaluators assert, based upon laboratory results, that deployment of rapid diagnostics might improve antibiotic prescribing, although findings are variable and not definitive. This RCT’s results considerably contribute to this current state of knowledge, in that they demonstrate very clear and significant advantages in time to diagnosis, antibiotic stewardship and reduction in the proportion of patients on initial inactive therapy.

Nonetheless, non-inferiority of clinical cure was not met and exploratory analyses suggested that the patients driving this result were predominantly those from whom a pathogen had been isolated and who had received antibiotics deemed ‘active and proportionate’ (data not shown). We offer several possible explanations.

First, this result remains within the bounds of chance variation.

Second, we have more closely examined our definition of ‘clinical cure’ in pneumonia, which was to the European Medicine Agency standard.19 The ‘cure’ assessment was made at 14 days by the Intensivist who had been responsible for the patient, who by then was unblinded to the trial arm. Assessment of pneumonia ‘Cure’ is acknowledged as subjective.20 Lack of blinding in the assessment of cure was a limitation of the design. We note that this judgement represents a composite outcome influenced by many factors, besides continuing infection. Third, we considered whether our algorithm’s recommendations prompted inferior treatment. This seems unlikely: cure was more frequent in those who received treatment consistent with the algorithm compared to those who did not (data not shown), suggesting that it was not driving failures. Fourth, we noted poorer clinical outcomes in the intervention arm concentrated at particular sites, whose in-depth analysis was not possible for want of statistical power.

Fifth, COVID-19: cure rates were worse for COVID-19 patients, and differences in cure rates between arms were more pronounced. Anecdotally, clinicians reported that COVID-19 complicated the judgement of clinical cure in this cohort.

Last, there is the disturbing possibility that HAP and VAP are not, ab initio, infections caused by the few species sought by culture or multiplex PCR. Rather, the early stages may entail aspiration events, with mixed oral anaerobes, or dysbiosis of a putative lung flora not routinely looked for by either standard or molecular techniques. In short, our results may reflect organisms and/or inflammatory processes, undetected by classical and molecular microbiology are additional important drivers of pneumonia. Metagenomics techniques may provide more insight into this possibility. If this is the case, then brief early broad antimicrobial therapy may be advantageous, just as it is universally accepted against the mixed flora typical of intra-abdominal sepsis. Results here indicated a greater usage of broad-spectrum antibiotics like carbapenems and piperacillin-tazobactam in the control arm.

Use of the Pneumonia Panel in COVID-19 patients revealed a subtly different epidemiology of HAP and VAP compared to non-COVID-19 patients. The Pneumonia Panel influenced antimicrobial prescribing in patients with both positive and negative results and increased the proportion of patients receiving appropriate therapy. The relatively low proportion of appropriate therapy overall reflects the uncertainties and challenges clinicians faced in the first wave of the COVID-19 pandemic.

Work package 4: behavioural study

Understanding antibiotic decision-making processes in ICU settings, where uncertainty is the norm, is key to optimising AMS programmes and the introduction of new rapid molecular microbiology diagnostic tests into clinical practice. Little is understood about clinicians’ perceptions of using these tests. Clinicians’ endorsement and uptake of rapid molecular diagnostics are crucial to maximise engagement with any future integration into routine care. Yet, adoption may be impeded if users harbour unaddressed concerns or if device usage is incompatible with local practice. Therefore, to support optimal implementation, it is vital we understand these better.

Aims

The overarching aim of WP4 was to assess clinicians’ willingness to adopt molecular diagnostics and treat patients based on their results, by exploring ICU clinicians’ attitudes towards molecular diagnostics and antibiotic prescribing both before and after deployment of the ‘Pneumonia Panel’. We conducted four separate but related studies which addressed the following objectives.

Objectives

1. To explore clinicians’ perceptions of antibiotic decision-making and the social and environmental aspects surrounding these perceptions (‘contextual factors’) (Study 1).
2. To explore clinicians’ general perceptions about the use of rapid molecular microbiology diagnostic tests as a prescribing aid (Study 2).
3. To explore clinicians’ perspectives of the use of the ‘Pneumonia Panel’ test in ICU after implementation in the INHALE RCT (WP3) (Study 3).
4. To explore clinicians’ perceptions about antibiotics and the ‘Pneumonia Panel’ at the time of prescribing, and how these and other contextual factors may influence their antibiotic decision-making for patients with suspected HAP/VAP (Study 4).

Study 1: exploring clinicians’ perceptions of antibiotic prescribing for patients with hospital-acquired pneumonia/ventilator-associated pneumonia in intensive care unit

Some text in this section has been reproduced with permission from Pandolfo et al.21 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) licence, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

Objective: Study 1 explored how clinicians’ beliefs, perceptions and other contextual factors influenced ICU clinicians’ antibiotic prescribing for HAP/VAP patients.

Method: Focus groups and interviews were conducted with clinicians in four English ICUs. Focus groups explored contextual factors perceived to influence antibiotic decisions. Interviews investigated antibiotic decision-making processes using two clinical vignettes depicting fictitious pneumonia patients (see Appendix 5). Data were analysed using thematic analysis, applying the Necessity Concerns Framework (NCF). The NCF has been used extensively to understand patients’ medication adherence, positing that decisions to start and continue with treatments are influenced by how individuals judge the necessity for treatment relative to their concerns about the actual or potential adverse consequences of doing so. This is the first study to apply the NCF to prescribing behaviour. All focus groups and interviews were audio-recorded, anonymised and transcribed.

Results: We conducted 4 focus groups and 34 semistructured interviews with 35 clinicians (Table 12).

Analysis of the transcripts identified two key themes: (1) clinicians’ perceptions of antibiotic necessity outweighing concerns, and (2) contextual factors (Figure 11).

FIGURE 11 Factors influencing ICU clinicians antibiotic prescribing in clinical uncertainty. (Figure not reproduced here; see the original article.)

Clinicians’ perceptions of antibiotic necessity outweighing concerns

Clinicians’ antibiotic decisions were influenced by their judgement of the necessity for prescribing/not prescribing, relative to their concerns about potential adverse consequences. Clinicians’ perceptions about the necessity for antibiotics were heavily influenced by their beliefs that antibiotic prescriptions would successfully protect both their patients from deterioration and themselves from the potential consequences of undertreatment (e.g. guilt, litigation). Clinicians also described concerns about prescribing antibiotics. These generally concentrated on AMR; however, the desire to protect the individual patient was often prioritised over these societal concerns. Few clinicians described antibiotic toxicity to be a key influencer. Clinical uncertainty frequently complicated the balancing of antibiotic necessity against concerns. Clinicians’ decisions to initiate or continue antibiotics often signified ‘erring on the side of caution’.21 This approach was reinforced by previous experiences of negative consequences (‘being burnt’) which motivated prescribing ‘just in case’ of infection.

Contextual factors

Data highlighted that prescribing decisions were also context dependent such as prescribing antibiotics out-of-hours, input from external team members, and local prescribing norms.

Discussion: Findings highlight the complex nature of antibiotic prescribing in ICU. Clinicians must juggle two sometimes contradictory imperatives: (1) protect the patient (and prescriber) now, versus (2) safeguard societal future (e.g. against AMR). Clinicians viewed antibiotics’ necessity (i.e. protection for their patients and themselves) as outweighing concerns about antibiotic toxicity and AMR. Findings suggest that stewardship initiatives may be more effective if they address clinicians’ desire to protect with a prescription.

Study 2: exploring clinicians’ general perceptions of rapid multiplex molecular diagnostic technology and its potential role in improving prescribing decisions

Objective: Study 2 was conducted to explore ICU clinicians’ general beliefs about rapid molecular diagnostics.

Methods: These data come from the Study 1 interviews but were analysed separately to explore clinicians’ beliefs about rapid molecular diagnostics. The data met Fine and Kurdek’s criteria for publishing multiple reports from one data set.22 Interview data were analysed using thematic analysis, applying the NCF.

Study 2 has been published by Pandolfo et al 2021.23

Results: Analysis was conducted on 34 semistructured interviews with 35 critical care doctors working in four ICUs in the UK (see Table 12). We identified two key themes reflecting clinicians’ general perceptions of rapid molecular microbiology diagnostic technology and its potential role in improving prescribing decisions: (1) clinicians’ perceived potential of molecular diagnostics to improve antibiotic prescribing (rapid molecular diagnostic necessity) and (2) clinicians’ concerns about how test results could be implemented into practice (rapid molecular diagnostic concerns).

Rapid molecular diagnostic necessity

Clinicians’ perceptions that rapid molecular diagnostic tests are necessary stemmed from beliefs that positive results would facilitate targeted antimicrobial therapy; that negative results would signal the absence of a pathogen, and consequently that having the molecular diagnostic results would bolster prescribing confidence.

Rapid molecular diagnostic concerns

Concerns about the use of rapid molecular diagnostic tests included unfamiliarity with the device’s capabilities, worry that it would detect non-pathogenic bacteria, uncertainty whether it would fail to detect pathogens, and discomfort with withholding antibiotics until receiving test results.

Discussion: Our findings highlight the importance of prescribers’ beliefs as determinants of molecular diagnostic uptake. Although clinicians were broadly open to using molecular diagnostics, they harboured concerns about their capabilities and integration into practice. This research highlights that there are still unanswered questions and uncertainties among clinicians regarding the place of rapid molecular diagnostic technologies in ICU.

Study 3: exploring clinicians’ perceptions of the ‘Pneumonia Panel’ in intensive care unit, within the context of their actual antibiotic prescribing within the INHALE randomised controlled trial (work package 3)

Objective: This study examines actual antibiotic prescribing in ICU settings within the context of the INHALE RCT (WP3). It explores how molecular diagnostics were applied in practice with particular emphasis on the specific beliefs that reinforced clinicians’ perceptions of the necessity to apply molecular diagnostics and those that enhanced concerns about doing so.

Method: Semistructured interviews were conducted with clinicians with experience of using the ‘Pneumonia Panel’ test for at least one INHALE intervention arm patient. Data were analysed using thematic analysis, applying the NCF, to examine factors reinforcing the necessity for, and increasing concerns about, use of the ‘Pneumonia Panel’.

Study 3 has been published by Stewart et al.24

Results: Twenty semistructured interviews with ICU clinicians (16 ICU consultants, 4 consultant clinical microbiologists) were transcribed and analysed. We identified two key themes relevant to clinicians’ perceptions about using the ‘Pneumonia Panel’ within context of their actual prescribing: (1) factors reinforcing the necessity for the ‘Pneumonia Panel’, and (2) factors increasing concerns about using the ‘Pneumonia Panel’.

Factors reinforcing the necessity for the ‘Pneumonia Panel’ test

Many clinicians described that the rapidity of the results of the ‘Pneumonia Panel’ enabled earlier refinement of antimicrobial therapy. Clinicians also described how often the results were used in combination with other evidence (e.g. inflammatory markers in blood tests). Clinicians felt that both positive and negative ‘Pneumonia Panel’ results increased their confidence in prescribing under conditions of clinical uncertainty, particularly when results corroborated the patient’s clinical picture and other results. Many perceived that positive ‘Pneumonia Panel’ results improved antibiotic choice and stewardship. Clinicians also valued negative results; some described de-escalating or stopping antibiotics as a result because bacterial respiratory infection was unlikely or interpreted negative results to signal a non-respiratory infection.

Factors increasing concerns about using the ‘Pneumonia Panel’ test

Clinicians had variable knowledge of the test’s inherent limitations; for example, some were unsure of the panel’s targets and were subsequently concerned about the improper treatment of drug-resistant organisms and atypical infections. Some highlighted doubts about the consistency and quality of the respiratory samples and their impact on ‘Pneumonia Panel’ result reliability. Many clinicians also had worries that false-positive/negative results may encourage antibiotic over- and undertreatment, respectively. There was uncertainty about how results would integrate with existing stewardship initiatives and communications and wanted clarification on the test’s evidence base for clinical usage.

Discussion: Clinicians perceived ‘Pneumonia Panel’ results to have improved their antibiotic decision-making and local AMS. However, much like our findings from Study 2, clinicians also harboured a number of concerns about the application of molecular diagnostics. The prevalence of these concerns suggests that there may be considerable barriers towards the implementation of these tests, and that experience of using molecular diagnostic technology does not necessarily simplify the decision to prescribe broad-spectrum antibiotics. It is essential that these barriers are addressed, possibly through the integration of behavioural support into implementation strategies.

Study 4: investigating clinicians’ ‘in the moment’ views about antibiotics and rapid molecular diagnostic tests, and how these and other contextual factors may relate to clinicians’ prescribing for patients with suspected hospital-acquired/ventilator-associated pneumonia

Objective: Study 4 investigated the specific factors influencing clinicians’ antibiotic prescribing decisions, and whether these varied for patient cases in the INHALE RCT (WP3) intervention arm compared with control arm. This study also explored clinicians’ perceptions of their prescribing adherence to local Trust guidelines and the INHALE trial prescribing algorithm recommendations.

Methods: EMAs were designed to capture ICU clinicians’ decision-making processes for specific antibiotic decisions for INHALE patients. Clinicians making antibiotic decisions in both study arms in the RCT were asked to complete a short 1-minute questionnaire within 24 hours of their prescribing decision (see Appendix 6). One questionnaire was completed per prescribing decision. Questionnaires explored factors influencing their prescribing, and clinicians’ self-reported adherence to (1) local Trust guidelines, (2) ‘Pneumonia Panel’ results and (3) INHALE trial prescribing algorithm (Figure 12). Descriptive statistics were run to present an overview of factors influencing prescribing decisions. We also explored whether these factors significantly differed between test arms using chi-square analyses.

Study 4 has been published by Stewart et al.25

FIGURE 12 Ecological momentary assessment questionnaire administration flow diagram. (Figure not reproduced here; see the original article.)

Results: We received 282 eligible questionnaires (159 intervention arm questionnaires; 123 control arm questionnaires) (Table 13).

Factors influencing antibiotic prescribing decisions

Most factors were only influential for a small proportion of decisions (Figure 13). ‘Trust guidelines/Unit SOPs’ was the most frequently reported factor to influence antibiotic decisions across the total sample (56.7%; N = 160). They influenced significantly more control arm antibiotic decisions (74.0%; N = 91), compared with intervention arm decisions (43.4%; N = 69), χ²(1)= 26.43, p < 0.001. No other variables differed significantly between arms (all p > 0.05).

FIGURE 13 The percentage of prescribing decisions influenced by each factor. (Figure not reproduced here; see the original article.)

Influence of staff

Clinicians often indicated other staff members to influence their prescribing decisions, mostly ICU consultants (63.48%, N = 179). Staff influence did not significantly differ between arms (Figure 14).

FIGURE 14 The percentage of total prescribing decisions (N = 289) influenced by other staff members. (Figure not reproduced here; see the original article.)

Influence of other contextual factors

The relevance of other contextual factors for prescribing decisions varied (Figure 15). Clinicians frequently identified the clinical deterioration of a patient and laboratory or radiological evidence of deterioration to be relevant factors for their antibiotic decisions. Significantly fewer clinicians agreed that the initial antibiotic prescription for respiratory tract infection or other infection was appropriate and did not need to be changed for control arm decisions (22.0%; N = 28), compared with intervention arm decisions (47.5%; N = 76), X²(1) = 18.67, p < 0.001.

FIGURE 15 The percentage of clinicians agreeing with each statement relating to their prescribing decision. RTI, respiratory tract infection. (Figure not reproduced here; see the original article.)

Clinicians’ confidence in, and perceived prescribing adherence to, Trust guidelines

Confidence in local trust prescribing guidelines was low, with clinicians reporting a lack of confidence in guidelines in 25.9% (N = 69) of cases from the whole sample. Only 66.4% (N = 75) of clinicians had confidence in guidelines in control arm cases, compared with 79.74% (N = 122) of intervention arm cases, χ²(1)= 6.05, p = 0.014. Clinicians reported a surprisingly high level of prescribing non-adherence to Trust guidelines. Across the total sample, 24.6% (N = 69) of prescribing decisions were reported to not be consistent with Trust guidelines.

Clinicians’ confidence in, and perceived prescribing adherence to, INHALE algorithm recommendations

Across intervention arm cases, clinicians’ confidence in, and prescribing adherence to, ‘Pneumonia Panel’ results and INHALE trial algorithm recommendations was relatively low. In 88.3% (N = 98) of cases, clinicians considered the INHALE trial prescribing algorithm to be easy to interpret and understand, and in 82.7% (N = 82) of cases, clinicians had confidence in what the algorithm was recommending (see Table 6). However, only 59.6% (N = 62) of clinicians perceived their decisions as adherent to the prescribing algorithm.

Discussion

Hospital guidelines support the choice of empirical antibiotics with an unknown organism; the INHALE trial prescribing algorithm guided choices against an identified organism. Findings reveal frequent distrust of local guidelines and algorithm recommendations, with consequent non-adherence. Distrust was rooted in prescribers’ beliefs about antibiotic prescribing, and the use of ‘Pneumonia Panel’ results for individual patients.

Overarching work package 4 conclusion

Taking together all findings, our data show that the availability of rapid molecular microbiology diagnostic tests does not necessarily lead to implementation in practice as a prescribing decision aid. Many clinicians are reluctant to eschew initial broad-spectrum antibiotics and stop them early, considering their use to be necessary to protect the patient and themselves, ‘erring on the side of caution’. The proximal need to protect was often perceived as more imperative than the more distal threat of antibiotic resistance. The introduction of new technology alone therefore does not deliver. To realise the potential for prescribing technologies such as the Pneumonia Panel to improve AMS, we need to look beyond the technology itself. There is a need for a ‘technology plus’ approach that acknowledges the challenges that clinicians face when applying technological solutions to the care of individual patients. Clinical decisions are strongly influenced by prescribers’ beliefs and ‘clinical instincts’ which are sometimes at odds with the scientific data. In such situations, the data and related guidelines may be less persuasive. There are implications for the design of support tools for clinicians to get the best from technological advances. Further, our work also highlights a need for ‘behaviourally intelligent’ guidelines, which reflect an understanding of human factors such as perceptions and beliefs.

Work package 5: health economics

Introduction

Hospital-acquired pneumonia and VAP have large implications for both health and healthcare resource use.26 For this reason, the INHALE programme included a health-economic component to evaluate the resource implications of molecular diagnostics and to inform healthcare decision-making. Although a separate WP, WP5 involved close collaboration with WPs 1, 2 and 3. Components of WP5 included: estimation of molecular diagnostics cost-per-test; trialling of health-economic data collection alongside WP1 and WP2; and conducting a cost-effective analysis alongside WP3. A further component included health-economic modelling of HAP and VAP. The aim of the health-economic work was to estimate the cost-effectiveness of the molecular diagnostic near patient test to determine whether the test represents an efficient use of healthcare resources.

Methods

Work carried out alongside work packages 1 and 2

Within WP2, the health economists inputted into the design of data collection proformas and databases to include variables relevant to health economics. Collected data included: length of stay in ICU; intensity of ICU stays (number of organs supported); usage of antimicrobials; total length of hospital stays; and tests such as X-rays and imaging. Additionally, links were established with hospitals involved in WP2 to establish acceptable methods for obtaining data on charges for the ICU and hospital stays. The feasibility of obtaining data on health-related quality of life (HRQoL) using the EQ-5D-5L at 21 days was investigated. Preliminary costing work was performed to establish an estimated cost-per-test of the diagnostic tests used in WP. This involved testing staff completing questionnaires to establish details of staff time and other resources required to provide tests. Details of costs for equipment and consumables were derived from the study team and the manufacturers of the tests.

Model development work

To inform a health-economic model of the cost-effectiveness of HAP/VAP, we conducted a review of models utilised in economic evaluation of HAP/VAP interventions. To inform the likely structure of any model, we also attempted to define a clinical pathway for the treatment of HAP/VAP within an ICU (see Appendix 7, Figure 17). This was informed by means of interviews with intensive care physicians. This clinical pathway, alongside lessons learnt from the review of models, was used to inform a simple model structure which was modelled in Microsoft Excel^® (Microsoft Corporation, Redmond, WA, USA) and was informed with data obtained from the review of modelling structures as well as directly from WP3 (see Appendix 8, Figure 18 and Table 23).

A review of previous health-economic models in HAP/VAP was published by Wagner et al.27

Economic evaluation alongside work package 3

The methods used in the economic analysis conducted alongside the clinical trial conducted in WP3 are described in detail in the health-economic analysis plan. The health-economic analysis was conducted from the perspective of NHS secondary care and compared the Pneumonia Panel supported by an algorithm translating its outputs into treatment advice to ‘standard-of-care', two separate economic analyses were conducted using the clinical trial’s coprimary outcomes. First, we calculated the additional cost per additional person on active and proportionate antimicrobial therapy within 24 hours of clinical diagnosis (‘stewardship’). A second analysis looked at additional cost per additional clinical cure of pneumonia at 14 days (‘cure’). In both cases, if one treatment strategy led to lower overall costs and was more effective than the other, we defined this as dominating the other treatment strategy (and, so by definition, cost-effective). Details of resource use included: days in ICU; total hospital stay; use of standard microbiology culture and susceptibility testing; use of Pneumonia Panel; prescribed antimicrobials; use of chest X-rays; and use of computed tomography (CT) scans. Costs for ICU and total hospital stays were obtained from hospital finance departments and represented hospital income. We also estimated ICU costs using NHS reference costs. For the cost of the Pneumonia Panel, we updated costing methodology developed in WP1 and WP2. Details of the methodology used to derive the Pneumonia Panel cost-per-test are given in Appendix 9. For other costs, we obtained relevant costs from NHS reference costs or directly from hospital trusts. Our base-case analysis included costs based on ICU costs and the costs for the Pneumonia Panel (intervention group only). Costs related to the use of other tests and antimicrobials were not included in the base case as they would be included in the ICU costs and hence would represent double counting. Additionally, we present a cost analysis that extends the base case to include all ICU admissions and income from the general hospital admission. This is referred to as total hospital cost. All costs identified were in 2020–1 Great British pounds, apart from early work to estimate cost of test used in WP1 and WP2 which were in 2018–9 Great British pounds.

The economic analysis alongside WP3 has been published by Wagner et al.28

Healthcare-related quality of life

We included an estimation of HRQoL using the EQ-5D-5L instrument taken at 21 days,29 either in hospital or via the 21-day telephone interview. For those in hospital this was limited to those considered well enough to complete the questionnaire. The EQ-5D-5L was valued using a published scoring method.30 We did not intend for this to be used in the economic evaluation as large numbers of individuals in each arm would not be expected to be able to complete this. Instead, we intended this to inform economic modelling.

Results

Work alongside work packages 1 and 2

Initial cost-per-test results constituted one element of the evidence considered in the selection of the diagnostic to be taken forward into WP3. Costs covered either purchase or leasing the machine, costs associated with consumables, quality assurance tests and staff time. Costs were compiled for the year 2018. Cost-per-test was £189 for the BioFire FilmArray Pneumonia Panel and £288 for the Curetis Unyvero. Cost-per-test was based on an annual throughput of 360 tests per year. By far the largest component of costs related to costs of equipment and consumables. Operator time was a negligible part of overall costs, at £2 per test.

Work package 2 showed that it was possible to collect health-economic resource use data as part of the clinical trial and that it was possible to collect data on hospital income for both the ICU stay and the wider inpatient stay. Results showed that ICU costs were high with mean adult costs of £42,136 (2017–8 Great British pounds) per person. Costs were considerably higher for those with VAP compared to HAP (£59k vs. £22k). Antibiotic costs constituted only a small component of total ICU costs, with mean total antibiotic costs for 21 days of £333 per adult. An EQ-5D-5L score was obtained in 43 of 107 adults at 21 days (40%).

Work alongside work package 3: within-trial analysis

Resource use

Estimates of the cost-per-test of the Pneumonia Panel were updated based on information available during WP3 (see Appendix 9, Table 25). The mean usage of diagnostic tests is shown in Table 14. Usage of tests other than the Pneumonia Panel was similar between arms, with no significant differences. A breakdown of all costs examined is given in Table 15. As expected, resources related to the use of tests and antibiotics are a small component of overall costs and costs are driven by stays in the ICU and other hospital departments. The costs of other resource use (barring Pneumonia Panel use) are similar and differences between groups are not statistically significant. For total costs in the base case (from randomisation to ICU discharge), the difference for the intervention groups was −£7802 (95% CI −15,696 to 92). When other inpatient costs are added to ICU costs, the estimated difference between the intervention and control arms increased to −£11,539 (95% CI −25,295 to 2217).

Economic evaluation of stewardship

The results of the economic analysis looking at stewardship are given in Table 16. Costs and proportional difference presented are for the difference between the intervention group and the control. ORs are compared to the control group. Base-case results are presented in the first row of the table, which show inpatient costs limited to ICU costs. Subsequent rows show a variety of sensitivity and subgroup analyses. In the base case, total costs are lower in the intervention arm (−£7373); however, this difference is not significant (95% CI −14,905 to 307). For the base-case analysis, we find that the intervention arm dominates: total costs are lower, and stewardship is improved. The finding of dominance is robust under the different sensitivity analyses conducted. The one exception to this is for the subgroup looking at participants with HAP only. In this case, the intervention group is both more costly and more effective. Cost reduction was highest among children compared to adults, and among those with VAP compared to those with HAP. The uncertainty in three of these analyses is plotted on the cost-effectiveness planes in Figure 16 (left column).

FIGURE 16 Cost-effectiveness plane resulting from the bootstrap resampling for stewardship and clinical cure, for base case, to 14 days, and total costs. (Figure not reproduced here; see the original article.)

Economic evaluation of clinical cure

The results of the economic analysis of clinical cure are given in Table 17. Base-case results are presented in the first row, again with inpatient costs limited to those of the ICU. We find that the intervention has lower total costs but is less effective across all presented analyses. The uncertainty in three of these analyses is plotted on the cost-effectiveness planes in Figure 16 (right column).

Investigating differences in intensive care unit costs

One uncertainty with the data was the degree to which results were being driven by differences in mortality between the study groups. To investigate this, Table 18 shows ICU cost analyses split by mortality at day 28 and cure status at 14 days. If results were driven purely by differences in mortality, we would expect to see differences disappear when we split by this this parameter. However, although Table 18 shows that cost differences are lower in individuals who died by day 28, they persist for both those alive and those who died at this time point (though no differences are statistically significant). Thus, differences in mortality may only partially explain any differences in costs. Similarly, observed differences in costs may have resulted from differences in cure rates at 14 days between groups; Table 18 also shows results divided by those who had and did not have clinical cure at 14 days. If results were driven by differences in clinical cure, we would expect differences to narrow when divided by clinical cure category. In the case of costs, differences persist when subdivided into cure and not cured; further, it appears that the biggest difference in costs is for those individuals who were not cured at 14 days where costs are substantially higher in the control group (though not statistically significant – the 95% CI includes zero).

Economic modelling

The results of a simple economic model using data from the health-economic within-trial analysis are given in Appendix 8 (Figures 18, 19 and Table 24). Data are used from the literature to estimate the cost per QALY using estimates from WP3. Broadly, conclusions are compatible with the ‘within trial’ results showing that the intervention group is associated with lower costs, but as estimates of clinical cure at 28 days are lower the estimated QALYs are also lower. The analysis also illustrates the considerable uncertainty in these results.

EuroQol-5 Dimensions, five-level version scores

EuroQol-5 Dimensions, five-level version scores were obtained for 78 individuals. These are presented in Appendix 10 (Table 26 and Figure 20).

Discussion

Summary of main findings

We found it was feasible to collect resource use and hospital finance data in INHALE. As expected, that the single most important item of resource use was the cost of ICU stays, and differences in this item far exceeded the unit cost of the Pneumonia Panel. For the RCT, we found a reduction in estimated ICU costs in the intervention arm; this difference persisted across a variety of assumptions and subgroup analyses, though differences were generally not statistically significant. For the cost-effectiveness analyses, results differed by type of analysis. For the stewardship outcome, we found that the intervention dominated control, as it offered both lower costs and better stewardship. For the clinical cure outcome, we did not find advantages for the intervention arm in terms of clinical cure and hence could not conclude an economic advantage for the Pneumonia Panel test on this basis.

Strengths of the study

The economic component was conducted alongside well-conducted clinical studies, including a large randomised controlled definitive trial. Hence, we were able to collect good-quality data. Data completeness was also good as each participating Trust had good engagement with the study. Through links with each Trust, we were able to obtain finance data on most participants, meaning that participants could be matched with a reliable estimate of trust income from that participant.

Limitations of study

Given the setting within critical care, it proved infeasible to collect HRQoL beyond a subset of patients; consequently, we could not use this as an outcome measure. Additionally, it was not feasible to follow up participants beyond the 21-day telephone interview, so longer-term outcomes relating to the study were not possible within the trial-based analysis. Long-term modelling would potentially be able to address these issues, and this was explored in Appendix 8. However, in the study, we did not have good data on the long-term outcomes of individuals who had HAP/VAP. As has been used in the literature, we substituted estimated QALYs from a study of sepsis. Therefore, we feel these results are interesting but inherently speculative. Consequently, the headline results we present in our conclusions were those provided by the ‘within-trial’ analysis.

A further limitation of the current study was that for the cost-effectiveness study looking at antibiotic stewardship, it was not possible within the bounds of the current study to quantify the benefits of improved stewardship. If better stewardship was able to reduce future antibiotic resistance, it would confer future benefits. Estimating a monetary value for reduced resistance would enable this to be incorporated into estimates of cost-effectiveness and would likely lead to better decisions regarding the implementation of interventions capable of improving stewardship.

Conclusion

The additional cost of the Pneumonia Panel (£198) represents a very small component of total ICU costs and are typically below 2% of total costs per ICU patient. We found a reduction in total costs with patients in the intervention arm averaging £33,148, and patients in the control arm averaging £40,951. However, it remains unclear how this arises. There was evidence of Pneumonia Panel being cost-effective in terms of stewardship, but not for clinical cure; consequently, we cannot conclude an economic advantage for the Pneumonia Panel. However, this does not take account of the potential benefit of improved stewardship in terms of reduced AMR.

Patient and public involvement

Patient and public involvement (PPI) in INHALE has been an integral part of the programme. The INHALE PPI panel have been actively engaged in all stages, from inception, at a workshop held during the design of the stage one application right up until the final stages, reviewing the outputs and taking part in the Heath Research Authority ‘Make it Public’ workshop, highlighting best practice of PPI in research.

Established in 2016, the PPI panel comprised of six members (seven initially, one resigned during 2021) meeting quarterly and contacted remotely to receive feedback on Programme updates and outputs from the Research team and contribute their ideas and suggestions for change. Three PPI representatives attended the biannual INHALE Research Programme Meetings. Two PPI panel members were part of the Trial Management Group (TMG) for the INHALE WP3 RCT and participated in TMG meetings. One independent PPI member also sat on the Programme Steering Committee (PSC), providing oversight. Meetings were initially face to face, but during the pandemic inclusivity of all panel members was maintained remotely by use of teleconferencing, captioning software, telephone and by mailing hard copy documents.

Being able to work so closely with the same PPI group, receive honest feedback, be held accountable for decisions in the research and have a group playing the critical friend were invaluable in the Programme and it led to a number of changes.

The PPI panel supported the substantial changes made by the Research team, such as the addition of more sites to WP1 and substitution of NovoDiag with ONT MinION. They also advocated for a diverse mix of hospitals to be carried forward into the trial, which the study team has reflected in the 12 sites initially opened for the trial.

A number of significant PPI proposals were implemented, including:

• requesting the addition of postal consent following their review of WP2 ethics approval and recommending a time limit for consent to be implemented, as well as wording changes
• suggesting a study website with a prominent PPI section
• recommending that participants be unblinded to the treatment arm in WP3
• raising concerns with the PSC and core team on the weighting of some scores after reviewing the INHALE diagnostic machine selection criteria.

They also reviewed and commented on all patient-facing information for the Programme including suggesting a participant thank you and newsletter that was disseminated to trial participants. These suggestions had a huge impact, and we were approached to present our PPI work at Health Research Authority Workshop for best practice in public involvement.

The PPI group have been supporting dissemination of results by advising on a trial infographic (see Appendix 11), attending workshops and public science festivals to publicise output from the programme. In addition, an evaluation has been produced by two members of the panel, identifying lessons learnt, based upon a questionnaire circulated to the panel and Research team and using the INVOLVE standards. This will be written up as a blog post.

Overall discussion

The INHALE programme has proven to be a successful piece of research that met the original objectives. At the start, several rapid molecular diagnostics for HAP and VAP were evaluated; both in terms of laboratory performance and within the broader context of treating this highly complex group of patients. The Pneumonia Panel emerged as the most suitable, and was chosen for the RCT, where the test placed at the point of care in ICU was used to guide treatment. Clinician perceptions of molecular diagnostics and antibiotic stewardship were monitored throughout and revealed useful insights into clinician prescribing behaviour. Costs and savings associated with use of the test were calculated.

The INHALE programme revealed important lessons about the implementation of rapid molecular diagnostics, not only for HAP and VAP in the context of critical care but for infectious disease diagnostics in general. One of the big successes of the RCT was placement of the Pneumonia Panel at the point of care within the participating ICUs rather than in the microbiology laboratory. This meant that the time advantage the test offers was maximised as time was not lost due to transport or from batching of samples. Furthermore, clinicians appeared to have better buy-in to the test as they felt more ownership of it. The enthusiasm among clinicians and the speed at which results were obtained no doubt contributed to successful recruitment during the WP3 trial. Following an initial extension due to COVID-19, the trial completed recruitment 2 months ahead of schedule. We believe that to have successfully embedded an infection diagnostic in the ICU setting was truly a ‘World First’ innovation. However, further work is needed for such a change to become commonplace, safe and tractable. Issues of oversight and relationships between test result and clinical action need to be considered before considering widespread implementation.

Use of the Pneumonia Panel clearly influenced antimicrobial prescribing and had a sizeable positive effect on AMS. This was despite clinicians expressing reservations regarding de-escalation of antibiotics for seriously ill patients. This finding corroborates other studies which also successfully used molecular diagnostics to improve antimicrobial prescribing.31,32 INHALE also showed that, despite its cost, the use of the Pneumonia Panel was responsible for a saving in overall ICU costs. This observation, though not yet fully explained, will undoubtedly help promote uptake of the test in healthcare systems.

Equality, diversity and inclusion

The research took place at a number of different hospital types, located in different parts of the country; hence providing broad participation. In WP3, there was a slight over-representation of Asian and Black ethnic groups, compared to the generational population of the UK. The trial also had a pragmatic design, accepting all patients with HAP or VAP, meaning its findings are applicable to a broad population.

Limitations

Despite achieving an improvement in AMS, the trial did not demonstrate equivalence of clinical cure. This result was surprising and notably key secondary outcomes such as SOFA and mortality also skewed to favour standard care. However, findings were not statistically significant, and it remains to be determined whether improved stewardship brought about a small but real negative effector whether we simply observed noise caused by participant’s various comorbidities, in ICU trials. Another significant limitation was the fact that the relatively low prevalence of acquired AMR in the UK meant that the test’s ability to quickly detect AMR genes could not be fully tested. The third limitation was caused by the COVID-19 pandemic, substantially changing the types of patients that were recruited. Indeed, some of the effects surrounding clinical cure were most pronounced in this population but the reasons why are not clear. Finally, in terms of cost findings, a limitation is the comparatively short follow-up meaning that cost differences that may have occurred over a longer follow-up may not be valued. Furthermore, improved stewardship may have long-term cost benefits in terms of reduced antibiotic resistance and associated costs. However, quantification of this is likely to be highly complex and unpredictable and was considered beyond the scope of the current work. One may imagine a scenario where improved AMS is associated with higher short-term costs but result in savings that are separated from the intervention, either temporally, geographically or both.

Conclusions

Overall, INHALE clearly demonstrates that molecular diagnostics for severe pneumonia are considerably faster and more sensitive than traditional culture-based methods. Use of such tests can influence antimicrobial prescribing in the ICU. However, the effect on clinical outcomes of de-escalating antimicrobial therapy as guided by the test is not clear. Doctors are generally positively disposed to such tests, but our behavioural work highlights that the availability of such tests does not necessarily lead to implementation in practice as a prescribing decision aid, with many clinicians hesitant to de-escalate antibiotics early based on test results. The optimal integration of such technology into practice will require a ‘technology plus’ approach that acknowledges the challenges that clinicians face when applying technological solutions to the care of individual patients. Use of molecular diagnostics can lead to cost savings, but the drivers behind these are not yet understood. Crucially, further work is needed to translate the results of molecular diagnostics to optimal therapy and improved clinical outcomes.

Future work

The central premise of any future must be to establish certainty on whether molecular test-driven improvement in antibiotic stewardship leads to altered clinical outcomes. If this is conclusively shown to be the case, then downstream work must explore the reasons why and develop prevention strategies. We have already secured funding through the NIHR’s Programme Development Grant scheme to perform further analysis of existing WP3 participants to understand whether worsening clinical cure was real and what the drivers were. We will collect more information about the participants and recalculate clinical cure using a newly developed more objective definition. We will explore relationships between the exact antibiotics prescribed and clinical outcomes. Current analysis looked at suitability of antibiotics at specific time points, but we will expand this to the whole duration of treatment. Should these post hoc analyses fail to yield conclusive results, then the next stage would be to carry out a second clinical trial, specifically designed and powered to look at differences in clinical outcomes between study arms, with a more rigorous objectively define primary outcome.

If additional work demonstrates that improved stewardship leads to worse outcomes, then one would assume the likely explanation for this would be the presence of additional organisms, not detected by culture or PCR, that nevertheless have a role in driving progression of pneumonia. Dysbiosis of the lung microbiome may also play a role. Therefore, diagnostic techniques such as metagenomics4,33 that provide a complete overview of the microbial composition of the lung may be helpful. Exploring the utility of these in management of pneumonia would be beneficial.

Our behavioural work highlights that the proximal need to protect the patient is often perceived as more imperative than the more distal threat of antibiotic resistance. Consequently, it would be useful for future work to focus on design and development of behavioural interventions/support materials to help clinicians make complex antibiotic prescribing decisions and implement such technology into practice. This could involve increasing concerns about the toxicity of broad-spectrum antibiotics.

In terms of health economics, useful future work would involve recalculating estimates of cost-effectiveness using the newly developed definition of cure which will form part of future work described above. Work to better characterise factors driving apparent reduction in costs in the intervention arm would also be beneficial. We also intend to develop the simple model presented here to include stewardship. Long-term work should focus on calculating how interventions such as this may impact the cost of AMR.

Implications

Until additional work exploring the impact the intervention on cure has been completed, the implications of this research are not fully clear. At present, we note that molecular diagnostics are faster and more sensitive than routine culture and can be successfully deployed at the point of care. Should our future work prove that there is no detriment to clinical outcomes resulting from diagnostic-guided prescribing, then we would wholeheartedly recommend the widespread introduction of these tests to the NHS. However, if that is not the case and detrimental effects observed from improved stewardship are demonstrated to be real, then this finding could have very wide-reaching implications for antimicrobial therapy and microbiological diagnosis of pneumonia, indicating a significant overhaul of current therapeutic practices may be required.