Methods

Parts of this chapter have been reproduced with permission from Kumar et al.1 and Taylor et al.48 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) licence, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

Study design

Magnetic resonance Enterography or ulTRasound In Crohn’s disease Extended Follow-up for predicting disabling disease was a multivariable prediction model, multicentre diagnostic inception cohort of adult patients with newly diagnosed CD. We enrolled patients who had already been recruited to the Magnetic resonance Enterography or ulTRasound In Crohn’s disease (METRIC) trial and extended their follow-up.35,49,50 To achieve an adequate sample size, we also included a separate retrospective cohort.1 We reported this trial according to TRIPOD reporting guidelines for prediction model development.51

Study population

Magnetic resonance Enterography or Ultrasound In Crohn’s Disease was a multicentre, prospective trial performed in nine NHS centres across England and Scotland designed to compare the diagnostic accuracy of MRE and ultrasound (US) for the location and extent of CD.35,50 Consenting adult patients presenting with either newly diagnosed CD or presenting with suspected relapse were recruited: all underwent both MRE and US. Patients were followed up for a minimum of 6 months. In METRIC-EF, we drew solely on the group of patients who were recruited into METRIC with a new diagnosis of CD (i.e. the ‘relapse cohort’ was excluded). We extended follow-up for the new diagnosis cohort to a minimum of 4 years.35 To achieve an adequate sample size, we supplemented newly diagnosed patients from METRIC with a carefully matched retrospectively identified group of patients also newly diagnosed with CD. A total of 147 (76%) patients were from the retrospective group. With ethical approval, additional patients recruited to the METRIC trial could be included via the retrospective arm if they did not respond to two direct invitations to take part. Because data from the retrospective cohort was pseudonymised to the central trial team, we were unable to record how many patients originally recruited to the METRIC trial were included in the retrospective cohort. The median number of years from recruitment (diagnosis) until the last outpatient visit was 6.3 [interquartile range (IQR) 5.2–7.3] years. The date of recruitment (time of diagnosis) ranged from 10 June 2009 to 5 December 2017. The date of the last outpatient visit ranged from 28 February 2016 to 20 October 2022.

Eligibility criteria

The study focused on newly diagnosed CD patients either (a) enrolled in METRIC (‘METRIC cohort’) or (b) imaged using MRE as part of their routine care at diagnosis (‘retrospective cohort’) (Figure 1).

FIGURE 1 Flow diagram outlining the stages of the METRIC-EF trial. BSG, British Society of Gastroenterology; BSGAR, British Society of Gastrointestinal and Abdominal Radiology. This figure has been reproduced with permission from Kumar et al.1 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) licence, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original text. (Figure not reproduced here; see the original article.)

Magnetic resonance Enterography or ulTRasound In Crohn’s disease cohort: inclusion criteria

All confirmed new diagnoses from METRIC were eligible for the present study; inclusion criteria were therefore equivalent to those of METRIC:

• Patients aged 16 years or more.
• New CD diagnoses (within 3 months of time of recruitment to METRIC), based on standard endoscopic, histological, clinical and radiological findings.

Additional retrospective cohort: Inclusion criteria

We added a retrospective cohort to the METRIC accruals to achieve the required sample size.

Inclusion criteria for the retrospective cohort were:

• Patients 16 year or more with newly diagnosed CD, based on endoscopic, histological, clinical and radiological findings
• MRE acquired according to METRIC standard minimum sequence data set, and performed either < 3 months before or after diagnosis
• Normal institutional practice is to perform MRE in all new diagnoses of CD.
• At least 4 years’ clinical follow-up data available

Sites that were not part of the original METRIC trial were eligible to be recruitment sites for the retrospective cohort if they fulfilled all eligibility criteria.

Exclusion criteria

Exclusion criteria for METRIC (and so carried forward) were:

• Any psychiatric or other disorder likely to impact on informed consent
• Evidence of severe (non-Crohn’s) comorbidities which makes it undesirable for the patient to participate in the study
• Pregnancy
• Contraindication to magnetic resonance imaging (MRI) (e.g. cardiac pacemaker, severe claustrophobia, inability to lie flat)
• Final diagnosis other than CD
• Enrolled in the METRIC study but not part of the final new diagnosis cohort

Ethics

Consent

The new diagnosis cohort patients recruited to METRIC were approached and consented (if willing) for participation in METRIC-EF at seven of the participating sites in the METRIC study. Patients declining participation were excluded. Two additional sites that were not originally METRIC recruitment sites were added for identification of patients to the retrospective cohort. We were granted permission to collate data from the retrospective cohort without direct patient consent (including patients recruited to the METRIC trial who did not respond to re-contact) as there was no direct patient intervention, and only pseudonymised data were collected by the Comprehensive Clinical Trials Unit (CCTU) at University College London (UCL).

Ethical permission

The METRIC-EF study achieved NHS Research Ethics Committee (NHS REC), London – Hampstead Research Ethics Committee approval on 26 October 2018 (IRAS 217422) and was conducted in accordance with the principles of good clinical practice. UCL’s CCTU supervised the study.

Magnetic resonance imaging

Sequences

Magnetic resonance Enterography or UlTrasound In Crohn’s disease patients underwent a standard minimum MRE sequence data set (1.5 T or 3 T) (Table 1). This was required for retrospective accruals, with the exception of diffusion-weighted imaging (DWI), which was not needed to calculate relevant activity and bowel damage indices.

We calculated established MRE activity indices as follows.

Magnetic resonance Enterography Global Score

This score encompasses aspects of both inflammatory activity and bowel damage and has been tested against several reference standards, including a composite clinical reference,52 FC,53,54 CRP54 and endoscopy (Table 2).54,55

Simplified Magnetic Resonance Index of Activity

The Simplified Magnetic Resonance Index of Activity (sMARIA) has been tested against endoscopic reference standards and is used increasingly to assess treatment response in clinical trials (Table 3).36–38,56,57

Lémann Index

The Lémann Index (LI) is based on comprehensive assessment of structural bowel damage, including stricturing, penetrating lesions (fistulae and abscesses), and surgical resection, and is applicable to different settings, such as early or advanced disease, patients with or without surgery, or with different CD locations and extension (Table 4).58 The score comprises several factors that can be assessed either clinically, or using imaging, or via endoscopy. We used the imaging-derived score. Since the anal canal was not imaged specifically for METRIC, we omitted this score.

Interpretation and blinding

Magnetic resonance enterography examinations were sent by each recruitment site to a central facility at University College London Hospitals NHS Foundation Trust (UCLH) through pseudonymised compact disc or digital versatile discs (CD/DVDs). Here, they were uploaded to an online viewing platform (3Dnet^®; Biotronics3D Ltd, London, UK) which offered the same functionality of a routinely used picture archiving and communication system (PACS) with the benefit of being accessible from any personal computer over the internet with appropriate password credentials.

Magnetic resonance enterography scans were interpreted by one from a pool of radiologists; all were GI radiologists and experienced in MRE, in both clinical and research settings. They all had achieved Fellow of the Royal College of Radiologists and had at least 1 year of subspecialty GI imaging experience. Eleven radiologists interpreted the MRE examinations from nine centres (range 6–36 reads per radiologist).

Radiologists were allocated MRE scans for scoring by the CTU; readers did not interpret cases from their own hospital and were blinded to all clinical information other than that relevant for the calculation of the relevant index (e.g. surgical history for LI). We used Research Electronic Data Capture to record the radiological parameters.

Assessment of disabling disease at follow-up

Time point of follow-up

Follow-up was for a minimum of 4 years: since participants were recruited to METRIC over 30-months, this corresponded to an average follow-up of approximately 5.5 years. This was felt to be sufficient time for clinically relevant complications of CD to manifest.24,59,60

Collection of data were completed by the research team at each recruitment site using study case report forms (CRFs) and retrospective data available in the patient hospital record.

Consensus panel assessment of disease outcome

Consensus panels were convened at each recruitment site. Panels comprised, as a minimum, one gastroenterologist and one radiologist, aided by the site research nurse. The consensus panels reviewed all available clinical information over the complete follow-up period. This included investigations such as CRP, FC, endoscopy (conventional and capsule), imaging [MRI, US, computed tomography (CT), fluoroscopy], surgical and histopathological findings, clinical activity scores (e.g. Harvey-Bradshaw Index) and overall clinical course, including outpatient and inpatient clinical records.

Using all the available data, the consensus panels recorded the presence or absence of disabling disease (see definitions below), as well as the Montreal classification and the date at which this end point was reached. Participant data for the predefined clinical predictors for model inclusion (see below) were also collated when available.

Primary definition of disabling disease

The primary definition of disabling disease was a modified version of that described by Beaugerie et al.25 The original definition was modified to clarify some of the symptoms and to permit use of disease-modifying therapy, since this has become common clinical practice. We also did not include events occurring within 90 days of diagnosis, to exclude events occurring within the initial diagnostic period. Disabling disease was therefore defined as any of any event reported at more than 90 days after diagnosis:

• Hospitalisation after CD diagnosis for flare or disease complication, as judged by the treating clinician.
• More than two corticosteroid courses required over 5 years and/or dependence on corticosteroids.
• Any intestinal resection > 50 cm, or surgical operation for perianal disease (examination under anaesthesia without seton placement did not meet this criterion; abscess drainage and/or seton placement did).
• Chronic disabling symptoms, defined as a cumulative time of over 12 months of one or more of:
  • Diarrhoea with nocturnal stool (getting up for a bowel movement after having gone to bed).
  • Urgency (defined as having to rush to the toilet for a bowel movement).
• Abdominal pain due to intestinal obstruction (requires imaging confirmation or surgical proof).
• Fever (documented tympanic temperature of > 38.0 °C or oral temperature of > 38.3 °C).
• Fatigue.
• Joint pain not due to an alternative cause.
• Uveitis.
• Pyoderma gangrenosum.

Alternative definitions of disabling disease

Since the Beaugerie criteria are imperfect, further definitions of adverse outcomes were also collected; specifically the Liège criteria,24 and Montreal behaviour criteria.61

The Liège criteria were met if any of the following occurred:

• development of complex perianal disease
• any colonic resection
• two or more SB resections
• a single SB resection of > 50 cm
• definitive stoma.

Complex perianal disease was defined as per the American Gastroenterological Association.62

The Montreal behaviour criteria classify CD as either inflammatory (B1), stricturing (B2) or penetrating (B3). Stricturing disease was defined as a fixed luminal narrowing of > 50% relative to normal proximal bowel. Penetrating disease was defined as an intra-abdominal or enterocutaneous fistula, inflammatory mass, or abscess.

Outcomes

Primary outcome

Comparative predictive ability of prognostic models incorporating MRE scores (MEGS, sMARIA and LI) in combination with clinical predictors versus a model using standard clinical characteristics alone to predict disabling CD [modified Beaugerie disabling disease (MBDD)] within 5 years of diagnosis.

Secondary outcomes

1. Comparative predictive ability of prognostic models incorporating MRE scores (MEGS, sMARIA and LI) in combination with clinical predictors versus models using standard characteristics alone to predict the development of disabling CD within 5 years of diagnosis, defined by Montreal behaviour and Liège criteria.
2. Identification of the best combination of individual MRE features for the prediction of MBDD within 5 years of diagnosis.
3. Evaluation of the average costs per-participant for the NHS, incurred within 5 years of CD diagnosis.
4. Investigation of the participant, disease phenotype, and imaging characteristics associated with higher costs within 5 years of CD diagnosis.

Sample size and justification

Assumptions

We assumed that the prevalence of MBDD was approximately 55–60%; this was informed primarily by the external validation cohort of the Beaugerie descriptors, in which 57% of 361 participants had developed disabling disease within 5 years of diagnosis.24 In support, a local audit of 33 newly diagnosed patients at 1 METRIC recruitment centre at the trial planning stage found 5 of 33 (15%) patients met the definition by mean 11.3 months, giving 16% at 1 year. Extrapolation to 5 years gave 58% prevalence, similar to that expected from the literature.24

The sample size was based on, including 207 participants newly diagnosed with CD; 207 participants provided 114–124 patients developed MBDD; the smaller proportion defines the minimum sample size for powering a modelling study. During the study, due to problems obtaining consent for additional follow-up due to the COVID-19 pandemic, the Trial Management Group (TMG) reduced the original target recruitment from 167 to 131 in the prospective METRIC cohort, with a corresponding increased target of 76 participants from the retrospective cohort. We anticipated that this sample size would provide between 114 and 124 patients developing MBDD. We would increase the number of the retrospective cohort to meet the 207-participant target if recruitment to the METRIC cohort was below 131.

Adequacy of this number of events/non-events

Calculating sample sizes for prognostic studies suffers from a relative lack of readily applied methods suitable for all study designs, since sample size for development depends on whether the primary aim is to select potential variables for a new model (via univariable significance within a data set) or to evaluate a model where the variables have been prespecified and are therefore fixed. In the present study, we fixed predictors since we were explicit that we would evaluate three MRE severity scores in the context of a model using fixed clinical predictors. Therefore, recommendations for sample sizes relevant to external validation were most appropriate. Accordingly, the literature suggested that we required 80–100 events for model evaluation where predictors were prespecified and fixed.63 This also provided sufficient power to assess whether the addition of the 3 MRE severity scores enhanced prediction, under the hitherto widely used ‘rule of thumb’ of 10–20 events per predictor.64 We are aware of recent methods to calculate model development and external validation sample size, but these were not reported in 2017, when the present study was powered.65

Power for secondary outcomes

Other definitions of adverse outcome

Development of Montreal severe disease was estimated to be 43% at 5 years.66

Development of Liège disabling disease was estimated to be 20% at 5 years.24 This provided approximately 41 events for the present study which was likely insufficient to develop meaningful prognostic models. Accordingly, we planned that analysis for this end point would be descriptive only, unless our assumptions proved incorrect and sufficient events satisfying this definition had been accumulated.

Identification of the most important magnetic resonance enterography variables for model inclusion

We used principal component analysis (PCA) to reduce the number of individual MRE features to ideally two or three eigenvector variables for subsequent addition to the clinical predictor only model. This allowed us to determine how adding MRE features affected model performance.

Retention

Participants did not undergo additional testing to enter this study. Only data obtained during routine clinical care were necessary to both define disabling disease and provide variables for model inclusion. Where participants were lost to local follow-up, participants’ general practitioner were contacted in an attempt to obtain routine clinical information, post consent (this was only applicable to the METRIC cohort and those patients on retrospective cohort who had provided consent).

Statistical methods: outcomes

Primary outcome

Comparative predictive ability of prognostic models incorporating MRE scores (MEGS, sMARIA and LI) versus a model based on clinical predictors alone for the development of MBDD within 5 years of diagnosis.

We developed a Royston–Parmar (RP) flexible parametric multivariable prognostic model using the following prespecified clinical predictors (based on a prior literature review and in consultation with the trial investigators):

• age at diagnosis (< 40, ≥ 40 years)
• smoking history
• sex
• disease status at diagnosis (stricturing disease, perianal disease, severe endoscopic disease)
• location of disease (ileal, colonic, ileocolonic, upper GI tract disease)
• initial need for steroid therapy
• weight loss of at least 5 kg prior to diagnosis
• CRP
• white blood cell (WBC) count
• FC
• haemoglobin
• platelet count.

There were five prespecified continuous predictors, including CRP level, WBC count, FC level, haemoglobin level and platelet count. We determined whether we should include the predictors as linear or to use fractional polynomials. Due to high levels of missing WBC count and FC levels, we could not investigate if fractional polynomial was appropriate, so assumed a linear relationship. For the remaining predictors, we calculated the best fractional polynomial models by searching through all power combinations. Then, we calculated p-values by comparing the deviance of the linear and fractional polynomial model 1 against the deviance of fractional polynomial model 2 (lowest deviance). We determined that retaining linear continuous predictors was the most efficient.

We retained categorical predictors as in the clinical report form, except for when modelling required us to combine specific levels of a predictor. For location of disease behaviour, we combined the ileal and upper tract levels due to small number of patients (N = 4) with disease in the upper tract.

Seventy-five per cent of participants had ≥ 1 predictor value missing. Because missing values were likely to be ‘missing at random’ based on other participant variables and to avoid loss in efficiency, we imputed values for smoking status, weight loss ≥ 5 kg prior to diagnosis, perianal disease, severe endoscopic disease, CRP level, WBC count, FC, haemoglobin level and platelet count using multiple imputation by chained equations [mi impute command in Stata 18 (StataCorp LP, College Station, TX, USA)].67 We created 20 imputed data sets from a set of imputation models constructed from all predictors and outcomes (event indicator and Nelson–Aalen estimator for time to event).

We based an improvement in model performance on an increase in the number of patients correctly predicted to develop MBDD, relative to the clinical predictor only model. We used sensitivity, specificity and net benefit as measures of model performance. We conducted internal validation using 200 bootstrap samples (sampling with replacement) or until estimates remained stable. We describe the results from internal validation in Appendix 8, Table 31. We did not adjust for optimism. Statistical significance was based on Wilson’s 95% CI. We calculated p-values for differences in sensitivity and specificity using McNemar’s test.

Secondary outcomes

Secondary outcome 1

Comparative predictive ability of prognostic models, including MRE scores (MEGS, sMARIA and LI) versus a model based on clinical predictors alone to predict the development of disabling CD within 5 years of diagnosis, defined by Montreal behaviour and Liège criteria. We conducted modelling using the same methods as in the primary outcome. Models were only developed if the number of patients developing disabling CD was adequate. Otherwise, we provided descriptive statistics.

Secondary outcome 2

Identification of the best combination of individual MRE features for predicting disabling CD (all definitions) within 5 years of new diagnosis. PCA was used to combine multiple MRE parameters into a small number of eigenscores variables. This allowed a larger number of features to be combined without compromising statistical power. The most influential imaging features were identified for further simplification of MRE variables included in modelling. Methods were as in the primary outcome, and the statistical significance of including MRE features were evaluated based on improvement of model fit [Bayesian information criterion (BIC)] in comparison to the standard model, with additional model performance reported as appropriate.

Model testing

To provide additional clinical relevance for potential model implementation, we formed a group from the trial group, including three gastroenterologists and two radiologists. The group in consensus defined a priori how the models could be best utilised in clinical practice. Specifically, following guidance from the study statisticians, we set two risk group definitions (RDs) for identifying patients at high and low risk of developing disabling disease which we felt would have clinical utility.

For RD1, the high-risk group included the top 40% of participants with the greatest predicted risk from the model. For RD2, the high-risk group included participants with an absolute risk ≥ 10%. The absolute risk threshold is determined by sorting the participants by predicted risk and then using the predicted risk of the eighth (10% of 81) participant who developed MBDD as the threshold.

Secondary outcomes 3 and 4; see Chapter 5.