Principal findings
Our systematic review and meta-analyses provide estimates of the accuracy of ADD-RS and D-dimer for AAS based upon cohort studies using consistent thresholds for positivity. The sensitivity and specificity (95% CrI) of ADD-RS greater than zero were 94.6% (90% to 97.5%) and 34.7% (20.7% to 51.2%), and of ADD-RS greater than one were 43.4% (31.2% to 57.1%) and 89.3% (80.4% to 94.8%). The sensitivity and specificity of D-dimer using the 500 ng/ml threshold for positivity were 96.5% (94.8% to 98%) and 56.2% (48.3% to 63.9%). Sensitivity analyses of ADD-RS limited to the six studies evaluating combinations of ADD-RS with D-dimer, and of D-dimer including studies with alternative thresholds for positivity showed similar estimates to the base-case analysis.
The sensitivities and specificities of ADD-RS greater than zero or D-dimer > 500 ng/ml were 99.8% (98.7% to 100%) and 21.8% (12.1% to 32.6%), ADD-RS greater than one or D-dimer > 500 ng/ml were 98.3% (94.9% to 99.5%) and 51.4% (38.7% to 64.1%) and the Canadian guideline-based strategy (ADD-RS > 1 or ADD-RS = 1 with D-dimer > 500 ng/ml) were 93.1% (87.1% to 96.3%) and 67.1% (54.4% to 77.7%).
These findings indicate that ADD-RS and D-dimer provide potentially useful information in the diagnostic assessment of AAS, and combinations of ADD-RS and D-dimer could be used to rule out AAS, albeit with a risk of missed diagnosis.
Our systematic review of biomarkers other than D-dimer produced much more limited data and no reliable estimates of diagnostic accuracy for use in clinical practice. We identified 38 case-control studies85–122 of biomarkers but only 13 cohort studies52,63,68,69,73,76–83 for inclusion in our review. Biomarker accuracy was generally modest, with sensitivity sufficient to rule out AAS only when a threshold was used that resulted in low specificity. An exception was the study of Wang et al.,73 reporting 99.1% sensitivity, 84.9% specificity for sST2, but this differed markedly from the findings reported by Morello et al.78 (sensitivity 58%, specificity 70.8%). No biomarkers, other than D-dimer, can currently be recommended for clinical use.
Decision-analytic modelling showed that if strategies were applied unselectively to all patients with possible AAS, then CTA for those with ADD-RS > 1 would be cost-effective at the £20,000/QALY threshold but would result in more than half of cases of AAS being missed. CTA for those with ADD-RS > 1 or ADD-RS = 1 with D-dimer > 500 ng/ml would be cost-effective at the £30,000/QALY threshold but would require a three- to fourfold increase in current CTA use, which is unlikely to be deliverable. If clinicians are able to use their judgement to exclude patients considered to have a zero risk of AAS, then a strategy of CTA for patients with ADD-RS > 1 or ADD-RS = 1 with D-dimer > 500ng/ml would be cost-effective and require a more modest (≈60%) increase in CTA capacity. A strategy of ADD-RS > 1 or D-dimer > 500 ng/ml is cost-effective at 2.95% prevalence of AAS, which is the prevalence of AAS in patients receiving CTA in the DAShED study.
At a threshold of £20,000/QALY, the individual EVPI was £4.46 per patient, and the population EVPI for patients presenting to the NHS with suspected AAS over 5 years was estimated as £17.75M, suggesting that further research to reduce the uncertainty would be valuable.
Contribution to existing knowledge
Our summary estimates of accuracy for ADD-RS (alone and with D-dimer) are similar to previous meta-analyses,20,123 although we report slightly lower sensitivity. We also evaluated a strategy based on the Canadian clinical practice guideline (ADD-RS > 1 or ADD-RS = 1 with D-dimer > 500 ng/ml), which has lower sensitivity but higher specificity than other strategies combining the ADD-RS with D-dimer. This offers an alternative strategy to patients who wish to avoid overinvestigation and populations with a low prevalence of AAS, where a strategy with low specificity would generate an acceptably low yield of positive imaging.
Previous meta-analyses of D-dimer for AAS have produced conflicting estimates of specificity. Yao et al.19 reported sensitivity and specificity of 96% and 70%, respectively, but included many case-control studies; while Asha et al.124 reported 98% and 42%, respectively, but included only four studies (1557 participants).34,59,62,72 Our base-case analysis included 18 cohort studies18,31–35,59,62–68,71–74 with 7978 participants and reported sensitivity and specificity of 96.5% and 56.2%, respectively. This is likely to be a more robust and reliable estimate of D-dimer accuracy than previous studies, by being based on a large number of cohort studies.
Our study is, to our knowledge, the first systematic review of biomarkers for AAS other than D-dimer. The conclusions were limited by the quality of the primary data, with 38 case-control studies excluded and only 13 cohort studies eligible for inclusion. We have, therefore, highlighted the need for more robust studies of biomarkers but are unable to identify any current biomarkers that are suitable for clinical practice, other than D-dimer.
Our decision-analytic modelling is, to our knowledge, the first to examine the cost-effectiveness of strategies for selecting patients with suspected AAS for CTA. Taylor and Iyer125 used decision-analytic modelling to compare testing strategies for AAS in terms of health outcomes but not costs. Their model suggested using low testing thresholds of 0.03% probability of AAS for CTA compared to no testing and 0.013% for D-dimer compared to no testing. These findings suggest that the benefits of accurate diagnosis substantially outweigh the risks of testing but do not take costs into account. Our analysis suggests that CTA for all is not cost-effective compared to alternative strategies when costs are considered rather than health outcomes alone. Furthermore, our base-case analysis showed that even if strategies are cost-effective, they may require an increase in CTA capacity that is unlikely to be deliverable in a typical hospital or across the NHS.
Strengths and weakness of the study/in relation to other studies
Our meta-analysis of ADD-RS and D-dimer only included cohort studies, thus avoiding the risk of bias associated with case-control studies, yet still included sufficient primary data to provide reasonably accurate and precise estimates of diagnostic accuracy. We were also able to use data shared by the primary study teams to estimate the accuracy of an additional strategy combining ADD-RS with D-dimer that was based on the Canadian clinical practice guideline.
There was potentially important heterogeneity between studies in both meta-analyses, especially in estimates of specificity, which increases the uncertainty around these estimates. This is discussed in detail in the primary publications.1,2 The heterogeneity may reflect differences in study design, particularly patient selection, with studies varying between those reporting populations with a low rate of imaging for the reference standard and AAS prevalence to those reporting populations with a higher rate of imaging and AAS prevalence. The wide range of prevalence in the primary studies limit our ability to generalise the findings from meta-analysis to specific groups, particularly low-prevalence populations, including many patients who would not normally receive imaging.
As noted previously, the biomarker systematic review was substantially limited by the quality of the primary data. We were unable to recommend any biomarkers for clinical use, other than D-dimer, and could not include biomarkers other than D-dimer in the decision-analytic modelling.
Our decision-analytic modelling is the first to examine the cost-effectiveness of alternative strategies for selecting patients with possible AAS to CTA. It is based upon robust estimates from our meta-analysis of the accuracy of the strategies and draws upon clinical expertise to ensure a model that reflects the complexities of the clinical problem while retaining transparency.
A key limitation in our understanding of AAS diagnosis relates to how clinicians determine whether a patient presenting with symptoms compatible with AAS is considered to require investigation for AAS. Our base-case analysis included all patients whose symptoms were compatible with AAS. However, the DAShED study33 showed that clinicians considered a substantial proportion of these patients to have a low or zero likelihood of AAS and only used CTA to investigate a minority for AAS. Our analysis showed that strategies involving the ADD-RS and D-dimer are cost-effective if limited to selected patients in whom clinical judgement suggests a meaningful risk of AAS. The limitation with this finding is that we do not know how clinicians make this judgement and whether their assessment of AAS risk is accurate.
A related limitation is that we did not compare the strategies to using unstructured clinical judgement to select patients for CTA or consider how clinical judgement could be used alongside the strategies. If we conclude that clinical judgement is required to select patients for diagnostic investigation with the ADD-RS and D-dimer, then we should consider whether the selection for CTA should be based on clinical judgement alone (or informed by ADD-RS and D-dimer). There is little evidence available to estimate the accuracy of unstructured clinical judgement for diagnosing AAS, so we were unable to evaluate it in our analysis.
It is also important to note that we assumed that the population for the modelling excluded patients whose frailty and/or comorbidities meant that they would not be eligible for surgery or endovascular repair if AAS were detected. We made this assumption because including such patients would increase the complexity and uncertainty in the analysis, while reducing the applicability of findings to the population most likely to benefit from AAS diagnosis. We felt that this assumption reflected clinical practice, which involves assessing the potential implications for treatment before ordering a diagnostic test. Our findings are, therefore, not applicable to patients with frailty or comorbidities that limit the treatment options to medical treatment alone. The decision to investigate such patients is likely to be individualised and involve consideration (and discussion with the patients) of whether extensive investigation is in the patient’s best interests.
Other limitations relate to uncertainties in the assumptions and estimates used in the model:
- Estimates of the effect of delayed treatment are inevitably based on limited observational data.
- We were unable to include any credible estimates of the benefits and harms arising from non-AAS diagnoses and incidental findings identified on CTA due to the variety of findings identified and uncertainty over their clinical significance.
Changes from the proposed project
We planned to include case-control studies and report their characteristics and findings but exclude them from the meta-analyses due to the potential high risk of bias of such studies.24,25 We also specified that studies would only be included if participants (cohort, cases or controls) presented with suspected AAS. We found that the reporting of case-control studies was often insufficient to determine whether the control group presented with suspected AAS or not, and therefore whether such studies should be included. We, therefore, made a post hoc decision to exclude case-control studies from the review. We reported the number and characteristics of case-control studies of alternative biomarkers to provide readers of this review with an insight into the large number of such studies but did not attempt to draw conclusions from the findings of these studies.
We made some changes to the proposed methods for meta-analysis:
- We planned to use the method of Steinhauser et al.126 but, instead, used the method of Jones et al. 2019.28 The Jones’ model has advantages over Steinhauser’s model in the following areas: it models the exact multinomial likelihoods of the spread of test results across thresholds, rather than requiring the normal approximations; it automatically accounts for within-study correlations resulting from studies reporting at more than one threshold; and it performs better with small counts.
- We planned to analyse sensitivity and specificity at multiple thresholds for D-dimer, but we found insufficient data to support this analysis. Most studies only reported one threshold, usually the 500 ng/ml threshold. The few studies that reported more than one threshold only reported a few variable thresholds, which were insufficient to support detailed analysis.
- We proposed to perform sensitivity analysis and listed the likely analyses as (1) studies with a sample based on suspicion of AAS rather than receipt of a reference standard test, (2) studies with > 90% of the sample receiving a definitive reference standard or adequate follow-up and (3) studies that evaluated a fully derived index test (including the threshold for positivity). We were unable to undertake analyses (1) and (2) because the primary study reporting of sample selection and reference standard testing was limited and inconsistent, and could not therefore support categorisation for sensitivity analysis. We were unable to undertake analysis (3) because most studies used the pre-specified 500 ng/ml threshold. We, therefore, used this threshold for the primary analysis and used sensitivity analysis to explore, including studies with other thresholds.
We made the following changes to the planned decision-analytic modelling:
- In accordance with the meta-analysis, we found insufficient data reporting D-dimer sensitivity and specificity at thresholds other than the 500 ng/ml threshold, so we did not explore the cost-effectiveness of using D-dimer at alternative thresholds for positivity.
- The systematic review identified no biomarkers (other than D-dimer) with sufficient robust evidence of accuracy to support clinical use, so we did not evaluate any strategies using alternative biomarkers in the decision-analytic modelling.
- We estimated the accuracy of strategies included in the modelling using data from cohort studies that reported ADD-RS and D-dimer in combination. We, therefore, did not need to undertake the planned exploratory analyses using indirect estimates of accuracy (estimates based on combining an estimate of ADD-RS accuracy with a combination of D-dimer accuracy) or estimates from case-control studies.
- We reviewed studies reporting the diagnostic yield of CTA for non-AAS pathology, along with data from the DAShED study33 and a single-centre study of CTA results.127 These showed that CTA identify a wide range of non-AAS pathology. For example, the 407 CTA reported in the DAShED study33 showed 201 non-AAS findings involving 61 different pathologies. We decided that it would not be possible to include any credible estimates of benefit or harm from these findings in the modelling, so we acknowledged this uncertainty and limitation of the model. Similarly, we were unable to include any estimate of benefit or harm from incidental CTA findings in the model but included a cost for the proportion of patients who would require a repeat CT scan for pulmonary nodules or similar equivocal findings.
Take-home messages
The ADD-RS and D-dimer, alone and in combination, provide useful diagnostic information, but their use in the diagnostic pathway for AAS (specifically ruling out AAS) depends upon how clinical suspicion of AAS is determined. D-dimer below 500 ng/ml in a patient with low clinical risk (ADD-RS zero) effectively rules out AAS (sensitivity 99.8%), but unselective application of this rule will lead to a high rate of negative CTA.
Clinical scores other than the ADD-RS and biomarkers other than D-dimer currently lack sufficient evidence of diagnostic accuracy for AAS to be recommended for clinical use.
In an unselected population with possible symptoms of AAS, current validated strategies struggle to achieve an acceptable balance between overinvestigation and risk of missed AAS. A strategy based on the Canadian guideline (CTA if ADD-RS > 1 or if ADD-RS = 1 with D-dimer > 500 ng/ml) appears cost-effective but would require over a quarter of a million CTAs and result in over 100 missed AAS across the NHS in England and Wales, if applied to an unselected population.
If clinicians can select a population with a higher prevalence of AAS without missing cases, then the strategy based on the Canadian guidelines or a strategy of CTA if ADD-RS > 1 or D-dimer > 500 ng/ml would be cost-effective and potentially deliverable without a substantial increase in CTA capacity. Further research is needed to determine how clinicians select this population.
For patients currently receiving CTA, the strategy of ADD-RS > 1 or D-dimer > 500 ng/ml could halve the number of CTA undertaken and would be cost-effective according to National Institute for Health and Care Excellence (NICE) thresholds for willingness to pay but would result in 143 additional missed cases of AAS per year across the NHS.