Discussion

Statement of principal findings

The ethical considerations identified in this review were derived from concerns expressed by patients, clinicians or other stakeholders. While the included studies did not always frame these directly as ethical issues, the qualitative synthesis highlights how such concerns translate into broader ethical implications for the responsible implementation of AI in medical screening.

The previous 2021 evidence map highlighted the limited availability of qualitative evidence on the ethical implications of AI-based technologies in screening, with most studies relying on broad surveys rather than using qualitative methods to elicit in-depth perspectives. The current review extends that evidence base by incorporating more recent qualitative studies and synthesising their findings, providing a consolidated account/comprehensive overview of clinician, patient and stakeholder concerns and priorities in this area and identifying a range of relevant themes related to the ethical application of AI-based technologies in screening.

The quality of the evidence

Overall, good quality evidence was found from 14 qualitative studies, as there were minor methodological concerns identified in the vast majority of studies. On an individual study level, the most common methodological concern identified (based on the CASP checklist) was that the relationship between the researcher and participants had not been adequately considered, but the impact of this on methodological rigour was considered to be minor. There were two studies with moderate concerns over methodological limitations.29,36 These were due to potential selection bias suspected, because the recruitment strategy had not been specified29 and due to a lack of sufficient detail over the data analysis method employed.36 Serious concerns over methodological limitations were identified in two studies. In one study, concerns were serious in addition to the relationship between the researcher and participants not having been discussed in the paper, the sampling method changed over time during the study, while not all participants appeared to have been asked all pre-specified interview questions, both of which were considered to have potentially introduced or to have been driven by bias.24 In another study, serious concerns were raised due to a lack of sufficient detail over both the recruitment strategy and the data analysis method in addition to the relationship between the researcher and participants not having been discussed.25

There were no concerns raised over applicability of the evidence from the vast majority of studies to the context of the current review. However, there were moderate concerns over applicability raised in relation to four studies. In one study, concerns were raised as the study examined barriers and facilitators to the implementation of two specific AI-assisted decision-making software: the ‘EyeWisdom’ for diabetic retinopathy (Vistel Technology, Beijing, China) and the ‘Dr. Wise^® Artificial Intelligence Assisted Diagnosis System for lung CT (Deepwise, Beijing, China) in the health setting in China, with some views expressed being of potentially limited applicability to UK healthcare practice and other AI software.32 Furthermore, moderate concerns were raised due to a lack of clarity about the stage of the screening or post-screening pathway to which the expressed views were related. It was also unclear whether all views were directly about the application of AI in image interpretation.24 One study explored experiences with three AI algorithms, but only one was related to imaging.36 Additionally, AI technology was applied for radiology report interpretation, raising questions about whether it was meant to assist clinicians with image interpretation or simply explain radiology reports to patients.37

In addition, minor concerns over applicability were raised in one study. These were due to the focus of the study being to explore experiences of implementing a specific AI-based imaging software tool [Veye Lung Nodules (VLN)] aiding the detection, classification and management of pulmonary nodules in CT scans of the chest; hence, the applicability of some expressed views are potentially limited to the specific tool and target condition examined.27 Nevertheless, evidence from these studies was considered to be relevant to the review question and was included in the present review. However, concerns were taken into account in the assessment of the quality element of relevance of the GRADE-CERQual and were reflected in the overall confidence in the themes to which these studies contributed.

On a review theme level, the level of confidence in the themes identified was mixed, ranging from high to low, reflecting the different levels of concern identified over methodological limitations in the contributing studies, coherence of the views expressed, relevance of the evidence to the review aim and the adequacy of evidence available to support each theme as per the GRADE-CERQual approach. The primary reasons for downgrading review findings were concerns over methodological limitations in the contributing studies and, occasionally, coherence, with participants within or across studies expressing opposing views and relevance. Evidence was rarely downgraded due to concerns over adequacy, as most themes were supported by a wealth of information.

Findings identified in the evidence synthesis

Several themes emerged, illustrating a number of issues related to the ethical application of AI-based technologies for medical image classification systems in screening. These were consistent with the views expressed by BMJ-TAG’s clinical experts.

Human in the loop and inability of artificial intelligence to make holistic assessments

As illustrated by high confidence evidence from 10 studies, both patients and professionals highlighted the need for maintaining the ‘human in the loop’ in AI-supported medical screening, not favouring the idea of autonomous AI. Concerns raised by patients and clinicians in the included literature were in line with the views of the BMJ-TAG’s clinical experts, who highlighted that patients’ concerns over the autonomy of AI systems and AI making stand-alone final decisions, rather than being used to help guide decisions made by a clinician, were common. Patients’ acceptability and perceptions of incorporating AI in their health care was highlighted by clinical experts as one of the potentially most challenging issues in the implementation of AI-based technologies in medical screening.

In this theme, the concerns expressed by patients and clinicians about autonomy and acceptability map onto the broader ethical principle of preserving human oversight in medical decision-making, reinforcing the view that AI should assist rather than replace human experts.

Liability/accountability

The issue of accountability and legal liability in the use of AI in medical settings was illustrated by 10 studies and confidence in the review finding was high. This issue appeared to be widely discussed across specialties, including diabetic retinopathy, breast cancer screening and PH diagnosis.

The BMJ-TAG’s clinical experts agreed that liability issues were important to consider, particularly in cases where AI tools produce errors. One clinical expert emphasised that if AI is incorporated in breast screening programmes in England, ultimate accountability should lie with NHS England, with NHS trusts being accountable for decisions made and individual doctors still responsible for making final decisions. It was emphasised that if the use of AI becomes approved for screening by NHS England, liability would still lie with the NHS trusts, doctors or the AI product developer and legal action against these parties could be taken if a mistake was made by AI. In line with the views expressed by participants in the evidence included in the current review, BMJ-TAG’s clinical experts had the view that clinicians should review decisions made by AI and have the confidence to identify and dismiss errors. Within this framework, it was highlighted that some clinicians may be hesitant to dismiss the potential errors made by AI and that users need to develop trust in using their own skills as well as being aware of the limitations of AI.

In this theme, the clinical and patients’ concerns about errors and hesitation in over-ruling AI decisions highlight the broader ethical consideration of accountability. Clear assignment of responsibility, whether it is to clinicians, NHS trusts or developers, is necessary to preserve trust and ensure ethical implementation.

Algorithmic bias

The issue of algorithmic bias and potential discrepancies between the data that AI algorithms are trained on and real-world data was illustrated by five studies and confidence in the review finding was moderate. Confidence in the review finding was lowered due to conflicting views expressed by some participants, with some professionals denying the existence of algorithmic bias or not recognising it as an ethical concern in the use of AI in screening. From an ethical perspective, unintended bias was highlighted by clinical experts as a potential harm of implementing AI in screening programmes, raising questions of fairness and equity.

The BMJ-TAG’s clinical experts highlighted the importance of knowing how an algorithm was trained, how the evidence used was collected, whether the results have been validated and whether the population upon which the model was trained is similar to the population in which it is going to be used. They highlighted that these are basic requirements when a new technology is applied. Moreover, it was also highlighted that clinicians may be experts in their field of work, for example breast radiology, but unlikely to be experts in machine learning or AI. Clinically, this creates concerns that healthcare teams may not be able to fully evaluate the above aspects and assist in overcoming the issue of bias.

For example, clinicians may not be able to ask the right questions to evaluate whether the population used to train an algorithm reflects the local community the tool is going to be used in and hence whether the algorithm could have lower accuracy in particular groups of patients. From the patients’ perspective, this raises concerns that individuals from minority or under-represented groups could receive less accurate or less timely results, undermining trust in the system. BMJ-TAG’s clinical experts also raised that data from populations with different ethnic backgrounds may not be applicable to a UK population and that, often, ethnic data are not documented in large data sets to avoid violating General Data Protection Regulation principles. Within this framework, clinical experts also highlighted the importance of clinicians being aware that any algorithm is not perfect and to align their expectations accordingly. It was emphasised that the population an AI algorithm was trained on needs to match the ethnic diversity of the population a screening tool will be used in. Bias in training data and validation was raised as an important issue to address to ensure that an AI algorithm is appropriate for the population it will be used in prior to implementation. Clinical experts raised that initially, AI tools are likely to be used in ‘shadow mode’, alongside clinicians, but it is not known what constitutes an appropriate length of time before an algorithm can be trusted. In contrast to the variability in the day-to-day performance of humans, AI algorithms are likely, once deployed, to consistently perform in the same way, further highlighting the need to recognise and address potential systematic algorithmic biases. Furthermore, one clinical expert raised that data used to train AI tools is from people who are participating in screening. Not attending screening may have been a result of various reasons which may lead to the under-representation of certain groups (those less likely to attend screening) in the images/data used to develop AI algorithms.

Data confidentiality, cybersecurity and consent

Issues of data confidentiality, cybersecurity and consent were illustrated in six studies. Confidence in this finding was low, primarily due to concerns over relevance in the contributing studies. Concerns over data confidentiality and consent highlighted in the current evidence were in line with the experience of BMJ-TAG’s clinical experts who emphasised that patients tend to be willing to consent to the use of their data if anonymised. They noted, however, that the definition of anonymised and pseudo-anonymised applied to large data sets required in the field of AI is still widely debated.

From an ethical perspective, the challenge lies in ensuring true anonymisation and safeguarding data security, as failures in this area could undermine public trust in screening programmes. From a clinical and patient perspective, concerns were more practical, focusing on patients’ willingness to share data and clinicians’ awareness of the complexities introduced when data sets are released to companies developing AI. In addition, the BMJ-TAG’s clinical experts highlighted that confidentiality risks can arise prior to implementation, for example when training data sets are released to the companies developing AI technologies. They also emphasised that what is considered ‘anonymous’ may be a grey area when it comes to retinal scans which, like fingerprints, are unique to each individual.

Transparency

The issue of transparency being essential in the use of AI for medical screening was supported by six studies. Confidence in the review finding was low, primarily due to concerns over methodological limitations and relevance identified in the contributing studies. Transparency was highlighted as an important issue both in terms of clearly communicating to patients being screened that AI was involved in the process as well as addressing the ‘black box’ nature of AI systems to ensure clarity in how decisions are made. This issue was in line with the view of BMJ-TAG’s clinical experts who emphasised that companies developing AI tools need to be transparent about their limitations.

In this theme, the ethical concern centres on the ‘black box’ nature of AI systems and the responsibility of developers to ensure openness about how decisions are made. From a patient and clinician perspective, the concern is more practical, with patients wanting clear communication when AI is used in their screening and clinicians wanting assurance that any imitations of AI tools are disclosed.

Risk to doctors’ competencies and impact on the profession

Concerns over the potential impact of AI-assisted tools on doctors’ competencies and the medical profession were highlighted across eight studies, generating a review theme of high confidence.

The BMJ-TAG’s clinical experts also acknowledged the concerns expressed by healthcare professionals in clinical practice around being replaced by AI. While, they considered widespread deskilling unlikely in large systematic screening programmes such as those existing in the UK, they noted the importance of maintaining skill levels as an ethical issue for implementation. Furthermore, the BMJ-TAG’s clinical experts suggested that there are numerous mechanisms by which skill levels can be maintained if deskilling is identified as an ethical issue. Contrary to the views of some participants in the evidence, the BMJ-TAG’s clinical experts thought that upskilling was improbable as AI tools are designed to present data in formats that are familiar to professionals. However, they emphasised that professionals would need to adapt their work with AI in ways that complement their expertise. Regarding AI replacing healthcare professionals, BMJ-TAG’s clinical experts raised that AI may increase the screening capacity by reducing the image interpretation workload and freeing resources that could be redeployed elsewhere in the screening programme, although they viewed this benefit as largely theoretical in practice. Nevertheless, the finding of the current review highlighted the importance assigned by patients on increased performance of AI-assisted screening over humans alone rather than on any operational benefits the implementation of AI may bring.

Trade-off between false positives and lesion detection

The possibility of overdiagnosis or misdiagnosis resulting from the introduction of AI was discussed across four studies. Confidence in the resulting review theme was moderate due to some concerns over methodological limitations and coherence with some participants expressing different views in terms of whether more false positives or negatives were preferable and the extent to which they were willing to risk false positives to ensure that all cancer cases are picked up.

Concerns emerging from the clinical evidence were in line with the view expressed by one of BMJ-TAG’s clinical expert (breast radiologist) who emphasised the need for an algorithm to reduce false-positive recalls. Nevertheless, BMJ-TAG’s clinical experts emphasised that a benefit of AI is that sensitivity and specificity can be predetermined and chosen as appropriate to the context in which AI is implemented in a screening programme. They emphasised that this can be extremely helpful and that it cannot be achieved by human graders. However, BMJ-TAG’s clinical experts emphasised that it is important to recognise that any technology, including AI, is extremely unlikely to detect all cancers and failure to do so cannot reasonably be described as an error.

In this theme, the clinical concern focused on balancing diagnostic accuracy, particularly the harms of false positives versus the benefits of detecting more cancers. The ethical consideration is proportionality, how much uncertainty and potential overdiagnosis patients and health systems should accept in exchange for maximising detection.

Lower tolerance for errors made by artificial intelligence versus humans

Within this framework, as demonstrated by two studies contributing to a review finding of moderate confidence, women’s acceptance of failure to detect breast cancer was lower for AI than for radiologists. BMJ-TAG’s clinical experts were aware of such concerns, highlighting the risk of perceiving an AI algorithm as perfect and the unrealistic expectations that may come with that. They highlighted that barriers to the adoption of AI in screening could be due to concerns over errors made by AI. This reflects an ethical concern about fairness and trust, as errors made by AI are judged more harshly than those made by humans, which has implications for patient acceptability and public trust.

Cases of reduced effectiveness

Clinicians, stakeholders and patients across three studies highlighted cases of reduced accuracy with AI-based radiology tools, including misclassifications and potential ‘blind spots’ in complex areas like blood vessels, airways and dense breast tissue. It is important to note that confidence in this review finding was low. That was partially due to concerns over the adequacy of the evidence available to support the theme and concerns over relevance associated with two studies where the views expressed were in relation to specific AI tools used in the context of the studies. From an ethical perspective, these concerns point to the risk of over-reliance on AI systems that may not consistently perform at the standard of human experts, raising questions of patient safety and equity in screening outcomes.

Lack of familiarity and trust as a barrier to implementation

An issue highlighted across three studies was clinicians’ and patients’ lack of familiarity with AI software as a potential barrier to implementation. This theme was consistent with the experience of BMJ-TAG’s clinical experts who highlighted that clinicians currently do not understand how AI will work in practice and that often, despite evidence of robust diagnostic accuracy being provided by the technology, concerns over reliability and safety remain. From a clinical and patient perspective, these concerns reflect uncertainty about the day-to-day use of AI in routine care. Ethically, they point to the principle of trustworthiness; if users cannot understand or rely on the technology, then its deployment risks undermining confidence in screening programmes. Nevertheless, clinical experts emphasised that there is currently not enough peer-reviewed evidence to support the deployment of AI-assisted tools on a large national screening programme.

Systemic integration of artificial intelligence

Finally, issues and requirements for systemic integration of AI-assisted tools in screening emerged from seven studies. These included ethical concerns related to healthcare inequality, as resource-limited hospitals may not benefit equally from AI advancements. Patients’ safety was also a concern, with delays in AI-generated reports potentially compromising timely care. The ethical principle of trust is crucial, requiring rigorous testing, transparency in AI decision-making, and clear guidelines to ensure that AI remains an assistive tool rather than replacing human judgement.

Furthermore, issues related to the implementation of AI-assisted tools to current screening pathways in the UK were highlighted during discussion with BMJ-TAG’s clinical experts, who emphasised there are concerns about how AI-assisted tools can be integrated into the existing IT infrastructure used for screening so that no risks are introduced. In the context of breast cancer screening, it was emphasised that the current infrastructure is not standardised across the UK, as not all NHS trusts are on the latest version of Windows, and Picture Archiving and Communications System (PACS) software for images is not consistently rolled out across the NHS. Thus, based on their current infrastructure, not all trusts would be able to incorporate AI in screening. Specifically, BMJ-TAG’s clinical experts highlighted that the current infrastructure for breast screening was developed in 1988, and it is still in the process of being modernised and thus may not be able to support AI systems. To overcome this, a modified version of an agreed AI algorithm may be required for it to be able to run in the NHS. This modified version would then need to be appropriately validated, otherwise it is likely to lead to further issues. In addition, it was highlighted that virtual servers would be required for AI software to run, which current IT departments do not have the capacity to implement. Implementation would also be dependent on individual NHS trusts purchasing AI algorithms as currently there is no national roll-out in place. Within this context, equality issues may arise as the likelihood of implementation may depend on how motivated different organisations/trusts are towards using AI and updating their current software to support it. The size of the organisation and the amount of funds available to invest in upgrading are also likely to have an impact, as a result of a lack of standardisation and budget priorities across the NHS.

Currently, as highlighted by one of BMJ-TAG’s clinical experts, AI-based technology is being used in the Leeds Teaching Hospitals NHS Trust. In Leeds, it is mainly used for processing, for example to overbook breast screening clinics, with women predicted to be most likely to not attend screening, rather than the diagnostic interpretation of mammograms. Furthermore, in Leeds, AI is currently being used to measure breast density of mammograms and to assess the quality of mammograms, but the use of AI for diagnostic interpretation in screening is not currently allowed.

In the context of diabetic retinopathy, it was highlighted that currently there are only two software systems used in screening, but the way to integrate AI-assisted tools with existing software needs to be considered. Moreover, the precise role of AI in the diabetic retinopathy screening pathway has not yet been fully determined, although its current proposed use is as a replacement for level 1 graders to detect disease versus no disease with no major issues.

Strengths and limitations of the assessment

In summary, this review drew on 14 primary qualitative studies that were relevant to the research question. However, as described in Statement of principal findings, the quality of the evidence varied; while most studies raised only minor methodological concerns, two studies had moderate limitations, while a further two studies had serious limitations (based on the CASP checklist), including potential selection bias, limited reporting of recruitment and data analysis methods, and insufficient discussion of the researcher–participant relationships. These limitations did not alter the synthesis process itself, as themes were identified consistently across studies. However, they did affect the overall confidence that can be placed in some of the individual findings (based on the GRADE-CERQual approach). The synthesis should therefore be interpreted as providing an indicative overview of emerging ethical considerations, with the strength of the evidence supporting each theme varying according to the quality of the contributing studies. This distinction can be important for policy-makers, who should take into account both the nature of the themes identified and the strength of the evidence base underpinning them.

Also, due to its qualitative nature, the review relied on thematic analysis, which is inherently interpretative and may reflect researcher subjectivity, and did not include formal input from ethicists, which could have strengthened the assessment of ethical issues.

Despite these limitations, the current review provides a useful synthesis of existing evidence and highlights a number of key ethical considerations related to the use of AI-based technologies for medical image classification in screening. These insights can be utilised to inform stakeholders, help them navigate the ethical implementation of AI in screening programmes in the UK and guide responsible innovation. Furthermore, given the rapidly growing volume of evidence on the topic, future evidence reviews may be able to provide more in-depth insights on ethical issues surrounding the use of AI-based technologies for medical image classification systems in screening, which are particularly relevant to specific conditions such as breast cancer or diabetic retinopathy, which the current evidence may not have been able to uncover.

Uncertainties

The review has the following methodological limitations: only EMBASE, Ovid MEDLINE, PsycInfo, CINAHL electronic databases were searched; searches were limited to records published from June 2020 onwards (updating the UK NSC 2021 evidence map search that included papers published since 2000) and included only peer-reviewed, English-language journal articles; and only 20% of the titles and abstracts were double-screened, meaning relevant studies outside these parameters may have been missed or some selection bias may have been introduced. The volume of published evidence identified, although sufficient (based on prespecified criteria) to only include qualitative studies and exclude survey and questionnaire studies, was insufficient to enable the review to focus on the target conditions of breast cancer and diabetic retinopathy. As prespecified in the review protocol, the 20% validation was also applied in the quality assurance of data extraction and thematic analysis due to time constraints. A future review could potentially aim for double-reviewing 100% of all steps of the review process to ensure accuracy and consistency. Nevertheless, qualitative thematic analysis is interpretative/subjective in nature, which may limit the reproducibility of findings despite double-reviewing.38 In addition, expanding the scope to include non-English literature and grey literature could further broaden the evidence base in future reviews.

Other relevant factors

The BMJ-TAG’s proposed mapping of the emergent themes to the UK NSC ethics framework is included in Table 14.

Patient and public involvement

No patient and public involvement occurred for this research.

Equality, diversity and inclusion

This review was conducted in line with NIHR guidance on equality, diversity and inclusion (EDI). We considered EDI in the scope of the evidence by including studies across different relevant screening contexts and populations where available. However, the evidence base was relatively limited, with no included studies explicitly examining issues relating to minority or under-represented groups. Nevertheless, a key theme emerging from the qualitative evidence was the importance of addressing potential bias in AI algorithm training data sets, particularly where under-represented groups may not be adequately included. In addition, BMJ-TAG’s clinical experts emphasised that algorithms must be trained and validated on data sets, reflecting the ethnic diversity of the populations in which they will be applied, to ensure appropriateness prior to implementation. These concerns highlight the relevance of EDI in the design, evaluation and implementation of AI-based screening tools and point to a need for future research that examines these issues directly.