About this chapter
Part of this chapter has been reproduced with additions and minor changes with permission from Meiksin et al.57 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes additions and minor changes to the original text.
Results of the search
A total of 49,473 references were retrieved by the original bibliographic databases, clinical trials register and Google searches. Of these, 28,747 (58%) were identified as duplicates, leaving 20,726 references. The updated search identified 5317 additional references, giving a total of 26,043 references identified. The number of references before and after deduplication are given in Table 1, as are the new references identified from the updated searches. Requests from subject matter experts (see Appendix 5) did not identify any additional eligible reports.
| Database/register name | References from initial search (n) | References once duplicates removed (n) | References from updated search (n) |
|---|---|---|---|
| ProQuest ASSIA | 1812 | 142 | NA |
| Campbell Library | 0 | 0 | 0 |
| EBSCO CINAHL Plus | 3061 | 977 | 406 |
| Wiley Online Library The Cochrane Library | 378 | 125 | 83 |
| CRD databases | 95 | 77 | NA |
| EPPI-Centre database of health promotion research (Bibliomap) | 7 | 0 | 0 |
| ProQuest Dissertations & Theses Global | 2231 | 1427 | 243 |
| OvidSP EconLit | 56 | 53 | 9 |
| OvidSP EMBASE | 5995 | 2289 | 805 |
| OvidSP Global Health | 1893 | 302 | 95 |
| OvidSP HMIC | 90 | 33 | NA |
| ProQuest IBSS | 2503 | 968 | NA |
| OvidSP MEDLINE ALL | 4701 | 4596 | 1279 |
| OpenGrey | 87 | 50 | 0 |
| OvidSP PsycINFO | 4854 | 2675 | 472 |
| Web of Science Science Citation Index Expanded | 3212 | 1031 | 185 |
| Elsevier Scopus | 10537 | 3729 | 981 |
| OvidSP Social Policy & Practice | 204 | 100 | 15 |
| Web of Science Social Sciences Citation Index Expanded | 4365 | 809 | 339 |
| ProQuest Sociological Abstracts | 3314 | 1277 | NA |
| ClinicalTrials.gov | 58 | 58 | 405 |
| World Health Organization ICTRP | 15 | 3 | NA |
| Eppi-Centre TRoPHI | 0 | 0 | 0 |
| 5 | 5 | NA | |
| Total | 49,473 | 20,726 | 5317 |
Screening
When piloting the process for screening by title and abstract, the rate of agreement for initial screening between two screeners, based on a pilot screen of 87 references, was 99%. Given this level of agreement, we moved to a system of one reviewer independently screening each reference, as set out in the protocol. Figure 1 describes the results of screening. Of the 20,726 references from the initial search screened by title and abstract, 20,497 (98.9%) were excluded on this basis. Full reports were sought for the remaining 229 references.
FIGURE 1 Literature search and screening. EE, economic evaluation; OE, outcome evaluation; PE, process evaluation; ToC, theory of change. This figure has been adapted with permission from Melendez-Torres et al.58 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. (Figure not reproduced here; see the original article.)
Of these 229 references, we obtained full study reports for 168 and online registrations for 58. No reports or other information could be found for three references. Of the 226 references for which there was a report or online registration, 183 reports were excluded: three based on population, 104 based on intervention, 49 based on study design (of which two were systematic reviews), 12 based on outcomes and 15 because these were belatedly identified as duplicates. In addition, 17 online registrations were excluded: in the case of six, although the registration itself contained insufficient information to be included, the study appeared relevant and we noted that our searches had already identified the report of results from the registered study. For a further 11 online registrations, the study appeared to be of potential relevance, but did not contain sufficient information to be included and no reports could be found.
Overall, 26 reports were included after screening full reports and online registrations.96–121 This included two reports that were included via two online registrations.101,102 In addition, one report not found from electronic searches was identified via checking the reference lists of included studies.122 The journal in which this study was published was indexed in at least one database included in our search, and so we did not hand-search the journal. Of the included study reports, 24 reported on theories of change,96–107,110–113,115–122 10 on process evaluations,96,98,103,107,108,111,118,119,121,122 12 on outcome evaluations99–102,109,110,113–117,122 and none on economic evaluations.
The updated search identified 5317 new references, of which 121 were retained after screening by title and abstract. Of these, three were unobtainable and 118 were screened by full text. Ten eligible reports on 10 studies of 10 interventions were identified for inclusion.115,123–131 Of these, five were new reports on studies of interventions already included from the original searches115,123,127,128,130 and five reported on new interventions.124–126,129,131 The updated search added nine reports on theories of change,123–131 two on process evaluations,123,130 four on outcome evaluations123–125,127 and one on economic evaluation.132
After original and updated searches, a total of 37 reports were included: 33 reporting on theories of change,96–105,107,110–113,115–131 12 on process evaluations,96,98,103,107,108,111,118,119,121–123,130 16 on outcome evaluations99–102,109,110,113–117,122–125,127 and one on economic evaluation.132 Table 2 shows an overview of interventions examined in the review and the included theory, process evaluation, outcome evaluation and economic evaluation reports on each.
| Interventions examined in the review | Included reports on theories of change (n = 33) | Included reports on process evaluations (n = 12) | Included reports on outcome evaluations (n = 16) | Included reports on economic evaluations (n = 1) |
|---|---|---|---|---|
| China–Gate HIV Prevention Program online intervention (no name) | Cheng et al.124 2019 | Cheng et al.124 2019 | ||
| Cognitive Vaccine Approach (tailored and non-tailored versions) | Davidovich et al.102 2006 | Davidovich et al.102 2006 | ||
| Gay Cruise | Kok et al.105 2006 | |||
| HealthMindr | Sullivan et al.118 2017 | Sullivan et al.118 2017 | ||
| Jones et al.128 2020 | ||||
| Hot and Safe M4M (website name) | Carpenter et al.100 2010 | Carpenter et al.100 2010 | ||
| Internet-based safer sex intervention (no name) | Milam et al.109 2014a (published abstract) | |||
| Milam et al.110 2016a (published paper) | Milam et al.110 2016a (published paper) | |||
| Keep it Up! | Mustanski et al.122 2013 (intervention version 1.0) | Mustanski et al.122 2013 (intervention version 1.0) | Mustanski et al.122 2013 (intervention version 1.0) | |
| Greene et al.103 2016 (intervention version 1.5, identical to 1.0) | Greene et al.103 2016 (intervention version 1.5, identical to 1.0) | |||
| Mustanski et al.112 2017b (intervention version 2.0) | ||||
| Mustanski et al.113 2018b (intervention version 2.0) | Mustanski et al.113 2018b (intervention version 2.0) | |||
| Madkins et al.130 2019b (intervention version 2.0) | Madkins et al.130 2019b (intervention version 2.0) | |||
| Mobile Technology and Incentives (MOTIVES) | Linnemayr et al.106 2018 | |||
| myDEx | Bauermeister et al.97 2017c | |||
| Bauermeister et al.123 2019c | Bauermeister et al.123 2019c | Bauermeister et al.123 2019c | ||
| MyPEEPS Mobile | Kuhns et al.129 2020 | |||
| Online mindfulness-based cognitive therapy (no name) | Avellar96 2016 | Avellar96 2016 | ||
| People Like Us | Tan et al.131 2020 | |||
| Queer Sex Ed | Mustanski et al.111 2015 | Mustanski et al.111 2015 | ||
| Rainbow SPARX | Lucassen et al.108 2015d | Lucassen et al.108 2015d | ||
| Lucassen et al.107 2015d | ||||
| Role-playing game | Coulter et al.126 2019 | |||
| Safe Behaviour and Screening | Chiou et al.125 2020 | Chiou et al.125 2020 | ||
| Sex Positive! | Hirshfield et al.104 2016e | |||
| Hirshfield et al.127 2019e | Hirshfield et al.127 2019e | |||
| Sexpulse | Rosser et al.116 2010f | Rosser et al.116 2010f | ||
| Wilkerson et al.120 2011f | ||||
| Smartphone self-monitoring (no name) | Swendeman et al.119 2015 | Swendeman et al.119 2015 | ||
| Socially Optimized Learning in Virtual Environments (SOLVE) | Christensen et al.101 2013 | Christensen et al.101 2013 | ||
| TXT-Auto | Reback et al.114 2017g (published abstract) | |||
| Reback et al.115 2019g (published paper) | Reback et al.115 2019g (published paper) | |||
| Reback et al.132 2019g | ||||
| Wyoming Rural AIDS Prevention Project (WRAPP) | Bowen et al.98 2007 (no name; preliminary work to WRAPP intervention) | Bowen et al.98 2007 (no name; preliminary work to WRAPP intervention) | ||
| Bowen et al.99 2008 | Bowen et al.99 2008 | |||
| Williams et al.121 2010 (Hope Project; extends WRAPP) | Williams et al.121 2010 (Hope Project; extends WRAPP) | |||
| Schonnesson et al.117 2016 (SMART; Swedish adaptation of WRAPP) | Schonnesson et al.117 2016 (SMART; Swedish adaptation of WRAPP) |
Included reports and interventions
The 37 reports included in this review reported on 28 unique studies96–132 focused on 23 interventions. Seven reports were study protocols, which were included because they reported on theory of change;97,104,106,112,126,128,129 for four studies, only a protocol was included.106,126,128,129 Seven studies were each featured in more than one report: one was featured in a protocol and two subsequent empirical papers,112,113,130 two were each featured in a protocol and one subsequent empirical paper,97,104,123,127 one was featured in a published abstract and two subsequent full empirical reports,114,115,132 one was featured in a published abstract and one subsequent full empirical report,109,110 and two were each featured in two empirical reports.107,108,116,120
In the case of reports of three interventions, reports presented multiple versions of very similar interventions.98,99,102,112,113,117,121,122,130 One report by Davidovich et al.102 presented two versions of the Cognitive Vaccine Approach intervention: one version offered all content to all users and the other tailored the content delivered based on an initial user assessment. We determined that this was a core difference and therefore treated these as two unique interventions. In the cases of the Keep it Up!103,112,113,122,130 and the Wyoming Rural Acquired immunodeficiency syndrome Prevention Project (WRAPP)98,99,117,121 interventions, author descriptions suggest that intervention content was very similar across versions, and so we treated Keep it Up! and WRAPP as single interventions. Therefore, the included reports report on 23 unique interventions. Of these, nine interventions are featured in more than one included report: two in corresponding published abstracts and papers,109,110,114,115 two in two unique papers reporting results from the same study,107,108,116,120 two in a protocol and corresponding paper reporting results from the same study,97,104 one in a protocol and results paper from two different studies,118,128 one in five reports on three studies of two versions of the same intervention103,112,113,122,130 and one in four reports on four studies of four versions of the same intervention.98,99,117,121
Table 3 describes the characteristics of the included interventions. Thirty-three theory reports described 22 distinct theories of change underpinning all 23 interventions,96–107,110–113,115–131 with the two versions of the Cognitive Vaccine Approach intervention sharing a single theory of change.102 Twelve process evaluation reports presented empirical results on eight interventions.96,98,103,107,108,111,118,119,121–123,130 Sixteen outcome evaluations presented empirical results on 13 interventions.99–102,109,110,113–117,122–125,127 One economic evaluation presented empirical results on one intervention.132 Of the 37 included reports, 10 (27%) reported on theories of change only;97,104–106,112,120,126,128,129,131 one (3%) reported on a process evaluation only;108 two (5%; both abstracts) reported on outcome evaluations only;109,114 one (3%) reported on economic evaluation only;132 nine reported on both theories of change and process evaluations (24%);96,98,103,107,111,118,119,121,130 12 (32%) reported on both theories of change and outcome evaluations;99–102,110,113,115–117,124,125,127 and two (5%) reported on theories of change, process evaluations and outcome evaluations.122,123
| Intervention name and study report(s) describing this intervention | Location: country (region) | Target population | Providers and organisation | Intervention development | Intervention aims, components, content and activities | Intervention timing and duration | Interactive or non-interactive? | Technology |
|---|---|---|---|---|---|---|---|---|
| China–Gate HIV Prevention Program Online Intervention Cheng et al.124 | China | MSM | Not stated | Intervention was based on formative research and reviewed by professional and community experts. Gay community representatives provided the scenarios presented in part I which were reviewed by the research team and target population | Two-part interactive HIV prevention intervention delivered via a popular website for gay men in China. Part I comprised realistic interactive scenarios addressing sexual behaviour (CAI, condom breakage, encountering sex partner in a pub and commercial sexual encounter) and HIV testing, and presenting peer attitudes towards behavioural decisions. Delivering its content via e-mail, part II presented visually appealing HIV information tailored for MSM addressing HIV/AIDS basic knowledge and transmission, local epidemic data for MSM and misconceptions about sexual behaviours | Part I delivered immediately after completing baseline survey. Following completion of part I, part II was delivered in three parts, each delivered weekly | Interactive | Internet |
| Cognitive Vaccine Approach (tailored and non-tailored versions) Davidovich et al.102 | The Netherlands | Single gay men | Not stated | Content was based on past research on determinants of sexual risk behaviour in steady relationships and on the intervention’s theory of change. To address concern that the impact would be limited by messaging that was too lengthy, the tailored version was designed to address user-specific needs | There were two versions of this online HIV prevention intervention promoting negotiated safety (i.e. unprotected anal intercourse between steady partners of a concordant negative HIV status). A non-tailored version delivered all modules, and a tailored version delivered general content considered relevant for all users in addition to selected modules considered relevant based on a baseline questionnaire. Information modules addressed how to practise negotiated safety; motivation modules addressed HIV transmission risk via steady partners, HIV testing and sexual agreements and stressed the consequences of HIV infection; and skills modules taught skills for negotiated safety | Users spent an estimated mean time of 30 minutes in the non-tailored version and 10–30 minutes in the tailored version | Non-interactive other than baseline questionnaire used in the tailored version | Internet |
| Gay Cruise Kok et al.105 | The Netherlands | MISM | Website operated by a lesbian and gay health service provider organisation in Amsterdam; intervention linked to and promoted by a popular MSM e-dating website | Used ‘intervention mapping’, a systematic method for developing interventions informed by evidence, theory and stakeholder input. Steps included conducting a needs assessment, establishing programme objectives and translating relevant theoretical models into intervention strategies that determined programme content. Materials were piloted with 15 professionals and 15 members of the target population | The Gay Cruise was an online interactive simulated cruise ship. Content included four ‘trips’ on the cruise ship and a follow-up quiz. Mimicking human face-to-face interaction via facial expressions, speech and tone, characters engaged in scripted, tailored dialogue with the user. Users selected a personal guide from among four virtual characters designed to be attractive. Intervention content (including videos, activities and virtual conversations) addressed HIV, partner communication and condom use. Users could tailor some content, for example by choosing either a ‘hot’ or ‘love’ movie to watch | Users could stop and continue at any point | Interactive | Internet |
| HealthMindr Sullivan et al.118 and Jones et al.128 | USA, with location varying by study: Atlanta, GA, and Seattle, WA;118 and Atlanta, GA, Jackson, MS, and Washington, DC128 | MSM | Not stated | Separate focus groups with MSM, HIV testing counsellors and key informants to identify preferences and requirements fed into an initial version of the app, which was then tested with focus group discussions. These phases fed into the beta version of the app | This multifeature mobile HIV prevention app included monthly risk-assessment quizzes followed by tailored HIV prevention recommendations; PrEP and nPEP resources including self-assessments; map locations and details of HIV testing locations; resources to create a HIV testing plan with reminders; ordering of free condoms and at-home HIV test kits; substance use and mental health screeners and directory; HIV treatment locator; health insurance resources; a frequently asked questions section; and a tool to submit questions to study staffa | Participants were asked to keep the app on their phones for 4 months118 or for an unspecified amount of time during a 12-month study128 | Interactive | Smartphone/mobile app |
| Hot and Safe M4M (website name) Carpenter et al.100 | USA | YMSM, including minority MSM | Not stated | Pilot tested with a sample of 21 MSM in New York City using a desktop computer at a community organisation specialising in HIV research and intervention development. Minor content revisions were made based on findings | This website-based intervention aimed to reduce HIV/STIs via modules addressing information about risk factors, skills (e.g. partner communication) and motivation. Multimedia content included didactic materials, quizzes, interactive exercises and audio from simulated peers. The approach was non-judgemental and emphasised both responsibility and freedom of choice. User assessments informed motivational exercises tailored to the user’s readiness to change, as well as tailored feedback | Seven brief sequential modules completed within 1 week. Authors’ description suggests intervention took approximately 1.5 hours | Interactive | Internet |
| Internet-based safer sex intervention (no name) Milam et al.109,110 | USA (southern California) | HIV-positive MSM | Three clinic sites that were part of a HIV clinical research network | Messages were adapted from an existing intervention effective in reducing unsafe sex. Subsequent pre testing in focus groups with HIV-positive MSM informed changes to content and approach | This intervention aimed to reduce HIV/STI transmission by HIV-positive MSM by targeting condom use, disclosure to sex partners, antiretroviral therapy initiation, and reduced use of drugs and alcohol. Based on their responses to monthly sexual behaviour surveys, users were directed to static web pages tailored to their risk of transmission. Tailored messaging took into account each user’s current behaviour and intent related to the targeted behaviour change | Brief intervention provided monthly for 1 year | Monthly risk assessment surveys informed other content, which was not interactive | Internet |
| Keep it Up! Mustanski et al.,112,113,122 Greene et al.103 and Madkins et al.130 | USA, with location varying by study: Chicago, IL;103,122 recruited from Atlanta, GA; Chicago, IL; and New York, NY; and via local and national advertising112,113,130 | Ethnically and racially diverse YMSM | Not specified. Some participants were recruited from community-based organisations providing HIV testing and counselling | Developed in partnership with community-based organisations providing HIV testing to the LGBT community and with engagement of diverse YMSM, and informed by formative mixed-methods research | Multimodule HIV prevention intervention for YMSM, with content designed to appeal to users from all racial and ethnic groups. Online modules were based on situations and settings relevant to YMSM, and used a variety of media and methods such as video, animation and games. Modules addressed, among other topics, condom use; triggers for unprotected sex; obtaining support; communication; the effects of mood, drug and alcohol abuse and sexual arousal; power dynamics in relationships; and the limits of serosorting. Users developed a HIV/STI prevention plan, and goals were suggested tailored to users’ baseline risks. In the intervention’s first iteration, a booster session revisited goals and provided tailored feedback to address obstacles and set new or reaffirm existing goals.122 In the second iteration, two booster sessions reinforced learning, introduced new skills and provided an opportunity to review earlier goals112,113 | Seven modules completed across three sessions. In the first iteration, these took 2 hours and were followed by a booster session at 6 weeks.122 In the second iteration, modules were followed by booster sessions at 3 and 6 months, with either the initial seven modules112 or the full intervention113 lasting 2 hours. In Mustanski et al.,112,113,122 and Madkins et al.,122,130 but not in Greene et al.,103 modules had to be done at least 24 hours apart | Interactive | Internet |
| MOTIVES Linnemayr et al.106 | USA (Los Angeles County, CA) | Latino/a MSM and transgender women | Recruitment via HIV testing sites of a community-based organisation offering programming and services primarily to the Latino/a population, including LGBT community and people living with HIV | Approach was informed by formative focus groups | This text message-based HIV prevention intervention drew on behavioural economics to optimise engagement and knowledge retention. Weekly, the user received a text message providing HIV prevention information, followed by a message 2 days later asking a question about that information. After sending their response, the user received a message indicating whether or not they were correct and providing a link with more information. A correct response increased their chance of winning an upcoming prize draw. Users also received HIV testing reminders via text message every 2.5 months | Information-related text messages weekly for 1 year; four prize draws (one every 3 months); and a HIV testing reminder every 2.5 months | Interactive | Text messaging |
| myDEx Bauermeister et al.97,123 | USA | Young adult MSM aged 18–24 years | Not stated | A sociodemographically diverse youth advisory board of three YMSM provided input on content and delivery and trained developers on same-sex attraction and YMSM dating behaviours | This module-based comprehensive sex education intervention aimed to improve psychological well-being and reduce HIV risk by targeting condom use, HIV/STI testing, unprotected anal intercourse, PrEP and alcohol/drug use before sex. Content within each session was organised into three levels: a core message, deeper discussion of relevant topics and an activity. Content used storytelling, case scenarios, motivational interviewing strategies, graphics and videos, and it was tailored to the user via personalisation, content matching and feedback to maximise persuasiveness and relevance. Interactive activities included role-play scenarios, a diary, quizzes and opportunities to develop dating strategies | Six sessions, each lasting 10 minutes | Interactive | Internet |
| MyPEEPS Mobile Kuhns et al.129 | USA | Young sexual minority men aged 13–18 years | Not stated | Adapted to e-health version for younger and more ethnically diverse users from effective group-based HIV prevention curriculum, featuring characters based on YMSM who took part in the formative phase of the development of the group-based intervention. Mobile version was previously tested for feasibility, acceptability and usability | Designed for less sexually experienced users, this interactive, modular HIV prevention intervention aimed to reduce sexual risk and promote health behaviours. Delivered via games, scenarios and role playing, and facilitated through the stories of four ‘peeps’ (YMSM characters), content addressed information on HIV/STIs among YMSM, minority stress, condom use, emotional regulation and negotiating interpersonal and substance-related risks. ‘Bottom Line’, a goal-setting activity running throughout the intervention, asked users to establish and regularly reconsider their limits and the risks they are willing to accept for different types of sexual acts | 21 activities divided into four sequential modules, accessible throughout 3-month period | Interactive | Internet |
| Online mindfulness-based cognitive therapy (no name) Avellar96 | USA | Same-sex attracted men with a range of bullying experiences during grade school and high school | Not stated | Modelled on an existing mindfulness-based cognitive therapy protocol for depression and anxiety symptoms, using that programme’s audio files and handouts, and with much of the intervention content drawn from the mindfulness-based cognitive therapy trainers’ manual | Module-based intervention integrating mindfulness and cognitive–behavioural techniques to improve mental health. Each weekly session began with an audio introduction and an outline of the session. Sessions introduced practices and skills to alleviate unpleasant thoughts, feelings and situations. Content included lectures, activities (including meditations), exercises, handouts, weekly homework assignments and both audio and video content, and directed participants to resources in the public domain, including a mindfulness meditation by a private psychologist | Eight weekly sessions, each lasting approximately 50–90 minutes | Interactive | Internet |
| People Like Us Tan et al.131 | Singapore | HIV-negative gay, bisexual and queer men aged 18–29 years | A community-based organisation developed the web series and was involved in the trial | Intervention includes videos from a previously launched web series developed by a community-based organisation | This educational, web-based drama miniseries featured stories of six ethnically and socioeconomically diverse gay, bisexual and queer men negotiating sexual health, mental health and relationship issues. Each video ended with community-based organisation representatives who summarised the episode and addressed key points on mental and sexual health. Although the planned evaluation focused on HIV/STI testing, key sexual health messages also addressed HIV/STI risk, homophobia and coming out, and safer sex negotiation and behaviours (including modelling safer sex behaviours) | Six videos, each lasting approximately 10 minutes, watched over 1 week | Non-interactive | Internet |
| Queer Sex Ed Mustanki et al.111 | USA | LGBT youth | Not stated | Informed by mixed-methods research | Multimedia, comprehensive sexual health curriculum including STI prevention. Comprised an introduction and five modules, moderated by a female-bodied avatar called ‘Ed’ who introduced the programme and provided a brief overview at the start of each module. Modules used varied media formats, each ending with a quiz. Content covered understanding and accepting one’s sexual orientation and gender identity, sexuality education (e.g. pleasure, anatomy and STI risk), healthy relationships, safer sex, and sexual health improvement goal-setting | Five modules worked through at the user’s own pace. Users could save their work and log back in at another time to continue. The intervention took a mean of 107.8 minutes to complete | Interactive | Internet |
| Rainbow SPARX Lucassen et al.107,108 | New Zealand (Auckland) | Sexual minority youth with depressive symptoms | Participants could complete the programme at home, at a youth-led organisation promoting the study, at a selected secondary school or on a dedicated computer at the research centre where the study was based | Rainbow SPARX was an adaptation for LGBT youth of the computerised CBT programme SPARX. Researchers and clinicians worked collaboratively with young people to develop SPARX, with young people’s feedback informing refinement and further improvement of prototypes. Separate consultations with sexual minority youth suggested the need for a specially adapted version for this population, which became Rainbow SPARX. Tailored content addressed issues and experiences of particular relevance to sexual minority youth. Changes were primarily script-related (accounting for 5.9% of the overall script) and included some changes to appearance. Mini-games, characters and homework tasks were unchanged | Computerised CBT programme to reduce depressive symptoms, designed as a multilevel game. Using interactive exercises and attractive graphics, the intervention presented the user’s avatar with challenges set in a fantasy world from which they had to eradicate gloom and negativity. Following an introduction and information about depression from a guide character, the user’s avatar entered each module and completed a mission. The guide then explained its relevance to real life, and homework tasks were set. Modules introduced CBT skills, each represented as a gem the user found and added to their ‘shield against depression’. CBT skills introduced included ‘relax’ (relaxation training), ‘do it’ (e.g. behavioural activation), ‘sort it’ (e.g. social skills training), ‘spot it’ (recognising or naming cognitive distortions), ‘solve it’ (problem-solving) and ‘swap it’ (e.g. cognitive restructuring) | Each of the seven modules took approximately 30 minutes. Users were instructed to complete 1 or 2 modules per week and to complete all within 2 months | Interactive | Computer (CD-ROM), with paper-based user notebook |
| Role-playing game Coulter et al.126 | USA | Sexual and gender minority youth aged 14–18 years | Not stated | Informed by interviews with 20 sexual and gender minority youth about gaming preferences, and developed in collaboration with professional developers of educational games | This computer game aimed to improve the health of bullied sexual and gender minority youth by improving help-seeking and productive coping strategies and reducing substance use, victimisation and mental health issues. The user played a customisable character who builds a team with non-playable characters to defeat robots in the ‘Holochamber Challenge’. The user was tasked with helping each non-playable character with challenges they faced such as bullying, confidence or anger; finding support or resources for them; and helping them to use these. If successful, the character joined their team. At the end of the game, positive endings were presented for those characters whom the user successfully helped and negative endings (e.g. getting into a fight) were presented for the others. The user could replay and receive hints to help the characters they had not successfully helped | Participants received a link to download the game following the baseline survey and could play an unlimited number of times. Questions about the intervention were presented at the 4- or 8-week follow-up, depending on when a participant reported having played the game (4 or 8 weeks) | Interactive | Computer download |
| Safe behaviour and screening Chiou et al.125 | Taiwan (Province of China) | MSM | Not stated | Informed by existing qualitative literature and by formative research. The latter included interviews with 10 MSM to inform initial development, then recommendations for refinement from five MSM and four experts | HIV prevention app with the following features: (1) log to record sexual behaviour and recreational drug use, which can inform output tables/figures showing changes over time, and links to PrEP resources; (2) information on HIV/STIs, safe sex strategies including partner communication, recreational drug use including alternative strategies to enhance arousal before sex, and PrEP; (3) recommendations, links and a log to promote and record testing; (4) search, messaging and message board to interact with other users; and (5) presentation of most popular users, message boards and testing locations | App was used for 6 months; quiz and prize activity related to HIV testing, safe sex and drug use were conducted every 3 weeks | Interactive | Smartphone/mobile app |
| Sex Positive! Hirshfield et al.104,127 | USA | MSM living with HIV | Not stated | Core intervention videos were newly produced, based in part on videos from the earlier HIV Big Deal project, which showed effectiveness in reducing instances of CAI. Content was informed by a community advisory committee. A video from the HIV Big Deal project was edited to create three booster videos; the fourth booster video came from a video-sharing website | Sex Positive! aimed to prevent onward HIV transmission among MSM living with HIV, and targeted treatment adherence, mental health, substance use, sexual behaviour and interpersonal violence outcomes. The intervention’s dramatic video series ‘Just a Guy’ followed ‘Guy’, a gay man living with HIV in Brooklyn, New York. The intervention used modelling to demonstrate risk reduction and health behaviours, including HIV disclosure and discussions about safer sex. Four follow-up booster videos, featuring dramatised vignettes showing HIV-negative MSM asking and disclosing STI status in realistic situations, and a segment on social support for people living with HIV, aimed to help sustain the intervention’s impact over time | Six videos, delivered weekly for 3 months, and four booster videos delivered weekly starting at 6 months. Full intervention was delivered over a 1-year period | Not interactive | Internet |
| Sexpulse Rosser et al.116 and Wilkerson et al.120 | USA | MISM | Not stated | Designed by health professionals, computer scientists and e-learning specialists, and developed by an e-learning company. Sexpulse was informed by formative research with 2716 MISM (recruited online) and developed by adapting an existing MSM sexual health curriculum from a seminar to an online setting. Module prototypes were reviewed by experts, tested with MISM and refined | Sexpulse was a flexible, modular HIV prevention intervention that aimed to reduce instances of unprotected anal intercourse. It incorporated video segments, interactive text and animations. Examples of modules included a ‘hot sex calculator’ demonstrating decision-making, a virtual gym where users could explore body image concerns, an online chat simulation to explore evasive and ambiguous communication and a ‘reflective journey’ exploring past experiences, long-term goals and spirituality. The intervention addressed a range of topics including mental, emotional and physical health; intimacy; relationships; sexuality; and spirituality. Modules were supplemented with frequently asked questions, virtual peers sharing their experiences, cartoons and interactive polls | Multimodule intervention completed over a 7-day period | Interactive | Internet |
| Smartphone self-monitoring (no name) Swendeman et al.119 | USA (Los Angeles, CA) | People living with HIV | Did not specify a provider organisation for intervention delivery, but seemed to be the research team | Not stated | Self-monitoring intervention to support self-management in medication adherence, mental health, substance use and sexual risk behaviours. Users completed smartphone-based self-monitoring surveys daily (alcohol, tobacco and other drug use, sexual behaviours and medication adherence) and four times per day (physical and mental health), with reporting on stressful events and text diary entries at any time. Customisable alarms prompted users to fill in surveys and users could access a web-based visualisation tool to view their survey responses over time and by location, as well as to view associations between variables | Self-monitoring daily and four times per day, with reporting on stressful events and text diary entries at any time. Intervention duration was not specified, but the last follow-up assessment specified took place at 6 weeks | Interactive | Smartphone/mobile app |
| SOLVE Christensen et al.101 | USA | Young adult MSM | Not stated | This interactive, media-based intervention was informed by an approach previously developed and tested by a co-author of the included report and their colleagues, and was delivered in SOLVE as a downloadable three-dimensional animated game. Content was based on qualitative and quantitative pilot studies | SOLVE aimed to decrease instances of unprotected anal intercourse, thereby reducing HIV risk. In this three-dimensional animated game, the user took the role of a customisable avatar and made decisions that affected the narrative in simulated settings presenting risky situations and barriers to safer sex that young adult MSM typically confront on first dates or ‘hook-ups’. The intervention simulated shame-inducing situations, and the avatar and other guide characters modelled acceptance and normalisation of the user’s desires. At decision points, these characters used an ‘ICAP’ process involving: ‘(I) interrupting automatic risky choices, (C) challenging those choices with persuasive messages, (A) acknowledging, accepting and sharing MSM’s emotions/motives (e.g. desires for men) and (P) providing a way and skills for MSM to be safe’101 | 30 minutes | Interactive | Computer download |
| TXT-Auto Reback et al.114,115,132 | USA (Hollywood area of Los Angeles County, CA) | Out-of-treatment methamphetamine-using MSM | Research activities took place at a community research centre with a long history of working with methamphetamine-using MSM | Pilot research identified peak times for high-risk activities. Text messages were written in collaboration with community/peer focus groups. A mobile-health development company programmed the text messaging software and hosted the system | TXT-Auto aimed to reduce substance use and HIV risk by decreasing methamphetamine use and instances of sex during methamphetamine use and CAI. A baseline survey assessed the user’s risk profile in relation to HIV status, antiretroviral therapy adherence, drug use and sexual behaviours. Users then received five automated, unidirectional, scripted text messages per day, which included both general messages and messages tailored to their risk profile. A brief weekly text-based assessment asking about methamphetamine use and HIV sexual behaviours in the past 7 days aimed to increase self-monitoring | Five messages per day for 8 weeks, delivered at peak hours of high-risk activities (Monday and Tuesday 12.00–20.00, Wednesday and Thursday 12.00–01.00, Friday 12.00–02.00, Saturday 15.30–02.00, and Sunday 15:30–00.00). Weekly self-monitoring assessments | Baseline assessment and weekly self-monitoring assessments were interactive | Text messaging |
| WRAPP (and linked interventions)b Bowen et al.,98,99 Williams et al.121 and Schonnesson et al.117 | Varied by study: USA (rural areas)98,99,121 and Sweden117 | Varied by study: sexually active, internet-using MSM117 in rural areas98,121 | Not stated | Content was identified from focus groups and from a web-based assessment. Intervention format was informed by two additional focus groups. The Swedish adaptation117 was informed by 20 in-depth interviews with Swedish MSM (HIV positive and HIV negative) and a presentation of the intervention to professionals from HIV prevention and treatment organisations. Information tailored to the Swedish context was reviewed by an experienced HIV physician | Online modular HIV risk reduction intervention with informational content tailored for rural MSM and presented as conversations between gay men. Dialogue was interspersed with interactive activities and graphics. The first module primarily addressed HIV prevention during sex and living with HIV, and it featured links to informational websites. Author descriptions suggest the second module changed across iterations. It initially98 focused on maintaining a HIV-negative status and addressed safer sex and types and correct use of condoms. In subsequent iterations,99,117,121 this module aimed to increase motivation, and a third module targeting behavioural skills was introduced. Both allowed users to print a summary of their responses to interactive components. The ‘motivation’ module helped users identify reasons for not using condoms and ways to address these to support the user’s pursuit of their life goals. The ‘behaviour’ module addressed approaches for reducing sexual risk with partners met online or in a bar. A version adapted for Sweden117 used the Swedish language, was tailored to Swedish health services, and added to the ‘knowledge’ module information about STIs and Swedish public health legislation | Initially98 included two 20-minute modules completed at least 24 hours apart; users were encouraged to complete all within 7 days. Subsequently,99,117,121 three modules each contained two 20-minute sessions. Initially, modules had to be completed at least 48 hours apart and results found that users took an average of 19.39 days to complete them all.99 Subsequently,121 each module had to be completed within 14 days; and later,117 sessions had to be completed 24–48 hours apart | Interactive | Internet |
Years of report publication
Reports were published between 2006 and 2020, with more than two-thirds (26/37)96,97,103,104,106–108,110–115,117–119,123–132 published from 2015 onwards (Figure 2).
FIGURE 2 Number of included reports, by year of publication. (Figure not reproduced here; see the original article.)
Targeted health outcomes
In total, 20 interventions addressed sexual health outcomes,97–106,109–125,127–132 10 addressed substance use97,103,104,106,109,110,112–115,119,122,123,125–127,129,130,132 and seven addressed mental health.96,97,104,106–108,119,123,126,127 Thirteen addressed a single health outcome of interest for this review,96,98–102,105,107,108,111,116–118,120,121,124,128,131 six addressed two of these outcomes103,109,110,112–115,122,125,126,129,130,132 (with five addressing sexual health and substance use,103,109,110,112–115,122,125,129,130,132 and one addressing mental health and substance use126) and four addressed all three outcomes.97,104,106,119,123,127
Intervention development
Most interventions were informed by formative research, consultations and/or pilot testing with participants drawn from the target population (see Table 3). For example, gay community representatives provided the scenarios presented in the China–Gate HIV Prevention Program Online Intervention for MSM,124 and development of myDEx was informed by input from a sociodemographically diverse youth advisory board of three young MSM.97 Gay Cruise developers used intervention mapping, a systematic method for developing interventions informed by evidence, theory and stakeholder input, and piloted materials with 15 professionals and 15 members of the target population.105 The HealthMindr mobile app was informed by separate focus groups with MSM, HIV testing counsellors and key informants, then tested in focus group discussions.118 Several interventions were informed by or adapted from other existing interventions. For example, the online mindfulness-based cognitive therapy intervention was modelled on an existing mindfulness-based cognitive therapy protocol and drew much of its content from the materials and trainers’ manual from that programme,96 and Rainbow SPARX was an adaptation for lesbian, gay, bisexual and transgender (LGBT) youth of the computerised cognitive–behavioural therapy (CBT) programme SPARX.108
Intervention timing and duration
Interventions varied in the timing, intensity and duration of delivery (see Table 3). For example, TXT-Auto involved text message delivery five times per day over an 8-week period,114,115,132 whereas Gay Cruise involved downloading a game that users could play at their own pace over an indefinite period of time.105 Three interventions were delivered via smartphone/mobile apps,118,119,125,128 which participants were to use over periods ranging from 6 weeks118 to 1 year.128 The content of several interventions was delivered weekly over a defined period of time,96,104,106,124,127 ranging from 3 weeks (for part II of the China–Gate HIV Prevention Program Online Intervention)124 to 1 year [for the Mobile Technology and Incentives (MOTIVES) intervention].106 When authors reported on the expected or actual amount of time taken for users to work through defined programme content, this ranged from 10 minutes for the tailored version of the Cognitive Vaccine Approach to 3.5 hours for Rainbow SPARX.107,108
Target populations
All included studies met our inclusion criteria stipulating that MSM account for 50% or more of the sample. Of the 23 included interventions, three targeted LGBT or sexual minority youth (aged 13–19,107,108 14–18126 and 16–20 years111) or young men (aged 13–18 years);129 four targeted young adult MSM aged 18–2497,101,103,112,113,122,123,130 or 18–39 years;100 one targeted HIV-negative young adult gay, bisexual and queer men (aged 18–29 years);131 three targeted MSM104,109,110,127 or people more generally119 living with HIV; two targeted single gay men;102 four targeted MSM/same-sex attracted men generally;96,118,124,125,128 two targeted men who use the internet to seek sex with men;105,116,120 one targeted sexually active, internet-using MSM,117 with an earlier iteration targeting this population in rural areas;98,121 one targeted Latino/a MSM and transgender women;106 and one targeted out-of-treatment methamphetamine-using MSM.114,115,132
Geographical location of studies and implementation
Three-quarters (21/28) of included studies took place in the USA.96–101,103,104,106,109–116,118–123,126–130,132 Two studies (7%) took place in the Netherlands,102,105 and one (4%) took place in each of the following: China,124 New Zealand,107,108 Singapore,131 Sweden117 and Taiwan (Province of China)125 (Figure 3).
FIGURE 3 Spread of studies by country. (Figure not reproduced here; see the original article.)
Intervention mode of delivery and interactivity
The 23 included interventions were delivered by a variety of electronic methods (Figure 4). Most were delivered via the internet (n = 15, or 65%),96–100,102–105,109–113,116,117,120–124,127,129–131 with others split across the following platforms: smartphone/mobile app (n = 3, or 13%),118,119,125,128 text messaging (n = 2, or 9%),106,114,115,132 computer download (n = 2, or 9%)101,126 and CD-ROM (n = 1, or 4%).107,108
FIGURE 4 Electronic mode of delivery, by intervention. (Figure not reproduced here; see the original article.)
The vast majority of interventions (20/23, or 87%) were interactive,96–101,103,105–108,111–113,116–123,128,130 a further three (13%) used participant assessments to inform the delivery of non-interactive content102,109,110,114,115,132 and the remaining three (13%) were not interactive.102,104,127,131