About this chapter
Parts of this chapter have been reproduced or adapted with permission from Meiksin et al.1 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes additions and minor changes to the original text.
Reports included in the theory of change synthesis
Thirty-three reports were included in the theory of change synthesis, representing 28 unique studies of 23 interventions.96–106,108,110–113,115–128,130,131 Seven reports were study protocols.97,104,106,112,126,128,129 Two interventions (a tailored and a non-tailored version of the Cognitive Vaccine Approach, both reported by Davidovich et al.102) shared a single theory of change, resulting in 22 unique theories of change included in the synthesis.
For each intervention theory of change, summaries of that theory of change, the evidence supporting it, how it was developed and the existing scientific theories on which it draws are provided in Appendix 11.
Quality of studies
Agreement between independent reviewer assessments on each theory report was good, ranging from agreement on none of the five quality criteria in one case120 to agreement on all five in 15 cases97,99,101,102,107,110,113,115,117,119,123–125,127,131 with agreement on four or five criteria for more than three-quarters of reports (n = 26; 79%).97,99,101–104,107,110,112,113,115–119,121–131 See Appendix 12 for quality assessment results for each theory report. All disagreements on independent quality assessments were resolved by discussion.
Quality varied notably across theory reports and clustered towards low and medium quality, with 14 reports (42%) meeting none or one of the criteria assessed,110,112,113,115,116,118,120–122,125,127,129–131 14 (42%) meeting two or three criteria96,98–100,103–106,108,111,117,124,126,128 and only five (15%) meeting four or all five criteria.97,101,102,119,123
Quality varied across the five criteria assessed. Nearly three-quarters of theory reports (n = 23; 70%) clearly defined the constructs or concepts that made up the theory of change,96–99,101–105,107,111,117,119,121–126,128–131 and about half described a pathway from intervention components to intended outcomes (n = 16; 48%).96,99–107,117,119,123,124,126,128 Slightly more than one-third clearly described how theoretical constructs were inter-related (n = 12; 36%).97–102,111,117,119,123,126,128 Only four (12%) described biological, psychological or social processes or mechanisms underlying these inter-relationships, and the same number described how mechanisms and outcomes might vary by context.97,102,106,123
Scientific theories informing intervention design
Authors cited a number of existing scientific theories informing intervention theories of change. Interventions often drew on more than one such theory. The information–motivation–behavioural skills (IMB) model and social cognitive theory were the most commonly cited, with the former informing seven interventions97,99,100,102,111–113,117,121–123,125,130 and the latter informing eight.98,101,104,105,110,115,118,126–128 The IMB model was initially developed to inform HIV prevention, but has since been advanced as a model for conceptualising individual and social determinants of health behaviours more broadly.133 Social cognitive theory incorporates individual, social and structural factors shaping health behaviours, and techniques for changing health behaviours.134 One report citing social cognitive theory also cited its predecessor,104 social learning theory,135 and one report cited the social–personal framework, which the authors described as building on social learning theory.129 In addition, the smartphone self-monitoring intervention was underpinned by the theorised role of self-monitoring in supporting self-management,119 and we noted that self-monitoring is a core construct of social cognitive theory.136
Other scientific theories of behaviour informed between one and three interventions each: the health belief model,102,115 which focuses on the role of individual beliefs about health problems;137 the theory of planned behaviour,101,102,124 which takes into account individual and social factors and theorises that intentions and perceptions of behavioural control are the direct precursors to behaviour;138 and social support theory,115 which theorises that support or perceptions of support from people who are trusted can reduce the stress of, and improve the ability to cope with, difficult events.139 Content for the Sexpulse intervention was informed by the sexual health model, which identifies 10 components essential to healthy sexuality,140 theorising that sexually healthy persons are more likely to make sexually healthy decisions.116 In addition to social cognitive theory, the role-playing game intervention126 was informed by stress and coping theory and by the emotional learning framework. In regard to bullied youth, the former posits that youths’ appraisals of their experience predict their coping strategies, with those who blame themselves, perceive little control or view bullying as a threat instead of a challenge tending to use non-productive coping strategies.126 The emotional learning framework specifies four health-promoting competencies, described as ‘awareness of self and others, responsible decision making, positive attitudes and values, and social interaction skills’.126
The MOTIVES intervention was rooted in behavioural economics,106 which examines how actors make decisions other than via conscious reasoning. Two interventions were underpinned by theory on interactions between cognitive and emotional factors: myDEx was informed by a ‘dual processing, cognitive–emotional decision making framework’,97 which recognises that affective states (emotions) and cognitive states (thinking) both influence decision-making, and that the former might be processed more quickly than the latter. The Socially Optimized Learning in Virtual Environments (SOLVE) intervention was informed by learning from neuroscience that highlights the important role that emotions play in decision-making.101
Scientific theories of how behaviour might be actively changed were also cited, such as the transtheoretical model,105,110 which maps stages and processes by which people change their behaviours.141 The Hot and Safe M4M intervention incorporated strategies from motivational interviewing by assessing, and delivering exercises based on, participants’ ‘readiness to change’.100 Two interventions were rooted in CBT, which examines inter-relationships between cognitions and behaviours and how these may be modified: the online mindfulness-based cognitive therapy intervention integrated mindfulness and CBT approaches96 and the Rainbow SPARX intervention delivered CBT using a computer game format.108
Some study reports suggested potential synergies when interventions were informed by multiple complementary theories. For example, the Cognitive Vaccine Approach drew on elements of the theory of planned behaviour and the health belief model to operationalise the motivation construct in the IMB model,102 while Reback et al.115 noted that the theories informing message content in the TXT-Auto intervention complement each other.
As evidence to support intervention components and theories of change, theory reports often cited evaluations of earlier iterations of the current intervention or of similar interventions, or previous research on the scientific theories in question. When authors discussed how the intervention theory of change was developed, these discussions were commonly informed by formative research,105,111,115 existing interventions,96 literature on the needs of the target group,102,105 components of the scientific theories underpinning the intervention102,108,115 or a combination of these.102,105,115
Theory of change synthesis
All intervention theories of change identified intended intervention components, mediators and outcomes that could be incorporated into an intervention-specific theory of change diagram, except for the theory underpinning the internet-based safer sex intervention,110 which did not identify mediators. Two theories of change also identified participant characteristics theorised to moderate the relationship between the intervention and its intended outcomes.97,106,123
Our grouping of theories of change inductively, based on their key constructs, resulted in three groups of intervention theories of change. The largest, the ‘cognitive/skills’ theory of change grouping, was informed primarily by social cognitive theory and the IMB model.97–100,102–105,111–113,117,118,121,122,124,125,129,131 The second grouping drew on two intervention theories of change that were driven primarily by self-monitoring (the ‘self-monitoring’ theory of change grouping),115,119 and the third drew on two intervention theories of change that were based on cognitive therapy approaches (the ‘cognitive therapy’ theory of change grouping).96,107 Five intervention theories of change did not fall within any of these three inductive groupings.101,106,110,116,120,126 The ‘cognitive therapy’ theory of change grouping comprised only mental health interventions. The other two theory of change groupings were not associated with particular intended outcomes.
The intervention typology developed at an earlier stage of the review (see Chapter 4) categorised included interventions into two overarching categories: ‘time-limited/modular’ (guiding participants sequentially through intervention content from beginning to end) and ‘open-ended’ (not designed as fixed and sequenced bodies of learning that all participants were intended to work through), each of which contained two or three intervention subtypes. Most time-limited/modular interventions (n = 12, 71%) used intervention theories of change that fell within the ‘cognitive/skills’ theory of change grouping.97–100,102–105,111–113,117,121–124,127,129–131 Theories of change underpinning open-ended interventions were more varied: among the two interventions in which content was organised by assessment, one fell within the ‘self-monitoring’ theory of change grouping114,115 and the other did not fall within a grouping.109,110 Among the four interventions delivering general content, two fell within the ‘cognitive/skills’ theory of change grouping,118,125,128 one fell within the ‘self-monitoring’ theory of change grouping119 and one did not fall within any grouping.106 Appendix 13 shows the theory of change grouping and targeted outcome(s) for each intervention included in the review, categorising interventions by type.
To demonstrate the methodological process of synthesising individual intervention theory of change diagrams to develop overarching theory of change diagrams, Appendix 8 presents the individual and synthesised theory of change diagrams for one inductive grouping of theories of change.
‘Cognitive/skills’ synthesised theory of change
We drew on 13 intervention theories of change, which varied in quality from low to high, to develop an initial diagram for the ‘cognitive/skills’ grouping (Figure 5).97–100,102–105,111–113,117,118,121–125,127–131 These theories of change underpinned 14 interventions: China–Gate HIV Prevention Program Online Intervention,124 Cognitive Vaccine Approach [(1) tailored and (2) non-tailored versions],102 Gay Cruise,105 Keep it Up!,103,112,113,122 HealthMindr,118 Hot and Safe M4M,100 myDEx,97 MyPEEPS Mobile,129 People Like Us,131 Queer Sex Ed,111 Safe Behaviour and Screening,125 Sex Positive!104 and WRAPP.98,99,117,121
FIGURE 5 ‘Cognitive/skills’ synthesised theory of change. MH, mental health; SH, sexual health; SU, substance use. (Figure not reproduced here; see the original article.)
Ten of the 13 intervention theories of change in this grouping referenced social cognitive theory98,99,104,105,117,118,121,127,128 and/or the IMB model,97–100,102,103,111–113,117,121–123,125,130 often in combination with other scientific theories. Three referenced the theory of planned behaviour102,124,129 and one referenced no existing scientific theories, but shared key constructs with other intervention theories of change in this grouping.131 Although other interventions also drew on these scientific theories, the intervention theories of change in this grouping shared, as core components, constructs that are key to these three scientific theories. All interventions represented in this grouping targeted sexual health outcomes, either alone or in combination with substance use or both substance use and mental health. Of the 12 time-limited interventions in this grouping, guiding participants sequentially through intervention content from beginning to end, seven were online modular interventions,97–100,103,105,111–113,121–124,129,130 four were non-interactive interventions102,104,127,131 and one was a computer game.105 The remaining two interventions in this grouping were open-ended and of the ‘general content’ subtype (i.e. with content not organised by an assessment).118,125,128
The MOTIVES intervention106 and the internet-based safer sex intervention110 theories of change, assessed as being of medium and low quality, respectively, were primarily rooted in different approaches or scientific theories and were therefore not included in the ‘cognitive/skills’ grouping. However, these theories of change included components both overlapping with and complementing those in the ‘cognitive/skills’ grouping. Using the line-of-argument approach, we therefore drew on the theories of change underpinning MOTIVES and the internet-based safer sex intervention to augment findings within this grouping, and these additions (theorised moderators and the provision of information based on assessed stage of change) appear in the synthesised theory of change diagram.
Interventions in this grouping provided information and exercises and incorporated techniques to model, or demonstrate, desired behaviours. Included in the exercises are decisional balance exercises, which weigh the pros and cons of a particular behaviour; this recurred as an intervention activity in theory reports.
These activities were theorised to affect mediators including knowledge, outcome expectancies (i.e. positive and negative expectations of a particular behaviour),134 attitudes (including internalised homophobia),111,131 perceived norms, perceived vulnerability/risk (a construct combining, via reciprocal translation, perceived vulnerability102,103,111–113,122,130 and risk perceptions105,124,131) and behavioural skills. In addition, two interventions whose theories of change informed this grouping aimed to affect connectedness to the LGBT community,111,131 portrayed as a mediator in the synthesised diagram, although the reports were not explicit about how intervention activities aimed to engender this connectedness or how it might be related to sexual health outcomes.
Individual theories of change differed as to what specific activities led to the development of behavioural skills. We inferred that exercises would develop behavioural skills, indicating this as an inference by use of a dotted line in Figure 5, although behavioural skills could be developed through other activities. For example, in the video-based Sex Positive! intervention, characters modelled behaviours and this was theorised to increase self-regulatory skills.104 Theory reports did not describe clear and recurring pathways from modelling to mediators, but we inferred (denoted by use of a dotted line) from reciprocal translation of recurring descriptions that these aimed to promote self-efficacy.104,105
Although information provision was consistently linked to knowledge, theory reports provided no consensus on the activities that were intended to modify outcome expectancies, attitudes, perceived norms and perceived vulnerability. These mediators are shown in Figure 5 by the use of three hollow block arrows, indicating that they stem from intervention activities generally. Knowledge and behavioural skills were described as affecting behaviour either directly or via self-efficacy, so both pathways are shown in the synthesised model.
Outcome expectancies tended to be linked to motivation in theory of change descriptions, although the theorised relationship of motivation to other constructs varied. Outcome expectancies were theorised as a component of motivation in the theory of change for the WRAPP intervention99 and, although the report for the Sex Positive! intervention does not explicitly link these two constructs, the measures used to assess outcome expectancies in that report reflect users’ motivation [e.g. ‘I am more likely to have anal insertive sex (top) without a condom while drinking or high’] and intentions (e.g. ‘I am more likely to use a condom with men who are HIV-negative or of unknown status’).104 The theory of change for the HealthMindr app highlighted outcome expectancies (referred to as ‘outcome expectations’)118 as a key mediator without discussing motivation. Given these differences, we drew on line-of-argument synthesis to include outcome expectancies as a mediator and inferred that it influences motivation in the synthesised diagram, indicating this as an inference by use of a dotted line in Figure 5.
The concept of self-regulation appeared variously as a mediator underpinning different theories of change. It is not included as a distinct construct in the synthesised diagram because its theorised role was not consistent. Self-regulatory skills were specified in the Sex Positive! intervention’s theory of change104 and are subsumed under behavioural skills in the synthesised diagram. Self-regulation was a key mechanism of change for HealthMindr,118 but did not recur in this way in other intervention theories within the theory grouping.
Motivation and self-efficacy both featured prominently in intervention theories of change within this grouping, often on the pathway between distal mediators and behaviour change. Although some reports suggested that motivation and self-efficacy influence each other in one direction or the other,99,100 most did not address their inter-relationship, either portraying the constructs as independent111,112 or including only one.102,104,105,118 The synthesised theory of change diagram, therefore, portrays both constructs as affecting behaviour independently via a line-of-argument synthesis.
Although some intervention theories of change included either motivation124,131 or intentions,98–100,117,121 those that included both variably portrayed their relationship as motivation influencing intentions,97,123 intentions influencing motivation111 or both combined into one construct.102,125 Given these differences, we have combined these constructs via reciprocal translation in the synthesised diagram, in which the combined construct of motivation/intentions is theorised to be engendered by exercises and influenced by outcome expectancies, attitudes, perceived norms and perceived vulnerability.
We interpreted the construct of perceived behavioural control97,102 to be similar to the concept of self-efficacy because the latter was variously described as including an ‘ability to refuse to have anal sex if a condom was unavailable’,99 ‘confidence in practicing safer sex behaviours’122 and/or the ‘ability to avoid the situational temptation to have unprotected sex’.122 Self-efficacy and perceived control were therefore merged via reciprocal translation in the synthesised theory of change diagram.
Theorised moderators of the relationship between intervention activities and outcomes are shown at the top right of Figure 5, and include participants’ acculturation to the local majority culture; sociodemographic characteristics; psychological risk correlates, including mental health and substance use/abuse; and, for sexual health interventions, sexual partner type (casual encounter, romantic interest, or friends with benefits). In a few cases, theory reports discussed the direction of moderation and/or how these might operate. Bauermeister et al.97 theorised that MSM’s ability to enact sexual risk reduction behaviours might vary by partner type and that MSM experiencing stressors related to being a sexual minority, experiencing psychological distress, or using alcohol or drugs might have less behavioural control, limiting the extent to which the intervention might promote their ability to ‘regulate their affective motivations’97 and enact sexual risk reduction behaviours. As noted previously, the theory of change underpinning the MOTIVES intervention was primarily rooted in a different approach and did not sit squarely within this grouping, but included components overlapping with and complementing those in this grouping.106 It was therefore used to further develop this synthesised theory of change via line-of-argument synthesis. Linnemayr et al.106 theorised that participants in the MOTIVES intervention who were more acculturated to the USA and could more easily communicate with health-care providers might face fewer barriers to HIV testing, thus benefiting less from the intervention.
Finally, some interventions in this grouping,100,122 and the two not included in the grouping but augmenting its synthesised theory of change,106,110 aimed to boost intervention impact, for example by recruiting participants at a critical biographical point for behaviour change (e.g. on receipt of a HIV-negative test result in the case of the Keep it Up! intervention),122 delivering content based on a participant’s assessed stage of change100,110 or using prize lotteries to incentivise knowledge retention.106
‘Self-monitoring’ synthesised theory of change
We defined a grouping focused on ‘self-monitoring’ (Figure 6) by synthesising theories underpinning the smartphone self-monitoring intervention,119 an open-ended intervention with general content addressing all three health outcomes examined in our review among people living with HIV (with a theory report assessed as being of high quality), and the TXT-Auto intervention,115 an open-ended sexual health and substance use intervention for out-of-treatment men using methamphetamines, with content tailored by a user’s risk profile (with a theory of change description assessed as being of medium quality). The theory report for the smartphone self-monitoring intervention offered a more detailed outline of the self-monitoring behavioural change pathway;119 this was incorporated into the synthesised theory of change diagram.
FIGURE 6 ‘Self-monitoring’ synthesised theory of change. (Figure not reproduced here; see the original article.)
Both interventions in this grouping asked the user questions about their behaviour to prompt self-monitoring, and both were theorised to be driven by self-monitoring as a key mechanism. Frequency of self-monitoring varied by study and outcome, ranging from four times per day119 to weekly.115 The theory report for the smartphone self-monitoring intervention suggested the need for users to first establish criteria for their desired behaviour, such as personal norms or standards, against which they could assess their actual behaviour.119 However, there was no evidence that this was a separate stage of the intervention; therefore, this stage does not appear in the synthesised diagram.
The synthesised theory of change posits that behavioural questions result in self-monitoring, which prompts reflection in terms of pre-established criteria. This reflection is theorised to result in either self-reward or self-critique, generating enhanced self-regulation, a theoretical mediator of behaviour change.
Although both theory reports included in this grouping suggested that processes of change are more complex than depicted in this synthesised theory of change, neither provided further details on other mechanisms. The theory of change underpinning TXT-Auto suggested that the intervention worked via two non-intersecting pathways, one captured in the ‘cognitive/skills’ synthesised theory of change and one featuring self-monitoring. Arguing that pathways from self-monitoring to behaviour change are underdeveloped, Swendeman et al.’s119 research drew on qualitative data from their study to further explicate the theory of change for the smartphone self-monitoring intervention. This was not included in our synthesis, which focused on a priori intervention theories of change, rather than on theories refined through empirical research.
‘Cognitive therapy’ synthesised theory of change
We drew on theories of change underpinning the online mindfulness-based cognitive therapy intervention,96 an online modular intervention for same-sex attracted men, and Rainbow SPARX,108 a computer game intervention for sexual minority youth, to develop a ‘cognitive therapy’ theory of change grouping. Descriptions of both interventions’ theories of change were assessed as being of medium quality. These interventions were the only two in this review that targeted only mental health outcomes; both interventions drew on cognitive therapy techniques and aimed to reduce depression and improve mental health. CBT skills were at the core of Rainbow SPARX, while the online mindfulness-based cognitive therapy intervention combined mindfulness with cognitive–behavioural techniques. Both interventions aimed to improve emotional health via relaxation training and by addressing negative cognitions (including internalised homophobia), utilising differing pathways for the latter.
The synthesised theory of change diagram (Figure 7) depicts activities and mediators stemming from these two approaches. Activities include CBT and mindfulness-based cognitive therapy. Although we do not detail all activities comprising each of these approaches in the synthesised theory of change diagram, we include key components of each: behavioural training (an element of CBT), an emphasis on prioritising ‘being’ over ‘doing’ (an element of mindfulness-based cognitive therapy)96 and relaxation training (an element of both approaches).
FIGURE 7 ‘Cognitive therapy’ synthesised theory of change. (Figure not reproduced here; see the original article.)
Theory of change descriptions for both interventions referred to recognising, paying attention to or developing awareness of thoughts, feelings and situations, with Rainbow SPARX107 then challenging negative cognitions and the online mindfulness-based cognitive therapy intervention focusing on accepting and ‘letting go’ of these negative cognitions.96 We use differently shaded block arrows in Figure 7 to delineate these distinct pathways. Among the negative cognitions to be addressed, both interventions sought to reduce internalised homophobia and mitigate its effects on health.96,107 The report for Rainbow SPARX suggested that this was addressed by providing information to help promote young people spending more time with those who accepted them and reducing exposure to homophobic bullying.107 The report for the online mindfulness-based cognitive therapy intervention did not make clear how this would be achieved.96 Descriptions of both interventions’ theories of change suggested that the reframing of distressing emotions was theorised as a key mechanism for improving mental health.
Intervention theories of change not included in synthesised models
Although informed by the sexual health model, the theory of change underpinning the online modular sexual health intervention Sexpulse (as described in theory reports assessed as being of low quality) did not specify components or mechanisms and so could not be synthesised.116,120
Four other interventions, with theory reports ranging in quality from low to high, did not fit within any of the three inductive theory of change groupings.101,106,110,126 These interventions cut across the targeted health outcomes considered in this review and represented three of the five categories of the intervention typology: computer games,101,126 ongoing interventions with content organised by assessment109,110 and ongoing interventions with general content.106 Although some components of their theories of change overlapped with the three groupings described above, theories of change for these interventions featured elements that differentiated them from those included in these groupings. The MOTIVES106 intervention was based on behavioural economics and the SOLVE intervention was driven by the theorised role of emotions in decision-making, featuring ‘shame associated with sexual stigma’101 as a key mediator. Drawing on social cognitive theory, stress and coping theory and the social and emotional learning framework, the role-playing game (described in a theory report assessed as being of medium quality) aimed to improve mental health and reduce substance use via use of productive coping strategies with a focus on help-seeking.126 Informed by the transtheoretical model, the theory report (assessed as being of low quality) for the internet-based safer sex intervention110 suggested that the intervention was primarily based on a stages-of-change approach, tailoring content based on responses to monthly sexual behaviour surveys. Although not included in the grouping, the MOTIVES intervention106 theory report (assessed as being of medium quality) and the internet-based safer sex intervention110 theory report informed the ‘cognitive/skills’ synthesised theory of change, as described above.
Feedback from stakeholder consultation on theory of change synthesis
‘Cognitive/skills’ synthesised theory of change
The ‘cognitive/skills’ synthesised theory of change resonated most strongly with stakeholders and was thought to be the best reflection of how face-to-face interventions are currently conceptualised in the UK. Recruitment at a critical point in potential participants’ lives was felt to be important to the success of interventions. Ensuring that interventions met individuals’ needs at various stages of change was also recognised as important, and several stakeholders felt that including prizes would boost engagement.
The mediators and moderators were felt to be well considered and likely to be central to how these types of interventions could be designed and targeted, reflecting much current practice for face-to-face interventions.
‘Self-monitoring’ synthesised theory of change
At the time of the consultation, the draft ‘self-monitoring’ synthesised theory of change diagram included an additional activity component, ‘define criteria (perceived norms, personal standard)’, which was later removed from the final diagram because the intervention theory of change from which it derived did not explicitly include a criteria-setting activity.119 This activity and its relationship to a theorised mediator were shown with dotted lines, denoting that they were inferred from, rather than explicitly described in, the author narrative.119
The ‘self-monitoring’ synthesised theory of change did not align with stakeholders’ views on how best to conceptualise interventions for addressing syndemics among MSM. Stakeholders thought that approaches that relied on this theory of change risked encouraging participants to become focused on self-monitoring and comparing their behaviours with goals implicitly in line with public health and LGBT community injunctive norms more widely. Stakeholders felt that interventions would perform better if the motivation for behaviour change was improvement in well-being, rather than comparison with behavioural norms. Stakeholders were concerned that self-monitoring approaches could amplify existing stigmas surrounding behaviour, especially relating to drug use and sexual activity, with potentially negative impacts on the community as a whole. There was, however, acknowledgement that goal-setting was an important part of existing interventions and should be an integral part of any potential e-health interventions. However, it was felt that this should be a secondary activity or component, rather than the primary focus of an intervention.
‘Cognitive therapy’ synthesised theory of change
The ‘cognitive therapy’ synthesised theory of change was felt to be useful in interventions targeting mental health, but less useful for interventions addressing sexual health and substance use. Stakeholders valued the inclusion of CBT and mindfulness approaches among the intervention theories of change reviewed, which are likely to be recognised by the client populations they serve. There were concerns, however, that this model would not apply to sexual health and substance use, and so an intervention based on these theories of change might focus on mental health at the expense of these other health domains.