This study assessed the feasibility and acceptability of a theory-informed web and app intervention that sought to mobilise social support from a participant’s social network to help them with weight loss goals. In phase 1, we developed the app and website in collaboration with potential users. In phase 2, we completed a feasibility trial. The research questions primarily focus on the progression criteria and related feasibility measures. However, BMI, diet and physical activity were assessed as potential primary outcomes for a full evaluation. Other outcomes included weight, waist circumference, waist-to-hip ratio, quality of life, capability well-being, mental health, social support, motivation, self-efficacy, alcohol use and smoking. Data on resource use, participant-borne costs and cost of the intervention were collected and a process evaluation assessed both the programme theory and the evaluation methods.
The main study findings will be considered below in relation to the study objectives and the trial progression criteria. Key learning points and considerations for a future evaluation will also be discussed.
Summary of main findings in relation to the progression criteria and study objectives
The progression criteria that were agreed with the independent Trial Steering Committee have all been met, with the exception of obtaining a source of funding for the treatment costs, which will be finalised later (Table 39).
| Progression criterion | Method of assessment | Was the progression criterion achieved? |
|---|---|---|
| 1. Are appropriate and effective routes of recruitment available to achieve a powered sample size in a full trial? (Research Objective 2) | Coming close to the sample size, as judged by the TSC, with reasonable expectations of being able to address any recruitment issues | Yes (see Chapter 4, Recruitment, and Chapter 6, Recruitment and retention)Our target sample size was 120 participants. We recruited 109 participants (91% of our target). Reaching our target was affected by a 2-month delay in receiving approval from the NHS Research and Development office. We would most likely would have achieved the remaining participants without the delay. We could also have achieved the full sample size if we had initially timed our newspaper advert for before the summer holiday break. Although the recruitment rate started quite slowly, the rate of recruitment then was in line with or slightly exceeded the target rateThe most successful routes of recruitment were via the SPCRN, online Gumtree adverts and newspaper adverts |
| 2. Are participants willing to be randomised to the intervention? (Research Objective 2) | Recruitment experiences of the study team and field workers Insight from qualitative interviews with participants | Yes (see Chapter 4, Recruitment, and Chapter 6, Recruitment and retention) Both groups were recruited and randomly allocated as intended. No issues were identified with this process |
| 3. Are appropriate retention rates achieved at 12-month follow-up? (Research Objective 2) | Measured using the following scale in both the intervention and control group at 12 months: if ≥ 70% followed up, proceed; if 50–69% followed up, discuss with TSC; if ≤ 49% followed up, do not proceed | Yes (see Chapter 4, Retention, and Chapter 6, Recruitment and retention) We achieved an overall follow-up rate of 77.1% of our baseline sample (84 of 109 participants). Although retention rates were different between the intervention group (71.2%) and the control group (88.9%), both met the progression criteria |
| 4. Is the intervention feasible to deliver and acceptable to participants and their helpers? (Research Objective 2) | USE questionnaire Participant/helper interviews | Yes (see Chapter 5 and Chapter 6, Exposure to the intervention)We have sufficient insight from participant/helper interviews to support the feasibility and acceptability of the intervention. The majority of interview participants engaged with the concept of the HelpMeDoIt! intervention, whether they harnessed social support via the app or outside the app. App use data also demonstrated some participants nominating one or more helpers and using the app as designedResponses to the USE questionnaire were mixed. As a data collection measure, we feel that the USE questionnaire lacked a ‘not applicable’ response option. This would have increased the response rate by providing a suitable response to some ‘not applicable’ questions. Findings suggested that the majority of participants found the app/website moderately easy to use and quite easy to learn. However, the majority did not find the app/website useful and were moderately unsatisfied. The qualitative findings suggest that the reasons for dissatisfaction and lack of usefulness were related to the technical issues experienced with the app. This is further supported by USE data showing that participants who used the app most frequently, and with a lack of technical issues, provided the most positive scores for usefulness, ease of use and learning, and satisfaction. This demonstrates that when the app operated effectively it was perceived positively by participants |
| 5. Do the majority (> 50%) of participants within the intervention group visit the app at least twice or do 25% of participants randomised use it three or more times? (Research Objectives 2 and 8) | App use statistics and/or participant interviews | Yes (see Chapter 6, Exposure to the intervention) Both criteria were met:66% of all intervention participants (including those who withdrew from the study) visited the app twice or more52% visited the app three times or more |
| 6. Are identified barriers and challenges to implementation of the intervention planned for and surmountable? (Research Objective 4) | Process evaluation | Yes (see Chapter 6, Recommendations to improve the intervention) The process evaluation highlighted numerous strengths and challenges for delivering a future app and website-based intervention for weight loss. These include: StrengthsIntervention was developed in collaboration with users, external software collaborators and PPI representativesIntervention was founded on strong evidence base for behaviour changeSufficient data were gathered to demonstrate that a weight loss intervention delivered via app and website is feasibleHelpMeDoIt! demonstrated higher levels of engagement than current commercial appsOpportunities to improve the participant and helper engagement with the intervention based on feasibility findingsSuccessful routes of recruitment and retention were identifiedWeaknessesPotential technical issues with the intervention hindered early engagement with the interventionAlthough the sample is generally representative of UK, the current findings relate to the Glasgow area of Scotland onlyChallenges were experienced in gathering insight from helpers (owing to helpers not being consented study participants)OpportunitiesCurrent climate in which almost 9 in 10 adults own a smartphone and have access to the internetCurrent opportunities for the development of digital health-related interventionsPotential for development of a minimal resource and cost-effective weight loss interventionOngoing collaboration with external software collaboratorsThreatsRapidly changing technology that conflicts with the timeframe of conducting a robust RCTChallenges to future hosting, implementation and maintenance of the interventionThe potential need for a large sample size to detect powered changePlan of action based on the findings of the feasibility trialRefine the programme theory (see Chapter 7, Planned dissemination of finding)Refine the intervention according to the findings listed in Chapter 6 |
| 7. Do the data collection procedures effectively collect the data required for a full trial? – successful completion of at least one data collection method (BMI, physical activity or healthy eating) at both baseline and at 12 months in those retained measured using the following scale (Research Objective 5) | If > 90% of at least one data collection measure, completed proceed If 70–89% of at least one data collection measure completed, discuss strategies for improvement in future trial with TSC If < 70% of all three data collection measures completed, do not proceed without further modification and pilot | Yes (see Chapter 4, Exploratory primary outcomes, and Chapter 6, General insights from the study team). Strategies for improvement were discussed with the TSC:Of the 84 participants retained at 12-month follow-up, 69 (82%) had their BMI objectively measured by a field worker. An additional 14 had their BMI calculated via self-reported weight using the ‘minimum data set’. Two participants did not provide weight at follow-up and therefore did not have their follow-up BMI calculated69 (82%) participants retained at follow-up had all three measures collected by a field worker. The remaining 15 provided only self-reported weight, or 7-day physical activity recall, or DINE dietary dataThere were issues with obtaining valid accelerometery data with only 39 (46%) participants providing both baseline and follow-up data |
| 8. Are the intervention costs of a full trial covered? | Identification of a source to pay access and treatment costs | Ongoing: The costs of delivering the intervention in a full trial will be low as the intervention is already developed. They will cover hosting and technical support. We are currently in the process of identifying a source of funding to cover these costs as well as to host the intervention should it be rolled out |
Objective 1: to develop an internet and web-based intervention that enables participants to set and monitor goals and facilitate effective social support
Phase 1 of the study involved designing the app and website with comprehensive consultation with potential users. In this phase we also completed initial testing of the website and app to assess their usability and acceptability. We combined four approaches to intervention development, which included an overarching intervention development framework (the 6SQuID model)74 and a review of the literature to identify relevant theories and behaviour change techniques that could be employed,16 as well as digital health-based approaches (Person-Based Approach and BIT model).75,76 User input was at the centre of the whole development process.
Initial development of the intervention was completed in tandem with a user development panel, the study team and the software company. This was an iterative process; as different aspects were developed, they were discussed in the development panel and updated according to what was agreed. In addition, a testing group of users helped with the initial evaluation of early-stage versions of the app and website. Although users were asked to provide input and feedback from a ‘helper’ perspective, this overall phase lacked input from individuals who formally identified as a helper. This is a consideration for future work. Once the beta version of the app and website had been developed, it was tested fully by both groups, and feedback was given in focus groups or individual interviews. The majority of the feedback was positive and we updated both the app and the website in anticipation of the feasibility trial. An expert heuristic evaluation was also completed at this point, which was very positive. During this phase we also further developed the HelpMeDoIt! programme theory and logic model, which incorporated behaviour change techniques and relevant theory, and addressed multiple contextual factors.
Objectives 2 and 3: to investigate recruitment and retention; to explore the potential of the intervention to reach traditionally ‘hard to reach’ groups (e.g. lower socioeconomic groups)
The trial methods and design were acceptable and feasible to implement. Although the recruitment rate was slow at first, it then was in line with or slightly exceeded the target rate. The average number of participants recruited per month was 17 (minimum 6 and maximum 27) and we recruited and successfully randomised 109 participants in the specified time frame (progression criterion 1). This was slightly short of our original target of 120. However, the trial team and steering committee felt that this number was sufficient to answer the research questions, so recruitment was stopped at 109. Participants also found randomisation acceptable, and none refused (progression criterion 2). This study has shown that it is feasible to recruit participants and to follow them up. There was an imbalance in follow-up rates between the randomised groups, which could introduce bias in a larger study if it is not addressed. A number of key learning points related to recruitment will inform a future study, and these are discussed in Chapter 7, Limitations of the study.
The sample was broadly representative of the target population; however, as is often seen in studies of interventions for obesity, more women than men were recruited.91 The majority of men were recruited via online Gumtree adverts. The participants in the sample were relatively well educated, with 42% having degree-level education or higher. The age range was good, although we were not successful in recruiting younger people (i.e. those aged < 25 years). Participants were relatively active at baseline, with those in the intervention arm achieving an average of 50 minutes and those in the control arm an average of 36 minutes of moderate to vigorous physical activity per day. Standard cut-off points were used for the accelerometer data.98 Participants may have increased their physical activity in response to using the activity monitor.130
Recruiting and engaging participants from lower socioeconomic groups131 in research remains a challenge, and many current interventions widen inequalities.132 However, we were particularly successful in recruiting participants from across the socioeconomic spectrum, with over one-third from the highest quintile of deprivation in Scotland and a further 21% from the next most deprived quintile. There were, however, slightly higher dropout rates among those from the most deprived group.
Baseline characteristics were broadly similar between the arms, although participants in the intervention arm were more likely to have higher levels of education and physical activity. Retention rates were good, with 84 (77%) participants followed up at 12 months (progression criterion 3). This included 14 participants who provided a minimum data set over the telephone. The split between intervention (11 of 52 participants, 21%) and control (3 of 32 participants, 9%) broadly reflected the 2 : 1 recruitment ratio. The waiting list control design, and the technical issues affecting intervention participants, may have affected the engagement of some participants, leading them to opt for the minimum data set. Those in the intervention group were slightly less likely to be followed up at 12 months. This may reflect either (1) a tendency for people who had not engaged with the intervention, or had experienced technical difficulties, to be less inclined to continue participating in the trial; or (2) the motivation of control participants to remain in the trial to receive the intervention. The participants least likely to complete follow-up data were younger, male, self-employed and deprived and had a higher BMI. Those currently attending a weight loss group were more likely to be followed up, as were those who complied with the baseline physical activity assessment by wearing a physical activity monitor. Both of these factors may represent markers of increased motivation to take part in the trial.
Objectives 2 and 4: to investigate the feasibility and acceptability of the intervention; to explore the barriers to and facilitators of implementing the intervention
Although engagement with the app was moderate for participants and low for helpers, the qualitative data indicated that most of the participants and helpers interviewed engaged with the concept of the intervention around mobilising social support to help with goal achievement. Helpers engaged with participants outside the app, and some participants sought social support from informal helpers who were not signed up to the study. This is a positive finding, considering that social support was the unique aspect of the HelpMeDoIt! intervention. Apart from some technical issues with the app that are described in Chapter 6, participants found the app and website acceptable and in the main were very positive about the intervention (progression criterion 4). The USE questionnaire data were not as positive as we had hoped; however, it is likely that the technical difficulties we experienced with the app, which were ultimately resolved, will have negatively influenced these responses. Participants who used the app most frequently, and after the technical issues had been resolved, provided more positive scores for usefulness, ease of use and satisfaction. This demonstrates that when the app operated effectively, it was perceived positively by participants. Interview data were more positive, but this may reflect interviews being conducted with participants who were potentially more motivated and engaged with the intervention. The qualitative data have provided a number of suggested refinements to improve the intervention (progression criterion 6). These would be implemented before delivery in a future study and are discussed in Chapters 5 and 6.
Adherence to the intervention was superior to that seen in general app use.94 However, commercial apps lack a recruitment phase, as in this study. Of the 54 (74%) participants who downloaded the app, 89% (n = 48) used it twice or more and 70% (n = 38) used it three times or more (progression criterion 5). In the study logic model, this level of participant engagement with the app and website was seen as necessary in order to receive a ‘dose’ of aspects of the intervention, in particular to receive key information about the importance of monitoring, setting realistic goals and social support to help with making lifestyle changes. The central aspect of the intervention was the support from the individual participant’s social network. It was not regarded as key whether this was via the app or another method, for example seeing someone face to face, sending them a text message or telephoning them. The interviews indicated that, in many cases, even though the helpers did not use the app, they gave the participants support outside the app. In the logic model, the app, and indeed the intervention as a whole, was seen as a prompt to help mobilise people’s social networks to help them in their behaviour change goals. The intervention helps start the conversation by getting the need for help ‘on the radar’. The process evaluation data indicated that this was achieved. There was also some evidence of changes in helper behaviour as a result of being nominated to help the participant. This ‘spillover’ effect could potentially be an important mechanism of impact of the intervention.
A recent expert international workshop considered the issue of engagement and concluded that engagement is complex, has many different aspects and cannot be evaluated purely on the basis of how often a person uses an app. Members of the workshop differentiated between the digital elements of the intervention and the behaviour change elements and concluded that continued engagement with the digital intervention (app/website) is not always needed for behaviour change, as delivery of the behaviour change elements via the digital intervention could kick start a process of behaviour change of developing new skills and new habits.95 This is particularly relevant to HelpMeDoIt!, where a brief interaction with the website or app can lead to the engagement of significant and sustained social support from existing, stable and durable social resources, the effects of which on resultant behaviour are not dependent on further app use.
The key barrier to implementing the intervention was the technical issues, described in detail in Chapters 5 and 6. Participants identified a number of other barriers to changing their behaviour, including various contextual factors such as time and money issues. They also described personal barriers, including willpower, motivation and a lack of routine. Some participants also described barriers to nominating helpers. Participants made a number of suggestions to facilitate use of the app, including receiving more reminders, receiving additional guidance on use and being able to choose what information to share on diet and weight.
The data suggest that the app seems to have worked particularly well for a minority and that the engagement rates are above industry norms. An intervention such as this has the very real potential to be cost-effective even if a relatively low percentage of the initial users engage with it repeatedly.
Objective 5: to assess the feasibility and acceptability of different outcome measures for diet and physical activity in this population
One of the key objectives of the study was to inform the choice of primary outcome in a future evaluation. Three potential primary outcomes were tested: BMI, physical activity and diet. Physical activity and diet were measured in two different ways: physical activity via a self-report measure and accelerometery, and diet by two different self-report measures. One of the criteria for selection for a future study was feasibility of the measures including data completeness (progression criterion 7). The measures were generally feasible to use, and qualitative data indicated that they were acceptable to participants and not burdensome. Data completeness was good for all of the measures apart from the accelerometry data. BMI data were collected in 98% of those who were followed up. The 7-day physical activity recall (PAR) data were collected for 96% of those followed up; however, valid accelerometry data (provided at baseline and follow-up) were available for only 46% of the sample. However, when these data were obtained, they provided the most robust evidence for an intervention effect in this trial. This measure required a greater level of commitment from study participants to obtain valid data for analysis. A wrist-worn accelerometer may have had better compliance if it had been possible to collect physical activity data in a less invasive way, for example via participants’ smartphones, throughout the follow-up period; this might have reduced the burden on participants, and provided objective physical activity data for those who did not engage with follow-up data collection. In addition, in other similar studies, strategies such as giving vouchers for returning the accelerometer have led to improved adherence and return rates, so vouchers combined with other methods could significantly improve return rates.128,133 In a future trial, accelerometry data could be collected for a subsample to validate the self-report physical activity measure, where self-report data are collected before sending out accelerometers. Alternatively, accelerometry data could be collected for the full sample but not analysed and instead used to increase the accuracy of the self-report.
For the diet measures, the collection of diet data using the 24-hour multiple pass recall was not considered feasible because, although we managed to obtain data at baseline from 74% of the participants, only 31% of the questionnaires were completed properly. In addition, this outcome also required a large amount of resources in terms of data collection and processing and, because of these issues, was not used at follow-up. The other dietary measure, the DINE questionnaire, was feasible and was completed by 96% of participants at follow-up.
Objective 6: to use outcome data (diet, physical activity, body mass index) to help decide on a primary outcome and to estimate the potential effect size of the intervention to facilitate the calculation of an appropriate sample size for a full trial
The study was not powered to assess effectiveness. Although these results should be treated with caution owing to the numbers contributing to these analyses, the objective physical activity data showed moderate to large effect size estimates across several measures, particularly the daily step count and sedentary time. These findings were amplified in per-protocol analyses, and appeared strongest in those with lower levels of physical activity at baseline. In addition, there was no evidence to suggest that self-report physical activity was different between those who did and those who did not provide valid accelerometry data, thereby increasing confidence in these results. There was also some suggestion that those who used the app more achieved better outcomes in terms of BMI, objectively measured physical activity and diet, which could potentially reflect an effect of the intervention, but would also be consistent with those who are most motivated to engage with the app or to provide follow-up data achieving better outcomes.
With regard to choice of the primary outcome for a full trial, BMI was feasible to collect and was objectively measured. Measuring diet and physical activity accurately is challenging.134,135 Although accelerometry was objective, valid data were available for only just under half the participants. However, methods to address issues with return of the accelerometer and compliance with the data collection protocol have been used successfully in other trials and could be applied in this population.128,133 Including interim data collection points could alleviate problems due to missing outcome data, and offering further incentives for completion of follow-up assessments could improve data collection rates. The effectiveness of these measures could be tested as part of an internal pilot within a future trial.
In terms of sample size for a future trial, there are three key parameters, namely (1) the minimum clinically important difference (MCID), in this case percentage weight loss (and its standard deviation); (2) proportion of recruited participants who engage with the intervention (among whom the MCID will apply, with zero weight loss assumed among those who do not); and (3) loss to follow-up.
If we take the MCID as 5% weight loss with a SD of 7.3% (as observed in this feasibility study; effect size 0.68), then a sample size of 47 per group would be required to achieve 90% power to detect a difference, with no loss to follow-up and 100% engagement. However, if only two-thirds of participants engaged with the intervention, as occurred in this feasibility study, then the population effect size is 0.46 (0.68 × 0.67) and 101 per group would be required. This assumes complete follow-up; allowing for 20% loss to follow-up, a study of approximately 127 per group would be required.
Recognising that it is highly risky to base sample size calculations on feasibility study data (where estimates of key parameters have wide CIs), a highly cautious set of assumptions may be to assume the SD of weight change to be 10%, 50% engagement with the intervention (giving a population effect size of 0.25) and 40% loss to follow-up, for which a sample size of 564 per group (1128 total) would be required to achieve 90% power.
Given that the intervention is very low cost and has the potential to reach large numbers, it would feasibly be cost-effective at a population level if the achieved MCID was 3% among those who engaged, and/or if engagement levels were < 50%. This would equate to population effect sizes of ≤ 0.15, and thus sample sizes at 90% power > 2000. Appendix 10 includes a range of potential sample sizes across different assumptions for the three key parameters.
Objective 7: to assess data collection tools and obtain estimates of key cost drivers to inform the design of a future cost-effectiveness analysis
The health economics feasibility component assessed procedures, validity and reliability of resource use data collection, quality of life and capability well-being tools in addition to prior estimation of parameters such as the variance of the outcome variable. Without conducting formal between-group statistical comparisons, it can be seen that health and social care resource use, food and drink and lifestyle activity spend patterns were broadly similar between the groups. The HelpMeDoIt! mean EQ-5D-3L values across groups were slightly lower than published population norms; however, values exhibited typical variance around mean values. The ICECAP-A instrument identified and measured responses across all levels of capability well-being across the control and intervention arms. The results reveal that the ICECAP-A was able to distinguish between different levels of self-reported capability well-being in this population group.
Overall, the results of the health economics feasibility study showed that the questionnaires designed for measuring health and social care resource use, food, drink and lifestyle activity spend, quality of life (EQ-5D-3L) and capability well-being (ICECAP-A) would be suitable for inclusion in a full study with some slight changes to the resource use questions. The costs of developing the app and website were mostly upfront, which meant that the cost per participant for the intervention was relatively high in this study. In a future trial, the delivery of the intervention would be a low per-participant cost, which would mainly cover the technical support and ongoing hosting. The low expected cost per participant would permit even modest reductions in BMI, equivalent to the low population effect sizes discussed in the previous section, to attain cost-effectiveness in a future trial.
Objectives 8, 9 and 10: to investigate how participants and helpers engage with goal-setting, monitoring and social support using new technologies and how these elements interact within a behaviour change intervention; to develop a conceptual model of how the key mechanisms of goal-setting, monitoring by self and others, social support and behaviour change are facilitated by the intervention; to test the logic model and theoretical basis of the intervention in stages 1 and 2
The above objectives broadly relate to the intervention theory and logic model. These objectives were addressed in both Chapters 5 and 6. Overall, participants and helpers engaged with the intervention concept but not necessarily via the app. Data from the interviews led to support for the hypothesised intervention logic model and programme theory and has led to further refinements (see Figure 10). Key changes relate to the importance of some of the hypothesised mediators and intermediate outcomes. Eight of the 11 proposed mediators of change were identified as being central to the HelpMeDoIt! intervention. These were increased social support; increased engagement with helpers via the app; increased interaction with helpers not via the app; reflecting and setting on-going goals; increased action planning; increased self-monitoring; increased skills and knowledge; and increased motivation. Increased motivation, increased social support and increased goal-setting and self-monitoring were identified as key mediators. Two out of four intermediate outcomes were also supported by the findings, namely improved social support and healthy habit formation. Although supported by the evidence base, improved self-efficacy136 and improved self-esteem137 do not appear to be integral processes. Indications in the qualitative data were that those with the strongest social support lost the most weight; they also reported more rigorous goal-setting and monitoring and had more successful workarounds for their identified barriers. This provides further support for the logic model. A number of key contextual factors were also identified that were incorporated into the revised logic model (see Figure 10).
Objective 11: to explore the characteristics of participants’ social networks and the influence social networks have on participant experiences and outcomes of the intervention
As noted at the beginning of this report, objective 11 was not part of the original funding application. Although some aspects of this objective were briefly mentioned within the qualitative findings, data for this additional objective will be analysed and published at a later date.
Objective 12: to assess whether an effectiveness trial is warranted
The results of the study indicate that an effectiveness trial is feasible and warranted. All of the progression criteria have been met, apart from obtaining intervention costs (progression criterion 8). We are in the process of negotiating with relevant bodies to cover the treatment costs associated with the intervention and to host the website and app in any future roll-out.
Strengths of the study
The intervention was developed with significant user involvement from the outset and using recommended frameworks for developing complex interventions and digital interventions. The extended development and early stage testing in stage 1, which incorporated the views of users throughout the process, led to the development of a high-quality app and website that were acceptable to users. The intervention was theory based and the programme theory and logic model was refined and tested in the feasibility trial. We used a mixed-methods approach to assessing the feasibility and acceptability as well as the potential impact of the intervention. We were able to triangulate these data to strengthen the internal and external validity of the findings.
The feasibility trial used rigorous methods for data collection and analyses. Objective measures were used to assess weight, BMI and physical activity, and self-report measures were chosen based on previous evidence of validity and reliability. The study also included measures of quality of life and capability well-being and collected cost data to inform a future cost-effectiveness analysis. A key strength is that we were able to successfully recruit a large number of participants from lower socioeconomic groups. The process evaluation allowed us to address the progression criteria, in particular to explore the feasibility and acceptability of the intervention and trial methods as well as to obtain feedback on and suggestions for improving the intervention and enhancing engagement. The qualitative methods were robust and the large number of interviews gave extensive, in-depth accounts of the experiences of both the study participants and their helpers. These data will be used to further refine the intervention ready for further evaluation. Further strengths of the trial were that all key progression criteria were achieved, recruitment rates were adequate, retention was good and engagement with the intervention was acceptable. The results are reported in line with CONSORT (Consolidated Standards of Reporting Trials) guidelines.
Limitations of the study
A key limitation of the study was the low engagement of helpers via the app. Although this was disappointing, the qualitative data indicated that the programme theory of the intervention was delivered, as the app facilitated engagement with helpers that then occurred outside the app. Technical issues with the app during the early stages of the trial led to a number of users disengaging with the intervention and perhaps also with the trial. Some participants and helpers chose to interact outside the app for this reason. The study could have benefited from greater helper input during the development stage. Although we did cover the ‘helper aspect’ of the intervention with users in stage 1, as well as gather ‘think aloud’ feedback from individuals tasked with giving feedback on the ‘helper’ aspects of the app and website, individuals who formally identified as helpers were not included in the development process. In addition, during the stage 2 follow-up, ethical constraints meant that we were unable to contact helpers directly to ask if they would be willing to be interviewed, which meant that the number of helper interviews was smaller than anticipated.
Potential sources of bias include that it is likely that more motivated individuals enrolled in the study, skewing the engagement data, although it is likely that similarly motivated individuals would engage with any future study or roll-out of the intervention. It is also likely that the more motivated and engaged individuals took part in the interviews, so they may have given more positive views of the intervention. We recruited only four participants for interview at 12 months, which meant that we obtained only limited insights into longer-term weight maintenance behaviours.
Another limitation relates to the geography, as people were recruited only from Greater Glasgow and Clyde. This is a large area including rural and urban areas and a range of demographics. However, it has poorer health than other areas of the UK with similar levels of deprivation and this could have an impact on the generalisability of the study approach.
Attrition was higher in the intervention group than in the control group. Attrition is typically higher in control groups for weight loss studies. The higher attrition in the intervention group for our study could be attributed to technical issues with the app, although this could also be explained by keen participants remaining in the control group in order to receive the intervention. The retention rate in the intervention arm may be more likely to be what would be seen in a full trial. One final limitation of the study is that 12 participants self-reported BMI, meaning that, overall, when considering obtaining an objective measure of BMI, our retention rate was 63% (69/109), so methods for improving this will need to be explored.
Key learning points
There are a number of key learnings from the feasibility trial that would be incorporated into the design of any future study. The focus of this feasibility trial has been to explore parameters for a full trial and, although our findings are mostly positive, a full trial may not be the only option for further evaluating the HelpMeDoIt! intervention. Other options include further refinement of the intervention with users, particularly focusing on the helper element and additional feasibility testing, which we plan to do in advance of applying for funding for a full trial.
Other key learnings have been presented in Chapter 6. With regard to evaluation design, we would include an online eligibility form to screen expressions of interest and additional administrative support to arrange recruitment and follow-up appointments. In terms of methods of recruitment, we would use the same methods, but focus more on primary care from the start, as this was the most effective recruitment method (after slight modifications to paperwork).
With regard to the intervention, a number of findings will inform the refinement of the app and website (see Chapter 6, Recommendations to improve the intervention). The option of accessing guidance via the app without the need for a website will be explored with users. Following refinement, we would ensure that there was a testing phase once the app was launched in the app store before allowing participants access, in order to smooth out any technical issues. A number of insights from the qualitative data will help to enhance both helper and participant engagement with the intervention. One key suggestion is to download the app when the field worker is present so that participants can be shown what to do and any issues can be ironed out at that point. We will also look to give further guidance on the participant–helper interactions, for example how to approach helpers and how to most effectively give and receive help. The helper nomination process could be streamlined and we would look to improve this. We could also provide guidance on how to pick a good helper based on our process evaluation findings. Another suggestion was to allow participants in the intervention arm to be peer supporters for other participants. This could potentially address the issue whereby a small minority of participants did not have anyone to nominate or they found it difficult to ask for help. It could also mean that the intervention effect could be enhanced by adding further opportunities for social support.
Planned dissemination of findings
Various routes of dissemination are planned for the findings of this study. First, in addition to the published protocol,73 we will publish the main outcomes in an open-access peer-reviewed journal. We have presented our methods and findings at numerous conferences, covering expertise in both behavioural medicine and social network analysis. We will continue to present findings at future conferences. Additional routes of dissemination have included five public engagement events with children and adults, and discussion of our study with other stakeholders at government events and other networking events.
Conclusions
This study has shown that the HelpMeDoIt! intervention is acceptable and feasible to deliver, although further work is needed to explore the helper role. It is also feasible to recruit participants, and to follow them up and collect outcomes at 12 months, suggesting that a large-scale study would be possible. We found evidence of promise for the intervention in both the qualitative and quantitative data. Some further refinement of both the trial methods and the intervention is needed, based on the results of the feasibility trial. The intervention refinements suggested are not fundamental changes to the content or mechanisms of the intervention. Based on the findings from the process evaluation, some changes are needed to the programme theory, the logic model and the intervention content, as well as functionality of the app and website to enhance engagement and improve the intervention. In particular, refinement is needed to improve helper engagement. These changes do not affect the key elements of the intervention: social support, goal-setting and self-monitoring. The results from the feasibility trial support the case for further evaluation of the HelpMeDoIt! Intervention.
A key challenge of working in digital health is that apps can rapidly become obsolete. However, although technology is continually advancing and the platform may change, the key issue is what the platform is delivering (i.e. the behaviour change approaches including mobilising people’s close social networks to assist with behaviour change). These are the key elements of this intervention, which could be delivered by an updated refined app or via a new platform. It is also possible to continue to improve the app during any evaluation as long as the key functions of the intervention are retained and delivered.138 As Hawe et al.138 have argued, the form of the intervention can be flexible as long as the key functions of the programme theory are delivered.
While the aim of this research was to develop an intervention that maximised effect size and participation rate, an intervention such as this has a potentially high reach and can therefore tolerate small effects and low use while still remaining cost-effective. It will also be important to consider the contribution of such digital health approaches within the wider ecological public health context for weight management and sustained weight reduction.
Implications for health care
This was a feasibility study. However, if effectiveness was demonstrated in a full trial there are several implications, including the following.
- HelpMeDoIt! may have the potential to deliver a low-cost, potentially high-reach intervention for adults with obesity.
- HelpMeDoIt! could be used as a complementary intervention used alongside other health-care or lifestyle services, including other apps.
- HelpMeDoIt! may have potential to positively influence the lifestyle of individuals in a participant’s broader social network.
- This approach to mobilising social support could be used in other areas of health behaviour change.
Recommendations for research
- To further understand the motivation and engagement of helpers in relation to providing social support to participants.
- To assess the effectiveness and cost-effectiveness of the HelpMeDoIt! intervention after further refinement of the intervention.
- To further explore the key mechanisms of change identified by the HelpMeDoIt! feasibility findings.