Results

Part 1: stage 2 outcome results

The key feasibility outcomes are reported in this chapter as well as in Chapters 5 and 6. In this section, we describe the recruitment rates and retention in the trial (progression criteria 1–3), as well as the characteristics of participants. The quantitative analyses relating to the exploratory primary outcomes and secondary outcomes are presented, followed by the health economic analyses. Key data are provided in this chapter, with the full statistical outputs presented in Appendix 8.

Recruitment

One hundred and eighty-eight individuals were screened for inclusion in the study, of whom 156 were identified as eligible. A total of 109 participants (70% of those eligible) proceeded to be randomised between 22 April 2016 and 22 October 2016 (Figure 5). This was slightly short of the target of 120 participants randomised. Of the 109 participants, 73 participants were allocated to the intervention group and 36 participants were allocated to the control group. Figure 6 shows the cumulative numbers randomised over time during this period. Although initially a slow start, the recruitment rate continued in line with or quicker than target, which is positive sign for a future trial. No individual signed up as both a participant and a helper.

FIGURE 5 The CONSORT flow diagram for the HelpMeDoIt! study. (Figure not reproduced here; see the original article.)

FIGURE 6 Recruitment rate in comparison with target rate (dashed line) over 6 months. (Figure not reproduced here; see the original article.)

Baseline characteristics

Key baseline characteristics of the randomised population are reported in Tables 10–12 (see Appendix 8, Tables 49–65, for additional baseline summaries). Overall, more women (69.7%, n = 76) than men were recruited, and the average age was 47 years [standard deviation (SD) 10.7 years]. The average BMI was 37.6 kg/m² (SD 5.9 kg/m²) and just over one-quarter were currently attending a weight loss group. We recruited across the spectrum of levels of deprivation, with over one-third being from the highest quintile of socioeconomic deprivation. Most participants were employed and the majority were married. Over 60% had received higher education. Participants’ daily moderate to vigorous activity was a mean of 45 minutes per day (SD 32.9 minutes per day).

Randomised groups were generally well matched, although intervention group participants were more likely to have higher levels of education. They were also more likely to have higher levels of physical activity (measured objectively), as well as more variability between participants. They also had more sedentary time. Slightly more people in the control group were attending a weight loss group and those in the control group were more likely to weigh themselves regularly (see Appendix 8, Table 50).

Retention

Table 13 shows the follow-up rates at 12 months in each group. Overall, 77.1% of participants were followed up, achieving the progression criterion of at least 70%. Both randomised groups achieved the overall target of 70%.

Table 14 shows selected baseline data for those who were successfully followed up at 12 months compared with those who were not followed up. Also reported are the odds ratios for being successfully followed up associated with these baseline characteristics, derived from univariate logistic regression models (see Appendix 8, Tables 68–84 and 85–100, for additional summaries).

Those in the intervention group were less likely to be followed up at 12 months. This may reflect a tendency for those who disengaged from the intervention to be disinclined to continue to participate in the trial. Older participants, and women, were more likely to be followed up, as expected. Those who were self-employed at baseline were less likely to be followed up, as were those living in one of the 20% most deprived areas of Scotland. Follow-up rates were lower for those who had higher weight, BMI or waist circumference, perhaps reflecting a greater tendency for these participants to withdraw from the intervention. Those currently attending a weight loss group were more likely to be followed up, as were those who complied with the baseline physical activity assessment by wearing a physical activity monitor. Both of these factors may represent markers of increased motivation to take part in the trial.

Exploratory primary outcomes

As a feasibility study, the trial was powered not to detect statistically significant changes, but to explore the feasibility and potential sensitivity of measures for use in a definitive trial. This study explored the use of three primary outcomes: (1) physical activity (2) dietary measures and (3) BMI.

Physical activity

Table 15 summarises physical activity measures derived from the 7-day physical activity recall, and from accelerometers, in the randomised groups, along with estimated between-group differences, with 95% CIs and p-values, derived from linear regression models of 12-month outcomes, with randomised group, baseline score, age, gender and BMI of ≥ 40 kg/m² as predictor variables.

For each estimated intervention effect estimate, the standardised effect size is reported with a 95% CI. The original plan had been to derive this effect size estimate by dividing the intervention effect estimate and confidence limits on the original scale by the standard deviation of the outcome measure at baseline in the total study population. However, it was recognised that for some measures, there was considerably more variability between individuals in baseline scores (or follow-up scores) than there was between individuals in the change from baseline. For an individual, a change in outcome equal to the standard deviation in the whole population might be an unachievable goal, but a change in outcome equal to the standard deviation of the distribution of within-person changes may be much more attainable, so it was felt that standardising between-group differences relative to the pooled standard deviation of changes from baseline would be more relevant.

As part of the modelling process, outcomes were initially analysed on their original scale. The residuals from each regression model were visually assessed for normality. For those measures where substantial non-normality was observed, data transformations (at follow-up and at baseline) were attempted to improve model fit; if no suitable transformation could be found, then changes from baseline to follow-up are compared between groups using the Wilcoxon–Mann–Whitney test. See Appendix 8, Tables 105 and 106, for additional summaries.

Only 39 participants (36% of those randomised, 46% of those followed up at 12 months) provided valid activity monitor data at both baseline and 12 months, in comparison with 81 (74% of those randomised, 96% of those followed up) who provided self-report physical activity data at both time points.

Both groups showed reductions in objectively measured physical activity (moderate to vigorous physical activity and step counts), but the intervention group had higher mean values, and showed greater variation, at each time point. Adjusted analyses suggest effect sizes of around 0.4, but unadjusted analyses give effect size estimates close to zero. Similar reductions in physical activity in both groups were seen in self-reported energy expenditure data, with intervention effect estimates close to zero. The intervention group had lower objectively measured sedentary times at 12 months, with 50–60 fewer minutes per day spent in sedentary activities relative to the control group. This represents an effect size estimate of around 0.7. We also examined whether self-report physical activity differed between those who did and those who did not provide valid accelerometry data (Table 16). There was no difference.

The results seem to offer evidence that self-report physical activity was no different between those who did and those who did not provide accelerometry data. This increases confidence in the differences found between the groups in the accelerometry data.

Diet

Table 17 summarises dietary outcome measures derived from the DINE questionnaire. These have been analysed in the same way as the physical activity data. Additional data summaries can be found in Appendix 8, Tables 107 and 108.

Dietary scores were available for 81 (74% of those randomised, 96% of those followed up) participants at baseline and 12 months. These generally improved or remained stable in both groups between baseline and 12 months, with the exception of the fibre score in the intervention group. Both groups achieved an increase in healthy eating score (difference between fibre score and fat score), although this was greater in the control group, with an effect size in favour of control. The intervention appeared most promising in terms of the unsaturated fat score, with an effect size of approximately 0.45. Fruit and vegetable scores were slightly higher in the intervention group, but fizzy drink and sugar scores were skewed towards zero and changed little in both groups during the study, and therefore show little promise as outcomes for a future study.

Body mass index

Table 18 summarises BMI data, which were analysed in the same way as the other primary outcomes (see also Appendix 8, Table 109). Data were available for 82 participants (75% of those randomised, 98% of those followed up). Both groups showed similar reductions in BMI during the study of approximately 1 kg/m².

Table 19 presents thresholds of weight loss for each group. A greater proportion of participants in the control group gained weight than in the intervention group (47% vs. 34%). A greater proportion of participants in the intervention group lost < 10 kg than in the control group (52% vs. 31%). However, weight loss of > 10 kg was more frequent in the control group than in the intervention group (22% vs. 14%).

Per-protocol analyses

Per-protocol analyses were carried out after excluding those who had either failed to provide follow-up data or failed to comply with the intervention (for those in the intervention group). Twenty-five participants did not provide any outcome data at the 12-month follow-up, although for objectively measured physical activity measures the follow-up rates were much lower (70 participants failed to provide valid activity monitor data for analysis). Fifteen participants in the intervention group showed a lack of adherence to the intervention, defined as failure to use the app at least twice.

Tables 20 and 21 show the intervention effect estimates for each primary outcome measure using the full analysis set and the per-protocol population.

Objective physical activity measures collected with an activity monitor showed greater between-group differences in the per-protocol population, with effect sizes increasing from 0.35–0.76 to 0.56–0.95. In particular, the average daily step count showed an increase of approximately 1400 steps per day (p = 0.033), and average sedentary time was reduced by approximately 70 minutes per day (p = 0.011), suggesting clinically relevant changes in physical activity among those who engaged with the intervention. However, self-reported physical activity continued to show little effect of the intervention in the per-protocol analysis.

For dietary outcomes and BMI, the per-protocol analysis showed little evidence of positive intervention effects.

Multiple imputation analyses

Because fewer than 80% of participants provided data for all primary outcome measures, multiple imputation analysis was carried out, as specified in the statistical analysis plan. Multiple imputation was implemented using chained equations. For each outcome studied, 10 imputed data sets were generated, imputed using the outcome variable at 12 months, age, gender, baseline BMI, and the baseline value of the relevant outcome measure.

Table 22 shows the results of the adjusted analyses from the original analysis and the analysis using multiple imputation (see Appendix 8, Table 132).

The estimated intervention effects for objectively measured physical activity were reduced in the multiple imputation analyses, generally by about half. This suggests that these results are sensitive to missing data; there was a large number of missing data for these outcomes. Other outcome measures were much less affected by missing data, with results using multiple imputation being generally similar to those of the original analyses.

Secondary outcomes

Tables 23 and 24 report the other outcome measures at baseline and follow-up and model estimates (see Appendix 8, Tables 110–120, for additional summaries).

Weight showed similar results to BMI, with slight reductions in both study groups, and little to suggest a greater reduction in the intervention group. Waist circumference and waist-to-hip ratio demonstrated similar patterns within groups, with larger reductions observed in the control group.

General Health Questionnaire (GHQ) scores were slightly in favour of the intervention group, although the effect sizes were small (around 0.2). GHQ caseness was 50% in both groups at follow-up. Quality of life (EQ-5D-3L) health utility scores reduced slightly in both groups, but EQ-5D-3L visual analogue scale (VAS) scores increased slightly in both groups. Between-group differences were in favour of the intervention, with effect size estimates of around 0.25–0.30. ICECAP-A scores changed little during the study and showed no signs of between-group differences (see Table 24). The feasibility of the EQ-5D-3L and ICECAP-A measures is reported in Part 2: health economics analysis.

There was no clear pattern of positive benefits across the other questionnaire measures used in the study (see Appendix 8, Table 99), with the possible exception of questionnaires measuring motivation to eat a healthy diet [Treatment Self-Regulation Questionnaire Concerning the Motivation for Eating a Healthy Diet (TSRD), effect size approximately 0.4] or to exercise regularly [Treatment Self-Regulation Questionnaire Concerning the Motivation for Exercising Regularly (TSRE), effect size approximately 0.2].

Alcohol use (AUDIT-C score) and smoking status (HSI) were collected at 12 months only. This was to explore their feasibility and potential use in a definitive trial. AUDIT-C scores were generally low and smoking was uncommon in both groups, with no evidence of any differences between the groups (see Appendix 8, Table 114).

Potential moderators

For primary outcome measures, intervention effect moderation was assessed with respect to age, gender, socioeconomic status and the baseline measurement of the outcome. This was limited to analyses of the objective measures of physical activity (average daily sedentary time and average daily step count), as these showed the largest effect sizes in earlier analyses. This was achieved by fitting regression models with main effects for randomised group and the moderator variable of interest, plus their interaction. All models were adjusted for the baseline measurement of the outcome, age, gender and BMI, if these were not already included in the model.

Table 25 shows the interaction test p-values in relation to each potential moderator. There was no evidence to suggest any effect moderation with respect to age, gender or socioeconomic status. For sedentary time (p = 0.021), and to a lesser extent daily step counts (p = 0.11), there was some evidence that any intervention effect was moderated by the baseline activity measurement.

Figure 7 shows the nature of the interactions identified in Table 25. For those with higher levels of sedentary time at baseline, the intervention appears to reduce sedentary time relative to the control group, while having less of an effect in those with low sedentary times at baseline. For those with low step counts at baseline, the intervention appears to increase step counts relative to the control group, whereas outcomes appear to be less affected by the intervention for those with higher baseline step counts.

FIGURE 7 Intervention effect moderation by baseline measures. (a) Estimated intervention effect for daily sedentary time, in relation to baseline daily sedentary time; and (b) estimated intervention effect for daily step counts, in relation to baseline daily step count. (Figure not reproduced here; see the original article.)

These figures highlighted the presence of some outliers at baseline, which may have an undue influence on these findings. Figure 8 shows the results of these analyses after two individuals were removed from the sedentary time analysis (with the highest and lowest baseline values) and one individual was removed from the step count analysis (with the highest baseline value). In general, the patterns of results are much the same, with the intervention showing most evidence of an effect for those with high levels of sedentary behaviour and low levels of physical activity at baseline.

FIGURE 8 Intervention effect moderation by baseline measures, sensitivity analyses. Individuals with extreme baseline values were removed from the analysis. (a) Sedentary minutes per day (highest and lowest baseline values removed); and (b) step count per day (highest baseline value removed). (Figure not reproduced here; see the original article.)

These apparent interactions may reflect floor (for sedentary time) and ceiling (for step counts) effects, with those who are more physically active, or less sedentary, having less room for improvement through the intervention.

Safety outcomes

There were no serious adverse events reported during the study.

Summary of quantitative analyses results

Recruitment was close (91%) to the target of 120 participants over 6 months, suggesting that the study could recruit large numbers of participants if run over a longer period at multiple study sites. In terms of sample characteristics, more than twice as many women as men were recruited, with an average age of 47 (range 25–68) years; most were employed and married. The average BMI was 37.6 kg/m². One-third of those recruited were from the most deprived quintile and over 60% had received higher education.

Retention at 12 months was above the target of 70% overall, and in both randomised groups, although it was lower in the intervention group, with a lower 95% confidence limit of approximately 60%. Any future study may require measures to increase follow-up of participants who become disengaged from the intervention.

Collection of data was good for self-reported outcomes, with > 90% of those followed up providing data for weight, and physical activity and dietary questionnaires. Data on objectively measured physical activity using an accelerometer were poorly collected, with fewer than half of those followed up providing valid data at both baseline and 12 months.

The study was underpowered to detect effects, so caution is required when interpreting these results. Point estimates of standardised intervention effect sizes were generally small (often less than ± 0.3), or small and in favour of the control group. However, CIs were generally wide and were therefore consistent with clinically relevant benefits. The one exception to this was the objective physical activity data, which showed moderate to large effect size estimates across several measures, particularly the daily step count and sedentary time. These findings were amplified in per-protocol analyses, and appeared strongest in those with lower levels of physical activity at baseline. There was no evidence to suggest that self-report physical activity was different between those who did and those who did not provide valid accelerometry data, which increases confidence in these results. However, these outcomes were poorly completed, and these findings were sensitive to missing data, being greatly reduced in multiple imputation analyses.

Part 2: health economics analysis

Data were collected for 109 participants at baseline: 36 in the control group and 73 in the intervention group. All costs were collected, with the exception of five participants who did not provide expenditure on food purchases. At 12-month follow-up, data were collected for 78 participants (72% of baseline sample) in relation to the EQ-5D-3L, ICECAP-A and expenditure on food purchases. Fewer participants (n = 69) provided data on health, social and personal resource use, which were gathered using a CRF. This was because a proportion of participants at follow-up, who were at risk of dropping out of the study, completed a ‘minimum data set’ that omitted the additional questions linked to resource use. The following section summarises the health economics findings in relation to (1) intervention costs; (2) health, social and personal resource use; and (3) quality-of-life and capability well-being measures.

Figure 9 shows the enrolment for the resource use CRF, food cost questionnaire and EQ-5D-3L and ICECAP-A data.

FIGURE 9 Flow chart of enrolment and follow-up of participants. (Figure not reproduced here; see the original article.)

Table 26 shows the number of participants taking weight loss medication in both groups at baseline and follow-up.

Intervention costs

The intervention cost comprised the cost of developing a mobile app and website and the cost of maintaining and updating the app. Table 27 shows the cost of the intervention, including app development and maintenance, alongside the research costs. The cost per participant for the current study was high, at £740 (see Table 27: calculated by the fixed cost of the study plus variable cost of the study divided between 109 participants); however, these costs involved the upfront cost of intervention and software development. Now that the app and website are developed, a future trial would simply need some refinement based on the feasibility results (£10,000), ongoing costs for hosting and support (£10,000), and costs of the study (variable depending on sample size). This would mean a low cost per participant in a future trial, at approximately £370 per participant for a sample of 400 or £220 per participant for a sample of 1000. If rolled out ‘in the real world’, only hosting (£10,000) and maintenance costs (£10,000) would be required, resulting in a low cost, for example, of £2 per participant per 10,000 users.

Health, social and personal resource use

Health and social resource use

The most frequently used health and social resources within the 3-month period prior to data collection were visits to GP, practice nurse and physiotherapy (see Appendix 9). The data identified individual participants with high attendances at GP, practice nurse and physiotherapy; these outliers giving rise to right-skewed data are a typical pattern occurring in resource use data. The mean cost of baseline health service resource use for the control and intervention groups at baseline was £285 (SD £571) and £80 (SD £119), respectively. The mean cost of follow-up health service resource use for the control and intervention groups was £553 (SD £1554) and £152 (SD £224), respectively. No formal statistical tests of the difference in costs were conducted owing to lack of power.

Data revealed that the HelpMeDoIt! resource use data collection instrument identified and measured resource use items across a range of health services in both arms including GP, nurse, physiotherapist, hospital consultants for various specialties, community psychiatric nurse, A&E and hospital inpatient stay. NHS weight management services were also used by both control and intervention arm participants at follow-up only.

Personal resource use

The total weekly cost of participant food and drink at baseline for the control and intervention groups was £98.93 and £104.75, respectively (see Appendix 9). Overall, grocery costs were the highest component of weekly costs at baseline and follow-up (64% and 58% at baseline for the control and intervention groups, respectively; 67% and 61% at follow-up for the control and intervention groups, respectively). Meals out constituted between 20–22% and 18–24% of weekly food expenditure at baseline and follow-up for control and intervention, respectively. Alcohol spend was the smallest component of weekly food and drink cost, at between 5% and 8% at baseline and follow-up, in both arms. Overall, these weekly food and drink data collection questions appeared feasible for gathering information on food and drink spend.

The mean spend on lifestyle activities (including slimming clubs, health clubs, gyms, swimming pools and exercise classes) in the 3 months prior to baseline and follow-up was £43.29 and £30.08 at baseline for the control and intervention groups, respectively. At follow-up this spend was £51.46 and £55.23 in the control and intervention groups, respectively. These lifestyle costs exhibited the same pattern of right skewness as health services data, including a small number of high values. This method of directly collecting lifestyle spend, pre-empted by a closed-ended yes/no question, proved acceptable to participants.

Quality-of-life and capability well-being measures

EuroQoL measure of quality of life

Responses to each level of all domains of the EQ-5D-3L are shown in Table 28. The majority of responses are in the level 1 (no problems) category across all five attributes, with the exception of the attributes ‘pain and discomfort’ and ‘anxiety’. These two are the only attributes that report higher responses in values in the level 2 and 3 categories (some/extreme problems). The EQ-5D-3L health index and visual analogue (VAS) values for each treatment group at baseline and follow-up are shown in Table 29. Table 29 shows that the EQ-5D-3L index values from baseline to follow-up are in the range 0.67–0.79 across control and intervention groups. These index and VAS values are slightly lower than the population norms reported for this age group.126

ICECAP-A measure of capability well-being

The ICECAP-A instrument feasibly identified and measured responses across all levels of capability well-being across the control and intervention groups. The highest percentage of responses typically lie in level 3, with few responses in level 1 (see Appendix 9). These results reveal that the ICECAP-A instrument is able to distinguish between different levels of self-reported capability well-being in this population group. No data were collected for the domain ‘love, friendship and support’. This question had been omitted from the questionnaire in error.

Summary of health economics feasibility findings

The aim of this feasibility study was to assess procedures, validity and reliability of tools, estimation of the recruitment rate, and prior estimation of parameters such as the variance of the outcome variable.127 The resource use patterns were similar between the groups; the main items of resource use were GP, practice nurse, physiotherapist, A&E and hospitalisations. In reviewing the data, three cases had outlier cost as a result of hospitalisation. Such outlier observations can have a strong influence on the results of an economic evaluation. The cost of lifestyle activities in the 3 months prior to follow-up was in the range £50–55. The mean weekly cost of food and drink at follow-up was in the range of £75–100 across both groups, with the majority of food expenditure on groceries, followed by meals out, takeaways and alcohol spend. This pattern was the same across both groups at baseline and at follow-up.

The EQ-5D-3L and VAS and utility index values reported values on the lower end of typical ‘healthy population’ values when compared with published population norms126 and revealed expected variation in values between the measures, with the VAS (non-preference weighted) reporting lower values. There were no implausible data for the EQ-5D-3L. An improvement is observed in both groups for EQ-5D-3L. Given the high completeness rates, the EQ-5D-3L appears to be an appropriate tool for detecting changes in quality of life in a weight loss intervention. As above, although data for the ‘love, friendship and support’ domain of the ICECAP-A questionnaire were accidentally omitted, all other ICECAP-A domains were collected correctly, with no implausible data.

The appropriateness of the data collection method was assessed by completion rates, missing data and implausible values. Response rate at baseline was high, with only one and four participants in the control and intervention groups, respectively, not completing questionnaires. Loss to follow-up was seven (22%) and four (12%) for cost data and outcome data, respectively, in the control group (n = 32), and 23 (31%) and 17 (23%) in the intervention group (n = 73). At follow-up, some participants in both groups were ‘at risk of dropping out’; these participants agreed to complete a shorter follow-up (minimum data set) over the telephone (n = 14). These participants completed the EQ-5D-3L, ICECAP-A and food/grocery expenditure, but did not include the additional CRF questions linked to resource use. However, loss to follow-up was not specific to the health economic aspect of the study. Implausible values in our results regarding resource use and outcome measures were not identified. Data were collected by interview, so the participants who attended an interview responded to all questions; however, one challenging issue for data processing was blank responses in the questionnaire. For many responders, if their response was ‘no’ or if they had used no resource they left the response section blank; this blank response can be misleading with respect to the type of missingness, so a re-design of this section is advised for any future study. Such a re-design would involve asking the participant to write answers for responses such as ‘no’ or ‘not applicable’ or adding a zero value. Some data for ‘other health staff’ were collected using open-ended questions and in some cases the exact meaning of the response was unclear. In the design of a full study, the questions should not be open-ended.

Overall, the results of the health economics feasibility study showed that the questionnaires designed for measuring resource use, lifestyle and grocery spend, EQ-5D-3L and ICECAP-A would be suitable for inclusion in a full study, with some minor re-design of the resource use questions. Although the per-participant costs of the intervention were high, this cost of development is mostly upfront. The intervention refinement and ongoing hosting in a future trial will deliver a low cost per participant intervention.