Methods

In this chapter we present the objectives of the study and then describe the methods that were used to achieve the objectives. The methods are divided into three main sections: quantitative (trial) methods, embedded qualitative study methods and economic evaluation methods. We conducted a process evaluation, drawing on both quantitative and qualitative data, and the methods for this are presented in relevant sections below, and drawn upon in the corresponding results chapters. The process evaluation aimed to evaluate the trial procedures, assess the fidelity of the intervention and ascertain the reasons for the intervention’s success or failure. It drew on quantitative data collected to assess the fidelity, dose and reach, including socioeconomic patterning, of the intervention, as well as qualitative data to assess the acceptability and perceived impacts of the trial and intervention, and the reasons for success and failure of each component thereof. An initial analysis of process data was conducted before the main outcome analysis to avoid biased interpretation of process data. This allowed process data to provide prospective insights into why we might have expected to see positive or negative overall effects and generate hypotheses about how variability in outcomes may have emerged.69 They thus informed our secondary, exploratory analyses.

The proposed methods for all analyses were documented in our protocol.1 Statistical and economic evaluation analysis plans were further pre-specified in greater detail and approved by the Trial Steering Committee (TSC) prior to the analyses being commenced.

Objectives of the randomised controlled trial

The overall aim of the study was to evaluate the effects on health and well-being of a domiciliary welfare rights advice service for independently living, socioeconomically disadvantaged older people (aged ≥ 60 years), recruited from general (primary care) practices.

Primary objective

To establish the effects on HRQoL of a domiciliary welfare rights advice service targeting independently living, socioeconomically disadvantaged older people (aged ≥ 60 years) identified via primary care, compared with usual practice.

Secondary objectives

• To establish the cost consequences and the cost-effectiveness of a domiciliary welfare rights advice service targeting independently living, socioeconomically disadvantaged older people (aged ≥ 60 years) identified via primary care, compared with usual practice.
• To establish the acceptability to trial participants and relevant professionals of a domiciliary welfare rights advice service targeting independently living, socioeconomically disadvantaged older people (aged ≥ 60 years) identified via primary care.
• To identify the unanticipated consequences (positive or negative) of a domiciliary welfare rights advice service targeting independently living, socioeconomically disadvantaged older people (aged ≥ 60 years) identified via primary care.

Trial design

A pragmatic, individually randomised, single-blinded (researchers), parallel-group, wait-list controlled trial of domiciliary welfare rights advice versus usual care, with embedded economic and quantitative and qualitative process evaluations (Figure 1).

FIGURE 1 Overview of study design. (Figure not reproduced here; see the original article.)

Setting

The trial took place in socioeconomically disadvantaged areas in the North East of England, which included urban, rural and semirural areas, with no previous access to welfare rights advice services targeted to primary care patients.

The study was planned to take place in 10 of the 12 local authority districts in the North East of England (Stockton, Darlington, Middlesbrough, County Durham, Sunderland, South Tyneside, North Tyneside, Newcastle, Northumberland and Gateshead). Social services departments in these local authorities had agreed in principle to provide domiciliary welfare rights advice services. The intention was to recruit two general practices per local authority district with the help of the Primary Care Research Network – Northern and Yorkshire (PCRN-NY). Lists of all general practices from the local authority districts were obtained from the PCRN-NY and ranked according to deprivation score using the 2010 English Index of Multiple Deprivation (IMD) calculated at middle-layer super output area (MSOA) level for practice premises (main surgery) postcodes, in accordance with the method of Griffin et al.72 Deprivation covers a broad range of issues and refers to unmet needs caused by a lack of resources of all kinds, not just financial. The English Indices of Deprivation attempt to measure a broad concept of multiple deprivation, made up of several distinct dimensions, or domains, of deprivation. Super output areas are geographical units for the collection and publication of small area statistics. There are currently two layers of super output area: lower-layer super output areas (LSOAs) and MSOAs. LSOAs have an average of 1500 residents and 650 households. MSOAs have a minimum of 5000 residents and 2000 households, with an average population size of 7500, and fit within local authority boundaries. WRAs established whether or not any of the practices in the most socioeconomically deprived two-fifths of the IMD distribution had existing dedicated or targeted welfare rights advice services. Those practices in the lower two-fifths of the deprivation ranking distribution, without existing dedicated or targeted welfare rights advice services, were eligible for inclusion and were sent a letter by the PCRN-NY inviting them to participate (see Appendix 1), along with an information sheet describing the study (see Appendix 2) and an expression of interest form to return directly to the PCRN-NY.

The PCRN-NY provided the study team with a list of all eligible practices that had indicated a willingness to participate in research. If more than two general practices per local authority expressed willingness to participate in the research, the intent was to list the practices randomly and then contact them sequentially until two practices from each local authority district had agreed to participate in the trial. Fourteen eligible practices originally volunteered to participate in the research, representing only eight of the local authority areas for which we had secured the services of domiciliary WRAs. As we did not achieve our target of two general practices per local authority area immediately, we took two further courses of action: (1) we asked those practices that had originally volunteered if they would be willing to recruit additional patients (specifically, twice the number originally requested); and (2) we attempted to recruit further eligible practices through informal research networks across the North East of England. The latter strategy resulted in three further practices volunteering to participate from local authorities already involved in the study. However, it was also necessary to ask three general practices that had expressed willingness to do so, to recruit double the number of participants (Table 1). Owing to the problems with general practice recruitment, the welfare rights advice services in two local authorities that were willing to provide them were ultimately not required and the study took place in eight local authority districts.

Participants and recruitment

Participants were volunteer patients aged ≥ 60 years (one individual per household) who were not resident in a nursing home or in hospital, were not terminally ill (as assessed by their GP) and were fluent in written and spoken English.

The PCRN-NY and the Comprehensive Local Research Networks offered personnel and financial resources to participating general practices in the North East of England to identify and approach research study participants. Each participating practice was asked to generate a random sample of up to 300 people aged ≥ 60 years from their practice register. Practice staff scrutinised their list to identify any patients terminally ill or known to be resident in hospital or long-term care, who were excluded. Those who were assessed as not able to participate in the research due to poor mental or physical health, as assessed by their GP, were excluded from the study. Practice staff also checked to ensure that only one person per household had been selected for this list. If two people from the same address were found, one was selected by practice staff at random (by flipping a coin) to be retained and the other was removed from the sample list. Not all practices had sufficient patients registered to be able to generate a list of 300 people aged ≥ 60 years from their practice register; in these circumstances, they listed as many as possible.

This list of up to 300 names (of people believed to meet the eligibility criteria) per practice was randomly ordered and the first 100 patients on the list were sent a letter (see Appendix 3) and patient information sheet (see Appendix 4) signed by the senior GP partner on behalf of the practice, inviting them to participate in the trial. The figure of 100 patients was determined on the basis of the recruitment rate in the pilot study and anticipated to yield the required sample in an optimistic, best-case scenario. The letter on practice-headed paper explained that, unless the participant objected by returning the opt-out form (see Appendix 5) to the practice in the stamped addressed envelope within 2 weeks, their name and contact details would be passed to the research team, who would then contact them directly to discuss the trial further and seek informed consent.

After 2 weeks, the names, addresses and telephone numbers of those who had not opted out were passed to the research team. Research interviewers contacted these individuals by telephone to arrange, if acceptable, a face-to-face meeting at a mutually convenient time in the participant’s own home. At the initial appointment, research interviewers sought written informed consent (see Appendix 6) and then proceeded to collect baseline data.

We monitored recruitment continuously to assess whether or not and when further patients should be invited by each practice. When it became apparent that we would need to approach more than 100 patients to achieve our proposed sample size, practices were asked to write to additional patients from their original list of 300. In two cases, practices contacted all remaining patients aged ≥ 60 years. In these cases, the total number of patients invited was 387 and 330 (practices 1 and 12, respectively). In the three practices that had offered to recruit additional participants, recruitment numbers were doubled at each stage of the process.

Randomisation

Following written consent and baseline assessment, participants were randomised in a 1 : 1 ratio to intervention or control condition, stratified by practice. Research interviewers notified the project administrator after each baseline assessment that a new participant had been successfully recruited. The administrator held sequential allocation tables for each practice, independently generated from random numbers prior to recruitment [generated by a statistician (AB) using Stata version 12 software (StataCorp, College Station, TX, USA]. The administrator allocated all participants to the intervention or control group in the chronological sequence in which they were recruited and immediately sent each participant a standard letter (see Appendix 7) informing them of their group allocation. Only the project administrator had access to the allocation tables, and the allocation was thus concealed from the research team, data collectors and statisticians. The allocation was revealed to the participant and (in the case of intervention group participants) the relevant WRAs. The administrator immediately informed the appropriate local WRA team member of the contact details of each newly allocated intervention group participant and requested that they should be visited for a welfare assessment within 2 weeks. The WRAs were sent lists of control group participants to assess 24 months later, once follow-up had been completed.

Blinding

Research interviewers, who collected data from participants at baseline and 24 months, were not notified of the allocation status of participants to ensure that they remained blinded for the duration of the study.

Intervention

Here the intervention is described as designed and intended, according to Consolidated Standards of Reporting Trials (CONSORT) and Template for Intervention Description and Replication (TIDieR) guidelines.73,74 In Chapter 3, variations of this intention are documented and the implications are identified. The rationale underpinning the intervention components was rehearsed in Chapter 1.

The domiciliary welfare rights advice service comprised face-to-face welfare rights advice consultations and active assistance with benefit claims, delivered in participants’ own homes and tailored to their individual needs by a qualified WRA employed by local authority departments, or their contracted services, in the North East of England. In one local authority, welfare rights advice services were contracted and delivered by a third-sector organisation. Following randomisation, intervention group participants were given an appointment in their own home with a WRA within 2 weeks, during which participants underwent a full benefit entitlement assessment involving assessment of financial, material and welfare status; assessment of previous benefit entitlement and claims; discussion of current entitlement and options for action, including new claims (financial and non-financial). Active assistance with benefit claims and other welfare issues was given, which included completion of benefit application forms on behalf of the participant. Complex claims or those referred for further assessment or tribunal were managed as per their usual practice by WRAs. It was anticipated that initial assessments would last up to 60 minutes. Participants were followed up at home or by telephone as required by WRAs until they no longer required assistance (cases are usually ‘closed’ once all claims and appeals have been concluded), which could take several months.11

In area K, one local authority WRA provided the intervention from the start of the project until April 2013, when budget reviews restricted the availability of the WRA for the Do-Well study. A freelance WRA took over and completed the intervention in this area. In area G, the local authority WRA volunteered to provide the intervention for the Do-Well study but withdrew before recruitment began; therefore, a freelance WRA provided the intervention in this area. In area J, a local authority WRA provided the intervention at baseline but was unable to continue at 24 months (i.e. to deliver the intervention to control participants) because welfare rights posts had been lost and a new team had been formed that combined local welfare provision with supported housing. One WRA was retained by the local authority to deal with complex welfare cases but did not have the capacity to continue providing the intervention for the Do-Well study; therefore, a freelance WRA provided the intervention at the 24-month follow-up. In area L, three local authority WRAs provided the intervention throughout the study. In area M, the intervention was provided by paid staff from the Citizens Advice Bureau. In areas H and F, one local authority WRA within each area respectively provided the intervention for the duration of the study. In area E, two local authority WRAs provided the intervention for the duration for the study.

Intervention reach

Intervention reach was assessed as the proportion of those eligible to receive the intervention who actually received it. Causes of participants not receiving the intervention as intended (e.g. participants withdrawing, refusing or being lost to follow-up) and reasons given by participants (in interviews and questionnaires) were recorded when available. We also analysed the socioeconomic patterning of receipt of the intervention, and receipt of welfare benefits using IMD 201072 scores assigned at household level by matching postcode to IMD score at LSOA level.

Eligibility and receipt of welfare benefits was assessed, by type of benefit, as a proportion of those assessed by WRAs in the intervention group at baseline. This was also assessed in the control group at the 24-month follow-up.

At the outset, the intervention was primarily to be funded and provided by welfare rights advice services in the 10 participating local authority areas in the North East. However, a contingency fund of ≈£28,000 was also secured from the North East Strategic Health Authority in 2012, prior to the start of the study, to fund any excess intervention costs and the costs of any training for WRAs and GPs to ensure the effective delivery of this service with a high level of fidelity.

Training and quality control

To ensure consistent approaches to medical assessments related to relevant claims, each participating general practice was visited by a member of the research team and provided with an information pack and guidance on the completion of medical information on benefit application forms (see Appendix 8). This information was developed by the study team, with the help of a senior WRA who was involved in training and quality control in our pilot RCT, and a GP who was a member of the study team for the pilot RCT.

All participating WRAs were invited to attend information sharing events prior to intervention delivery to agree the intervention protocol, which specified the procedures for delivering welfare rights advice and the follow-up actions in a standardised manner. Intervention procedure checklists (see Appendix 9) were then given to all WRAs prior to the commencement of the study to ensure consistent delivery. Study staff closely monitored the progress of intervention delivery and maintained regular contact with all WRAs.

Fidelity assessment

We assessed the fidelity of the intervention in a number of ways. First, we asked WRAs to record the date and time of each initial welfare rights advice assessment, so we could assess whether or not these were delivered on time (within 2 weeks of the baseline study data collection). Second, we assessed whether or not initial welfare rights advice assessments were delivered as intended. This assessment was carried out, as unobtrusively as possible, by analysing audio-recordings of WRAs undertaking this aspect of intervention delivery with a small subsample of study participants. We aimed to analyse one initial welfare advice consultation per WRA (i.e. n = 19). Audio-recorded consultations were assessed by a senior WRA from a local authority not involved in the Do-Well study, selected for her extensive experience as a WRA, supervisor and welfare advice team manager. Each WRA was contacted shortly after the start of the study and asked to audio-record the next consultation undertaken with a consenting participant, and to record a brief commentary explaining the case and the actions recommended. We explained to WRAs that, as part of the research, we needed to understand how they interacted with clients and how outcomes were achieved. We avoided explicit reference to judgements about their practice, as this might have led to practice that was different from usual. Each participant was informed about the nature of the fidelity assessments, and written consent was obtained for audio-recording of the consultation. If a participant refused consent, the WRA was advised to ask the next participant they were due to visit for an initial assessment. The fidelity assessments took place between December 2012 and February 2013. Each consultation was analysed against a checklist of criteria developed by the study team (see Appendix 10). This checklist assessed the overall quality of the consultation, whether or not the correct assessment of needs had been made and that, if warranted, the correct benefits were applied for according to standard procedures. The senior WRA who listened to the recordings completed a confidentiality agreement prior to assessment, as it was felt not possible to fully anonymise the recordings. All data were held in accordance with the Data Protection Act 199875 and all sound files were individually password protected. After analysis of the fidelity assessment, all recorded files were destroyed. The research team was provided with a summary of each consultation based on assessment against the criteria of needs assessment, appropriate benefits applied for and a brief commentary on the justification for the outcomes achieved.

Comparator (wait-list control condition)

Participants randomised to the control group received ‘usual care’ (standard practice) from both health and welfare rights advice services after randomisation until they had completed their 24-month follow-up assessment. They were given no advice regarding welfare rights as a part of the study intervention during this period. However, they were free to seek welfare rights advice independently from their local authority or any third-sector provider at any time. Participants who sought independent advice remained in the trial and were analysed in the control arm on the intention-to-treat principle, and details of any advice and ensuing claims and outcomes were recorded at the 24-month follow-up assessment. Following the 24-month follow-up assessment, participants in the control arm received the intervention, as delivered to the intervention group (described above), including all follow-up visits by WRAs and assistance with claims and appeals over the following months, until all claims had been resolved.

The participants were informed during recruitment, via the patient information sheet (see Appendix 4), that we wanted to recruit 750 people into the study. They were told that one group of 375 people would be given an appointment with the welfare advice service straight away and another group of 375 people would be given an appointment around 24 months later. Potential participants learned that the group they would be put in would be decided by chance, like tossing a coin, but that everyone would receive an appointment within 24 months. Once individuals agreed to participate in the study and completed baseline assessment, they were sent a letter (see Appendix 7) detailing which group they had been assigned to. Wait-list control participants were told that the group they had been assigned to would mean that they would see a WRA approximately 24 months after they entered the study.

Both intervention and control group participants remained clients of the welfare advice service beyond the end of the trial, if necessary, until such time as their help was no longer needed, as per usual welfare rights advice service protocols.

Data collection

Primary outcome measure

The primary outcome measure was HRQoL, measured using the CASP-19 questionnaire.57,63 CASP-19 comprises 19 questions in the four domains of control, autonomy, self-realisation and pleasure. The range of the scale is 0–57, with higher values indicating better quality of life.63 CASP-19 was administered by face-to-face interview at baseline (pre randomisation) and at follow-up 24 months post randomisation, and by postal questionnaire (with two reminders at fortnightly intervals including one duplicate questionnaire) at 12 months post randomisation (see Appendix 11).

Secondary outcome measures

The following secondary outcomes were collected by face-to-face interview at baseline (pre randomisation) and at follow-up 24 months post randomisation (see Appendix 11).

Mental health

Mental health was measured by the Patient Health Questionnaire-9 (PHQ-9) depression scale.76–78 The PHQ-9 examines nine mental health problems. The potential range of the scale is 0–27, with lower values indicating fewer depressive symptoms.

Perceived financial well-being

Perceived financial well-being was measured by the Affordability Index.79 The 13-item Affordability Index has a potential score range of 4–20, with lower scores indicating fewer financial problems.

Standard of living index

The standard of living index is a 0–24 scale assessing ownership of 24 household items, based on questions used in the British General Household Survey.80 A higher score indicates a higher standard of living. The items enumerated are listed in question 42 in Appendix 11.

Social support and participation

Social support and participation was measured by social interaction and strength of confiding relationships.11 The social interaction score has a potential range of 0–27, with higher scores indicating a higher level of social engagement and support. An isolation indicator was created from one item on this scale, categorising whether or not participants reported that they did not see friends and relatives as often as they wished.

General health status

General health status was measured by the EuroQol-5 Dimensions-3 Levels (EQ-5D-3L) instrument.81,82 The EQ-5D-3L is a five-item scale with three levels of response. The tool has been extensively used in studies and full details are reported on the website of the EuroQoL Group (www.euroqol.org/).

Health-related behaviours

Health-related behaviours were assessed by self-report, to measure change in key indicator behaviours, such as smoking, alcohol consumption,83 diet (consumption of key food groups) and physical activity [Physical Activity Scale for the Elderly (PASE)],84 as in our pilot RCT.11 The diet score had a potential range of 15–75, with higher scores indicating a healthier diet in terms of salt, fat and sugar consumption. The physical activity score had a potential range of 0–400+, with higher scores indicating higher levels of activity.

Mortality

Mortality was assessed by identifying deaths at 12 months and 24 months from GP records. General practices were sent a list of participants and asked to check their status. This was carried out prior to commencing the 12- and 24-month follow-up assessments, in order to avoid distressing any recently bereaved relatives.

Financial status

Financial status was measured using an assessment tool developed and used in our pilot RCT.11 This included data on all sources of household income, including benefits, major outgoings (rent/mortgage, fuel bills, etc.), debts and capital assets (i.e. home and savings). As well as these data, at follow-up detailed data were collected (by WRAs) on new benefits received since baseline, including one-off (lump sum) payments and regular, weekly or monthly income.

Independence

Independence was measured by assessing living arrangements and carer status using the following categories: living independently or with carer support, in own home, with relations, in a care home or hospital. We also assessed (by self-report) the number of hours of home care received per week.

Harms

At the outset, we were not aware of any major risks or harms associated with the delivery of the intervention. However, it was possible that older people could spend additional resources in ways that would be potentially harmful. These might include spending additional financial resources on alcohol or tobacco (with known risks of chronic diseases), ‘luxury’ foods, high in fat and sugar (e.g. chocolate), or gambling, which can be addictive and financially ruinous. It was also possible that the increased independence and mobility (which we hypothesised would be associated with access to additional resources) could result in greater environmental exposures outside the home, resulting in infectious diseases or accidental injury. Furthermore, the intervention could lead to greater use of car travel, resulting in lower levels of physical activity. It was also possible that older people would feel obliged to share any additional income with family members, who may be equally socioeconomically disadvantaged, thus negating the beneficial impacts on themselves. Older people may also be more vulnerable to external pressures, such as cold-calling from salespeople. We assessed potential adverse outcomes by (1) identifying negative (unhealthy) changes in all primary and secondary outcome measures; and (2) including additional, semistructured open questions in follow-up questionnaires and interviews on other, potential, unanticipated outcomes in order to document these and develop explanations.

Other quantitative data collected

Demographic variables were collected to characterise the trial participants and adjust for potential confounding in analyses, including age, sex, ethnicity, educational level attained, employment status and living arrangements (number of household members, paying for accommodation, and whether or not emotional support was available). In addition, two scales were used.

Functional ability was measured by the modified Townsend Activities of Daily Living scale,85 which assesses a person’s ability to perform eight activities. The possible range of scores is 0–16, with higher values indicating a greater ability to perform activities of daily living.

Life events score was measured by recording eight potentially serious events, including bereavement and significant illness, that might have occurred in the past 7 months, as well as their impact on the individual.86

Sample size

A minimum of 318 participants needed to be followed up in each of the intervention and control arms (a total of 636) to provide 90% power at 5% significance level to detect a 1.5-unit difference in mean CASP-19 score57,63 at 24 months between the intervention and control groups, assuming a standard deviation (SD) of 8.7 and a correlation between baseline and 24 months of 0.74. The estimates of SD and correlation coefficients came from the results of ELSA (wave 4) restricting the analyses to those aged ≥ 60 years.87 Estimating an attrition rate between baseline and 24-month follow-up of 15% (as experienced in our pilot RCT),11 we needed to recruit 750 participants to the study (375 to each group).

There has been no published work to establish a meaningful or clinically important difference on the CASP-19 scale. However, we based the above acceptable difference on data from waves 1 and 2 of the ELSA in those aged ≥ 60 years. We investigated the adjusted mean difference in CASP-19 at wave 2 between groups whose social or health circumstances had changed.64 Examples of changes in CASP-19 score associated with changes in health or social circumstances that we might have expected to see in the proposed trial included ‘developed limiting illness’, –2.8 units; ‘developed depression’, –2.7 units; ‘lost access to car’, –1.8 units; and ‘increased chance will not meet financial needs’, –1.1 units. These differences on the CASP-19 scale suggest that a difference of 1.5 units would represent a minimally clinically important difference.65 The chosen sample size also provided power to demonstrate some clinically significant differences in secondary outcomes. For example, 750 participants would provide 90% power to detect a difference between a prevalence of 11% and 4% of clinically significant depressive symptoms (PHQ-9 score of ≥ 10).

Data handling, record keeping and sharing

Baseline and 24-month data were entered directly during interviews, via a tablet computer, into a secure custom-built database for processing and management using a bespoke content management system on Microsoft SQL^® Server and ASP.NET (Microsoft Corporation, Redmond, WA, USA). A record was maintained of any changes made to the data post entry. All personal information obtained for the study was held securely at Newcastle University and was treated as strictly confidential. The project administrators undertook entry and verification of data from 12-month posted self-completion questionnaires into a Microsoft Excel^® (Microsoft Corporation, Redmond, WA, USA) spreadsheet, with a 10% random sample checked for data entry accuracy. The data collection and transfer in this study complied with NRES88 and Caldicott guidelines89 and the Data Protection Act 1998.75

All patients were allocated a unique study identifier, which was used on all data collection forms and questionnaires to preserve confidentiality; names or addresses did not appear on completed questionnaires or on other data collection forms. Only a limited number of members of the research team were able to link the unique identifier to patient-identifiable details (name, address and telephone number) that were held on a password-protected Microsoft Access^® (Microsoft Corporation, Redmond, WA, USA) database. All study documentation was held in secure offices that were not open to the public, and all members of the research team with access to identifiable or anonymised data operated to a signed code of confidentiality. Transmission of original or hard copy records (e.g. questionnaires, interview recordings) was by secure fax, post or hand delivery by members of the research team or by the WRAs. Participants were informed in the patient information sheet (see Appendix 4) about the transfer of information to the research team and about levels of access to patient identifiable data, and were asked to consent to this.

At the end of the study, original questionnaires, interview transcripts, consent forms and final versions of all data sets will be securely archived in the Institute for Health & Society for 5 years following publication of the last paper or report from the study, in line with sponsor policy and Newcastle Clinical Trials Unit standard operating procedures. This also allows any queries or concerns about the data, conduct or conclusions of the study to be resolved. Anonymised data will subsequently be securely archived and made available for secondary analysis as appropriate. Details of data sharing arrangements are available from the corresponding author.

Trial data analysis

In this section, the methods used to organise and analyse the data are described. This is divided into three sections, covering the quantitative study, economic evaluation and qualitative analysis.

Statistical analysis

Analysis populations

The analysis used the intention-to-treat population, which comprised all participants in the group to which they were randomised, regardless of the intervention that they received. The number of participants who did not receive the intervention to which they were randomised is reported.

Analysis data sets

As interviewers collected most of the data, it was not expected that there would be many missing data on items in particular scales. However, unless specified otherwise by the scale developers, when no more than 20% of items were missing or uninterpretable on specific scales, the scores were calculated by using the mean value of the respondent-specific completed responses on the rest of the scale to replace the missing items.90

Depending on the extent of missing data on the primary outcome at 24 months, the use of multiple imputation was considered for the primary outcome CASP-19. It was decided that multiple imputation using iterative chained equations was appropriate to obtain a complete data set for the primary outcome at 12 and 24 months, conditional on survival to 12 or 24 months.91 The variables considered for the multiple imputation model were those thought a priori to be associated with CASP-19 at 12 and 24 months, as well as variables that were predictors of missingness of CASP-19 scores.92 The final model included baseline characteristics (e.g. age, gender, education, living alone), and CASP-19 at baseline and 12 months. Twenty multiple imputation data sets were produced.

The primary analysis of study end points (listed in the next section) was after the use of the simple imputation method described above to estimate missing items. A sensitivity analysis was carried out to explore the effect of this simple imputation approach. The results of the CASP-19 are also reported after using multiple imputation.

Descriptive analyses (baseline)

Baseline characteristics of the study population were summarised separately within each randomised group. This included primary and secondary outcome variables and covariates. Continuous variables were summarised by numbers of observations and mean and SD, or median and interquartile range (IQR), depending on whether or not the distribution was symmetric. The numbers of observations and percentages are reported for categorical variables. No significance testing for any baseline imbalance was carried out, but any noted differences are reported descriptively.

When covariates were available in both the Do-Well data set and the national ELSA (wave 4),87 their distributions were compared to determine the representativeness of the Do-Well participants.

Analyses of study end points

Descriptive analysis of benefits received

The numbers and percentages of any financial and non-financial benefits (e.g. aids and adaptations) received since baseline were summarised separately within each of the randomised groups.

Primary effectiveness end point

The primary end point, CASP-19, was compared at 24 months between the intervention and control groups using multiple linear regression with adjustment for baseline value and general practice (the stratification variable). The results are reported as a difference in means with a 95% confidence interval (CI). An adjusted analysis included the life events score and functional ability score at 24 months, and baseline covariates age, gender, education and whether or not living alone in the regression model. Bootstrap estimation was used if the distribution was skewed. A similar comparison was also made at 12 months (in this analysis, the life events and functional ability scores at baseline were used instead of those at 24 months).

Secondary effectiveness end points

The following continuous outcomes were assessed at 24 months:

• mental health PHQ-9
• Affordability Index
• Standard of Living Index
• social interaction score
• alcohol (timeline follow back)
• dietary intake score
• physical activity (PASE)
• receiving care – hours per week.

Each secondary outcome was compared at 24 months between the intervention and control groups using multiple linear regression with adjustment for baseline and general practice (the stratification variable). The results are reported as a difference in means with a 95% CI. The adjusted analyses also included the baseline covariates of age, gender, education and whether or not living alone in the regression model. Bootstrap estimation was used for CIs when the distribution was skewed.

The following categorical secondary outcomes were assessed at 24 months:

• living independently (yes or no)
• mortality
• change in smoking status since baseline
• whether or not seeing friends and relations as often as wished.

We used logistic regression with adjustment for general practice (stratification variable) to compare proportions between the intervention and control group. The results are reported as odds ratios with 95% CIs. An adjusted analysis in the regression model also included general practice (stratification variable), age, gender, education and whether or not living alone.

Exploratory analyses

Exploratory analyses were performed in which the linear model for the primary outcome contained terms for intervention, other key variables (sex, age in years and education) and the interaction between them.

In addition, within the intervention group, multiple linear regression explored whether the mean primary outcome (CASP-19) at 24 months differed, first between those receiving and not receiving welfare rights advice, and then between those receiving and not receiving any extra welfare benefits. As a final exploratory analysis, a comparison was made between the CASP-19 scores at 24 months for those in the intervention arm who had previously been awarded a financial welfare benefit and those in the control arm who were later awarded a financial benefit. As they were all eligible for financial welfare benefits, these participants should be similar in their socioeconomic and health profiles. All of these models also included baseline CASP-19 score, general practice, age, gender, level of educational attainment and whether or not living alone.

Analysis of trial process

The analysis for the process evaluation consisted of descriptive statistics for the recruitment and flow of participants through the trial (trial implementation), the time in days between randomisation and visit of WRA, whether or not advice was received, the benefits recorded (receipt of intervention) and the length and number of WRA consultations (dose).

Qualitative study

Sampling

Qualitative interviews were conducted with trial and professional participants. Semistructured interviews with 50 purposively sampled trial participants took place in two phases between January and July 2014 (30 intervention participants) and February and May 2015 (20 control participants). The trial participants were identified from the trial database and recruited to achieve a maximum variation sample with respect to group allocation, gender, age, receipt of benefits and any unanticipated consequences of the intervention identified at follow-up. A sample of 17 professional participants was interviewed between February and June 2015, selected on the basis of their roles in service commissioning, policy, strategy and service delivery. The professional participants were recruited from the Department for Work and Pensions, local authority and Citizens Advice Bureaux welfare rights services, primary care, public health and the third sector. One group of professional participants was directly associated with intervention delivery (i.e. WRAs) and these are identified throughout the report by the code WRA DEL.

Recruitment

All trial participants were asked during the baseline trial assessment and consent procedures if they would be willing to participate in qualitative interviews. Altogether, 657 (327 intervention and 330 control) participants consented to being contacted for a qualitative interview. This excludes one intervention participant who declined a WRA visit and another participant who withdrew consent before their allocation to a trial arm.

Those selected for interview (trial participants and professional participants) were sent a letter of invitation (see Appendix 12) and a participant information sheet (see Appendix 13) by the research team. The researchers’ contact details were provided so that those approached to participate could ask any questions before deciding to participate. Separate written informed consent was obtained for the qualitative interviews (see Appendix 14). Sampling and interviews with trial and professional participants continued until data saturation was achieved.93 During the initial qualitative fieldwork with trial participants, the partners of trial participants were often present during the interviews and made valuable contributions. After establishing that such partners were also willing to take part, we sought an amendment to the study ethics so that partners who were willing to take part could also be consented and included (see Appendices 15 and 16 for the partner information sheet and consent form). The intervention participants were interviewed between January and July 2014, after their baseline interview was completed and after they had received their welfare rights advice consultation. The control participants were interviewed between February and May 2015, after their 24-month follow-up interview and after receiving the intervention.

Data collection

The interviews with trial participants explored the acceptability of the intervention and research design, and the perceived benefits and unanticipated consequences of the intervention.

The interviews with professional participants explored the acceptability of the intervention, training and research; the fidelity of the intervention; and the probable implications of the intervention for translation into routine policy and practice, both within the North East and more widely.

All interviews were digitally recorded (with permission) and transcribed verbatim. The transcripts were checked by EN, MS and SL for accuracy of transcription. The data were anonymised, and pseudonyms were applied to preserve anonymity.

Analysis

Qualitative data were collected and analysed iteratively so that themes that emerged in early interviews were explored in later ones. A coding framework was developed, applied to an initial 10 interviews, and then revised to accommodate new insights from the initial analysis. Three interviews were double-coded by MS and SL to ensure reliability of the coding framework that was then applied to all the interview data. The data were analysed thematically using the Framework method94 with constant comparison95 and deviant case analysis96 to enhance validity. NVivo 10 software (QSR International, Warrington, UK) was used to code and manage the data.

Economic evaluation

The relative efficiency of the domiciliary welfare rights advice intervention was assessed in within-trial cost–consequences and cost–utility analyses using data collected during the 24 months of the trial follow-up. For the cost–consequences analysis, the costs and consequences of the intervention are reported separately. The cost–utility analysis combined cost data and HRQoL measures to estimate the incremental cost per quality-adjusted life-year (QALY) gained.

Estimates of cost took the perspectives of the public sector services (for the service delivery costs of the Do-Well intervention) and the Treasury (for additional benefits awarded). Sensitivity analyses were performed to assess the impact of different data sources and varying key assumptions and parameters on the cost-effectiveness of the intervention.

Intervention costs

Identification and measurement

Intervention group

The intervention costs were based on resource use and associated costs collected during the trial (supplemented by routine data sources where necessary) for every participant in the intervention group. The intervention costs included the additional time inputs required by WRAs to deliver the intervention. The service costs of delivering the intervention also required data on staff salaries (collected from local authorities) and standard reimbursement rates for the travel distances from participating localities, as well as data on typical caseloads. These data were used to estimate the average total time spent with participants and to calculate the monetary cost of a specific resource use and the average total cost per case of delivering the intervention to participants.

Time spent on visits to clients and additional time inputs were recorded by WRAs in open-format case diaries as part of the casework contact sheet (CCS) for participants in the intervention group (see Appendix 17). The self-completed case diaries enabled WRAs to report categories of their time spent on the intervention [e.g. home visit(s), telephone call(s), writing letter(s)/e-mail(s) and administrative tasks (such as form filling or referring participants to or making follow-up contacts with other agencies)].

Salary costs (including National Insurance and pension) were averaged and a cost per minute was estimated assuming that WRAs worked 46 weeks per year and 37 hours per week. The cost per minute was then applied to the actual time spent for each participant. Table 2 shows the information provided by participating local authorities and the equivalent average cost per minute of staff time and reimbursement rate for travel used in the analyses, together with sensitivity analyses. Only three local authorities were able to supply relevant data, and in two cases this was incomplete. When data were not available from a local authority, this is marked as not available.

Given that the intervention was assumed to cause no harm to participants, no additional individual-level resource use (e.g. for adverse events) was collected.

Control group

Ideally, full details of number of tasks and specific durations would have also been collected for participants in the control group to estimate the difference in costs associated with the delivery of the intervention compared with usual practice. Although case-specific activities and durations were collected by means of case diaries in the intervention group, similar information on resource use was not collected from the control group. In the absence of the full details in either arm, the economic analyses considered only the additional activities associated with the delivery of the intervention at home (i.e. travel duration and distance travelled) to calculate the incremental cost between arms.

Aggregation of costs and sensitivity analyses

The data on costs for each participant were summed to produce a total cost per participant. These data were used to estimate the mean total costs for each arm of the trial and incremental costs. All costs are reported in UK pounds sterling (£) for the 2013/14 financial year. As all costs associated with the intervention were assumed to occur during the first year of the 24 months trial period, costs were not discounted.

Given the likely skewed distribution of the cost data, bootstrapping was used to produce 95% CIs. Sensitivity analyses allowed for effects of a variation in the average WRA salary at a range of –20% to +20%. Additionally, a higher rate of reimbursement per mile travelled (£0.594/mile) was considered to account for higher reimbursement rates usually used in health-related and social sciences research.

Outcomes

Our outcome assessments captured the number of new benefits awarded to participants, as well as changes in HRQoL between baseline and 24-month follow-up assessment. New benefit claims could have occurred as a result of the delivery of the domiciliary advice in the intervention group or due to participants in the control group independently seeking welfare rights advice. However, it was expected that the number of new claims in the control arm would be lower than those in the intervention group. As the period of study was 2 years, all outcomes (financial, non-financial, HRQoL) occurring in the second year were discounted at 1.5%, the recommended rate for public health interventions.97 A sensitivity analysis investigated the impact of discounting at the UK Treasury rate of 3.5%.

Financial and non-financial benefits

Financial benefits included newly awarded monetary payments or tax credits, while non-financial benefits included newly provided aids and adaptations. Whereas financial benefits have an immediate effect on an individual’s income, non-financial benefits do not involve payments to individuals, but capture transfers in some way that may impact on individuals’ income or well-being in the longer term. To avoid a systematic bias when using data from different sources, the initial analysis used questionnaire data for both trial arms when assessing the number of new benefits awarded and weekly amounts per financial benefit. These data were combined with information on durations for financial benefits collected by means of the CCS in the intervention group.

Intervention group

Gains in financial benefits (in terms of weekly amounts for newly awarded benefits) for participants in the intervention arm were self-reported at 24 months and additionally recorded by WRAs in the CCS. The costs of these were presented from the perspective of the UK Treasury.

The total monetary gain in financial benefits per participant was estimated, based on available information on the weekly amount as reported by participants in the 24-month questionnaire and the duration of the award during the follow-up period as recorded by WRAs in the CCS. The duration of financial benefits paid to participants in the intervention group during the trial period was calculated as the number of weeks between the date of the award as documented in the CCS and the end of the study follow-up. Using CCS data allowed for the inclusion of additional information on the payment of arrears (as lump sums) that was not available from questionnaire data.

Control group

For the control group, newly awarded benefits were identified by comparing information collected on welfare benefits received at baseline and at 24-month follow-up.

As data on the duration of each financial benefit were not collected for the control group, we inferred durations for financial benefits received by participants in the control group using information on durations in the intervention group. We used median durations that were less prone to outliers in the data.

Comparison between trial arms

The data described above and multiplying the duration (in weeks) by the amount per week for newly awarded claims provided the total value of each financial benefit per participant during the trial follow-up period. The average values of benefits are reported for those participants who received new benefits in either trial arm. Comparisons between trial arms were possible after mapping questionnaire questions capturing different types of benefits onto all benefits recorded in the CCS (Table 3).

Sensitivity analyses considered IQR values of durations (first and third quartile of the duration distribution observed in the intervention group) to determine if results were robust to a variation in durations when comparing consequences between trial arms. The information on arrears payments was not included in any of the analyses because the relevant data were not available for the control group, and including arrears payments would have increased the apparent effectiveness of the intervention.

Non-financial benefits

Newly awarded non-financial benefits included Blue Badge car parking permits and new household aids and adaptations provided by the Community Equipment Service, which help people to remain independent at home. A list of aids and adaptations items was included in the questionnaire at both baseline and 24-month follow-up (see Appendix 11). Information on relevant non-financial benefits awarded in the intervention group was also recorded by WRAs in the CCS. However, using cross-validation between the CCS and data collected in the 24-month questionnaire revealed discrepancies in the numbers of claims reported by WRAs and those self-reported by participants in the intervention group, while the numbers of new claims for non-financial benefits obtained from questionnaire data for both trial arms were very similar. Therefore, outcomes associated with non-financial benefits were based on questionnaire data only capturing frequencies of newly awarded non-financial benefits for both trial arms.

Unlike the financial benefits, for which weekly amounts per benefit were reported by participants, unit costs were not collected during the trial for the non-financial benefits. Participating local authorities and public sources provided information on unit costs for aids and adaptations that were part of the non-financial benefits and listed separately in the baseline and 24-month questionnaires. The Department of Health and Social Care’s Community Equipment Services National Catalogue and Prescription Scheme (URL: http://psnc.org.uk/services-commissioning/locally-commissioned-services/community-equipment-services/) was used to provide nationally representative unit cost data for Community Equipment Services, which were used to calculate cost implications in absence of unit cost data from the participating local authorities.

Owing to the large number of items listed as aids and adaptations and a lack of variation in the numbers of newly claimed items between trial arms, the costs of installation of those aids and adaptations were not included in the analysis.

Table 4 gives an overview of questions capturing non-financial benefits (including aids and adaptations) claimed at baseline and 24-month follow-up. The associated unit costs for aids and adaptations are shown in Appendix 18.

Health-related quality of life

Changes in HRQoL over the 24-month trial period were captured by means of the EQ-5D-3L questionnaire.81 The EQ-5D-3L is a self-completion questionnaire comprising five dimensions of quality of life: mobility, self-care, usual activities, pain/discomfort and anxiety/depression. Using the EQ-5D-3L version gave respondents three response options for each of the dimensions: no problems, some problems and severe problems. The responses were recorded as part of the trial questionnaire at baseline and 24-month follow-up and reported separately for each of the five questions in order to observe how responses changed.

Using UK population tariffs, the responses to the EQ-5D-3L questionnaire were converted into scores of participant-specific health state utilities at each time point. The EQ-5D-3L scores were then transformed into QALYs using the ‘area under the curve’ method.98 From this, the mean QALY score for each group was calculated, along with incremental QALYs gained to capture the change in HRQoL between trial arms over the trial period.

The differences in mean QALYs gained between the two trial arms were estimated both without adjusting for baseline characteristics and with adjusting QALYs for baseline EQ-5D-3L, age, and gender, to account for any imbalance in the characteristics between the two groups at baseline. Imprecision surrounding incremental QALYs was estimated using bootstrapping and the level of imprecision was presented as 95% CIs.99

Missing data

To assess patterns in missing EQ-5D-3L data, we tested whether or not data were ‘missing at random’100 by creating a binary variable for missing EQ-5D-3L scores at follow-up and analysing the correlation with the primary study outcome, CASP-19 score, at follow-up. If data were missing at random, missing data were addressed by the application of multiple imputation,91 linking the missing EQ-5D-3L data at follow-up to baseline EQ-5D-3L score and a binary variable capturing whether or not participants reported requiring help at home.

Data analysis

The economic analyses estimated differences in costs and outcomes between the intervention and usual care (control group). The measures of variance for all outcomes involved bootstrapping estimates of costs, consequences (financial and non-financial benefits, QALYs gained) and incremental cost per QALY gained. All data were analysed in Stata version 13 software (StataCorp, College Station, TX, USA). In the base-case analysis (complete-case analysis), missing data were assumed to be missing at random.

Cost–consequences analysis

For the cost–consequences analysis, costs and consequences (financial and non-financial benefits as well as HRQoL) of the intervention were assessed separately. Therefore, the implications for participants’ income, based on additional benefits received during the 24-month trial period as well as changes in EQ-5D-3L, were presented in a disaggregated way and presented in the form of a balance sheet. In the balance sheet, outcomes are reported depending on which arm of the trial they favoured. Thus, the cost–consequences analysis can be used to show the trade-offs between different outcomes deemed of importance.

Cost–utility analysis

The cost–utility analysis combined cost data and HRQoL measures to estimate the incremental cost per QALY gained by the intervention group compared with the control group at 24 months. The incremental cost-effectiveness of the intervention compared with usual care was estimated using seemingly unrelated regression (SUR).101 This allowed for the simultaneous estimation of costs and QALYs gained, which were calculated at individual level, and accounted for unobserved individual characteristics that could affect both costs and QALYs and lead to the potential correlation of these two variables. Using SUR controlled for the potential bias in estimates and ensured efficient estimation.102 Additionally, the estimation controlled for the baseline covariates of age, gender and EQ-5D-3L.

For the bootstrapping, a common random seed was set and used for regressions of both costs and QALYs, to ensure that each bootstrap iteration for the differences in costs and QALYs was comparable. The results are based on 1000 bootstrap iterations. For each iteration, the SUR analysis was run on the data set, non-parametric bootstrap samples were drawn from the SUR residuals for both trial arms, predicted values of incremental costs and QALYs were calculated using the bootstrapped residuals, and differences in mean costs and QALYs between the intervention and control groups were estimated with 95% CIs to account for uncertainty surrounding the incremental cost-effectiveness ratio (ICER).

The results of the analysis are presented as point estimates for each of the 1000 bootstrap iterations in a cost-effectiveness plane describing the ICERs from each of the 1000 iterations. Finally, the ICERs calculated were compared against willingness-to-pay thresholds of relevance to UK decision-makers [e.g. the National Institute for Health and Care Excellence (NICE)’s current threshold of £20,000–30,000 as society’s willingness to pay (WTP) for one QALY gained]. The probability for the intervention to be cost-effective at different WTP thresholds was then presented in a cost-effectiveness acceptability curve (CEAC).

Ethics arrangements, governance and sponsor

A favourable ethics opinion was received from the UK NRES Committee South West – Exeter (reference number 11/SW/0260). The trial was registered as Current Controlled Trials ISRCTN37380518. The sponsor of the research was NHS North of Tyne.

Changes to protocol

In October 2011 (protocol 1–2), the protocol, participant information sheet and informed consent form were amended to include the possibility of long-term follow-up. In the event of an inconclusive outcome to the study at 24-month follow-up, it would be desirable to follow up participants at a later time point. Therefore, we amended the protocol to seek, at baseline, participants’ consent to contact them again at a later date. It was recognised that long-term follow-up would require further funding and a new protocol.

In May 2013 (protocol 2–3), the protocol was amended to correct some typographical errors. In addition, the covering letters designed to be sent out with the 12-month follow-up questionnaire (cover letter, reminder letter 1 and reminder letter 2) were added.

In August 2013 (protocol 3–3.1) the protocol was amended to increase the number of participants able to take part in the qualitative element of the study from 30 trial participants and 10 professional participants, to up to 70 trial participants and up to 20 professional participants, subject to data saturation.

In February 2014 (protocol 3–4), the protocol was amended to allow us to include other household members in our qualitative interviews. This involved the production of a separate participant information sheet and consent form for other household members (see Appendices 15 and 16).

Protocol version 4, our final approved version, is available at www.nets.nihr.ac.uk/projects/phr/09300902.103

Independent Trial Steering Committee

An independent TSC provided overall supervision of the trial. The independent TSC comprised Professor Stephen Walters (chairperson), Professor of Medical Statistics and Clinical Trials, School of Health and Related Research (ScHaRR), University of Sheffield; Professor Colin Green (deputy chairperson), Professor in Health Economics and Head of Health Economics Group, Peninsula College of Medicine and Dentistry, University of Exeter; Mr Nick Whitton, Head of Commissioning for Adult Services, Adult and Community Services, Durham County Council; Mrs Sally West, Strategy Adviser – Income and Poverty; Age UK; Dr Anne Corden, Senior Research Fellow, Social Policy Research Unit, University of York; Christine Smith (lay member); John Marshall (lay member); Professor Martin White (Chief Investigator); the study statistician (Ms Denise Howel); and the study project manager (Dr Catherine Haighton). Observers from the National Institute for Health Research Public Health Research programme and our sponsor were invited to all independent TSC meetings. Following the initial pre-study meeting, the independent TSC met annually. Its role was to monitor progress and supervise the trial to ensure that it was conducted to high standards in accordance with the protocol, the principles of good clinical practice, relevant regulations and guidelines, and with regard to participant safety and well-being. A written charter was agreed and used by the independent TSC (see Appendix 19).

Data Monitoring and Ethics Committee

This was a low-risk trial and major safety data were not anticipated; therefore, a Data Monitoring and Ethics Committee was not convened.

Patient and public involvement

We engaged with service users from the outset and throughout the study. The research design was discussed with a representative sample of low-income older people to assess its acceptability. We undertook a simulation experiment in the context of a focus group discussion with a representative sample of low-income older people (discussed in detail earlier, see Length of follow-up). These members of the public provided valuable insight and advice regarding the study design and patient-relevant end points at the outline stage. Throughout the study, direct patient involvement, via a number of focus groups with Age UK members, also supported the development of suitable letters of invitation to participate, participant information sheets and covering letters explaining the study instruments. There were also two lay members of the TSC (Christine Smith and John Marshall), who provided valuable input throughout the course of the study.