Work package 4: feasibility of a trial to compare the effectiveness of supported housing and floating outreach services

In WP4, we aimed to assess the feasibility, sample size and outcomes for a large-scale trial to compare the clinical effectiveness and cost-effectiveness of two commonly used models of mental health supported accommodation: supported housing and floating outreach. Our aim was to establish whether or not participant recruitment and randomisation to different types of supported accommodation was possible, including the acceptability to service users, clinicians and service providers, and the feasibility given the complex logistics involved. Our evaluation included a qualitative component to assess service user and staff experiences of the trial, including the process of randomisation and their views on the usefulness and feasibility of a larger-scale trial.

Methods

Trial design

This was an individually randomised, parallel-group trial.

Sample size

As this was a feasibility study, a formal sample size calculation was not conducted. The original target was to recruit and randomise 60 participants from across the three study sites.

Setting

The feasibility trial was conducted in three sites that provided both types of supported accommodation and where the study team had good links (i.e. North London – Camden and Islington; East London – Tower Hamlets, Newham and Hackney; Gloucester and Cheltenham).

Inclusion/exclusion criteria

All service users in the three study sites referred to either supported housing or floating outreach services who had housing rights in the local area and were subject to the Care Programme Approach were eligible for inclusion. Those who lacked capacity to give informed consent were excluded.

Recruitment process

Each of the three sites had a system for referral of service users to local supported accommodation services. All those referred to supported housing or floating outreach were considered for potential participation in the study. The research team first met with the relevant staff at each site to explain the purposes of the study and clarified the local referral processes. A researcher who liaised with the key personnel involved in co-ordinating the referrals system and the local clinicians making referrals was based at each study site. They gained details of any new referrals and contacted the referring clinicians to discuss whether or not the individual might be appropriate for participation in the study and, if so, the researcher contacted them to gain informed consent for participation. When the clinical team felt that it was not appropriate for the person to be approached about the study, that individual was not contacted by the researcher. Recruitment took place over 12 months from June 2015.

After 6 months, we decided to adjust our approach to try to increase recruitment. In addition to the processes described above, researchers met with the managers of acute inpatient wards and community mental teams operating across the three sites to identify any individuals being considered for referral to supported housing who had not yet been referred. Service users who did not consent to randomisation were offered to participate in a naturalistic cohort whereby they gave informed consent to complete the same research interviews as trial participants but their supported accommodation was not allocated randomly.

Allocation of participants to trial arms

Participants in the trial were randomly allocated on an equal basis to receive either a local supported housing or a floating outreach service. Computer-generated randomisation was conducted independently of the research team by a statistician from the Pragmatic Clinical Trials Unit based at Queen Mary University of London and stratified by site. The outcome of randomisation was communicated to the QuEST project manager who then communicated it to the local referrals co-ordinator, and referrer, to process the participant’s supported accommodation allocation accordingly.

Comparison services

Supported housing services provided a constant level of staffing on-site to a number of service users living in individual or shared tenancies with the expectation of move-on within 2 years. Floating outreach services provided visiting support of flexible intensity to service users living in a permanent (non-time-limited) independent tenancy.

Content of care was assessed using the QuIRC-SA,20 completed with the service manager once for each service, 6 months after the participant was randomised (assuming that they had moved to the service). This was completed only once per service.

Data collection

We collected the following metrics to inform the feasibility of a larger trial: number of referrals to the trial; number recruited; attrition (i.e. number who withdrew consent to continue with the research, declined to move to the allocated service or could not be located at follow-up); and time from recruitment to moving into either type of supported accommodation. We assessed the feasibility of potential outcome measures through collection of data from service users, support staff and service managers, at recruitment and 6 and 12 months after recruitment (Table 4).

Masking of researchers

The feasibility of using a telephone interview to collect follow-up data from service users was also explored. At the 12-month follow-up, the researcher met with the service user participant, took informed consent, then telephoned a second researcher (blind to the participant’s supported accommodation allocation) who completed one instrument from the interview battery (MANSA) with the participant. This measure was selected as all others would have invalidated the blinding by revealing the participant’s allocation.

Qualitative evaluation

We aimed to recruit five participants who had agreed to randomisation and five who had not but who had agreed to be followed up naturalistically, along with their respective referrers, to assess their experiences of the trial, the process of randomisation and their views on the usefulness and feasibility of a larger-scale trial. The aim was to elicit and detail common themes systematically that could inform the feasibility of a larger trial. lnterviews were recorded, independently transcribed and anonymised. The text data were entered into a software package (NVivo 10) to assist management and coding. The interviews were analysed using thematic content analysis; a coding frame was developed by one of the researchers (Rose McGranahan), with supervision from Stefan Priebe and Gerard Leavey, which was expanded and modified to include further codes as new themes and subthemes emerged in the course of interviews and analysis.

Data analysis

We followed Consolidated Standards of Reporting Trials (CONSORT) guidelines on the analysis of feasibility trials for the presentation of our results.46 However, our analysis was mainly descriptive and focused on the recruitment rate, acceptability of randomisation to participants and staff, ease of collection of data, characteristics of participants, other baseline and outcome variables, the feasibility of masking outcome assessments, loss to follow-up and any adverse events.

Cost-effectiveness

Service use in the period before follow-up was measured using the CSRI35 and combined with unit costs. The service costs were derived from expenditure data. Cost-effectiveness of the two types of service was to be assessed by combining service costs with the composite outcomes described above. We also planned to use the Short Form questionnaire-12 items47 and the EuroQol-5 Dimensions (EQ-5D)48 to derive quality-adjusted life-years. The use of both is appropriate because there are concerns about using the EQ-5D in this population.49 Uncertainty around the cost-effectiveness estimate was to be assessed using cost-effectiveness planes and cost-effectiveness acceptability curves.

Results

Feasibility metrics

We screened 1432 potential participants, of whom 87 were ineligible (not on a Care Programme Approach, n = 63; no recourse to public funds/housing, n = 24), 456 were deemed inappropriate for participation by the research team (no plans for move-on/new admission, n = 194; already housed, n = 99; no response from clinical team, n = 60; own accommodation, n = 51; referral withdrawn, n = 22; previously screened, n = 13; eviction in process, n = 8; moved out of area, n = 5; and clinician refused access, n = 4) and 851 were deemed inappropriate for participation by the clinical team. The most common reasons were that the individual had a high level of support needs and was felt to be inappropriate for floating outreach (n = 524) or that they had low support needs and were thus inappropriate for supported housing (n = 137). In total, 17 service users consented to participate, with eight service users randomised and nine participating in the naturalistic arm. The CONSORT flow diagram is presented in Figure 3.

FIGURE 3 The WP4 CONSORT flow diagram. CMHT, community mental health team; DNA, did not attend; FO, floating outreach; SH, supported housing. (Figure not reproduced here; see the original article.)

Attrition

Of the 17 recruited participants, three were lost to follow-up [one randomised (one refused follow-up); two naturalistic (one death; one refused follow-up)].

Time from recruitment to moving

Of those randomised, three out of eight participants (38%) moved to their allocated accommodation. The median time from recruitment to moving was 4 months [interquartile range (IQR) 1.5–5.5 months].

Collection of outcome data

Demographic data for the naturalistic and randomised cohort are presented in Appendix 4, Table 22. A high level of data collection was achieved for each outcome, with completion ranging from 76% to 100% across all follow-up periods. At baseline, 100% of measures were completed, but completion rates dropped as the study progressed, primarily because of participant attrition (n = 3) described above. Of the measures, the Time Use Survey and the Scale to Assess Therapeutic Relationship (STAR) had the lowest completion rates. The completeness of data collection, organised by time point, is presented in Appendix 4, Table 22.

Our blinding procedures, tested at the 12-month follow-up, were successful. Telephone administration of the MANSA, by a researcher who was unaware of the participants’ allocation (supported housing/floating outreach), was completed without revealing the participants’ experimental condition.

Harms/unintended consequences

No harms or unintended consequences occurred during the course of WP4.

Economic evaluation

The CSRI and EQ-5D data were collected at baseline and 12-month follow-up. Owing to the very small numbers, it was not feasible to conduct the health economic analyses.

Qualitative findings

We carried out in-depth qualitative interviews with 11 service user participants and 10 staff [including care co-ordinators who referred participants to the study (n = 6), and those who knew of the study but did not refer participants (n = 4)]. Four main themes emerged from the service user and staff interviews that shed light on the impediments to recruitment: preference for a certain type of supported accommodation, rejection of randomisation, complexity of randomisation and value of the trial.

Preference for a certain type of supported accommodation

Staff interviews revealed a strong belief that supported housing and floating outreach offered quite different support to individuals and they therefore struggled to consider an individual as potentially suitable for either service. They voiced concerns that service users would be insufficiently or inappropriately supported in floating outreach and might be vulnerable to exploitation or at risk of relapse:

I suppose ultimately, when a decision is made to move someone into a . . . to allow them to go into an independent council flat with floating support, clinically we’ve already made the decision that you don’t think . . . it’s going to be a waste of resources for . . . Because there’s clinical reasons why you’d refer someone to a 9 to 5 project. I’d be slightly worried about medication compliance or maybe slightly worried about safeguarding issues.

They held a clear conviction that the two models operated as sequential components of a ‘step-down model’, whereby service users move from more highly supported to less supported accommodation as they gain the skills to live more independently. They spoke of how this system provided the opportunity for staff and service users to be confident that the person would be able to manage in an independent tenancy by the time they reached the point of referral to floating outreach:

. . . the structure we’ve got does work quite well because they are in [supported accommodation provider], they stay with the staff, they are tested in the 24 hour, they are tested in the low, and then off to their own flat. It’s not a bad programme really.

Although some service users had a clear preference for either floating outreach or supported housing, others appeared to see advantages and disadvantages for both types. This observation applied to participants who agreed to randomisation and those who joined the naturalistic group. Service users who expressed a preference for floating outreach felt that this model would allow them to be more independent and maintain control over their lives:

I’m [forties] years of age, I’m fed up of being monitored. I’m quite able, I can cook. I can clean. I can look after myself. I can wash my clothes. I can have a bath. I can do everything on my own.

Some consented to randomisation simply to increase their chance of being allocated to this form of support. For others, the preference for floating outreach was based on having more control over where they would live as they would be able to ‘bid’ for a tenancy in their preferred area. Sometimes this was in order to be close to friends and family, and sometimes it was part of their ‘risk-management strategy’, to avoid areas with known individuals who might exploit them or offer them illicit substances:

I like to be close to my family, you know, my daughter round, you know, my grandchildren, things like that. I thought [borough] or somewhere like that I’d like to live, if it was like that.

Well I was worried that I would end up in a bad area of town . . . I might get involved in drugs again.

Some service users acknowledged being influenced in their preference by family and staff and some reported previous negative experiences of other forms of supported accommodation and a lack information about supported housing that made them feel more positive about floating outreach as the preferred option.

Service users who expressed a preference for supported housing reported feeling concerned about the risks associated with more independent settings. Often they felt underprepared and expressed anxiety about relapse and ‘moving backwards’ if they were to move to a tenancy with floating outreach support:

I’m not ready for my flat yet, but everyone is saying I’m ready for it, but I’m not ready. I’m ready for it, but I’m ready for it now, but I just want that extra 6 months to make sure that I’m stable. I don’t want to get my flat and become unwell again. It costs the government so much money.

Many seemed influenced by their family’s or support workers’ views that they were not ready for floating outreach and some had previous negative experiences of floating outreach services. Some held a clear preference for a particular supported housing service in a specific location, close to family or friends. Others felt that the lack of available tenancies would mean that they would wait longer for a floating outreach option that supported housing. Some expressed a preference to live among others rather than alone:

I think supported housing is better for some people . . . I prefer supported because you’re surrounded with people.

Service user preferences, a lack of availability of independent tenancies leading to delays in individuals moving to floating outreach services and a perceived lack of staff resources to facilitate service users taking part in the feasibility trial were also cited by staff as impediments to recruitment into the study.

Rejection of randomisation

A number of service users and staff voiced ethics concerns about having accommodation decided by randomisation. Many service users suggested that housing was too important to decide by chance, and staff often reflected that a (perceived) lack of equipoise between supported housing and floating outreach services made random allocation inappropriate:

It’s a bit . . . We’re talking about someone’s home here, do you know what I mean? It’s a base need. It seems like something quite serious to flip a coin about, if you know what I mean?

So, yes I understand the randomisation process, but I would hate to think that it was to the detriment of the well-being of a client in a sense. There must be some clinical judgement based on where that client goes.

Specifically, staff suggested that the level of support and oversight in supported housing may be oppressive for an individual with relatively limited support needs, whereas a person with higher support needs may not receive the level of support necessary to facilitate recovery if they were randomised to their own tenancy with floating outreach. Service users also indicated that the individual and clinician should have the final say over housing and support arrangements. Similarly, staff voiced concerns about the randomisation aspect of the trial that was felt to negate the need for clinical judgement in deciding on supported accommodation options. Staff also felt that randomisation inappropriately excluded the service user from the process of decision-making about their accommodation.

Complexity of randomisation

Despite providing informed consent for participation at recruitment, one or two service users found it difficult to recall the processes relating to the process of randomisation when undertaking the qualitative interviews some weeks later. Some staff felt that the process of randomisation was too complicated for service users to understand and that this could lead to confusion or disappointment if they were allocated to a service they did not wish to move to. However, some staff also seemed to struggle with understanding the process and acknowledged this difficulty when they had tried to explain the study to potential participants:

The first time I heard about [the trial] I thought maybe it was a platform to find a way of how our clients can get accommodation easily. That’s what I initially thought, but obviously, as you indicated, it’s not about them, it’s about basically the support they can get once they get that accommodation. Yeah. That’s what I thought.

Value of a trial

Despite the many obstacles to recruitment we encountered, all those who participated in the qualitative interviews felt that it would be worthwhile to carry out a larger trial. Several service users felt that the results from such a study would be of use in understanding which models of supported accommodation worked best and many appeared to gain some satisfaction from feeling that they had contributed to this cause:

It’s helpful; you need to find out things about people who are unwell and to better things in the future to come through us who are unwell. I don’t mind helping that.

The majority of service users spoke positively about their experience of taking part in the trial, saying that they had enjoyed it. Staff also indicated that further research had potential to benefit service users, but were vague about how this might happen.

Links to other work packages

The QuIRC-SA and CAT-SA, developed in WP1, were used to assess service quality and service user experience in this WP. Findings from WP2i and WP3 indicated equipoise between supported housing and floating outreach, and service user and staff support for WP4.

Limitations

The main limitation was the major challenges we faced to recruitment, although, arguably, this was more of an outcome than a limitation. Although we made changes to our original recruitment strategy after 6 months, seeking referrals from inpatient units and community mental health teams directly, the delay in making this decision may have reduced access to potential participants. However, recruitment did not increase significantly after this point. We made extensive efforts to engage potential referrers but it is possible that we failed to make contact with some who were simply not known to us.

Key findings

• Our feasibility trial was successful in showing that a large-scale trial is not feasible.
• We screened 1432 potential participants, but only consented 17 to participate: eight service users were randomised and nine participated in the naturalistic group.
• The majority of potential participants were not approached as they were deemed to be clinically inappropriate for the study by their clinical team (n = 851).
• Of the 17 recruited participants, three were lost to follow-up (one randomised and two naturalistic).
• Of those randomised, three out of eight participants (38%) moved to their allocated accommodation.
• The median time from recruitment to moving was 4 months (IQR 1.5–5.5 months).
• Qualitative interviews indicated that participation was impeded by service user preferences for certain types of supported accommodation and staff views that individuals needed to graduate through the existing step-down pathway. There were concerns regarding whether or not it was appropriate for supported accommodation type to be decided through randomisation. In spite of this, participants broadly agreed that a larger trial would be valuable.