Measuring the effectiveness of peer support schemes in adult prisons in England and Wales

Overview

What was already known:

• There has been no evaluation of the effectiveness of peer support services for social care in prisons in England and Wales (see Chapter 3).
• Routine data sets have been used previously to measure health outcomes.
• While peer support is used for social care in many prisons in England and Wales, little is known about impact.

What this chapter adds:

• This chapter explores the relevant outcomes for peer support services in prisons, what existing data might be available to assess those outcomes and where there are data gaps.

Introduction

There is some evidence that peer support services in prisons can be effective in disease detection, improved prisoner mental health, pre- and post-release behaviour and improved knowledge and skills, but very little is known about the impact of their role for providing social care support (see Chapter 3). Indeed, the scoping review found no studies that have evaluated their effectiveness. However, there are several potential benefits and risks associated with peer-supported social care support in prisons against which their effectiveness can be measured (see Chapter 7).

This chapter addresses the following research question:

• What are the important outcomes of peer support initiatives for social care in adult prisons in England and Wales? (Including for the person receiving social care, the person delivering social care, prison community, staff and health and social care.)

During the scoping phase of this project, we discovered that there was no standard routine data specifically collected on peer support services for social care in prisons and that the outcomes against which to assess their effectiveness were not well identified. For this reason, we focused the research on establishing appropriate outcomes against which to assess these services and identifying what data might be available to measure them. The plan was then to investigate where there are data gaps (i.e. outcomes for which no data exist to measure them), understand the challenges of data collection and make recommendations towards improving the monitoring of these services and enabling them to become evaluable. This chapter presents findings relating to outcomes and data. Guidance for monitoring and evaluation leading from these findings is presented in Chapter 9.

Methods

This chapter draws on findings from the stakeholder workshop which was set up to explore questions about outcomes and data, interviews with national and prison leads, and viewing of relevant data sources to understand their use and potential (see Chapter 2 for details of methods and Report Supplementary Material 6 for demographics). Where gaps were identified, we held further one-to-one discussions with identified individuals (including three researchers, a local constabulary, and a senior social worker for prisons in a local authority).

Findings

Benefits, shortcomings and challenges of prison social care peer support initiatives

The benefits and risks of these schemes are described in Chapter 7.

The use of existing data for measuring and monitoring impact

Workshop attendees and interviewees confirmed that, at a national level, there is currently no formal monitoring and evaluation of the effectiveness of peer-supported social care in prisons and consequently no data being routinely collected for this purpose (see Chapter 5). Fundamentally, there is no agreement as to what a good peer support programme should look like.

There is potential for existing data to be used for measuring some outcomes. In Table 11, we list the different outcomes for peer-supported social care in prisons in broad categories derived from the benefits and risks described in Chapter 7. Against each of these we identify data that are potentially available to measure them. Further information about the available data sources is provided in Table 12, including their coverage and completeness.

There are only a few areas that can be addressed with available data and much data that exist will be in local surveys or questionnaires. Existing routine data sets do not appear to be used.

There is no national data collection that covers peer support programmes for social care in prisons, although data are collected in some prisons and local authorities, mainly for governance purposes. The types of data being collected are shown in Box 5.

• Numbers of buddies and recipients at regular time points.
• Local authority reviews obtaining feedback from buddies and recipients.
• Information gathered at monthly meetings.
• One-to-one check-ins with buddies and recipients.

Key gaps in data and information that have been identified are shown in Box 6. Some gaps in data will depend on location and whether there are any local monitoring programmes. For example, questionnaires exist in at least one local authority for monitoring buddy activity and experiences of prisoners both providing and receiving peer support but are not widely replicated across the country. Many of these gaps were confirmed by both the workshop and interviews.

• Guidance on what the service should look like.
• The hours buddies are working and the number of prisoners they support.
• Data from healthcare services on health outcomes.
• Data on how well social care needs of prisoners are being met by non-personal buddy support.
• Experiences of prison and other professional support staff.
• Data on the non-care impact on prisoners, including relationships and risks between prisoners.
• Softer outcomes such as dignity, self-respect and self-confidence.
• Monitoring activity and outcomes after transfer between prisons.
• Monitoring activity and outcomes after release, including continuity of care and monitoring some impact related to experiences in prison, such as employability.

There is very little information captured on outcomes for prisoners after release which could be partly due to logistics and problems with obtaining their engagement. Some information may come from the probation service systems and recidivism can be monitored using police national data. If the prison is in the same local authority as the place of residence and the prisoner has been receiving formal support under supervision of the local authority, then there ought to be some continuity in their care records, although this relies on any new needs identified while in prison being picked up. If the prisoner lives elsewhere, such continuity is put at risk. Similarly, information can be lost when prisoners move between prisons. Care needs, including needs for peer support, could be reported in the probation service’s nDelius system, which would allow for continuity after release or between prison establishments.

Operational data collected by local authorities are recording information on the support being provided and numbers both delivering and receiving support. This may go some way towards assessing whether needs are being met. At least one local authority has measured the impact of a buddy scheme on the numbers of prisoners needing formal social care support, although without an appropriate comparator. However, such data are difficult to capture in prisons where there is a high turnover of prisoners, for example, where prisoners are on remand or awaiting court orders.

Existing routine national data sets do not identify prisoners receiving or providing peer support. In routine hospital records [Hospital Episode Statistics (HES) and secondary users service (SUS)], prisoners can be identified by postcode matching.132 These data may allow the measurement of how well the overall social care needs in a prison are being met. So, for example, to identify admissions to hospital for conditions or incidents that may reflect poor care (falls, diabetes complications or self-harm) and whether peer support schemes might impact on this. However, the numbers of individuals involved in these schemes may be too small to detect any effect. Also, some caution would need to be exercised when using data sets that record clinical outcomes or use of external clinical resources, as better support for prisoners could increase rather than decrease the use of external services, for example, if clinical needs are better picked up by the buddies. Moreover, many hospital visits may be necessary regardless.

Incidents of prisoner self-harm or assault should be reported in the prison incident reporting system (IRS), but it is not always possible to identify individual prisoners from these records and there is inconsistency between prisons as what gets reported.131

Workshop participants and interviewees could not identify any examples of where established tools for measuring quality of life, mental health, or well-being were used outside individual research studies.

Developing data collection for improving the care and well-being of individuals

Workshop participants and interviewees mentioned how the care and well-being of prisoners with social care needs rely on capturing good information on the support that is being received. This would include fundamental information about an individual’s care needs, whether they are receiving support and the type of support, and picking up changes in their needs. Moreover, this should be reported in care plans that are passed over to the relevant authorities after release or to another prison after transfer. Suggestions were made about digitalised national records with, for example, details of referral, assessments, and peer support. This is important not just for making sure individuals receive the support they need (both in prison and later in the community), and reducing problems caused by loss of information but also facilitates an understanding of how effective support services are. However, people recognised that it would be a particular challenge to identify all the needs of prisoners who only receive informal care through peer support.

Well-being, quality of life and mental health measurement tools were cited as being useful for measuring effectiveness. Some have been used in research studies, but there is currently no guidance as to which to use routinely.

The value and feasibility of a national data programme for monitoring programmes and their evaluation

Workshop participants and interviewees mentioned a need for a national, standardised way of monitoring and recording data and indicators alongside mechanisms to enable national quality assurance. However, it was recognised that this would be challenging with regard to ownership and logistics. It would also need particular local arrangements as not all prisons are run in the same way.

For example, any evaluation using routine data sets such as HES or the Police National Computer would rely on having a process for identifying prisoners providing or receiving peer support in the data. At a national scale, this would mean establishing a national list of these prisoners from local data that is regularly maintained.

Viewing operational data on a national scale would be useful in assessing variability of provision, although this would require standardising the data and extracting it from numerous different local databases.

Evaluation challenges

Finding comparators for any analysis of impact using existing data would be a challenge as data collected by the services are not collected with evaluation in mind. Given the lack of funds and resources to support such schemes, there is also the issue about defining the counterfactual and where to measure from. For example, is the comparison against an absence of any support schemes (where prisoner needs might be met through informal arrangements with other prisoners) or where increased use of professional support services are filling the gap?

Also, there are many different ways peer support schemes are implemented and in how training is provided (see Chapter 5). For example, one region has a 14-day training course compared to another which is 2 days, and some buddies receive no training at all. These differences need to be reported and accommodated within any national evaluation. If all the data are amalgamated and viewed as a whole, then it would not show the full picture and might hide the best performers.

Practicalities of improving data collection: purpose and motivation

The role of commissioners of peer support services (e.g. the local authority) was discussed in the workshop. Commissioners should have an important role in improving data collections to help understand how effective peer support services are. There are similarities with some of the challenges of general support and social care when trying to justify what is often called non-cashable or non-tangible benefits. These are hard to articulate and when trying to build a case for quality improvement while money is tight, it needs to be robust to convince finance departments and budget setters.

It is therefore important to be clear about the overall aims of peer support in social care, the purpose of collecting data and how they would be used in the wider system. Indeed, anyone commissioning or collecting new data would need to understand this. There are also questions as to the proportionality of extra reporting and decisions about what could be mandated. These needs to be articulated in legislative terms and with reliable data.

Workshop participants raised the role of HMPPS and their need for good data on peer support schemes and their effectiveness in order to support decisions about whether to mandate them. The Care Quality Commission (CQC) and HMIP should also have an interest in good data on needs of prisoners and the support they receive in order to assist with their new inspection programme for social care provision in prisons.

Discussion

Key findings

We have identified a range of important outcomes for evaluating peer-supported social care services in prisons. These include outcomes for the wider system and society, for prisons and prison staff, and for prisoners both providing and receiving support. Recommendations for how to measure outcomes for these services are described in Chapter 10.

There is no routine national data collected with which to measure these outcomes. Some data are collected by some individual local authorities, but this does not seem to be widespread. Also, examples of such data we have found are largely operational rather than useful for evaluation, although some local authorities do undertake occasional surveys and questionnaires for gathering experiences of prisoners involved in these programmes.

There is a lack of data with which to measure many of the potential benefits and shortcomings of these services. There are particular data gaps that affect any ability to measure the impact on prisoners themselves, including safety concerns, the experiences of prison and other professional support staff, how well social care needs of prisoners are being met, and for monitoring outcomes on prisoners after release or prison transfer. Some aspects appear particularly difficult to evaluate, for example the impact on prisoners who do not have formal care plans and whose needs are met informally by buddies.

It may be feasible in research studies to use some of the existing routine data collections to infer possible impacts of these schemes, such as on hospital attendance, self-harm, and recidivism after release. However, without accurate information as to who is providing or receiving peer support, such analysis would only be achievable at the prison level.

There is a recognised need for better data and for there to be national standards for what that data should be and this would be easier if it was linked to national standards on service provision. There is a particular need to record when someone is in receipt of informal support as the responsibilities for capturing that data are not always clear.

However, a national data strategy may be some way off as there would need to be agreement between different stakeholders, such as prisons, local authorities, and inspectorates, as to what data to collect and how to report a list of prisoners providing and receiving peer support across prisons with very different systems. Also, the aims, responsibilities, accountabilities and resources required for this process would need to be defined.

How findings relate to previous research

It is understood that provision of social care across prisons varies, but the effectiveness of peer support schemes in comparison to other modes of delivery remain unstudied.1 Indeed, in the rapid systematic scoping review we were unable to identify any previous studies that have evaluated the effectiveness of peer-supported social care in prisons, both national and international (see Chapter 3).

Strengths and limitations

One strength of this study is that we have been able to gather views from a range of stakeholders, including commissioners of peer support services, prison staff, prisoners, academic researchers, national bodies, etc. Because of the lack of data and knowledge of the overall impact of these services, we have gone back to first principles to investigate what relevant outcomes should be and to explore how existing data sets might be able to measure these outcomes. This has allowed us to identify what data could be collected and how to enable these services to be evaluated in the future.

Because this is a rapid study, we have not been able to gain a comprehensive overview of how data are being collected and used across the country. For example, there may be local initiatives we are unaware of. However, none have been identified in the discussions with both local and national bodies and researchers with extensive knowledge of peer support schemes.

Implications

Better data not only help measure the effectiveness of peer support programmes but also enable improved monitoring of their delivery and helps provide care continuity when prisoners move prisons, are released or when buddies change. With a wide variation in the implementation of these schemes, better data would also help a better understanding of which have improved outcomes as well as impact on inequalities. Guidance on future monitoring and evaluation is provided in Chapter 10.

Future research

There is much scope for further research which could support local evaluation studies of whether peer-supported social care improves outcomes and for comparing delivery in different prisons. These could include studies of established well-being, quality of life or mental health tools,133–136 newly developed surveys tailored to these interventions and the use of routine data sets linking to cohorts of prisoners receiving or providing peer support services.

Conclusions

There is a range of outcomes which could be measured to both monitor and assess the value of peer-supported social care services. However, there is no standard data collection that can be used for used for this purpose at a national level and the effectiveness of these schemes remains little studied. Without better data collection many potential risks and benefits will be unmeasured. Some local areas collect better data than others and it is possible that much could be learnt from these. There is likely to be a particular challenge in establishing responsibilities and incentives for collecting data which is perhaps exacerbated by the number of different bodies responsible for operating such schemes.