Discussion and conclusion

Box 15 provides a summary of Chapter 10 of this report.

Summary of key points

• In analysing and interpreting our findings, we draw on two implementation frameworks: the ‘rational model’ of policy implementation and the NASSS framework (non-adoption, abandonment, scale-up, spread, sustainability). We use these frameworks to set out practical recommendations for implementing new and emerging technology in social care.
• We identified a number of gaps in the implementation of new and emerging technology in social care, relating to: identifying the problem, establishing/weighing decision criteria, generating/evaluating/choosing the best alternatives, implementing the decision and evaluating the decision. We outline questions to raise when exploring the potential for implementing new and emerging technology in relation to each of these aspects.
• While both the rational model and the NASSS framework are helpful for structuring and summarising the findings of this evaluation, this should perhaps not be at the expense of room for sites to try out something that might make a difference (without necessarily knowing the expected outcome in advance) and to learn by doing.

Chapters 4–9 set out findings with regard to our first six research questions. We found through our field work that the decision-making processes within social care organisations adopting IndependencePlus tended to be ad hoc rather than strategic, with sites often selecting a technology before identifying the exact need that it would fulfil within their organisations. Decision-makers had expected the technology that they had adopted to achieve a lot of different goals, including: improving care planning and reducing costs for the social care system, aiding in providing preventative care and responding to care needs, supporting independent living and providing reassurance for those who draw on care and support and their carers. However, these expectations were, on the whole, not met within case study sites, both due to the technological capability of the equipment and due to factors such as a lack of capacity among social care staff to analyse data and use it to improve care at scale, and a culture of responding to needs and crises rather than preventative care.

Building on these data, the current chapter seeks to draw these insights together in order to answer our final research question: how can the process of implementing new and emerging digital technology be improved? In the process, we draw on two frameworks from the broader literature around implementation in general, and around the implementation of technology in particular:

• the ‘rational model’ of policy implementation
• the NASSS framework.

Using both models, the chapter sets out a series of practical recommendations for local authorities and care providers seeking to implement new and emerging technology in future, and for technology providers/innovators seeking to work with local authorities and care providers. These messages will be fed back to participating sites to contribute to local discussions about current and future strategy. They will also form the basis of more practice-orientated outputs from the study (e.g. a checklist or toolkit that enables local authorities, care providers and technology providers to plan and structure future approaches to implementing new and emerging technology). Finally, we conclude with reflections on the strengths and limitations of this study, and on implications for future research.

To what extent is new and emerging technology implemented in a ‘rational’ way?

A common framework within the implementation literature is the ‘rational model’, usually associated with Simon (1945)57 (see Figure 1). While different versions of this have emerged over time, the model is usually presented as a series of stages to work through, with a clear statement of the problem to be solved, a weighing up of the options available, implementation of the chosen approach and evaluation of the impact/lessons learnt – with insights usually feeding back to inform future implementation. Ever since, this model has been subjected to a number of critiques – not least that it seems not to describe what actually tends to happen in practice (see Powell 201158 for further discussion). As a result, other writers set out more of an ‘incremental model’, usually associated with Lindblom (1959).59 Here, decision-makers tend to take a small number of incremental steps, but often remain broadly within the status quo – and implementation is more a matter of ‘muddling through’ than it is a neat, linear, transparent and rational process.

FIGURE 1 The ‘rational model’ of implementation (authors’ own version of Simon’s rational model). (Figure not reproduced here; see the original article.)

However, as Powell (2011, p.14)58 observes, criticisms of the rational model which focus on it not being a good description of what happens in practice perhaps miss the point. In his view, ‘much of this misses the difference between description and prescription … The approach essentially prescribes how policy ought to be made rather than describing how it is actually made in the real world. [Critics of the rational model] conclude that [it] is too idealistic, while the [incremental model] is more realistic but too conservative … In one sense, then, many critiques of the ‘rational’ model often fire blanks at a straw person as they conflate descriptive and prescriptive perspectives.’ Thus, despite its not describing how implementation actually tends to occur, we have still found this a helpful model for comparing the findings in our study with an ‘ideal type’ of how new ways of working might, could, and even perhaps should be implemented. To emphasise, this is not to criticise sites for not being ‘rational’, but rather an attempt to see if there are stages of the model that might help to reflect on local approaches to implementing IndependencePlus and identify improvements that could be made in future. Moreover, in the case of the present study, an alternative focus on ‘muddling through’ is hardly helpful, either – as sites were seeking to explore the possibilities offered by new and emerging technology precisely because it may hold the potential to be transformative (in one sense, the very opposite of ‘muddling through’).

Using the steps in Figure 1, a number of potential gaps are evident in our study:

• Identifying the problem: as set out in Chapter 4, different stakeholders were seeking to resolve a number of potentially different problems/seeking a number of different outcomes, with an apparent lack of a shared understanding of what success would look like.
• Establish/weigh decision criteria: this does not seem to have happened in our case studies, with decisions made on a more ad hoc basis.
• Generate/evaluate/choose the best alternatives: in our study there does not seem to have been a process for systematically considering different options, with decisions taken on a more pragmatic basis.
• Implement the decision: this is only one of the stages in the rational model, but in reality, it contains a range of complexities and a number of different issues to consider: from the engagement of staff, people drawing on care and support and carers (in the initial decision to implement and in the implementation itself) to subsequent training; from ethics and consent to the practical/technical functioning of the product; and from responding to the evolution of the technology to being clear about who was responsible for what in terms of the subsequent data.
• Evaluate the decision: sites sought to gain a number of different insights into how the technology was performing at local level, and decided to take part in this national study in order to reflect on their experiences and share key lessons with others. However, a lack of clarity around desired outcomes makes evaluation difficult (if not impossible), and any formal evaluation might have found it difficult to respond to the rapidly evolving nature of the product and the implications of COVID. Ultimately, however, sites decided that their experiments had not worked in practice, and have all moved on to consider other types of technology and different forms of transformation.

Insights from the NASSS framework

Drawing on insights from a series of apparent ‘failures’ of technology products, and a seeming tendency to overpromise and underdeliver, Greenhalgh et al.’s (2017) NASSS framework37 draws attention to the factors which can contribute to the adoption/non-adoption, abandonment and challenges to the scale-up, spread, and sustainability of health and social care technologies (see Figure 2).

FIGURE 2 Adapted from Greenhalgh et al.’s (2017) NASSS framework.37 Reproduced with permission from Greenhalgh et al.37. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. (Figure not reproduced here; see the original article.)

Reflecting on our findings in relation to the NASSS framework, there are a number of ways in which future processes of implementing new and emerging priorities might be improved:

• With the benefit of hindsight, there seems to have been a lack of clarity about the condition (or service setting) for which IndependencePlus might be most appropriate – and the change of tack part way through to focus more on monitoring of personal health data can only have compounded this issue.
• Sites felt that the technology itself simply did not meet their needs and expectations, in terms of either cost, utility or usability – and several people felt that the potential benefits of IndependencePlus had been ‘oversold’. In contrast, IndependencePlus feels that local authorities and care providers can have unrealistic expectations of what is possible from any one product and have reflected on the need for the company to do further work with future sites in order to clarify potential benefits and manage expectations.
• With a lack of clarity around desired outcomes, a degree of distrust between some participants and the technology company, changes in the nature of the product, practical problems on the ground and the unexpected and unprecedented challenges posed by COVID, the value proposition in our study was problematic and contested. It was also not always clear to us what was the responsibility of the developer and what was the responsibility of the case study sites, and how potential risks and benefits might be shared.
• With any technological innovation, more work was arguably needed with front-line staff, people who draw on care and support, and carers (adopters) – both when deciding whether new and emerging digital technology might offer a promising way forward and when implementing the subsequent product.
• The capacity of the organisation to innovate and its readiness for this form of digital, sensor-based technology was an important issue in our study. While all sites had taken a risk to experiment with new and emerging forms of technology (and were both generous and brave to share their learning with others via this evaluation), it seems clear that more work was required in terms of consultation and engagement, practical installation and maintenance of the product and securing buy-in from key stakeholders.
• Whatever sites had done with regard to the above domains, the wider system was arguably unfavourable to the delivery of some of their initial aspirations – particularly in situations where statutory input is relatively brief (preventing the ability to build up detailed knowledge of people’s routines over time) and/or crisis-focused (hindering more preventative approaches). Key fault lines between health and social care systems meant that care staff were unsure how to proceed when the nature of the product shifted to focus more fully on personal health data, leaving some workers feeling anxious and ill-equipped to respond to what the data may be telling them.
• Embedding and adapting over time was difficult given the practical difficulties that were encountered with the product itself, the impact of COVID and the frustrations experienced by a number of front-line staff. While all sites are committed to reflecting on their experiences – hence taking part in this study – and to exploring other forms of new and emerging technology in future, this particular experiment was deemed to have been unsuccessful in all three sites, and the pilots were abandoned.

While the NASSS framework was developed from secondary research and a series of empirical case studies concerned with technology implementation, we have reflected on the extent to which our focus in this study (the implementation of new and emerging technology) may potentially be subtly different. Although the framework provides a useful structure for drawing together our data and highlighting scope for improvements in the implementation process, we are also struck by one site’s observation (see Chapter 4) that:

We were all kind of, like, ‘I don’t know!’, like, ‘we don’t know what things will happen, we don’t know what the possibilities are until we have it’. So I think that was part of our interest was actually, like, we’re not sure what we’re looking for but we think we’ll know if we find it!

Although our attempts to utilise the rational model and the NASSS framework imply that sites should have a preconceived notion of what they are trying to achieve and a rationale as to why this was the best way of going about it, it is possible that we are presenting the nature of their experiments the wrong way round. In the above quote, the technology seemed promising and people gave it a go to see what it might achieve on their behalf, rather than knowing (or thinking they knew) what might be possible. Thus, both the rational model and the NASSS framework are helpful for structuring and summarising what we found, and for anticipating potential issues to tackle in advance, but this should perhaps not be at the expense of room for sites to (sometimes) experiment with and test out something that might make a difference and learning by doing.

Implications for future policy and practice

In many ways, our research identifies similar themes to many of the previous studies summarised in Chapter 1 of this report. For example, our findings echo several of the key messages that emerged from the Care City test bed evaluation that explored the implementation of digital innovations and the development of the workforce in health and social care in London.60 These included the importance of pursuing greater and continuous engagement with the workforce and those intended to benefit from the innovation, careful consideration of information governance, and increasing awareness of the consequences for existing workstreams of incorporating the innovation. With this in mind, our priority is not just to set out our findings via this report, but to use them to help avoid similar dilemmas and potential pitfalls in future.

Building on insights from the two analytical frameworks above, Table 9 sets out a series of implications for future practice. These offer no guarantees of successful implementation – not least because of the severe pressures which social care is facing. Instead, they are intended as questions which local authorities and providers might use as a checklist to reflect on their readiness when exploring future technology-based pilots. We will seek to share these with case study sites in the first instance, as part of feeding back our findings to those who took part. We will also work to develop these initial questions into a toolkit, explore collaborations with national policy and practice partners, and seek to promote this as a practical resource for others wishing to implement new and emerging technology in future.

Given that our findings tend to mirror much of the previous literature in terms of common implementation challenges and a tendency of some new technology to seem to ‘overpromise and underdeliver’, it is clear that even more work is needed to embed lessons learnt in policy and practice, hopefully preventing similar issues from arising in future. Crucially, this is not just about the actions of individual social care stakeholders. At a national level, for example, there are geographical disparities in the digital readiness of various local authorities and care providers. These can relate to the availability of reliable digital infrastructure, and a shortage of the necessary digital skills within the workforce.3 The government’s recent White Paper on social care reform has made commitments to address variations in digital infrastructure, also committing £500M for the necessary training of care staff, with a further £1M earmarked for a new Centre for Assistive and Accessible Technology.14 While such measures are welcome, some of our participants felt a degree of pressure to be adopting technological solutions to their current pressures, which might lead some organisations to make commitments to pilot interventions they are not yet fully equipped to support.10

Implications for future research

Our study looked at the implementation of one technology (IndependencePlus) at three case study sites in England. Although we endeavoured to include a broad range of stakeholders, there were limitations (as noted throughout this report) around the inclusion of front-line workers, carers and people who draw on care and support – all of whom should be included in future studies. Further research might usefully explore a number of different AI-based technologies and their introduction across sites representing a broader range of organisations and settings. Complementing the experiences of our case study sites, where the pilots were not felt to have been successful, there may also be scope to learn from implementation that is felt to have been positive, which can be used to expand the practical insights provided in Table 9. The fact that our study largely confirms already known implementation challenges perhaps raises an important question about the capacity of social care organisations to design pilots. Rather than conducting new research, perhaps a priority should be to make implementation support available on a more routine basis to help social care organisations with the design/implementation of new interventions, so that pilots are not ‘set up to fail’ due to known implementation problems that are not addressed at the outset. Of course, evaluation can be particularly challenging when, as in our study, there are competing notions of what success might look like – so practical support to clarify and measure desired outcomes might be useful.

Strengths and limitations

As set out in Chapter 1, the previous literature recognises that technology is not being used to its full potential in social care and identifies a series of gaps in our current knowledge. These include a lack of evidence on the expected or achieved impact of new technologies on people who draw on care and support and on carers, as well as a lack of evidence to help commissioners in implementing new and emerging technologies alongside existing services and understanding the potential impact on future service use and cost-effectiveness. There is also a wide range of technologies available, and little clarity as to how people wanting to explore the potential of these should go about it. We also know relatively little about the attitudes and perspectives of people who draw on care and support, carers and care staff, and about the impact on the wider workforce and services. There has also been more research in health than in social care. As a result of all this, there has often been positive policy rhetoric about the potential of technology, which tends to sit uncomfortably with a more mixed (perhaps even poor) track record in practice.

In seeking to fill some of these gaps, the current study took the decision to focus on the process of making decisions about/implementing new and emerging technologies, feeling that this way of framing the question might be more useful to local authorities and care providers than simply focusing on a piece of equipment/an innovation and asking ‘did it work?’ We believe that this decision was justified by our subsequent findings, which – despite a broadly negative experience in our case study sites – have generated a series of practical lessons to help others in future. We have also been able to gain insights from case study sites implementing a new and emerging technology (home-based sensors with AI capabilities), as well as from technology providers/innovators seeking to develop new ways of working. While there were some differences in interpretation (e.g. some sites feeling as if the technology had been ‘oversold’, and IndependencePlus feeling that some sites had unrealistic expectations), there was also significant agreement (e.g. about a number of the pilots not having a fully shared sense of what they were trying to achieve, or having full and shared clarity around desired outcomes).

At the same time, the challenges that we faced in conducting this research – in terms of the impact of COVID, the time which elapsed between the national prioritisation exercise and the subsequent research, and the myriad of practical difficulties which sites had faced – meant that the study is smaller than we had hoped (in terms of both the depth and breadth of case study sites). This has also meant that our reflections on early experiences of the pilots are based on the perceptions of senior decision-makers and operational staff, who fed back the messages they heard from people who draw on care and support, carers and front-line care staff (rather than us being able to hear these voices directly). While this is a key limitation, we still believe that the insights presented here are helpful to inform future practice, and the research conducted was all that was possible in an incredibly difficult external environment. Had we pushed any harder with already very stressed and inundated case study sites, we would have become just another burden for them at a time of unimaginable service pressure. In our view, this would not have been ethical – and would also have proved completely counterproductive (as sites would simply have withdrawn).

Summary

New and emerging technology, such as home sensors that use AI, has been identified as potentially promising in addressing a series of challenges facing adult social care. However, there is a lack of evidence to support its widespread use, and to support local authorities and care providers making decisions around the implementation of these technologies. Overall, the evidence base is sparse, methodological limitations of available studies are common, and there is a focus on health rather than social care outcomes. In response, this rapid evaluation sought to understand what senior decision-makers are hoping to achieve by adopting new and emerging technology, how they go about deciding which technology to adopt and which partners to engage, how these technologies are implemented, and the early experiences of people drawing on care and support, carers and front-line care staff.

After significant scoping work (a national prioritisation exercise, a review of the literature, engagement with key stakeholders and detailed design discussions with sites piloting this technology), the research went ahead in three case study sites. Due to a series of practical constraints, it drew mainly on the perspectives of senior decision-makers and operational leads, commenting on their own experiences and on those of people who draw on care and support, carers and care staff. This was supplemented by insights from the technology provider, and other key stakeholders involved in the development and delivery of technological innovation in adult social care. Overall, the research confirmed a number of common implementation challenges, but also adds early insights into slightly newer themes, such as the volume/complexity of data and subsequent analytical burden on untrained staff, or the challenges of implementing technology which tries to establish a long-term picture of someone’s routine in a system where interventions can often be short-term and crisis-focused.

Despite significant challenges in conducting such research in such a difficult context, a series of very clear and significant themes emerged, the study remains highly topical, and the approach adopted has helped to produce a series of tangible and significant findings. This has helped to fill some of the key gaps in the literature and to share practical lessons learnt with commissioners, providers, technology providers and policy-makers – especially at a time when technology has been so prominent during the pandemic and in recent government policy.14 Given that this study confirms so many common implementation barriers, this focus on sharing and embedding lessons learnt in order to help future implementation feels crucial – otherwise we run the risk that future studies of new and emerging technology will simply report similar challenges once again.