Periodic Reporting for period 4 - K4U (Knowledge For Use [K4U]: Making the Most of Social Science to Build Better Policies)
Reporting period: 2020-05-01 to 2021-10-31
Horizon 2020 urges ‘Research is an investment in our future.’ This investment will be wasted if we do not figure out better ways to put the vast store of knowledge accumulated to use to build better, fairer and more prosperous societies. That’s what K4U aimed to do—to figure out how to put knowledge to use to build more decent societies.. “I think of this as study in social technology,” says Nancy Cartwright, K4U project coordinator.
K4U brought together researchers from three institutes, the University of Durham and London School of Economics and Political Science in the United Kingdom and Ca’ Foscari University in Italy. The research team was highly interdisciplinary, ranging from philosophy to sociology and psychotherapy. Together the team examined 6 case studies of social interventions: international HIV/AIDS policies, British child welfare, EU mental health, reconfiguration in British health services, occupational health, work and well-being in Greater Manchester, UK, and climate services in a period of climate change. These studies provided an opportunity to carry out conceptual and theoretical analysis, research on objectivity and deliberation and an assessment of joined-up science knowledge.
Three kinds of findings are of special note:
A systems problem. One central difficulty in putting knowledge to use is understanding what knowledge is relevant and what to conclude from all the relevant knowledge together. For example, in the child welfare project K4U noted that when a child dies the usual procedure is to trace back to find who in Social Services is to blame. Instead, “Child protection is a systems problem,” explains K4U’s child protection expert Eileen Munro. “The system needs to be redesigned to make it harder for people to make mistakes.” In the case of international HIV/AIDS policies, K4U found that local knowledge systems knowledge was frequently not taken seriously, leading to “wasted efforts and spent hopes.” Similarly key workers in the Greater Manchester project addressing the interplay of unemployment and mental health were failing to work effectively with each other, without realising this. In the end, in 4 of its 6 case studies, K4U diagnosed that problems were being addressed at the wrong level, and better outcomes could be achieved by adjusting features in the underlying system rather than trying to eliminate immediate causes of mistakes.
Forecasting policy outcomes. A 2nd major K4U contribution was to develop a framework for bringing knowledge together to make better forecasts of policy outcomes. As a start, K4U made a thorough study of what can and cannot be learned from randomised controlled trials, which are good at providing average results but provide little help in burrowing down to predict if a programme will work in any particular site. As a tool for making predictions in new cases, working with the UK Centre of Excellence in Development, Impact and Learning, K4U developed a template for constructing an information-rich ‘theory of change’ for how a programme is supposed to succeeded in producing the intended outcomes in a target setting.
Objectivity. K4U developed a new account of objectivity that recognises that objectivity requires identifying the purposes the research should serve. Often well-conducted research meets the explicit requirements but misfires because it is not responsive to the broader context in which results are to be used. One simple example is in child welfare: research commissioned explicitly to find ways to get decisions made more quickly needs to recognise the implicit requirement this be done in ways that don’t jeopardise the quality of decisions. K4U also developed a set of recommendations for avoiding harms that might be created by biases in social activist research.
K4U brought together researchers from three institutes, the University of Durham and London School of Economics and Political Science in the United Kingdom and Ca’ Foscari University in Italy. The research team was highly interdisciplinary, ranging from philosophy to sociology and psychotherapy. Together the team examined 6 case studies of social interventions: international HIV/AIDS policies, British child welfare, EU mental health, reconfiguration in British health services, occupational health, work and well-being in Greater Manchester, UK, and climate services in a period of climate change. These studies provided an opportunity to carry out conceptual and theoretical analysis, research on objectivity and deliberation and an assessment of joined-up science knowledge.
Three kinds of findings are of special note:
A systems problem. One central difficulty in putting knowledge to use is understanding what knowledge is relevant and what to conclude from all the relevant knowledge together. For example, in the child welfare project K4U noted that when a child dies the usual procedure is to trace back to find who in Social Services is to blame. Instead, “Child protection is a systems problem,” explains K4U’s child protection expert Eileen Munro. “The system needs to be redesigned to make it harder for people to make mistakes.” In the case of international HIV/AIDS policies, K4U found that local knowledge systems knowledge was frequently not taken seriously, leading to “wasted efforts and spent hopes.” Similarly key workers in the Greater Manchester project addressing the interplay of unemployment and mental health were failing to work effectively with each other, without realising this. In the end, in 4 of its 6 case studies, K4U diagnosed that problems were being addressed at the wrong level, and better outcomes could be achieved by adjusting features in the underlying system rather than trying to eliminate immediate causes of mistakes.
Forecasting policy outcomes. A 2nd major K4U contribution was to develop a framework for bringing knowledge together to make better forecasts of policy outcomes. As a start, K4U made a thorough study of what can and cannot be learned from randomised controlled trials, which are good at providing average results but provide little help in burrowing down to predict if a programme will work in any particular site. As a tool for making predictions in new cases, working with the UK Centre of Excellence in Development, Impact and Learning, K4U developed a template for constructing an information-rich ‘theory of change’ for how a programme is supposed to succeeded in producing the intended outcomes in a target setting.
Objectivity. K4U developed a new account of objectivity that recognises that objectivity requires identifying the purposes the research should serve. Often well-conducted research meets the explicit requirements but misfires because it is not responsive to the broader context in which results are to be used. One simple example is in child welfare: research commissioned explicitly to find ways to get decisions made more quickly needs to recognise the implicit requirement this be done in ways that don’t jeopardise the quality of decisions. K4U also developed a set of recommendations for avoiding harms that might be created by biases in social activist research.
Much K4U research was conducted jointly with users. K4U child protection researchers worked with the Irish Government introducing Signs of Safety as a national programme. Others engaged with the UK Ministry of Defence on evidence. A third group worked with the UK Centre of Excellence in Development, Impact and Learning to develop better theories of change. And researchers in the case studies worked with stakeholders, in 4 of the studies uncovering how features of the local context were affecting outcomes.
K4U also learned lessons of importance to specific problems in its case studies, in tandem with stakeholders. For instance, K4U did theoretical analyses to learn how better to implement Signs of Safety and developed a conceptual lens on international HIV/AIDs policies with special attention to the importance of local context. Two studies using secondary data analysis enabled insights into discrepancies between employer policies for workers and workers’ needs. K4U also conducted interviews and focus groups to improve knowledge equality in multi-agency working. Also K4U research on climate services identified decision points where greater attention to user values is needed.
K4U also learned lessons of importance to specific problems in its case studies, in tandem with stakeholders. For instance, K4U did theoretical analyses to learn how better to implement Signs of Safety and developed a conceptual lens on international HIV/AIDs policies with special attention to the importance of local context. Two studies using secondary data analysis enabled insights into discrepancies between employer policies for workers and workers’ needs. K4U also conducted interviews and focus groups to improve knowledge equality in multi-agency working. Also K4U research on climate services identified decision points where greater attention to user values is needed.
K4U had a number of practice-oriented outcomes beyond the state of the art: K4U helped shape a decision tool used by the UK Ministry of Defence. In child protection, K4U suggested improvements to Signs of Safety and unearthed difficulties created by misfits between the programme and the underlying system into which it was set. Also, K4U published Improving Child Safety, a user-friendly book for practitioners. On policy prediction, K4U researchers published a 94-page paper ‘Making Predictions of Programme Success More Reliable’ for the Centre of Excellence for Development, Impact and Learning (supported by the UK Department for International Development) and also a practitioner-oriented methods brief. The study on the Manchester programme on depression and unemployment produced positive suggestions for programme improvement, and results from the study had effect on NICE guidelines for depression. (NICE sets standards for best practice in the UK National Health Service.)
On the academic front, K4U produced a better understanding of how underlying socio-economic systems afford causal pathways and uncovered the importance of knowledge inequality in hindering attempts at joined-up, cross-group cooperation. K4U research on policy prediction produced new arguments for mixed-methods research for policy deliberation and a new scheme for understanding the roles of different evidence in warranting judgments, as well as a better understanding of the roles randomised controlled trial results can play in warranting conclusions.
On the academic front, K4U produced a better understanding of how underlying socio-economic systems afford causal pathways and uncovered the importance of knowledge inequality in hindering attempts at joined-up, cross-group cooperation. K4U research on policy prediction produced new arguments for mixed-methods research for policy deliberation and a new scheme for understanding the roles of different evidence in warranting judgments, as well as a better understanding of the roles randomised controlled trial results can play in warranting conclusions.