Trials in an international context
Randomised controlled trials of social interventions; a pilot study of their history, prevalence, barriers and facilitators in an international context.
Although the use of experimental designs (such as randomised control trials, RCTs) is increasingly accepted in medicine, this is not the case in social policy circles. Arguments against such trials tend to draw on potential problems with feasibility, ethics, cost, public and professional acceptability, and generalisability. In the UK those testing new policies or programmes have tended to avoid such designs in favour of observational, “case study”, or qualitative methods. The problem with these methods is that, although they may be more attentive to context and implementation, they make it harder to reach clear or reliable conclusions about the effectiveness (and cost effectiveness) of the interventions in question.
Despite frequently cited difficulties in implementing experimental trials of social interventions, there are in fact many examples (in particular from the US) of “social” RCTs, and these may hold lessons for those seeking to develop and implement new trials of social policies elsewhere. Investigation of the perceptions and experiences of triallists, policymakers and other “users” of trials and of evaluative research provide useful insights into the conditions under which trials may and may not be feasible; about the barriers and facilitators to their development; and about the evaluation and use of different sorts of evidence within different policy sectors.
This pilot project had two components:
- Developing methods of “mapping” existing social trials - that is, working out how to identify as far as possible existing and ongoing social RCTs in the US, UK, Australia and other countries, in order to estimate how common RCTs of social policies really are; and
- Conducting pilot interviews with a sample of senior researchers involved in policy RCTs, and with policymakers/practitioners in selected countries, in order to identify the barriers and facilitators to RCTs and the experiences of those implementing or using them.
The project was conducted in 2008 by Kristin Liabo and Madeleine Stevens at the Social Science Research Unit at the Institute of Education in London, with funds from the MRC Social and Public Health Sciences Unit and the International Collaboration on Complex Interventions (funded by the Canadian Institute of Health Research). The principal investigators were Professor Sally Macintyre, Professor Mark Petticrew (London School of Hygiene and Tropical Medicine) and Professor Helen Roberts Social Science Research Unit). This work is published and available on PubMed.
Craig P, Cooper C, Gunnell D, Haw S, Lawson K, Macintyre S, Ogilvie D, Petticrew M, Reeves B, Sutton M, Thompson S. Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. Journal of Epidemiology & Community Health 2012;[epub ahead of print]pubmed
Roberts H, Petticrew M, Macintyre S, Liabo K. The Anglo-Saxon disease: a pilot study of the barriers and facilitators to the use of randomised controlled trials of social programmes in an international context. Journal of Epidemiology & Community Health 2012;[epub ahead of print]pubmed open access
Craig P, Cooper C, Gunnell D, Haw S, Lawson K, Macintyre S, Ogilvie D, Petticrew M, Reeves B, Sutton M, Thompson S. Using natural experiments to evaluate population health interventions. Glasgow, 2011open access
Macintyre S. Good intentions and received wisdom are not good enough: the need for controlled trials in public health. Journal of Epidemiology & Community Health 2011;65:564-7pubmed open access
Roberts H, Petticrew M, Macintyre S, Liabo K, Stevens M. Randomised controlled trials of social interventions: report of a pilot study of barriers and facilitators in an international context. MRC/CSO Social and Public Health Sciences Unit Occasional Paper 19, Glasgow, 2008