Counterterrorism evaluation Taking stock and looking ahead

by Jacopo Bellasio, Joanna Hofman, Antonia Ward, Fook Nederveen, Anna Knack, Arya Sofia Meranto, Stijn Hoorens

Source Link

Research Questions

What is meant by counterterrorism policy in the Netherlands and other countries? And what different types of counterterrorism and preventing and countering violent extremism measures, policies and interventions can be distinguished in the Netherlands and abroad? What evaluations of counterterrorism and preventing and countering violent extremism policies have been conducted over the last five years in the Netherlands and abroad? What can be said about counterterrorism and preventing and countering violent extremism evaluations characteristics? What are the differences and similarities between the identified evaluations of counterterrorism and preventing and countering violent extremism policies? What does the evaluation literature say about quality criteria for evaluations? To what extent do the identified counterterrorism and preventing and countering violent extremism evaluations meet these quality criteria?

In 2010, a study commissioned by the Research and Documentation Centre (Wetenschappelijk Onderzoek- en Documentatiecentrum, WODC) in the Netherlands first aimed to assess evaluation practice and culture in the fields of counterterrorism (CT) and preventing and countering violent extremism (PCVE). Seven years on from this initial study, in light of the continuous developments and investments occurring in the fields of CT and PCVE, there was a growing necessity for a renewed analysis of how strategies, policies and initiatives in these fields are evaluated and what advances could be made.

Mindful of this, in December 2017 the WODC commissioned RAND Europe to conduct a study aimed at investigating how evaluations of CT and PCVE policies in the Netherlands and abroad have been designed and conducted over the last five years, and what practical lessons can be drawn regarding such evaluations.

To achieve this goal, the study entailed the undertaking of three interconnected research tasks. First, the study opened with the production of an inventory of evaluations of CT and PCVE strategies, policies and interventions conducted since 1 January 2013. In parallel to this, the study team developed an analytical framework to be used for assessing evaluations collected. Lastly, the study team conducted a structured analysis and review of the evaluations inventory.

This report discusses the activities, results and findings of this study and presents recommendations for future work in this area. It is aimed at a specialist audience of academics, practitioners and policy-makers with an intimate understanding of evaluation, particularly in the context of CT and PCVE.

Key Findings

The scope, purpose and activities characterising CT have evolved in recent years in parallel with the growth of the PCVE field. This is both a response to changes in the threat landscape and a result of a growing understanding of terrorism and violent extremism. Dutch authorities have designed and implemented a wide array of CT and PCVE legislation and measures, in recognition of the need for an approach that goes beyond traditional CT measures.

The study produced an inventory comprising of 48 CT and PCVE evaluations manuscripts (38 in English, 6 in Dutch and 4 in German). These were analysed through an analytical framework to identify trends, patterns and characteristics of CT and PCVE evaluation.

When compared with results from previous reviews, results from this study suggest that a growing volume of CT and PCVE evaluations is being undertaken and that the majority of these rely on primary data from multiple sources, perspectives and methods. Nonetheless, there appear to be limits to the extent to which evaluation practice has advanced and grown evenly across all areas of CT and PCVE work as significant gaps and shortcomings continue to mar a number of evaluations (e.g. evaluations characterised by designs that undermine their ability to draw robust conclusions about an initiative's impact).

A number of issues and challenges continue to mar evaluations in the fields of CT and PCVE, not all of them are exclusive to these fields. Lessons and reflections identified from evaluations reviewed pertain to: (i) inherent complexities of the fields of CT and PCVE; (ii) challenges associated with measuring real-world phenomena; (iii) challenges associated with existing evaluation designs; (iv) practical difficulties of conducting evaluations; and (v) drawbacks and benefits of specific evaluation methods.

Recommendations

Continue to invest in the evaluation of planned and existing initiatives in the CT and PCVE policy areas, setting minimum quality and robustness requirements for future evaluations.

Develop new evaluation designs, frameworks and approaches for conducting evaluation in the CT and PCVE policy areas, encouraging the adoption wherever possible of (quasi-)experimental designs.

Encourage further research on the dynamics, drivers and factors governing the phenomena of radicalisation, violent extremism and terrorism.

Conduct regularly mapping and stocktaking exercises akin to the present study, providing resources and means required to gain access to CT and PCVE initiatives' evaluators and beneficiaries.