RRI TOOLS, a project to foster Responsible Research and Innovation for society, with society.

Final Report Summary - RRI TOOLS (RRI TOOLS, a project to foster Responsible Research and Innovation for society, with society.)

Executive Summary:
After three years of work, RRI Tools presents its contribution to the fostering of the responsible research and innovation concept. Beginning as a grouping of 26 partners and 19 Hubs across 30 European countries, it has contributed to building a Community of Practice nearing as of today 1.200 members worldwide and a Toolkit on RRI to assist everybody to apply this concept in practice.
Building upon previous efforts, the project has developed a working definition of RRI and created a catalogue of best practices to describe what RRI looks like in real life situations. Its collection of showcases exemplifies how RRI can be carried out by businesses, governments, funding bodies, research centres, civil society and educational institutions.
RRI Tools has also explored the benefits of RRI for a wide range of stakeholders - from those closest to research and innovation to those regulating its implications, as well as those affected by these implications - while also characterising the barriers that could stand in the way of it.
As a result, the project has identified the actions needed to make RRI a reality and has crafted the RRI Toolkit to help anyone who wants to make their research and innovation processes more responsible. The RRI Toolkit gathers over 500 resources developed by experts in a diversity of fields as well as by the project itself. These resources are useful for a number of goals, ranging from increasing awareness of RRI to implementing it and evaluating its outcomes. The accompanying How-Tos explain, step by step, how to use many of these resources to put RRI in practice in a range of situations for different actors. This is perhaps the most practical achievement of the project and will help people across the globe to quickly find advice and support on RRI.
To further the RRI message, the RRI Toolkit also provides educational and dissemination resources to help others develop their own training programmes, advocate and spread the word about this approach. This will help those who are new to RRI to make sense of the concept, understand how it can be applied and how it could be useful to their work. Finally, the creation of a growing, global RRI Community of Practice means people interested in RRI can engage through forums and networking events. This will ensure that the RRI Toolkit continues to evolve and acts as a source for the latest information on RRI in the future.
This future will undoubtedly bring changes. Science and technology are becoming more open, collaborative and connected than ever. R&I projects increasingly include a diverse range of partners and voices to plan, co-create and co-develop marketable solutions, adapt to social demands and anticipate unexpected impacts, transparently share knowledge and ideas with all actors, connect people and institutions in different disciplines, sectors and countries. This means we can better tackle the global challenges society faces, and use their results to openly inform responsive policies, future developments, and new educational approaches for a more knowledgeable society. This all resonates with RRI and puts its principles as cornerstones of the new science and technology trends.
Whatever the future may be, RRI principles are now firmly established at the heart of the research and innovation system. RRI Tools has contributed to set the stage; others should capitalise on this work to ensure that the word continues to spread. As science and technology progresses at a faster pace than ever before, it is essential that responsible research and innovation is embedded in governance structures around the world to make sure these developments work for the society of today and tomorrow.

Project Context and Objectives:
Project definition and objectives
The RRI Tools project was launched the 20th January 2014, with the main objective of developing a Toolkit on Responsible Research and Innovation (the RRI Toolkit) and a training programme to help implementing RRI in Europe. The ultimate goal of the project was to build a Community of Practice in Europe which ensures the use, evolution and enhancement of the RRI Toolkit and thus boosts RRI. This document reports the main activity undertaken by the project.
A wide European coverage has been possible due to the exceptionally large consortium of RRI Tools, which consists of 26 partners active in 30 countries from all over Europe. The consortium brings together considerable experience in the key components of RRI and a variety of representatives from stakeholders’ groups relevant to RRI. It consists of foundations, science centres, universities and research centres, a science shop, a chamber of commerce and a technological partner, plus related European networks.
The project aimed to be collaborative and inclusive, in order to increase creativity and shared ownership of the process. The RRI Toolkit was collaboratively designed by representatives of all the stakeholder groups involved in the Research and Innovation process. These include: policy makers, the research community, the education community, business and industry and civil society organizations. The project made a special focus on policy makers in order to impact significantly in the future governance of R&I.
19 RRI Hubs were created to reach 30 countries in the European Research Area. These hubs were responsible for implementing the project at the national and regional level, with the aim of bringing into being a European community of practice that draws together all people and organisations that are engaged in RRI and that can use and continuously contribute to the RRI Toolkit.
RRI Tools was structured around seven work packages designed to support the production and use of the "RRI Training and Dissemination Toolkit". The work was organized in 3 phases: analysis (WP1 and WP2), production (WP3) and implementation (WP4, which includes training and advocacy); communication (WP6), evaluation (WP5) and management (WP7) were transversal and continuous along the project.
1st Phase: ANALYSIS:The development of the RRI Toolkit started with an analysis of the state of the art of the RRI concept and the level of implementation of RRI and its different components, that lead to D1.2. Guidelines on the methodologies for the collection, classification and evaluation method for RRI Good Practices.
Running in parallel, WP2 was focused on identifying stakeholder groups that needed to be addressed by the consortium, and on analysing their needs in terms of implementation of RRI. Based on the outcomes of WP2, project partners could carry out work in WP3, WP4 and WP6. The stakeholder mapping was the first step in developing the methodology for the RRI stakeholder consultation process. This mapping helped to identify the institutions and individuals in each stakeholder group. This methodology, which was delivered as D2.1. Guidelines for the implementation of the stakeholders' consultation in relation to RRI, draws on the mapping as well as the working definition of RRI developed in WP1, and details how the five stakeholder groups would be consulted as to their needs and constraints.
The implementation of a Pan-European RRI stakeholders’ consultation was carried out by the 19 RRI Hubs, in the shape of interactive one-day meetings; from M9 to M11 27 Consultation Workshops were carried out across Europe; 411 stakeholders from 30 countries were involved. These workshops contributed to assessing the attitudes of stakeholders towards RRI and the extent to which they already perceive themselves as participating in the RRI process; helped to gather feedback on the project’s working definition of RRI; identified the main needs of the RRI Toolkit’s target users to increase their participation in the RRI process; contributed to a first collection of promising RRI practices. The Consultation Workshops also helped to identify the needs and constraints of all stakeholders involved in the RRI process; to receive ideas of potential tools for the RRI Toolkit; to gain an overview of the differences across stakeholder groups and EU countries; and to engage and mobilise participants acting as a catalyst in building an RRI Community of Practice (CoP) across Europe.
The first outcome of the consultations was D1.1. Policy Brief on the state of the art on RRI, which contains a common working definition of RRI: “Responsible Research and Innovation is a dynamic, iterative process by which all stakeholders involved in the R&I practice become mutually responsive and share responsibility regarding both the outcomes and process requirements.” After thorough analysis, the consultation also yielded D2.2. Report on the analysis of needs and constraints of the stakeholder groups in RRI practices in Europe, which allowed to fine tune the RRI Toolkit characteristics to the expectations of the future community of practice. As well, a first collection of 51 RRI “Promising Practices” was collected by the Hubs. With this material, WP1 elaborated D1.3. Report on the quality criteria of Good Practice Standards in RRI, and using the filter of these criteria 31 “Inspiring Practices” were selected and presented in D 1.4. Catalogue of Good Practices Standards.
2nd Phase: PRODUCTION: While all of this was happening, the actual production of the RRI Toolkit (WP3) also started in the first months of 2014. The first step was the design and implementation of the basic project’s website. This website, published in 21 languages, served as one of the main tools for information, communication and interaction, supporting other communication channels and providing a space where to inform about project contents.
This website, as explained in D3.1 - Defining the RRI Tools collaborative platform, turned into the Virtual Collaborative Platform which hosts the RRI Toolkit and sets the basis for a Community of Practice on RRI. As part of the project’s core strategy, RRI Tools relies for its implementation and dissemination on the federation of an “RRI Community of Practice”, composed by individuals and institutions with a shared interest in Responsible Research and Innovation. This Community of Practice finds a common ground in the resources provided by the RRI Toolkit, and in the virtual workspace that enables fruitful collaborations among users of these resources.
The mandate of the RRI Tools project was to develop a Toolkit that helps all those involved in R&I to familiarise themselves with RRI and make it a reality. When crafting the first version of the RRI Toolkit, the project found three main challenges: (1) A wealth of resources tackling diverse aspects of RRI is actually already available. These include expert literature, training materials, tools for practical implementation and self-assessment, and examples of good practice. The RRI Toolkit should gather all these useful resources in a single place and provide easy access to them; (2) RRI is a multifaceted concept covering a variety of topics, processes and outcomes. The RRI Toolkit should try to offer a digestible but holistic view of what RRI entails, connecting all the separate pieces; (3) RRI involves diverse actors who have various levels of expertise. The RRI Toolkit should address them all, but paying special attention to those more in need of guidance: newcomers to the concept.
With these challenges in mind, the project decided to build a Toolkit that includes both, already existing resources created by other initiatives and new resources developed by RRI Tools.
After an extensive collaborative work of intra and extra-consortium experts, the project compiled a first selection of already existing resources to be included in the RRI Toolkit. This first compilation of resources was then transformed into databases, and discussed within the Consortium in order to yield a refined version. Resources were grouped into four main categories: library elements, projects; inspiring practices and tools. Each of these resources is presented in the RRI Toolkit through a specific page, which gives a brief overview of each resource.
From the analysis of the Users’ obstacles and needs and the feedback received from external users and experts while designing the RRI Toolkit, two practical conclusions emerged in terms of developing new resources: the lack of awareness, knowledge, and acceptance of the RRI concept and the need of guidance on how to put the concept in practice. Based on these considerations, two sets of new resources were devised: (1) Introductory resources and (2) Implementation resources
The project decided to produce several introductory resources to overcome the current lack of understanding of RRI among all actors: a general home page and 11 specific sub-homes, videos and slide presentations explaining RRI to each group of stakeholders in their own terms, as well as describing each policy agenda within the RRI framework.
To complement the introductory resources and the compilation of already existing ones, the Consortium conducted a gap analysis to identify which non-existent resources were most urgently needed to implement RRI within each group of actors. The selected implementation resources are the “How-To application guidelines” which are meant to help users understand how they can solve a given challenge through examples and the contextualized use of specific resources of the RRI Toolkit. For doing so the How-Tos combine examples of what different institutions have done and explanations on how to use specific resources of the RRI Toolkit on each case. Users are introduced to concrete examples explaining how to address possible challenges they may face in their daily work.
To help visitors find what they need, the website includes several levels of navigation, first to understand what RRI is and then to browse through the Toolkit, depending on the experience and needs of users. All these features are available in 18 languages.
Given this complexity and multiplicity of contents, users, and potential applications, the RRI Toolkit offers 8 main entry gates:
• Home page: offers a first contact with RRI and the contents of the RRI Toolkit, and redirects users to the other entry gates.
• The introduction page – “What is RRI” – helps users understand what the emerging and evolving concept pf Responsible Research and Innovation covers
• Five pages are dedicated to each of the research and innovation stakeholders: policy makers, the research community, the education community, business and industry and civil society organisations. They explore what RRI means to them and suggest a selection of resources that would especially fit their needs.
• Six pages specifically address the policy agendas defined by the European Commission: ethics, gender equality, governance, open access, public engagement and science education. These pages relate the policy agendas to the holistic RRI approach, and include some recommended resources on each of these topics.
• 28 how-tos have been developed as practical guides to help visitors navigate through the RRI Toolkit and find concrete answers to specific challenges they might face. These include, e.g. how to support RRI at a national level, how to incorporate the RRI principles in a funding call, how to incorporate RRI in policy/ funding institutions, how to embed the RRI principles in research proposals and business plans or how to co-create community-based participatory research.
• The search engine allows users to look up the resources they need, filtering them by type of resource, expertise required, policy agenda target, stakeholder group, topic and more.
• The Self-reflection tool allows users reflect on the implementation of RRI in their daily work and on which strategies are most useful for their practices.
• Training and Communication pages. Throughout 2016 the project provided more than 90 training sessions on RRI and the RRI toolkit. The associated resources are available online for re-use, as well as presentations and videos to understand more in depth what RRI is about. Dissemination materials in several languages to advocate on RRI can be found on the media room as well as several channels to access the RRI Community of Practice (blog, forum, events).
All the RRI Toolkit resources are openly available to any visitor. However, RRI is much more than a wealth of resources and guidelines. Collaboration, cross-fertilization, and co-creation are at the heart of the RRI approach, which calls for all R&I actors being involved in the process of research and innovation.
To foster this multi-actor collaboration the RRI Toolkit incorporates a number of functionalities that allow users to upload resources to the Toolkit, share knowledge with a Community of Practice, find potential partners for new projects, and inform the Community on interesting events and news. Most of these functionalities are only available to the registered members of the Community
A complete description of the process of development, contents and structure of the RRI Toolkit can be found in D3.1 - Defining the RRI Tools collaborative platform and D3.2 – The RRI Toolkit
3rd Phase: OUTREACH: Training and advocacy are key aspects of the RRI Tools project. WP4 had the objective of developing and enabling both a training programme and an advocacy programme based on the RRI Toolkit, with the objective of encouraging the use and dissemination of the RRI Toolkit within the maximum number of EU Member States and Associated Countries.
The task of delivering training on responsible research and innovation is a daunting one, if one considers the vast numbers in the R&I communities (private and public) and key stakeholders who may need to be involved. Thus, a key part of WP4 has been to develop a cohort of people across the European Union who feel confident in their understanding of the RRI concept and can pass on that understanding to other relevant partners.
The RRI Tools training programme has been designed and implemented “for” and “with” constant participation of the RRI Community of Practice. Stakeholders received training that enabled them to train others, therefore turning them into active “RRI agents” whose base is the RRI Toolkit and who have a common objective of “spreading the word on RRI”. This way, active and continuous participation has been supported.
Right from the outset, RRI Tools decided that its training programme would be based on a modular approach, so that workshops could range from a brief introduction to RRI and the RRI Toolkit to immersive role-play and detailed discussion, depending on the time available and the requirements of participants. As explained in D4.1.- RRI Toolkit Training programme the project developed a series of learning outcomes to form the objectives of the training workshops. The training materials developed by the project are targeted at delivering these objectives.
Training resources are organized in three main modules, a set of eight showcases and additional videos and presentations. All training resources are openly accessible and can be used for complementing a training attended by the user, for online self-instruction, or for preparing training sessions.
Central to the training programme have been two “Train-the-trainers” workshops, held in February and July 2016. As well as providing training opportunities for all of the Hubs, they have been important forums to exchange experiences, develop new training ideas and generally to enhance the morale and cohesion of Project members for a very demanding training programme during 2016.
During the course of the Project, the RRI Tools consortium members, including Hubs and Networks, have been involved in training and advocacy events that range from one-on-one or small-team-meetings through to addressing audiences of several hundreds. Training events have tended to be with groups from a few to several tens of participants “learning” about RRI and being involved in simulated or real-life exercises to develop practical applications of what the RRI Tools project is providing.
Throughout 2016 the project has provided more than 105 training sessions on RRI and the RRI Toolkit all across Europe. These workshops have been organized by the 19 national RRI Tools Hubs. In addition the project has also carried out several training workshops for some of the main international organisations and networks working in the R&I field (in the context of their own annual meetings and conferences).
The project has also run two one-day training workshops – in June and October 2016 – for Commission staff from DG Research and other DGs. In October, in particular, participants were topic leaders developing their scoping papers with a view to including (aspects of) RRI in the Horizon 2020 calls that would be forthcoming in the next few years of the programme
Regarding the advocacy programme each of the Hubs carried out a series of meetings with relevant policy makers, often on a one-on-one basis, or with small teams. As described in D4.2 - RRI Toolkit advocacy programme the target audiences were policy-makers in national and regional government bodies, as well as the European Commission itself. As well as with governmental policy-makers, meetings were also held with leaders in academic institutions, funding agencies, learned and professional societies, formal and informal education establishments, and civil society organisations, recognising that – in line with RRI stakeholder involvement – these partners also need discussion about the aims and benefits of RRI. Additionally, more public, larger-scale, advocacy events were put on to enlarge the community of practice.
For a brief synopsis of the training and advocacy activities organised by the project see D4.3.- Report on the implementation of the training and advocacy programmes around Europe.
TRANSVERSAL TASKS:
Monitoring & Evaluation:
As for the transversal tasks, WP5 first delivered D5.1.- Report on coverage of RRI aspects in STI (Science, Technology, Innovation) evaluations which gives an overview of existing RRI evaluations, its different methods and aspects. D5.2.- Set of turn-key evaluation designs to perform internal formative evaluation processes was ready by month 6 and provides the explanation, guidance and materials needed for formative evaluation throughout the project.
D5.3.- Validation report on the RRI Toolkit describes how the RRI Toolkit has been monitored and evaluated. It shows results of the feedback directly gathered from its users via multiple channels and formats at different stages of the development process of the RRI Toolkit. The aim of the validation process was to suggest improvements and adaptations necessary to attract end users, for a sustainable integration and further development of the community of practice.
RRI is primarily about reflecting, sharing, creating, and learning in a collaborative group. This is why the project has also created a self-reflection tool (SRT), to help users to reflect upon “how RRI” their professional practice is. D5.4.- Proof of concept self-assessment benchmarking tool traces the development of the SRT from the early stages to the final version by explaining the different iterative design steps, the scope of the tool itself, as well as its structure and look and feel.
Dissemination:
Communication, awareness raising and mobilization of stakeholders were crucial in RRI Tools. WP6 was built on the dissemination efforts of the 26 members of the Consortium, and was focused on raising the awareness and visibility of the RRI concept and of the RRI Tools project among R&I stakeholders across Europe. The ultimate goal was to encourage users to join the RRI Community of Practice, making use and contributing to the RRI Toolkit
An array of communication and dissemination tools were launched at the beginning of the project, including the project´s website and its events section, the RRI Tools blog and newsletter, social media profiles - all of them regularly nurtured with fresh content; as well as press relations and articles, along with some non-digital promotional materials.
The community of registered users of the RRI Toolkit had reached 1.100 by the end of the project in December 2016, and keeps on growing since then. Thousands of followers from all over Europe keep informed about RRI Tools and the RRI Toolkit through social media and the newsletter.
By providing practical guidelines to facilitate communication activities and by constantly encouraging coordinated dissemination actions and the exchange of information and cross-fertilization of ideas within the 26 members of the consortium, a higher communication impact at the collective level has been reached.
Coordination & Management:
WP7 has overseen the correct development of the working plan, supervising all WPs to coordinate the efforts of all partners. Internal communication, the organisation of the official project meetings, the coordination of the hubs, and contractual financial and administrative management and quality control are tasks of WP7.
The last official meeting of the project was the RRI Tools Final Conference (Brussels, 21-22 November, 2016). The Final Conference was meant to be both a “celebration” of three years of intensive work and an “open market” to share and exchange ideas on how research and innovation can be further built upon more open and responsible foundation. More than 250 people attended, representing a full range of R&I actors at regional, national and European level. Nearly 80 speakers shared their views with the public on the future of openness of research and innovation and the social impact of science. The conference also hosted the Awards Ceremony of the European Foundations Award for Responsible Research and Innovation (EFARRI), an initiative that aims to recognise the best 3 RRI projects in Europe.

Project Results:
1.3. Description of main results and achievements
1.3.1. WP1: Compilation of good practices and development of good practice standards
1.3.1.1. Objectives and Milestones of WP1 (from the Dow)
The objectives of WP 1 are:
- Defining (in consensus) a working definition of RRI and its key components.
- Defining a methodology for data collection, classification and evaluation, based on formulated Good Practice Standards already being implemented at local, national and global level on the different key components
- Gathering Good Practice Standards on RRI on the different key components
- Identify weaknesses and strengths of available Good Practice Standards in RRI and develop a catalogue of Good Practice Standards in RRI
1.3.1.2. Partners involved in this WP
Leading partner: ATHENA
Other partners involved:
- ATHENA, CARIPLO, ECSITE, EUN, EXPERIMENTARIUM, IRSICAIXA and KBF – as policy agenda experts
- All RRI Tools National Hubs in the collection of RRI good practices
1.3.1.3. Progress towards objectives – tasks worked on and achievements made
T1.1 Consensus on the RRI Working definition (first sphere)
Task leader: ATHENA
Other partners involved
- FLC, UCL,
- ATHENA, CARIPLO, ECSITE, EUN, EXPERIMENTARIUM, IRSICAIXA and KBF – as policy agenda experts
The main tasks carried out have been aimed at drafting, sharing, validating and generally working towards a consortium-wide consensus on a working definition of RRI. ATHENA requested policy agenda experts in the Consortium to carry out an extensive research on the existing literature, including relevant papers, reports and existing tools from different sources on their respective areas of expertise. To fulfil this task, partners completed different “RRI matrices”.
An internal meeting took place in Amsterdam (March 2014) where members of UCL, ECSITE, FLC, KBF and ATHENA reviewed academic literature on RRI looking for the main arguments and main lines of reasoning and the real state of the art concepts in the discussions on RRI. They also started to draw the RRI model for the RRI Tools project considering the following questions: How should the model for the RRI tools project address the academic discussion, in other words, how should we find ways to put RRI into practice? How should the project deal with the dimensions of governance, ethics, open access, public engagement, science education and gender equality? How do the RRI dimensions relate to one another?
ATHENA made a compilation of all the gathered information in a document called “Background note on the RRI working definition” - a first draft of the working definition including the process requirements and outcomes that RRI should bring about.
The matrices and the first proposal of the working definition were discussed during an internal meeting (Barcelona, 22-23 April 2014 - 1st Advisory Board Meeting) with participants from UCL, IRSICAIXA, LCF, KBF, EUROSCIENCE, CVIVA, CIPAST, EBN, EUN, ECSITE, CARIPLO, EXPERIMENTARIUM and ATHENA. The Advisory Board members of RRI Tools were also present during part of this meeting. They joined the discussions about the working definition and provided feedback.
The main product of this task is D1.1.- Policy brief on the state of the art on RRI and a working definition of RRI.

Status and Deviations: This task is completed without deviations

T1.2 Methodologies for collection, classification and evaluation of RRI Good Practices
Task leader: ATHENA
Other partners involved:
- ATHENA, CARIPLO, ECSITE, EUN, EXPERIMENTARIUM, IRSICAIXA and KBF – as policy agenda experts
- All RRI Tools National Hubs in the collection of RRI good practices
ATHENA developed a methodology for identifying and classifying promising practices and developing quality criteria for good practice standards (D1.2). An internal workshop was organised in Copenhagen in June 2014 (1st Consortium Meeting) to train the RRI Tools National Hubs in executing this methodology. Hubs were invited to give feedback and to suggest adjustments/ improvements to the proposed methodology. The consortium´s policy agenda experts, as well as the rest of the partners, were asked to identify best practices according to this methodology.
The individual process requirements for RRI as identified in the working definition were further specified. To this end, ATHENA had identified criteria and indicators for each process requirement. An internal workshop was held in March 2015 in Amsterdam, with participants from FLC, ZSI, UCL, IRSICAIXA, KBF, CARIPLO and ATHENA to discuss this. The criteria were then tested with various audiences by IRSICAIXA, ATHENA and SCIENCE ANIMATION.
The main products of this task are D1.2.- Methodology for the collection and classification of RRI practices and D1.3.- Report on the quality criteria of Good Practice Standards in RRI

Status and Deviations: This task is completed without deviations

T1.3 Consensus of Working RRI definition and Collection of Good RRI Practices by the HUBS (second sphere)
Task leader: ATHENA
Other partners involved: All RRI Tools National Hubs
The RRI Tools National Hubs were responsible of carrying out this task under the instructions of ATHENA and ECSITE (WP1 and WP2 leaders). The final consensus on the working RRI definition and on the collection of good practices took place during the Hubs Consultation Workshops. The methodology applied was designed by WP1 and WP2 leaders and FLC, IRSICAIXA, ZSI and CVIVA as Hubs Coordinator. The full explanation of the Consultation Workshops is available in WP2- T2.1.

Status and Deviations: This task is completed without deviations

T1.4: Analysis of Good Practices to develop Good Practice Standards
Task leader: ATHENA
Other partners involved: All RRI National Hubs in the reporting of the promising practices to ATHENA
One of the main outcomes of the consultation workshops was the collection of case studies: a total of 51 promising practices were collected and analysed by the Hubs and sent to ATHENA, using an online tool. Of these 51, 31 were selected and presented in the D1.4 Catalogue of Good Practice standards in RRI. This catalogue is one of the most relevant inputs for WP3: the Promising Practices were included in the RRI Toolkit as one of the 4 main types of resources (see WP3 for more details)

Status and Deviations: This task is completed without deviations
1.3.1.4. Deliverables and Significant Results
The work done by ATHENA as WP leader is presented in four deliverables produced along the first year and a half of activity of the project.
D1.1 Policy brief on the state of the art on RRI and a working definition of RRI (M8)
The policy brief presents the working definition of RRI that, from its publication onwards, has been central to the project. This definition runs as follows:
Responsible Research and Innovation is a dynamic, iterative process by which all stakeholders involved in the R&I practice become mutually responsive and share responsibility regarding both the outcomes and process requirements.
The process requirements referred to are: (1) inclusion and diversity, (2) anticipation and reflection, (3) openness and transparency, and (4) responsiveness and adaptive change.
The outcomes concern either learning, ethically sound, socially desirable and sustainable R&I, and working towards the solution of the 7 Grand Challenges the EC has identified.
This definition builds on the existing scientific and policy literature and is discussed against the background sketch of the problems RRI is supposed to address. The policy brief also illustrates the concept of RRI through a number of brief case descriptions, and provides a concise overview of the RRI Tools project.
D1.2 Methodology for the collection and classification of RRI practices (M6)
The final methodology for the collection and classification of RRI good practices describes a collection phase, a classification phase, and an analysis phase:
- Identifying and collecting “promising practices” with help of the hubs and of the participants in the stakeholder consultation workshops (M9-M11) using especially designed question sheets;
- Selecting promising practices for deeper analysis for which a survey, based on the working definition (D1.1) is used, and then carrying out the surveys for these practices;
- Analysing the surveys (ATHENA) to come to a final selection of good RRI practices for the catalogue.
D1.3 Report on the quality criteria of Good Practice Standards in RRI (M15)
D1.3 presents the quality criteria of good practice standards, which have been based on the working definition of RRI (D1.1). As such, the criteria and sub criteria are formulated per process requirement and further specified in the form of questions (and sometimes examples). Although no independent criteria are formulated for outcomes of R&I practices (for various reasons), the integrated nature of outcomes and processes has been made visible in the tables containing the quality criteria. The same also holds for the policy agendas.
It is not the intention to use these criteria and questions as a tick-box exercise – as that would not be consistent with what RRI stands for – but rather as a thinking exercise on whether and how a practice aims to be more responsible. The guiding principles we believe on are on the core of RRI and are presented in D1.3:
(1) democratic values regarding participation and power,
(2) social and moral values regarding the care for the future of our planet and its people
(3) individual and institutional values of open-mindedness or receptiveness to change.
D1.4 A catalogue of good RRI practices (M18)
This catalogue provides a compilation of good RRI practices. The aim of the document is to present examples of how RRI can be put into practice, so other stakeholders in R&I can learn from these. The practices brought together in this catalogue have been selected from a larger collection of so-called promising practices. These were collected through a consultation round held as part of the RRI Tools project by consortium partners from all around Europe. Five different types of stakeholders involved in research and innovation took part in this consultation round: researchers, policy makers, civil society organizations, science educators and business and industry representatives. After an initial selection procedure, a web-based survey was used to gain more in-depth information about five to ten practices from each Hub. The results of this survey were analyzed using, amongst other criteria, the criteria for RRI process requirements described in D1.3. In the end, 31 practices were included in the catalogue. These are presented by way of a short summary, project details and lessons learned. To make them accessible to a wider audience these practice descriptions have also been shared on the RRI Toolkit (for more details see WP3).

1.3.2. WP2: Assessing the needs and constraints of RRI stakeholders
1.3.2.1. Objectives and Milestones of the WP (from the Dow)
According to the DoW, the objectives of WP2 were:
- To design a stakeholder consultation methodology for the analysis of the needs and constraints of stakeholders
- To implement a stakeholder consultation
- To produce a comprehensive map of RRI Stakeholders with a description of the strengths and weakness of the relations between them, as well as an analysis of their needs and constraints in the implementation of RRI.
1.3.2.2. Partners involved in this WP
Leading partner: ECSITE
Other partners involved:
- CIPAST, EBN, ECSITE, EUN, IRSICAIXA, EUROSCIENCE and UCL, as stakeholder groups Experts
- ATHENA, CARIPLO, ECSITE, EUN, EXPERIMENTARIUM, IRSICAIXA and KBF – as policy agenda experts
- FLC, ECSITE, ZSI and CVIVA in the design of the methodology for the consultation Workshops
1.3.2.3. Progress towards objectives – tasks worked on and achievements made
The main contribution of WP2 was focused on identifying stakeholder groups that needed to be addressed by the consortium, and analysing their needs in terms of implementation of RRI. Based on the outcomes of WP2, project partners could carry out work in WP3, WP4 and WP6.
Task 2.1 Mapping of the RRI stakeholder groups and design of a massive RRI Stakeholders consultation process (M1-M8)
Task leader: ECSITE
Other partners involved:
- CIPAST, EBN, ECSITE, EUN, IRSICAIXA, EUROSCIENCE and UCL, as stakeholder groups Experts
- FLC, ECSITE, ZSI and CVIVA in the design of the methodology for the consultation Workshops
The stakeholder mapping was the first step in developing the methodology for a RRI stakeholder consultation (Hubs Consultation Workshop) process carried out from M9 to M11. The mapping helped to identify the institutions and individuals in each stakeholder group and gave an overview of the potential issues that could arise during the consultations. ECSITE with the contribution of several partners (in particular those responsible for the RRI’s Stakeholders Groups as well as those responsible for the different policy agendas carried out the mapping. The mapping was also discussed and validated at the project’s first workshop in April 2014 (1st Advisory Board Meeting), where the Advisory Board members provided their input.
ECSITE, ZSI UCL and ATHENA then developed the consultation methodology that was implemented across the 19 Hubs. This methodology drew on the mapping as well as the working definition of RRI developed in WP1, and details how the five stakeholder groups would be consulted as to their needs and constraints. The consultation took the form of heterogeneous workshops, bringing stakeholders together in order to take advantage of group dynamics and collective reflection, and still ensuring that the voice of each stakeholder group in each hub area is heard. The workshop was structured around 4 sessions (“Introduction to Workshop and RRI Tools”, “RRI Working definition”, “Compilation of Promising Practices”, “Stakeholder Consultation on needs and constraints”) with 8 exercises with 8 hours of work. WP2 partners provided the Hubs with a Consultation Manual to guide them through the organization, moderation and reporting of the workshops.
The Consultation Workshop carried out in M9-M10 contributed to assessing the attitudes of stakeholders towards RRI and the extent to which they already perceive themselves as participating in the RRI process; helped to gather feedback on the project’s working definition of RRI; identified the main needs of the RRI Toolkit’s target users to increase their participation in the RRI process; contributed to a first collection of promising RRI practices. The Consultation Workshop has also helped to identify the needs and constraints of all stakeholders involved in the RRI process; to receive ideas of potential tools for the RRI Toolkit; to gain an overview of the differences across stakeholder groups and EU countries; and to engage and mobilise participants acting as a catalyst in building an RRI Community of Practice (CoP) across Europe.

Status and Deviations: This task was completed without deviations
T2.2 Implementation of a European RRI Stakeholders’ consultation
Task leader: ECSITE
Other partners involved: All RRI National Hubs
ECSITE contributed actively to the creation of the Hubs Manual for the Consultation workshops to guide the hubs during the organization, implementation and reporting phase of their stakeholder consultations. A specific manual with the all the information needed for the organization and collection of the outputs methodology, programme, dynamics, material and templates was prepared by the leaders of WP1, WP2, WP5, WP3 and CVIVA as European Hubs Coordinator.
Based on the outcomes of the Task 2.1 each of the Hubs organised at least one stakeholder workshop (per country) during October/ November 2014. All Hubs were involved in the organization of the local workshops and in total, the Hubs carried out 27 workshops. ECSITE coordinated their implementation, supporting the Hubs during all the phases of the consultation process. Phone and skype calls were organized with several Hubs prior to the workshops.
The implementation of a European RRI stakeholders’ consultation (Task 2.2) was carried out in the shape of interactive one-day meetings across Europe from M9 to M11. These meetings, in which 30 countries were represented, brought together representatives from each stakeholder group, to discuss their understandings of RRI, what they say as the obstacles and opportunities in moving towards this approach and their ideas of practical measures to help the implementation of RRI. After monitoring the workshops, ECSITE shared with the project partners a brief report on the lessons learnt and things to improve.
411 representatives of various stakeholder groups attended the workshops around Europe: 52% male and 48% female. In the recruitment of participants, Hub coordinators aimed for an equal attendance of each of the identified stakeholder groups. After a careful reporting done from each Hub (one report per workshop), the outcomes of discussions were digested and used as the basis for filling in a feedback template which was sent to ECSITE and UCL for analysis.

Status and Deviations: This task was completed without deviations
Task 2.3 Analysis of the needs and constraints of the stakeholders’ groups in RRI practices in Europe
Task leader: UCL
Other partners involved: All RRI National Hubs (Report of the consultation workshopS)
UCL carried a detailed study of the outcomes of the 27 workshops. Deliverable 2.2 “Report on the analysis of opportunities, obstacles and needs of the stakeholder groups in RRI practices in Europe” contains the major results and anylysis of the consultations.
Status and Deviations: This task was completed without deviations

1.3.2.4. Deliverables and Significant Results
D2.1 Guidelines for the implementation of the stakeholder consultation in relation to RRI (M5)-ECSITE.
The document, delivered in May 2014 (M5), includes both the stakeholders mapping and the consultation methodology. The methodology for stakeholders’ consultations was tested (for fine-tuning) during the consortium meeting in Copenhagen (M6), during which the delegates of the 19 Hubs were also trained on its implementation. The consultations delivered clear results that were analysed and then implemented into other WPs (in particular, WP3 and WP4).
D2.2 Report on the analysis of needs and constraints of the stakeholder groups in RRI practices in Europe (M14)-UCL
This document presents the synthesis of the findings of the consultation workshops carried out within Task 2.2. The analysis of the outcomes significantly informed the production of the RRI Toolkit (WP3), as it addressed specific needs of each stakeholder group and of each geographical region. D2.2 provided a comprehensive synthesis of current thinking of European stakeholder groups on key questions related to RRI. The reports of the different workshops identified issues, needs, opportunities and obstacles for stakeholders. This review provided the whole consortium with valuable insights in terms of how the RRI Toolkit would need to be prepared and disseminated. Moreover, in providing a comprehensive review of current thinking on RRI across Europe, the report is of wider interest for the community beyond the RRI Tools consortium, as well as for future RRI-related EU-funded.
Both deliverables - D2.1 and D2.2 - had significant impact and contribution to the RRI Tools Project by informing and feeding into tasks in other WPs. Stakeholder mapping and analysis of their needs, opportunities and constraints have been the main pillars to produce the RRI Toolkit (WP3), develop Training on the RRI Toolkit and advocacy (WP4) and dissemination (WP6).

1.3.3. WP3: Production of the RRI Toolkit
1.3.3.1. Objectives and Milestones of the WP (from the Dow)
The three objectives of WP3 are:
- Integrating the needs of stakeholders in the design of the RRI Toolkit and designing and producing a set of tools to train and disseminate RRI
- Creating the website for the project that hosts the RRI Toolkit and the virtual collaborative platform
- Setting the basis for a Community of Practice on RRI.
1.3.3.2. Partners involved in this WP
Leading partner: FLC
Other partners involved:
- IRSICAIXA, ATHENA ECSITE, UCL, CARIPLO, KBF, EXPERIMENTARIUM, EUROSCIENCE, EUN, EBN and CIPAST as policy agenda and/or stakeholders experts.
- The entire Consortium in the compilation of resources and in providing feedback on the RRI Toolkit.
1.3.3.3. Progress towards objectives – tasks worked on and achievements made
T.3.1 Identification, prioritisation and conceptualisation of the tools (M3-M18)
Task leader: FLC
Other partners involved:
- IRSICAIXA, ATHENA ECSITE, UCL, CARIPLO, KBF, EXPERIMENTARIUM, EUROSCIENCE, EUN, EBN and CIPAST as policy agenda and/or stakeholders experts.
- The entire Consortium in the compilation of resources to be included in the RRI Toolkit.
Background analysis
A toolkit has to be useful and intuitive to its users. In the case of the RRI Toolkit, it was aimed to serve a variety of actors’ groups facing very different challenges. To achieve these goals we conducted a background analysis consisting on two processes:
1. A review of existing resources, toolkits, and portals for RRI and related topics;
2. A consultation on the obstacles and needs of each group of stakeholders.
1. Review of existing efforts
During the first year of the project, the RRI Tools Consortium compiled and reviewed a significant number of resources, toolkits, and digital platforms dealing with diverse aspects of RRI. This process led to the following conclusions:
- There is a lack of resources and portals that cover RRI as a whole and connect all its components.
- There are multiple resources, toolkits, projects, portals and online communities related to some aspects of RRI (e.g. gender, public engagement, open access), most developed by renowned experts in their own fields.
- Most of them have reduced visibility and/or activity, especially outside their very specific field and once funding ends and users’ collaboration remains purely digital
2. Users’ obstacles and needs
In order to understand how to best serve the interests of all R&I actors, one of the cornerstones of RRI Tools was the 2014 pan-European consultation to all stakeholders’ groups on the needs, opportunities, and constraints in implementing RRI. The consultation comprised 27 workshops attended by more than 400 participants from 30 countries of the European Research Area.
In addition, to complement these results with the views of several target groups especially prominent for the implementation of RRI, specific consultations on the RRI Toolkit were conducted for the following potential end users:
- National Contact Points (NCPs) (Tallin, June 2015; Madrid, December 2015)
- Research managers and administrators (EARMA Congress, Leiden, June 2015)
- EC officers and experts on RRI external to the project (Brussels, November 2015)
The results of this background analyses helped us decide the following Guiding principles to design the RRI Toolkit:
- Move beyond the RRI ideal: nothing is 100% RRI
- Avoid duplicating efforts and sum forces whenever possible.
- Provide an umbrella, alive portal that:
• Introduces and informs all actors into RRI and all its facets;
• Takes on board existing partial efforts and resources;
• Increases the visibility and accessibility of these existing efforts;
• Facilitates quick scanning of resources through simple, friendly summaries;
• Addresses needs of both beginners and practitioners;
• Organizes resources according to problems faced by different users;
• Promotes dialogue across all actors to foster the use of resources.
- Build a self-sustainable Community of Practice, both through face-to-face meetings, and an online collaborative platform linked to the RRI Toolkit
Based on these guiding principles, we decided to develop a Toolkit that includes both, already existing resources created by other initiatives and new resources developed by RRI Tools. To reach this goal the following subtasks were carried out:
A. Compilation and selection of already Existing Resources
B. Identification, conceptualisation and development of new resources
A. Compilation and selection of already Existing Resources
This section describes the criteria and process of compiling and selecting those resources developed by previous initiatives that were included in the first version of the RRI Toolkit (v1.0) before opening it up to the contributions of the incipient Community of Practice (RRI Toolkit v2.0)
What is an RRI resource?
A preliminary step before starting the selection process consisted on deciding what should be included under the general term of an ‘RRI resource’. For practical purposes we defined an RRI resource as any resource that can help an individual, project, or institution to learn about RRI and how to implement it, evaluate their performance, or train others on and advocate for its values.
This working definition emerged from the internal discussion raised during the process of compilation and selection of existing resources. Given the variety of formats and uses of the resources found during this process, we decided to stick to this practical definition to keep the scope of the RRI Toolkit content as wide and useful as possible, and to avoid narrowing too much the later contributions by the Community of Practice.
In practice this definition covers a wide variety of formats and aims: background documentation that helps understanding what RRI means and make actors aware of this concept and what it entails, inspiring practices to spark adaptation in other contexts, reflecting tools that help think how to be more responsible in R&I practice, training tools for teaching staff, implementing and methodological tools to put RRI into practice and conduct R&I activities in a more responsible way, evaluating tools to gauge the performance of current practices on different aspects of R&I, disseminating and advocating/ promoting tools to spread the word on RRI and its different aspects, etc.
Compilation and selection process
For each of the four types of resources – library elements, inspiring practices, projects and tools - we applied different compilation procedures and selection criteria. This decision was made due to the different nature and potential use of each typology of resource.
For a detailed description of the compilation procedures and selection criteria for each of the four types of resources see D3.2 – The RRI Toolkit
B. Identification, conceptualisation and development of new resources
From the analysis of the Users’ obstacles and needs and the feedback received from external users and experts while designing the RRI Toolkit, two practical conclusions emerged in terms of developing new resources:
- The lack of awareness, knowledge, and acceptance of the RRI concept.
- The need of guidance on how to put the concept in practice.
Based on these considerations, two sets of new resources were devised:
• Introductory resources that could explain the concept and its diverse facets to all groups of actors in their own terms.
• Implementation resources that would guide users to apply the RRI principles when dealing with the challenges they face in their daily work.
B.1. - Introductory resources
To overcome the current lack of understanding of RRI among all actors, we decided to produce a general home page and eleven specific subhomes for the digital platform, videos and slide presentations explaining RRI to each group of stakeholders in their own language, as well as describing each policy agenda within the RRI framework.
To conceptualize these resources we created 11 groups of experts within the Consortium that drafted 11 internal documents: 5 “Rationales for Stakeholders” and 6 “Policy Agenda Perspectives”. All these documents were later reviewed and enriched by the whole Consortium, the Advisory Board members, and some experts external to the project, both online and through face-to-face meetings.
The “Rationales” explained, for each stakeholders’ group and in their own language, what is RRI, what are the potential benefits of adopting it by that group, and how can it be put in practice by such group. They also included compelling narratives on examples of RRI and key messages targeted at that group, a list of key terms needed when talking about RRI to representatives of that group, a short explanation on some resources useful to apply the concept by that group, and a mapping of key actors to be contacted within that group when advocating on RRI.
The “Perspectives” gave, for each policy agenda, an overview of its meaning within the RRI framework, the main aspects considered in that agenda, some key messages and key terms used when talking about that policy agenda, and a short explanation on some relevant initiatives related to that policy agenda.
B.2. - Implementation resources
To complement the introductory resources and the compilation of already existing existent ones, the Consortium conducted a gap analysis to identify which non-existent resources were most urgently needed to implement RRI within each group of actors.
The selected implementation resources are the “How-To application guidelines” which are meant to help users understand how they can solve a given challenge through examples and the contextualized use of specific resources of the RRI Toolkit (see T.3.2 for more detailed info)

Status and Deviations: This task was completed without desviations
T.3.2 Design and production of the tools (M6-M30)
Task leader: FLC
Other partners involved:
- IRSICAIXA, ATHENA, ECSITE, UCL, ZSI, CARIPLO, KBF, EXPERIMENTARIUM, EUROSCIENCE, EUN and CIPAST as policy agenda and/or stakeholders experts.
- FLC, ATHENA, IRSICAIXA and CVIVA in the production of the Databases
- Rest of the Consortium for feedback on the new resources developed by the project
As explained in T.3.1 the RRI Toolkit includes both existing resources created by other initiatives and brand-new resources developed by RRI Tools:
A. Already existing resources compiled by RRI Tools:
In addition to the Catalogue of Inspiring Practices (compiled in WP1) the project has built three databases (library, projects, and tools). The complete process is described in D3.2. The RRI Toolkit
B. New tools by RRI Tools:
B.1. - Introductory Tools
As explained in T.3.1 the introductory tools – landing pages, videos and slide presentations – are based on 11 documents, the “Stakeholder Rationales” and the “Policy Agenda Perspectives” produced by the STK & PA experts within the Consortium. The tools were designed adapting the content of the documents to the communication language used in each format (web, video, and slide presentation). After the Consortium experts reviewed and validated the draft designs, the introductory tools were developed. All these materials are available at the stakeholder and policy agenda landing pages, and also at the training and communication webpages. Videos are also available at the RRI Tools YouTube channel.
B.2. - Implementation Tools
The How-To application guidelines provide users with practical guidance on how to address common challenges faced by R&I actors when putting the RRI principles into practice. For doing so the How-Tos combine examples of what different institutions have done and explanations on how to use specific resources of the RRI Toolkit on each case.
Users are introduced to concrete examples explaining how to adress possible challenges they may face in their dayly work. Thse include, e.g. how to support RRI at a national level, how to incorporate the RRI principles in a funding call, how to incorporate RRI in policy/ funding institutions, how to embed the RRi principles in research proposals and business plans or how to co-create community-based participatory research.
The How-To guidelines are divided in two groups: those that tackle general challenges faced by multiple groups of actors and those that deal with specific questions related to specific policy agendas (see complete list of application guidelines in D3.2 The RRI Toolkit – Tables 1 and 2)
These guidelines were developed by the STK&PA experts within the Consortium, with contributions from Advisory Board members and/or external experts in some cases.
In addition, some more implementation resources, focused on applying the RRI principles to secondary education and to science centres and museums, have been developed by EUN and ECSITE.
• The educational handbook “RRI in practice for schools. Handbook for teachers” produced by EUN with the aim of helping educators to develop and implement RRI practices in the classroom as well as to include self-reflection processes in their everyday practices.
The handbook contains a variety of resources. For a start, it includes a set of short exercises to be used as a starting point to analyze how RRI-oriented educators’ practices are, specifically focused on the mapping of RRI obstacles and opportunities, the development of school project simulations and of exercises involving different stakeholders.
The documents’ main feature is a set of guidelines and practice templates to integrate RRI in everyday school activities and, specifically, to support educators in designing academic practices integrating RRI dimensions and principles. The guidelines provide with instructions to set up a dynamic process of creation of educational resources that should equally foster capacity building and cooperation among teachers and among any other relevant stakeholders in the educational community.
Additionally, the set of templates is designed for different subjects and is addressed to both primary and secondary school levels, depending on the activity. Particularly, it contains a (1) generic template/guide that describes the other templates’ rationale and which can be used for any type of practice and (2) three extra templates designed for specific activities, namely: (a) Getting started with RRI (b) Experiments and labs activities and (c) Reflection and dissemination.
The handbook has been uploaded into the RRI Toolkit and references have been added to the following How-to guidelines: How to integrate RRI in secondary education and How to introduce RRI at school through project- and inquiry-based learning in STEM.
• A guide for science centers “Quick Start Guide for implementing RRI in science engagement organisations”, developed by ECSITE in close collaboration with several science centres and museums including AHHAA Science Centre (Estonia), Science Animation (France), Museo delle Scienze (Italy), Ciência Viva – Pavillion of Knowledge (Portugal), NEMO (the Netherlands) – members of the Ecsite network, partners of the RRI Tools project and beyond.
This Quick Start Guide is a free document that can be used to promote RRI in science engagement organisation (science centres, science festivals, etc) to offer advice gained from experiences throughout the RRI Tools project and other RRI projects, and to provide recommendations so that any user of the guide will be able to implement their own RRI activities.
• The “Cards for fosteRRing open conversations about research and innovation” developed by ECSITE. This set of cards presents different questions, statements & ideas aimed at fostering informal conversations and collective reflections about different aspects of Responsible Research and Innovation.
• The game “Roll the RRI Dice to spark good training conversations or creating case stories” developed by ECSITE. The dice game gives the oportunity to provoke conversations, reflections and create scenarios, especially thought for RRI Trainings and meetings. A ready to print-glue-use dice template plus a script with ideas of how to use it are included.
RRI Tools - A practical guide to Responsible Research and Innovation - Key lessons from RRI Tools
In November 2016 a quick introductory guide to the RRI Toolkit was published, explaining what responsible research and innovation means and why it is important for modern society. This guide explores RRI through the lens of the RRI Tools project and provides practical examples of its implementation through a number of case studies and an overview of the RRI Toolkit structure and main contents. A selection of ‘How To’ guidelines explain how to apply RRI to specific situations, including policy, research and business contexts. The guide also provides five recommendations that can help to make all types of research and innovation more responsible. The document explains how the RRI Tools project has laid the groundwork for more responsible, acceptable, and ethical science and technology development in Europe – in pursuit of a better, more sustainable and more equitable world.
Status and Deviations: This task was completed without deviations

T.3.3 Development of the multimedia 2.0 collaborative platform (M6-M30)
Task leader: FLC
Other partners involved:
- EVERIS has leaded the design and development of the first informative website and of the collaborative platform
- IRSICAIXA and EUROSCIENCE have contributed in the design of the first informative website and of the collaborative platform
- IRSICAIXA, ATHENA, ECSITE, UCL, CARIPLO, KBF, EXPERIMENTARIUM, EUROSCIENCE, EUN, EBN and CIPAST as a policy agenda and stakeholders experts in the design and development of contents of the collaborative platform
- The entire consortium has contributed in the translation of the first informative website and of the collaborative platform and to the feedback loops in the design process
This task aims to cover the objectives of creating the project’s website and the virtual Collaborative Platform, which hosts the RRI Toolkit and sets the basis for a Community of Practice on RRI. As part of the project’s core strategy, RRI Tools relies for its implementation and dissemination on the federation of an “RRI Community of Practice”, composed by individuals and institutions with a shared interest in Responsible Research and Innovation. This Community of Practice will find a common ground in the resources provided by the RRI Toolkit, and in the virtual workspace that enables fruitful collaborations among users of these resources.
A first version of the website, providing information on the project and means to interact with the members and to register to the project’s dissemination channels, was launched in M6, during ESOF 2014. In 2015, this website turned into the first version of the Collaborative Platform. During 2016 this beta-version was tested with help of experts and users, and the feedback from this test was used to build the final version which was released by M30. What follows is a more detailed description of the whole process.
1. Design and implementation of the first informative site.
Teams from FLC, Everis, IrsiCaixa and Euroscience worked together in the conceptualization of the first version of the website, meant to inform and to act as enabler of the construction of a preliminary network of contacts from stakeholders interested in the project.
To accomplish these objectives, the website had to host basic information about the project, the consortium and the concept of RRI; a registration form where users can subscribe to the RRI Tools dissemination channels; a repository of the project deliverables, published under license Creative Commons; a specific page to publish information about RRI Events; a specific page to share materials meant for press use; a set of basic informative pages about the forthcoming Toolkit and the role of the different stakeholder groups in RRI and the access to the website translation in all consortium languages
Everis lead the design and production of the site in collaboration with FLC and IRSICAIXA. This process comprised a series of iterations between the user experience, production and content teams, and came to a first milestone in June 2014, when the first version of the site was launched in coincidence with ESOF 2014.
All Consortium partners collaborated in the translation of the contents, acting as reviewers of the translations provided by the FLC’s subcontractor. In December 2014 the translated versions started to be released. After the publication of some of the translations, a concern was raised about the lack of a gender perspective in the language used in the translations. The project´s Management Board decided to address this issue and all translations were sent back for revision to all project partners whose languages are gendered.

2. Design and implementation of the collaborative platform: the RRI Toolkit.
Designing steps
Based on the findings of the background analysis described in T.3.1 as a first step we established a series of guiding principles to design the RRI Toolkit
In a second step, to further define the scope of the RRI Toolkit we conducted a workshop within the RRI Tools Consortium where representatives of each group of stakeholders were asked about the vision, mission, and values of the platform to be developed.
In a third step, aimed at finding a global vision, mission, and values of the RRI Toolkit, the expectations of all groups of actors were merged and analysed as a whole. Identifying common interests helped to define the priorities for the RRI Toolkit described in the next section
As a fourth step we did an internal survey within the Consortium to identify the top priorities regarding the Toolkit content and the Community of Practice.
The fifth and last step in the design of the Toolkit consisted on defining the main functionalities and associated navigation model of the platform. To do so, a second benchmark exercise was performed with a number of sites and service providers that offer their solutions in terms of functionalities. Then an ‘address and prioritize” exercise was carried out to match the vision, mission, and values identified before with the preferred functionalities detected in the benchmarking. Finally, assuming that it would not be possible to implement all the identified functionalities, they were prioritized and trimmed through a collaborative reflection process.
More details of the whole design process can be found in the reports D3.1 - Defining the RRI Tools collaborative platform and D3.2 – The RRI Toolkit (Annex I. Design of the Toolkit)
Regarding the language of the platform and its contents, the whole RRI Toolkit is available in English. In addition there are significant parts of the RRI Toolkit (home page, landing pages, ‘What is RRI?’ and Questions & Answers pages) available in most of the languages of the European Research Area. Members of the Community of Practice can upload resources and post content in the Forum in any language. This is so to facilitate access to the RRI Toolkit as much as possible and foster its consolidation as the main site on RRI in Europe, taking into account cultural differences in the familiarity of using foreign languages across countries in the continent.
Content of the Toolkit
The Toolkit is structured around a basic unit: the resource. There are four types of resources:
• Library elements: background documents to inform on RRI and related aspects;
• Projects: past and ongoing initiatives dealing with RRI or some of its features;
• Inspiring practices: examples of practical application of the RRI principles;
• Tools: resources to plan, implement, evaluate, train, and advocate on RRI.
The basic unit: the resource
Each resource is presented through a resource page, which gives a brief overview of the resource (title, author, format, content summary, URL to access the original resource, etc.). The purpose of these resource pages is two-fold:
1. Providing the basic information to allow users quickly decide whether a given resource is valuable for their purposes;
2. Offering visibility and quick access to the actual resource, redirecting users towards the original URL where such resource can be found.
Each resource is described by a number of fields: the taxonomy of the resource. This taxonomy is different for each type of resource. Each resource is tagged according to the contents introduced in each field of its taxonomy. This tagging is key to identify individually each resource and plays a critical role in searching resources through the RRI Toolkit.
Structure of the Toolkit: entry gates
The structure of the RRI Toolkit tries to address the main challenges detected during its design:
- The novelty and inherent complexity of the RRI concept, comprising process dimensions, outcomes, and policy agendas;
- The vast amount of resources compiled (as of January 2017, nearly 500) ;
- The variety of R&I actors from different groups that should serve;
- The heterogeneous backgrounds of these actors, from newcomers to experts;
- The diverse challenges these actors face in their daily work, and for which the Toolkit should offer an aid.
Given this complexity and multiplicity of contents, users, and potential applications, the RRI Toolkit offers seven entry gates:
• Home page: offers a first contact with RRI and the contents of the RRI Toolkit, and redirects users to the other entry gates;
• Stakeholder landing pages: provide a soft landing for newcomers, introducing RRI and the Toolkit contents tailored to the group of actors that the user belongs;
• Policy agenda landing pages: relate the six policy agendas defined by the European Commission to the holistic RRI approach for those interested in particular aspects of RRI;
• How-To application guidelines: help users understand how they can solve a given challenge through examples and the contextualized use of specific resources of the Toolkit;
• Search engine: allows a detailed search of the whole Toolkit with multiple customizable filters for those looking for specific resources;
• Self-reflection tool: provides room for thinking on the user’s own professional practice to analyse how responsible it is and ways to increase its level of RRI;
• Training and Communication pages: offer training opportunities, resources for designing customized training, channels to access the RRI Community of Practice (blog, forum, events), and dissemination materials to advocate on RRI for those more familiarized with this concept.
The top menu, always visible, provides a direct navigation route to all the entry gates and the registration/login access. The bottom menu informs on the RRI Tools project and offers typical functionalities like links to social media, a contact form, and a Q&A section. For a more detailed description of each of these sections/ entry gates, see D3.2 The RRI Toolkit
How to use the RRI Toolkit
There are multiple ways of using the RRI Toolkit, as many as potential users. However, most users may turn to the RRI Toolkit looking for particular resources or functionalities, and thus follow pretty defined navigation paths. The most commonly reported behaviours correspond to the following needs:
• For beginners: looking for an introduction to RRI
• For practitioners:
- Finding resources for a particular stakeholder group or policy agenda
- Doing an advanced search of resources in the RRI Toolkit
• For all sorts of users:
- Reflecting on and improving their professional practice
- Looking for help on the Toolkit structure and contents
For a brief description of each of these behaviours/ navigation paths see D3.2 The RRI Toolkit.

How to contribute to the Toolkit - Entering the Community of Practice
All the RRI Toolkit resources are openly available to any visitor. However, RRI is much more than a wealth of resources and guidelines. Collaboration, cross-fertilization, and co-creation are at the heart of the RRI approach, which calls for all R&I actors being involved in the process of research and innovation.
To foster this multi-actor collaboration the RRI Toolkit incorporates a number of functionalities that allow users to upload resources to the Toolkit, share knowledge with a Community of Practice, find potential partners for new projects, and inform the Community on interesting events and news. Most of these functionalities are only available to the registered members of the Community (see D3.2 The RRI Toolkit)
1.3.3.4. Deliverables and Significant Results
D3.1 Defining the RRI Tools collaborative platform (M17) – FLC
This document details the steps that have been followed to conceptualize the RRI Tools collaborative platform, transforming the informative site to develop and implement a virtual collaborative workspace around the resources contained in the RRI Toolkit. The described procedures have been around to identify the values of stakeholders, how these values are translated into the needs, and how the needs can be transformed in functionalities of the platform.
D3.2 The RRI Toolkit
This report offers a practical overview of the RRI Toolkit. The main sections of D3.2 explain the content of the RRI Toolkit, describing the types of resources included in it; the entries to access the RRI Toolkit, providing an overview of the structure and sections of the digital platform; the possible uses of the RRI Toolkit, exemplifying several of the multiple options that users have to learn, reflect, and find resources for their needs; and the ways of collaborating in the Community of Practice, describing how users can actively contribute to the further fostering of RRI.
In addition the annexes provide information on how the RRI Toolkit was designed in a collaborative and inclusive way, based on the detected needs of the different actors; how the existent resources were compiled and selected; and how the new resources developed by RRI Tools were identified and conceptualized.

1.3.4. WP4: Training on the RRI Toolkit and advocacy
1.3.4.1. Objectives and Milestones of the WP (from the Dow)
WP4 had the objectives of developing and enabling both a training programme and an advocacy programme based on the RRI Toolkit, with the objective of encouraging the use and dissemination of the RRI Toolkit within the máximum number of EU Member States and Associated Countries. To that end, the objectives included:
1. The development of materials for different formats of training modules
2. The implementation of an intensive training course to train trainers
3. The implementation of trainings in the different countries, organised by the RRI Hubs.
The RRI Tools training programme has been designed and implemented “for” and “with” constant participation of the RRI Community of Practice, based on the website developed in WP3. Stakeholders received training that enabled them to train others, therefore turning them into active “RRI agents” whose base is the RRI Toolkit and who have a common objective of “spreading the word on RRI”. This way, active and continuous participation has been supported.
1.3.4.2. Partners involved in this WP
Leading partner: UCL
Other partners involved: All other Consortium Partners
1.3.4.3. Progress towards objectives – tasks worked on and achievements made
At all stages, WP4 has taken input from the RRI Tools Advisory Board, and individual Hubs have benefitted from the expertise of their Hub members. This section should be read in conjunction with the reports D4.1 D4.2 and – especially – D4.3 - Report on the implementation of the training and advocacy programmes around Europe. For conciseness, many details contained in those reports are not repeated here.
The Tasks of WP4 were as follows:
T.4.1 Development and pilot of a training programme (M12-M26)
Task leader: UCL
Other partners involved: FLC, IRSICAIXA, ATHENA, ECSITE, UCL, ZSI, CARIPLO, KBF, EXPERIMENTARIUM and CVIVA
The RRI Tools approach and learning outcomes
Right from the outset, RRI Tools decided that its training programme would be based on a modular approach, so that workshops could range from a brief introduction to RRI and the RRI Toolkit to immersive role-play and detailed discussion, depending on the time available and the requirements of participants. The approach to training also recycled many of the resources developed by other WPs so that deep familiarity with RRI Tools itself could be engendered, and workshop participants could recapitulate (some of) the thinking and practice that had given rise to them.
As far as possible, RRI Tools decided that its workshops would work best if they were multi-stakeholder, since one of the vital features of RRI is the need for multiple perspectives and inputs into the research and innovation processes at all stages.
Following this, RRI Tools developed a series of Learning Outcomes that would apply to all stakeholders and to different stakeholder groups. These outcomes were initially drawn up by the UCL team and then trialled at a project meeting in Barcelona in September 2015. These outcomes were refined further as a result of feedback from Hubs at the Consortium meeting in Brussels in November 2015.
Training section in the RRI Toolkit
The project has developed a training section in the RRI Toolkit that consists of three webpages:
• The main training page offers an introduction to the training program developed by the project, its objectives and learning outcomes, and links to training experiences shared in the RRI Tools blog
• The when & where training page displays through a map and a calendar all the training sessions offered by RRI Tools during 2016. By clicking on a particular session, users can access the basic information (place, date, organizer, type of session) and contact details.
• The resources training page gathers the complete package of materials for the RRI Tools training program.
Training modules and resources
The RRI Tools training resources are organized in three main modules (“Explaining the RRI concept”, “Why is RRI important” and “Using the RRI Toolkit”), a set of eight showcases and additional videos and presentations. The different training resources have variously been developed from WPs 1, 2 and 3, and are briefly outlined in D4.3.
Apart from creating a definition of responsible research and innovation (RRI), and the accompanying policy and project briefs, another goal of WP1 was to identify a number of “Inspiring Practices” that would illustrate key aspects of RRI such that they could be used to “bring RRI to life”. From about 30 such practices, eight were then developed in detail to cover all of the six stakeholder groups and all of te RRI key agendas in such a way that they could (a) act as cases studies that could be gone through in detail, either in a workshop or as part of online study, and (b) upon which role-play exercises could be based that would simulate “real life” RRI situations.
The eight showcases developed are:
• The Challenge Driven Innovation scheme, based in Sweden
• The EPSRC’s Framework for Responsible Innovation, based in the UK
• The Fishery Benchmarking project of the IPMA, based in Portugal
• The Hao2 company, based in the UK
• The Knowledge for Climate (KvK) project, based in the Netherlands
• The Novo Nordisk Blueprint for Change, based in Denmark
• The Social Innovation Factory, based in Belgium
• The Xplore Health project, based in Spain
The showcases themselves also act as exemplars of how to use case study / role-play methodology for RRI training. For each showcase, several additional materials are available such as example ppt presentations, introductory videos and articles in the RRI Tools blog
All training resources developed by the project are openly accessible and can be used for complementing a training attended by the user, for online self-instruction, or for preparing training sessions. For this purpose, three examples of training session programs are also available on the webpage.
Status and Deviations: This task has been completed with no deviations

T.4.2 Development of an advocacy programme (M12-M26)
Task leader: UCL
Other partners involved:
- All the RRI Tools National Hubs in the elaboration of their respective Advocacy Plans
- FLC, CVIVA, ECSITE and EUROSCIENCE giving assistance and support in this process
During the course of the first two years of the RRI Tools Project, the outlines given in the DoW were developed into a set of key objectives. Each of the Hubs carried out a series of meetings with relevant policy makers, often on a one-on-one basis, or with small teams. The target audiences for the advocacy programme were policy-makers in national and regional government bodies, as well as the European Commission itself
The Project developed four key objectives for its advocacy programme:
1. To build knowledge of the Responsible Research and Innovation (RRI) Toolkit amongst relevant policymakers
2. To help policymakers understand their roles within RRI and enable them to know how to act
3. To encourage policymakers to engage with their local hubs on RRI issues
4. To involve policymakers in the community of practice being developed by the hubs with a view to securing the life of the hub/community of practice after the project funding.
As well as with governmental policy-makers, meetings were also held with leaders in academic institutions, funding agencies, learned and professional societies, formal and informal education establishments, and civil society organisations, recognising that – in line with RRI stakeholder involvement – these partners also need discussion about the aims and benefits of RRI. Additionally, more public, larger-scale, advocacy events were put on to enlarge the community of practice.
Status and Deviations: This task was completed without desviations
T.4.3 Train the trainers (M26)
Task leader: UCL
Other partners involved:
- FLC giving support in the organisation, with help of CVIVA as hubs coordinator
- All project partners attended the train the trainers workshops
The task of delivering training on responsible research and innovation (RRI) is a daunting one, if one considers the vast numbers in the R&I communities (private and public) and key stakeholders who may need to be involved. Thus, a key part of WP4 has been to develop a cohort of people across the European Union who feel confident in their understanding of the RRI concept and can pass on that understanding to other relevant partners.
Central to this programme have been two “Train-the-trainers” workshops, held in February and July 2016 (in the DoW only one such workshop was envisaged, but the project decided that two would be necessary). As well as providing training opportunities for all of the Hubs, they have been important forums to exchange experiences, develop new training ideas and generally to enhance the morale and cohesion of Project members for a very demanding training programme during 2016.
The first Train-the-trainers workshop (T3.1) was held at University College London and the Wellcome Trust from February 17 to 19, 2016. This was attended by 63 project members, representing all of the Hubs as well as all of the project partners. The second Train-the-trainers workshop (T3.2) was held at Cosmo Caixa, Barcelona, from July 4 to 6, 2016. This was attended by 62 project members, representing all of the Hubs as well as all of the project partners. A representative of the European Commission DG Research “Science-with-and-for-Society” team also attended both workshops as an active participant. T3.2 was combined with a meeting of the RRI Tools Advisory Board, which enabled Hub members to get insights into various RRI issues (including the reasons for and impact of Brexit) from the “experts”.
These workshops were also important in ensuring that Hubs were fully engaged in RRI Tools and had clear guidance on what was expected of them, and how they could achieve this. The meetings were also important for enthusing the team overall. A short video prepared from material filmed at T3.2 captures a flavour of this.
Status and Deviations: This task was completed without desviations
T.4.4 Implementation of the training and advocacy programme throughout Europe (M27-M36)
During the course of the Project, the RRI Tools consortium members, including Hubs and Networks, have been involved in training and advocacy events that range from one-on-one or small-team-meetings through to addressing audiences of several hundreds. Training events have tended to be with groups from a few to several tens of participants “learning” about RRI and being involved in simulated or real-life exercises to develop practical applications of what the RRI Tools project is providing. Advocacy may take place in individual face-to-face meetings or at larger gatherings where there is little time or opportunity for individual contact.
It is difficult to make a rough distinction between a workshop, where participants get to do some work, as against advocacy meetings, where participants talk, listen and discuss without necessarily being involved in practical exercises. In many ways, training workshops are also advocacy events, however; indeed, the best way to advocate for RRI is to make the person(s) with whom one is discussing feel that they can explain it and argue for it with others, and that they can find resources to support their arguments and help them when developing projects etc.
The DoW foresaw that RRI Tools would undertake to advocate at EU level and to adapt the training resources for an international context, as well as at the local and national level covered by the individual Hubs. In order to report we split the training and advocacy activities between:
• National, regional and local workshops - usually organised and serviced by just one Hub/partner.
• Workshops for international organisations, where several project partners have been involved in the meetings.
a) National/regional/ local training workshops held by the Hubs
Folllowing the T3.1 workshop, Hubs were sent “A briefing for Hubs on Training Workshops and Advocacy”, reported in D4.1 and D4.2. This set out the project requirements for a minimum of two major dissemination and training workshops per Hub. There were a total of 105 national/ regional / local training workshops held by the Hubs.
For a brief synopsis of the training and advocacy activities organised by each of the 19 RRI Tools Hubs see D4.3.
b) International training workshops and advocacy events
The importance of training and advocacy with international organisations is two-fold: firstly such organisations take RRI beyond national boundaries, and make use of pre-existing networks; secondly, such organisations are often seen as repositories of knowledge in themselves, and act as important “multipliers” for dissemination. Efforts for the international organisation training and advocacy events were led by UCL and LCF. But many other Hub members and partner organisations were also involved.
RRI Tools has been involved with several informal meetings at the European Commission, particularly with members of the Science-with-and-for-Society (SwafS) team. In addition, the project has run two one-day training workshops – in June and October 2016 – for Commission staff from DG Research and other DGs. In October, in particular, participants were topic leaders developing their scoping papers with a view to including (aspects of) RRI in the Horizon 2020 calls that would be forthcoming in the next few years of the programme.
RRI Tools carried out several training workshops for the main international networks and organisations working in the field (in the context of their main annual meetings and conferences) such as the network of Science-with-and-for-Society European National Contact points, the European Association of Research Managers and Administrators, the Living Knowledge network, the European Collective of Science Centres and Museums, the European Citizen Science Association, Public Communication of Science and Technology and the European Science Open Forum, among others.
D4.3 - and its Appendix II - gives details of meetings and workshops organised by or participated in by the RRI Tools project and its Hubs
Status and Deviations: This task was completed without desviations
T.4.5 Review and update training (quality control) (M27-M36)
The project is leaving an extremely useable legacy from the training and advocacy workpackage. This includes downloadable training modules, together with some sample presentations, so that future trainers can use them. The modules are all in a standard format so that future (and current) trainers have materials that will at least act as a starting point for RRI. It is also possible that they can be used for remote learning, and even as the basis for MOOCs.
Status and Deviations: This task was completed without desviations
1.3.4.4. Deliverables and Significant Results
The formal deliverables for WP4 consisted of three reports:
• D4.1 - on the development and pilot of the training programme;
• D4.2 - on the development of the advocacy programme;
• D4.3 - on the implementation of these programmes.
D4.1 and D4.2 were delivered according to a modified workplan, agreed with the Commission that reflected the additional effort required to develop the RRI Toolkit fully, and the additional activities carried out by the project

1.3.5. WP5_Monitoring and evaluation
1.3.5.1. Objectives and Milestones of the WP (from the Dow)
The objective of WP5 was to monitor and appraise the work done in the project and its major deliverables and outputs in terms of quality, relevance, effectiveness and impact-orientation and to contribute to the improvement of the project’s processes, services and results.
In particular, WP5 deals with:
• Quality control and evaluation of the project’s research, training and dissemination processes in terms of relevance, efficiency, effectiveness and impact-orientation (inward-looking formative evaluation) by considering how the dimensions of RRI are tackled within the project itself
• The user-friendliness, acceptance, and perceived usefulness of the RRI Toolkit in order to assess its potential effectiveness and impact by addressing its users directly (outward-looking RRI Toolkit validation)
• The provision of an online self-assessment tool to external stakeholders and users to enable them to test and benchmark if their concepts and daily performance are meeting standards of responsible research and innovation in terms of research ethics, engagement with civil society, adequate treatment of gender issues etc. (user-centred formative self-assessment)
1.3.5.2. Partners involved in this WP
Leading partner: ZSI
Other partners involved: FLC, EVERIS, IRSICAIXA, ATHENA, CARIPLO, EBN, ECSITE, EXPERIMENTARIUM, EUROSCIENCE, KBF and UCL
1.3.5.3. Progress towards objectives – tasks worked on and achievements made
T.5.1 Compilation and examination of already existing evaluation procedures concerning RRI issues (M1-M10)
Task leader: ZSI
Within this task ZSI reviewed published opinions on aspects of RRI in a meta-evaluation analysis. This task showed to what extent aspects of RRI have already been taken into account in research and innovation evaluation studies
In addition to an extensive database of resources and interviews with Experts, the main output of this task is D5.1.-Report on Coverage of RRI aspects in STI (Science, Technology, and Innovation) evaluations. It provides the backbone for the provision of methodologies and tools for internal formative evaluation requirements. In addition it contributed with the collection of material and interviews to other workpackages, in specific WP1.
Status and Deviations: This task is completed. No deviations.
T.5.2 Provision of methodologies and tools for internal formative evaluation requirements (M1-M18)
Task leader: UCL
Other partners involved: ZSI, with collaboration of LCF, IRSICAIXA and EUROSCIENCE
ZSI and UCL have developed different monitoring and evaluation designs, to be employed in the different work packages. Quick feedback on quality, relevance, efficiency, effectiveness and impact orientation was gathered. Therefore quantitative and qualitative evaluation instruments have been developed: feedback questionnaires at meetings, events and workshops; regular online surveys for WP leaders; website/login statistics (employed by WP6); media and social media tracking (gathered by WP6); pop-up questionnaires/ tool assessment questionnaires (developed and employed by WP3; and task 5.3). Additionally feedback workshops for trainers were designed.
Status and Deviations: This task is completed. Deliverable 5.2 was delayed from M6 to M18 following the request of the WP leader
T.5.3. Validation of the RRI Toolkit (M24-M36)
Task leader: ZSI
Other partners involved: FLC, IRSICAIXA, EVERIS, EUROSCIENCE, ATHENA, ECSITE, UCL, EBN, CIPAST, CARIPLO, KBF and EXPERIMENTARIUM
Formative evaluation cycles already started before the first beta-version of the RRI Toolkit went online. All meetings and workshops regarding the RRI Toolkit were evaluated and fed back to FLC and EVERIS, in charge of the development of the RRI Toolkit.
ZSI, LCF and EVERIS developed a short pop-up questionnaire to collect online feedback of users. There was not much response online, but ZSI also prepared paper and pen versions to gather as many quantitative feedback as possible. All hub partners were invited to distribute the questionnaire in their national training activities. This step was important and enabled to gather quantitative feedback. Additionally ZSI collected feedback at trainings and dissemination activities at different locations (e.g. STS Conference Graz, Expert workshop Brussels, ECSITE conference Graz).
At the second Train the trainers workshop in Barcelona (July 2016) ZSI organised one specific working session to gather feedback of power users within the RRI-Tools Hub community. This qualitative workshop showed that the RRI Toolkit, its design and usability were very much appreciated, but some suggestions for improvement on search functions still could be collected. Last bugs could be identified and solved later on. ZSI put together a preliminary report to immediately feed back to the programmers. Further, these results have been discussed within a virtual focus group with EVERIS and FLC.
Status and Deviations: This task is completed. D5.3 was shifted from April to September 2016 to allow time enough to gather feedback from users (taking into account that the first beta-version of the RRI Toolkit was openly launched in March 2016). By shifting, the deliverable outlines besides methodology also results of the evaluation process.
T.5.4. Designing a self-assessment tool to verify the compliance of own concepts with acknowledged RRI (M12-M36)
Leading partner: ZSI
Other partners involved:
- FLC, IRSICAIXA, EVERIS, ATHENA in the conceptualization of the tool
- ATHENA, CARIPLO, IRSICAIXA, ECSITE, EUN, EXPERIMENTARIUM and KBF as policy agenda experts
The design of the self-assessement tool started by collecting several resources (concept and content) and best practice examples (concept, technology) that could serve as a role model. Based on external sources as well as other relevant inputs from other WPs (especially WP1) a first rough concept for the tool was established.
A major shift was taken during this process of development and design. From the initial idea of assessing the compliance with RRI keys we moved away towards reflecting on RRI key dimensions and processes and thus, individually assess through reflection. This shift resulted from continuous improvement of the RRI concept and the difficulty of identifying indicators for RRI, since there is no “one-fits-all” approach within the concept. This feedback was gathered at conferences in Austria, presenting the concept of the tool, and agreed within the consortium.
From January until May 2015, ZSI did detailed desk research on theoretical concepts of RRI, which reflects additionally to the definitions and work done by Athena, and on good practice expamples of already existing self-assessment and self-reflection tools targeting various aspects of RRI. Within a workshop together with EVERIS and FLC a first concept of the tool was developed in June 2015. Based on this technical framework ZSI started to develop questions, which were improved by the internal policy agenda experts throughout 3 review cycles between August 2015 and November 2015. A first draft was agreed upon and ZSI started testing the question blocks at conference workshops and with external potential users.
Before coding by EVERIS started, the core policy agenda experts group within the consortium came together and agreed on all six question blocks within a one day workshop held by ZSI in Vienna (December 2015). Still feedback on the questions was constantly gathered and the blocks were adapted accordingly. ZSI presented the question blocks to all project partners during the first Train the trainer’s workshop in London (February 2016) and again, worked in the feedback.
The first online version of the self-reflection tool was launched in March 2016. By that time the idea still was that the SRT would suggest or recommend targeted resources based on the answers given by users to the questions. ZSI worked together with FLC and EVERIS on a tagging system. All questions were tagged according to the search function of the RRI Toolkit. This tagging was done by ZSI and the policy agenda experts. Still it showed, the tagging did not work and the team moved away from the original idea.
Based on this, the feedback at the second Train the trainers in Barcelona showed that there was a need for a completely new design. This was the reason for many new tasks and for the decision to shift D5.4 to December 2016. ZSI worked in close collaboration with EVERIS on the new design. 4 rounds of skype calls needed to be organised. The new tool was ready in October 2016. November was used to collect online feedback and to develop a booklet on the tool. Within three following conferences, the booklet was distributed and showed to be very effective for disseminating the self-reflection tool. The deliverable giving detailed information on the whole development process was submitted in December.
Status and Deviations: This task is completed. Deliverable 5.4 was delayed from M32 to December 2016 following the request of the WP leader
1.3.5.4. Deliverables and Significant Results
D5.1 Report on “Coverage of RRI aspects in STI (Science, Technology, Innovation) evaluations.
This deliverable gives an overview of existing RRI evaluations, its different methods and aspects. It aims to identify good practice examples of RRI evaluations by screening different sources. This work required the framing of the terms ‘RRI’ and ‘evaluation’. Only by defining these two terms, the focus of the search could be clarified as well as the selection criteria could be set.
D5.1 was deliverd in month 4 and gives a detailed outline on the results of the systematic search of resources for RRI evaluation, its selection criteria, and the analysis of all the qualitative and quantitative resources conducted within task 5.1
D5.2 Set of turn-key evaluation designs including questionnaires and templates to perform internal formative evaluation processes throughout the project.
D5.2 was ready by month 6 and provides the explanation, guidance and materials needed for formative evaluation throughout the project. It aims to provide project members with the material they need to gain feedback and to evaluate their WPs, at the earliest stages – while the project activities are developing and at a point when they can still be changed/ adapted/ improved – and for feedback to the project coordination team.
D5.3 - Validation report on the RRI Toolkit
This deliverable describes the way on how the RRI Toolkit was monitored and evaluated. It shows results of the feedback directly gathered from its users and partners via multiple channels and formats at different stages of the RRI Toolkit development process. The aim of the validation process was to suggest improvements and adaptations necessary to attract the end users, for a sustainable integration and further development of the community of practice. For that it is useful to have one transparent summarising document with information on the user-friendliness, acceptance, logical structure and perceived usefulness of the RRI Toolkit for creating awareness of possible issues amongst the project partners and, by launching changes and adaptations, and facilitating on a long run the sustainability of the RRI Toolkit.
Agreed within the consortium, D5.3 was shifted to later dates to be able to involve the results of all feedback workshops, focus groups on usability of the Toolkit as well as of the quantitative evaluation questionnaires.
D5.4 - Proof of concept self-assessment benchmarking tool
The aim of this deliverable is to trace the development of the self-reflection tool (SRT) from the early stages to the final version by explaining the different iterative design steps, the scope of the tool itself, its structure as well as its look-and feel.
The submission of D5.4 was delayed because the SRT had to be redesigned, as described in T5.4. Since the definitive online version of the tool was finalised at the end of October the deadline was agreed on being shifted to December 2016.

1.3.6. WP6_Dissemination
1.3.6.1. Objectives and Milestones of the WP (from the Dow)
The three main objectives of WP6 are:
• to increase knowledge and understanding of RRI
• to raise awareness and visibility of the RRI Tools project and the RRI Toolkit among all R&I stakeholders - policymakers, research community, industry and business, education community, civil society organizations - across Europe
• to induce users to join the RRI Community of Practice, making use and contributing to the participatory RRI Toolkit.
In order to reach these objectives, WP6 builds up on the dissemination efforts of the 26 members of the Consortium, and is focused on disseminating the results of the work undertaken by all the work packages as widely as possible, using a range of dissemination channels, including the website, blog, social media, newsletter, workshops, conferences, press relations. Dissemination is a cross-cutting aspect of the project, and has been carried out throughout the whole development of the project.
1.3.6.2. Partners involved in this WP
Leading partner: EUROSCIENCE
Other partners involved: FLC and all other project partners
1.3.6.3. Progress towards objectives – tasks worked on and achievements made
T.6.1 Dissemination Plan and Communication Manager
Task leader: EuroScience
Other partners involved: FLC and all RRI Tools National Hubs and Networks
EuroScience produced a first version of the dissemination plan of the RRI Tools project (D6.1) presenting the communication goals of the project and the means to achieve them. EuroScience introduced the main headlines of the dissemination strategy to the Hub members in a Consortium meeting in Copenhagen in June 2014. In September 2014, this dissemination strategy was complemented with a social media strategy, to provide support and give indications to the members of the project on how to best display RRI Tools in social media.
In February 2015, in order to further support and engage the RRI Tools Hubs in the dissemination of the project, EuroScience elaborated an updated advocacy and dissemination plan in collaboration with UCL and FLC which was then presented in the Hubs meeting held in Lisbon on 23-24 April 2015. This dissemination strategy for 2015 emphasized the “multiplier” role of the RRI Tools National Hubs and of the four European networks - EBN, EFC, EUN and ECSITE - and the importance of getting feedback, channel by channel, on the dissemination activities in preparation of the launch of the RRI Toolkit. The consortium also discussed about the best ways to foster the internal exchange of information for stronger dissemination.
As a consequence, over the summer 2015 a strong effort was put on internal communication and reciprocal exchange of information in order to increase productivity and to coordinate dissemination efforts within the Consortium. EuroScience provided the Consortium members with the following documents:
- A social media analysis (March 2014-May 2015) and action plan (July-October 2015) to reinforce RRI Tools presence in social media
- Advocacy and Dissemination plans of the Hubs: a cross-comparison to boost dissemination and foster cross-fertilization of ideas
- A one-page visual memo on communication guidelines to help the Hubs with the communication tasks expected from them.
Furthermore, shared documents were put into place, such as a shared events calendar where Hubs published their events, and which was used in turn by WP6 to update the RRI Tools events webpage, the newsletter, and the social media. From July 2015 on, members also shared on a dedicated Flickr account the photos of their events and on a dedicated SlideShare account the presentations they had made for RRI Tools.
Moreover, from September-October 2015 on, a monthly report monitoring the key communication results on Facebook, Twitter, the website, the blog, the newsletter, and Youtube, was shared by WP6 with the Consortium members, for all to get a sense of where we stood. In return, the Hubs and Networks were invited to fill in a brief monthly survey on their dissemination activities in the previous month. A compilation of the answers provided by the Hubs and Networks was shared with all on a monthly basis, as a complement to the ‘communication monitoring report’, to allow members to inspire one another and to fill in the potential gaps.
From October 2015 on, in preparation for the launch of the RRI Toolkit in early 2016, and in order to announce it to the widest audience and to increase participation in the platform, EuroScience developed a plan for all the project members to follow. A strategy and a timeline were shared with the Consortium members in a one-page visual document. All were invited to list the categories and numbers of contacts to which they planned to send the announcement of the launch of the RRI Toolkit on a shared document.
In addition, they were invited to identify and recruit ‘multipliers’ (i.e. organizations representing a wide audience at the European, national, regional or local level) that would accept to spread the word on the RRI Toolkit to their contacts and networks. In parallel, EuroScience listed its own existing contacts and built a list of 500+ multipliers, focusing on the European level. EuroScience also identified several ambassadors that would be likely not only to spread the word on the, but also to introduce the RRI Toolkit to their colleagues in their own organization and have it adopted.
In December 2015, WP6 elaborated an updated version of the Dissemination Plan (D6.3) that was initially delivered in June 2014, presenting the achievements made in 2015 and the plans for 2016, briefly described above.
As a complement to the timeline and the action plan provided in autumn 2015, in preparation for the launch of the first beta version of the RRI Toolkit, officially released on 7 March 2016, the WP6 team provided the whole consortium in early February 2016 with two template e-mail messages, ‘sharers’ for social media and a template press release, as detailed in the deliverable D6.4 on RRI dissemination materials, handed over in April 2016.
On the day of the launch, EuroScience circulated the announcement of the launch of the RRI Toolkit to 500+ European stakeholders and 250 journalists. As a complement, EuroScience also provided the Consortium members with a template e-mail invitation for contributors to add their own tools to the Toolkit. In parallel, e-mail invitations were systematically sent by EuroScience to the potential contributors they had identified.
In addition EuroScience made sure the latest updates on RRI, RRI Tools and the RRI Toolkit were shared with the wider community on social media on a daily basis over the period - 4+ messages a day being published on Twitter and one post a day on Facebook and LinkedIn - as well as in the RRI Tools blog, updated on a weekly basis (from January 2015), and the monthly newsletter (from June 2015).
Contacts with stakeholders were not only maintained through frequent updates on the newsletter, the blog, the forum, Twitter, Facebook, and LinkedIn, but also through personal e-mails following up on the launch of the RRI Toolkit and informing about the different ways users can contribute to the RRI Toolkit and spread the word on it in their organization.
Status and Deviations: This task is completed with no deviations.
T.6.2 Organisation and participation in conferences, workshops and different related events
Task leader: EuroScience
Other partners involved: The entire Consortium
Following the decentralized dissemination strategy, in order to raise the RRI Tools project’s awareness and to invite all stakeholders to contribute, RRI Tools members endeavoured to meet the stakeholders where they are, in the conferences and events. All consortium partners were encouraged to organize dissemination events in their own terms, and to participate as much as possible in events organized by others.
In the RRI Tools’ website, the Events section reflects not only those events organized by, or participated in, but also other RRI Events that can be of interest to the Community of Practice.
EuroScience brought support to the Hubs in organizing events along the whole period. In a first stage of the project by making recommendations on the communication actions to perform before, during and after the stakeholders’ workshop consultations planned in autumn 2014 across Europe. This brief guide was completed with the feedbacks of FLC and the Hubs in July 2014, before being integrated in the Consultation Workshops Manual and put into practice by the Hubs. In a second stage, EuroScience provided Hubs with communication guidelines to help them make the most of their training sessions in terms of dissemination, before, during, and after the session, in a document meant to be concise and practical.
As WP6 leader EuroScience took the lead in organising several promotion activities such as the following:
• Promotion of RRI Tools at the ESOF 2014, held from 21 to 26 June in Copenhagen. The largest multidisciplinary meeting on research and innovation in Europe, ESOF is held every other year in a European city and gathers over 4,500 participants. EuroScience had a stand at ESOF 2014, where it introduced the project to the visitors and circulated the first RRI Tools leaflets.
• Participation of RRI Tools in Science & You (Nancy, France –June 3-5, 2015), an event on science communication, counting 1,000 attendees. There, EuroScience showcased a selection of inspiring RRI initiatives on its stand, inviting visitors to have a look at a photo exhibition on RRI and add the RRI practices they were aware of, in a participatory activity designed to engage them in the RRI Toolkit.
• The latest EuroScience Open Forum (ESOF) edition was held in Manchester from 24 to 27 July 2016. There, EuroScience organized an RRI training session, involving UCL and FLC, during the pre-conference special event ‘Opening Doors on Responsible Research and Innovation’. Twenty-five attendees participated in the workshop, discussed the concept of RRI further and, through group work, considered the opportunities and challenges RRI presents to research managers and examined the resources available through the RRI Toolkit. RRI Tools gained also exposure at the EuroScience stand, where the Communication team introduced the RRI Toolkit to the visitors, and guided them through the RRI Tools website.
EuroScience also recruited RRI Tools ‘ambassadors’ in several targeted organizations showing interest in socially responsible, self-reflective and responsive research. The role of these ambassadors is to advocate for RRI in their organization and make it benefit from the Toolkit. Contacts were established within the University of Strasbourg, the international funders’ group Belmont Forum, and CIRAD, the French agricultural research and international cooperation organization from 2015 on.
T.6.3 Publication of dissemination materials
Task leader: EuroScience
Other partners involved:
- FLC and IrsiCaixa in the production of the dissemination materials
- Rest of the Consortium as disseminators
A first set of dissemination materials (A5 leaflet, roll-up and A3 poster) was produced over summer 2014 to raise the profile of the RRI Tools project among stakeholders and to inform them about its aim: build with their participation a Responsible Research and Innovation Toolkit, to be available online at www.rri-tools.eu. These initial dissemination materials were translated and adapted by the RRI Tools National Hubs, before they used them to display the project on their premises, at conferences, and at the stakeholders’ consultation workshops they organized in autumn 2014.
A blog and a newsletter soon complemented the leaflet, roll-up and poster. In January 2015, Euroscience contributed to the elaboration of a blog, in collaboration with FLC and EVERIS for the technical aspects of the blog, hosted on the RRI Tools website. EuroScience commissioned and scheduled editorial content, wrote 26 contributions, and published the articles (137) written by the other contributors (see the full list of blog articles published in 2. Use and dissemination of foreground. Table 2). Several improvements were brought to the blog over the summer 2015, such as adding new tags and new categories to highlight the diversity of topics covered and make it more attractive to the reader. The contributors were members of the Consortium as well as guest authors from European projects on RRI and organizations that are developing interesting RRI resources. The topics treated were closely monitored in order to ensure a balanced coverage of the different aspects of RRI.
EuroScience also designed a newsletter, with feedbacks from FLC, and managed its production and distribution. EuroScience released a first newsletter in February 2015, followed by a second one in June 2015. Monthly newsletters followed from June 2015 on until December 2016 (see the full list of newsletters published in 2. Use and dissemination of foreground. Table 3.). Every publication was shared with the Hubs and Networks, who were invited to re-use parts of it in their own newsletters, and well as with the members RRI Community of practice (those that had registered in the newsletter distribution list)
Twelve A3 exhibition posters, showcasing a selection of RRI promising practices, among the ones collected by the Hubs and R&I stakeholders across Europe in early 2015, were made available to the Consortium members in June 2015 for re-use in their own events and exhibition stands.
In April-May 2016, EuroScience produced a new set of communication materials, comprising a poster, a leaflet and a roll-up; collaboratively with the Consortium members to ensure these materials answer their needs. These communication tools aimed to raise awareness and deepen the understanding of RRI, as well as to reflect the new visual identity and changes brought to the RRI Tools platform. The nineteen Hubs were invited to translate and re-use these communication materials at the 90+ training workshops and events held across Europe in 2016.
In July 2016, the Media Room was also revamped to display the various dissemination tools available in different languages in an attractive way and foster their use by a wider community.
EuroScience also managed the RRI Tools Youtube channel, uploading and tagging the 82 videos produced throughout the project duration to introduce the concept of RRI and to describe the various features of the RRI Toolkit (see the full list of videos in 2. Use and dissemination of foreground. Table 5.).
Status and Deviations: This task is completed with no deviations

T.6.4 Relations with the media
Task leader: EuroScience
Other partners involved: The entire Consortium
EuroScience produced several press releases to be spread to media contacts and shared with the Hubs for translation, adaptation and dissemination to their own contacts.
- Following the kick-off meeting on 20-21 January 2014 in Brussels, EuroScience produced a first press release announcing the project launch.
- A second press release, based on the key findings of the stakeholders’ workshops report D2.2 on obstacles, opportunities and ideas for RRI, followed in June 2015. UCL, the author of the report D2.2 made a 4-page summary of this report, serving as a source of inspiration to Euroscience, who also collected feedback from LCF and CVIVA, when drafting the press release.
- A third press release announced on 7 March 2016 the official launch of the beta version of the RRI Toolkit, publicly available at www.rri-tools.eu. On the day of the launch, EuroScience circulated the announcement of the launch of the RRI Toolkit to 500+ European stakeholders and 250 journalists.
- A fourth and final press release, published in November 2016, announced that the full version of the RRI Toolkit was now available, featuring new guidelines, information page on RRI, training and communication resources.
RRI and the RRI Tools project were showcased in three special issues of the EuroScientist magazine - an online magazine dealing with the relationship between science and society and the issues affecting scientists in their careers across Europe. EuroScience commissioned contributions and followed up the production of these three special issues.
• A first special issue on RRI was released in October 2014, with contributions from members of the project (EuroScience, LCF, IRSICAIXA, UCL, ATHENA and Science Animation) as well as experts external to the project (G. Laroche, R. von Schomberg, P. Benoît-Joly). It offered crossed views by academics, experts and practitioners on the many facets of RRI.
• A second special issue was published in May 2016, shortly after the release of the Toolkit. It covered notably the launch of the RRI Toolkit, the 2016 RRI training programme and the European Foundations Awards for Responsible Research and Innovation.
• A third and last special issue, dedicated to Responsible Research and Innovation and the relations between science and society, published in December 2016, explored RRI implementation and its future.
In addition, several articles accounted for the project launch in early 2014. EuroScientist’s original content was re-used to raise the visibility of the project and the RRI Toolkit, through other dissemination channels, such as the newsletter and social media.
Besides being highlighted in EuroScientist, RRI Tools got a notably press coverage. For a detailed list of press cuttings on RRI Tools, please refer to 2. Use and dissemination of foreground. Table 4. Moreover, an independent review of the RRI Toolkit is to be published in the Journal of Responsible Innovation in their winter 2017 special issue.

1.3.6.4. Deliverables and Significant Results
D6.1. RRI Toolkit Dissemination Plan (M6):
EuroScience released a dissemination plan (D6.1) at the end of June 2014, entitled “Communication and Dissemination Plan - Disseminating and learning from the others: a two-way communication“. This document features the objectives and leading dissemination principles, the distribution of roles and responsibilities in the Consortium and the communication means to be used. Since the release of D6.1 the document has been complemented with several follow-up reports, such as the social media strategy (September 2014), the advocacy and dissemination plan for 2015 (February 2015) drafted in collaboration with UCL, and a social media analysis and action plan (June 2015).
D6.2. Project Information sheet (M12):
In December 2014, EuroScience drafted a first-year activity report, “RRI Tools: looking back at the first year of the project“. This document lists the main achievements in 2014, from the stakeholders’ consultation workshops and the Consortium meetings, to the various dissemination tools, and the events in which RRI Tools participated. The hub members also provided testimonials on this first year of activity, as seen from their perspective. This document proved to be useful to newcomers to the project, as it gave them a sense of where the project stood at a glance.
D6.3. RRI Toolkit Dissemination Plan (updated) (M24):
In December 2015, half-way in the project, EuroScience released an updated version of the dissemination plan D6.1 under the name of “RRI Toolkit Dissemination Plan (update) - 2016, a crucial year: dissemination gaining momentum for the launch of the Toolkit“. Taking into account the beta version of the RRI Toolkit about to be released in early 2016, this report recalls some of the principles guiding the RRI Tools dissemination strategy, describes in more detail the communication channels and their improvement in 2015, and presents the plans for 2016.
D6.4 RRI Toolkit Dissemination materials (M24):
First dissemination materials had been elaborated in June 2014 to showcase the new RRI Tools project and its aim: producing a participatory Toolkit on Responsible Research and Innovation with the European stakeholders‘ participation. Now that the first beta version of the RRI Toolkit was launched in early 2016, a new set of materials was needed to promote it. In April 2016, shortly after the official launch of the RRI Toolkit to the public (7 March 2016), EuroScience drafted a resport on the campaign and the communication tools put into place to advertise the release of the RRI Toolkit, under the title of “RRI Toolkit Dissemination Materials: New materials to accompany the launch of the Responsible Research and Innovation Toolkit“. The report lists the array of dissemination materials made available to the Consortium, including several template e-mailings, social media ‘sharers’, a press release, several videos, as well as a new leaflet, poster, and roll-up. These last three items were elaborated on the basis of the needs and expectations expressed by the Consortium members.
D6.5 Project Information sheet (update) (M24):
A first beta version of the RRI Toolkit was about to be launched in early 2016. A one-page visually attractive memo which could be re-used in A3 or A4 format for poster sessions, for instance, was needed by the Consortium. To that end, the project information sheet D6.5 briefly presents the project, the soon-to-be RRI Toolkit, and the incoming advocacy and training programme in 2016, introducing the new visual identity of the RRI Toolkit.
D6.6 Report of the different events organised within the project or with the participation of the project partners (M36):
At the end of the three-year project, EuroScience built a report of the different events organised across Europe by the 26 project members or with their involvement. Events were indeed one of the key means to reach out to stakeholders and inform them about RRI Tools and the RRI Toolkit. The consortium members accounted for the events they took part in, either as organizer or participant, throughout the project. On the basis of this information, EuroScience put together a global outlook on these activities, presenting in more detail the type of events, the players involved, and their time and geographical distribution. In total, the 26 partners within the Consortium organized or participated in 366 events during the three years of the project, reaching around 9,200 people in 34 countries and 115 cities all across Europe, contributing to a wide dissemination of the project and a vast training on the RRI concept.
D6.7 Project information sheet (updated version) (M36):
The project came to an end in December 2016. An updated version of the project information sheet (D6.5 - initially released in December 2015) summarizes the main achievements of the three-years, from the stakeholders‘ consultation workshops in 2014 to the building of the RRI Community and the RRI Toolkit in 2015-2016, and the communication and training activities carried out throughout the whole period. The main ways to stay updated with the RRI Toolkit beyond the end of the project are also outlined. This project information sheet D6.7 was built up on the eye-catching infographics designed by Ubik Media in collaboration with LCF, in preparation for the RRI Tools final conference held on 21-22 November 2016 in Brussels.

Potential Impact:
1.4. Potential impact
RRI Tools: tacking stock
After three years of work, RRI Tools presents its contribution to the fostering of the responsible research and innovation concept. Beginning as a grouping of 26 partners and 19 hubs across 30 European countries, it has contributed to building a Community of Practice and a Toolkit on RRI to assist everybody to apply this concept in practice.
Building upon previous efforts, the project has developed a working definition of RRI and created a catalogue of best practices to describe what RRI looks like in real life situations. Its collection of showcases exemplifies how RRI can be carried out by businesses, governments, funding bodies, research centres, civil society and educational institutions. RRI Tools has also explored the benefits of RRI for a wide range of stakeholders - from those closest to research and innovation to those regulating its implications, as well as those affected by these implications - while also characterising the barriers that could stand in the way of it.
As a result, the project has identified the actions needed to make RRI a reality and has crafted a Toolkit to help anyone who wants to make their research and innovation processes more responsible. The RRI Toolkit gathers over 500 resources developed by experts in a diversity of fields as well as by the project itself. These resources are useful for a number of goals, ranging from increasing awareness of RRI to implementing it and evaluating its outcomes. The accompanying How-Tos explain, step by step, how to use many of these resources to put RRI in practice in a range of situations for different actors. This is perhaps the most practical achievement of the project and will help people across the globe to quickly find advice and support on RRI.
To further the RRI message, the RRI Toolkit also provides educational resources to help others develop their own training programmes, advocate and spread the word about this approach. This will help those who are new to RRI to make sense of the concept, understand how it can be applied and how it could be useful to their work. Finally, the creation of a growing, global RRI Community of Practice means people interested in RRI can engage through forums and networking events. This will ensure that the Toolkit continues to evolve and acts as a source for the latest information on RRI in the future.
This future will undoubtedly bring changes. Science and technology are becoming more open, collaborative and connected than ever. R&I projects increasingly include a diverse range of partners and voices to plan, co-create and co-develop marketable solutions, adapt to social demands and anticipate unexpected impacts, transparently share knowledge and ideas with all actors, connect people and institutions in different disciplines, sectors and countries. This means we can better tackle the global challenges society faces, and use their results to openly inform responsive policies, future developments, and new educational approaches for a more knowledgeable society. This all resonates with RRI and puts its principles as cornerstones of the new science and technology trends.
These transformations are reflected in the three goals set by EC Commissioner Carlos Moedas for research and innovation in the EU: Open Innovation, Open Science and Open to the World. In contrast to classical closed innovation principles, Open Innovation is about sharing the knowledge of many different players to co-develop products and services that better meet society’s needs. Open Science describes a whole new approach to the scientific process, based on new methods of doing research and sharing scientific knowledge, moving away from the traditional way of communicating research. Open to the World means promoting international cooperation in research and innovation, using the knowledge, talent and resources available worldwide to more effectively take on challenges that do not stop at national borders. Summarised as the three O’s, these goals share many similarities and are mutually supportive with the principles of RRI. All these elements will gradually feed into the shaping of a policy for Responsible Research and Innovation and contribute to putting citizens at the center of the attention of policy makers and R&I organizations, for example through citizen science activities and user-led innovations.
Whatever the future may be, RRI principles are now firmly established at the heart of the research and innovation system. RRI Tools has contributed to set the stage; others should capitalise on this work to ensure that the word continues to spread. As science and technology progresses at a faster pace than ever before, it is essential that responsible research and innovation is embedded in governance structures around the world to make sure these developments work for the society of today and tomorrow.
RRI Tools: Building a community of practice on RRI
The ultimate goal of the RRI Tools project was to build a Community of Practice in Europe to ensure the use, evolution and enhancement of the RRI Toolkit and thus boost RRI. To reach that goal the project developed an ambitious training and advocacy programme all across Europe, together with a strong emphasis on communication and dissemination activities.
Training and Advocacy:
The training and advocacy programme activities far exceeded what had been envisaged in the DoW, in terms of both numbers and quality.
The task of delivering training on responsible research and innovation (RRI) is a daunting one, if one considers the vast numbers in the R&I communities (private and public) and key stakeholders who may need to be involved. Thus, a key part of the training and advocacy programme was to develop a cohort of people across the European Union who feel confident in their understanding of the RRI concept and can pass on that understanding to other relevant partners. With that aim the project organised two Train-the-trainers workshops (in the DoW only one such workshop was envisaged, but the project decided that two would be necessary). 80 people overall attended one or other of the two. This represents a considerable training force for future RRI training events and projects, a resource that needs maintaining and supporting in the future.
Throughout 2016 the project trained +3000 trainees in more than 100 training sessions on RRI and the RRI Toolkit all over Europe. The 19 national hubs facilitated these training events and have furthermore held +200 dissemination events and +100 advocacy meetings.
The project has also run two one-day training workshops – in June and October 2016 – for Commission staff from DG Research and other DGs. In October, in particular, participants were topic leaders developing their scoping papers with a view to including (aspects of) RRI in the Horizon 2020 calls that would be forthcoming in the next few years of the programme
In addition the project has also carried out several training workshops for some of the main international organisations and networks working in the R&I field (in the context of their own annual meetings and conferences). The importance of training and advocacy with international organisations is two-fold: firstly such organisations take RRI beyond national boundaries, and make use of pre-existing networks; secondly, such organisations are often seen as repositories of knowledge in themselves, and act as important “multipliers” for dissemination.
The full list of events organised by the project is included in xxxx.
Dissemination:
Mobilization of stakeholders, awareness raising and dissemination activities were crucial in RRI Tools. The communication strategy of the project was built on the dissemination efforts of the 26 members of the Consortium, and was focused on raising the awareness and visibility of the RRI concept and of the RRI Toolkit among R&I stakeholders across Europe. The ultimate goal was to encourage users to join the Community of Practice, making use and contributing to the RRI Toolkit
An array of communication and dissemination tools were launched at the beginning of the project - all of them regularly nurtured with fresh content; as well as press relations and articles, along with some non-digital promotional materials.
Main online dissemination channels launched by RRI Tools were:
• The RRI Tools website, blog, and newsletter,
• The Events section in the RRI Tools website, which reflects not only those events organized by, or participated in, by the project but also other RRI Events that can be of interest to the Community of Practice.
• The Media Room in the RRI Tools website, which provides access to the dissemination channels and materials developed by the project.
• Social Media: specific RRI Tools profiles in Facebook, Twitter, Linkedin, Youtube, Flickr and SlideShare
Thousands of followers from all over Europe keep informed about RRI Tools and the RRI Toolkit through social media.
• On LinkedIn, members went from 95 in December 2014 to 245 in December 2015 and 492 in December 2016.
• On Facebook, the RRI Tools fans went from 157 fans in December 2014 to 480 in December 2015 and 959 in December 2016.
• On Twitter, the RRI Tools followers went from 341 followers in December 2014 to 1,136 in December 2015 and 2,378 in December 2016.

Figure 3. A growing audience on RRI Tools social media
Newsletter subscribers went from 222 subscribers in February 2015, to 800 in December 2015, and 1,914 in December 2016. As for the community of registered users of the RRI Toolkit, they had reached 1.100 by the end of the project in December 2016, and numbers keep on growing since then
The RRI Tools blog was launched in January 2015. A total of 137 articles were published between January 2015 and December 2016 (see the full list of blog articles published in “2. Use and dissemination of foreground”- Table 2). The contributors were members of the Consortium as well as guest authors from European projects on RRI and organizations that are developing interesting RRI resources. The topics treated were closely monitored in order to ensure a balanced coverage of the different aspects of RRI.
A newsletter was also designed. The first number was released in February 2015, followed by a second one in June 2015. Monthly newsletters followed from June 2015 on until December 2016 (see the full list of newsletters published in “2. Use and dissemination of foreground”- Table 3). Every publication was shared with the Hubs and Networks, who were invited to re-use parts of it in their own newsletters, and well as with the members RRI Community of practice.
Several press releases to be spread to media contacts and shared with the Hubs for translation, adaptation and dissemination to their own contacts, were produced along the whole period:
• Following the kick-off meeting on 20-21 January 2014 in Brussels, the first press release produced announced the project launch.
- A second press release, based on the key findings of the stakeholders workshops report D2.2 on obstacles, opportunities and ideas for RRI, followed in June 2015.
- A third press release announced on 7 March 2016 the official launch of the beta version of the RRI Toolkit. On the day of the launch, the announcement of the launch of the RRI Toolkit was circulated to 500+ European stakeholders and 250 journalists.
- A fourth and final press release, published in November 2016, announced that the full version of the RRI Toolkit was already available.
RRI and the RRI Tools project were showcased in three special issues of the EuroScientist magazine - an online magazine dealing with the relationship between science and society and the issues affecting scientists in their careers across Europe.
• A first special issue on RRI was released in October 2014, with contributions from members of the project as well as experts external to the project. It offered crossed views by academics, experts and practitioners on the many facets of RRI.
• A second special issue was published in May 2016, shortly after the release of the Toolkit. It covered the launch of the RRI Toolkit, the 2016 RRI training programme and the European Foundations Awards for Responsible Research and Innovation.
• A third and last special issue, dedicated to Responsible Research and Innovation and the relations between science and society, published in December 2016, explored RRI implementation and its future.
Besides being highlighted in EuroScientist, RRI Tools got a notably press coverage. Please refer to ”2. Use and dissemination of foreground” - Table 4 for a detailed list of press cuttings on RRI Tools. Moreover, an independent review of the RRI Toolkit is to be published in the Journal of Responsible Innovation in their winter 2017 special issue.
Throughout the project duration 82 videos were produced and uploaded into the RRI Tools Youtube channel, to introduce the concept of RRI and to describe the various features of the RRI Toolkit (see the full list of videos in “2. Use and dissemination of foreground” - Table 5.).
An array of dissemination and promotional materials was also produced. The first set of dissemination materials (A5 leaflet, roll-up and A3 poster) was produced in summer 2014 to raise the profile of the RRI Tools project among stakeholders and to inform them about its aim: build with their participation a Responsible Research and Innovation Toolkit. These initial dissemination materials were translated and adapted by the RRI Tools National Hubs, before they used them to display the project on their premises, at conferences, and at the stakeholders’ consultation workshops they organized in autumn 2014.
Twelve A3 exhibition posters, showcasing a selection of RRI promising practices, among the ones collected by the Hubs and R&I stakeholders across Europe in early 2015, were made available to the Consortium members in June 2015 for re-use in their own events and exhibition stands.
In April-May 2016, a new set of communication materials, comprising a poster, a leaflet and a roll-up was produced. These communication tools aimed to raise awareness and deepen the understanding of RRI, as well as to reflect the new visual identity and changes brought to the RRI Tools platform. The nineteen Hubs were invited to translate and re-use these communication materials at the 100+ training workshops and events held across Europe in 2016. In July 2016, the Media Room was also revamped to display the various dissemination tools available in different languages in an attractive way and foster their use by a wider community
Following the decentralized dissemination strategy, in order to raise the RRI Tools project’s awareness and to invite all stakeholders to contribute, RRI Tools members endeavoured to meet the stakeholders where they are, in the conferences and events. All consortium partners were encouraged to organize dissemination events in their own terms, and to participate as much as possible in events organized by others. For a detailed list of the events organised by the project please refer to 2. Use and dissemination of foreground. Table 1.
The RRI Tools Final Conference:
The biggest event organised by the project was the RRI Tools Final Conference conceived as a main advocacy and dissemination event (21st- 22nd November 2016, Sheraton Hotel, Brussels). The Final Conference was meant to be both a “celebration” of three years of intensive work and an "open market" to share and exchange ideas on how research and innovation can be built further on more open and responsible foundations. More than 250 people attended, representing a full range of R&I actors at regional, national and European levels. Nearly 80 speakers shared their views on the future of openness of research and innovation and the social impact of science. The conference also hosted, in the evening of Monday 21st, the Ceremony of the European Foundations Award for Responsible Research and Innovation, a sister initiative that aimed to recognise the best 3 RRI projects in Europe.
All the information about the Final Conference can be found in a specific website on the RRI Tools platform. The full programme, videos, pictures, power point presentations, main results of the parallel sessions and the RRI Tools Booklet are available there.
In November 2016, in preparation of the Final Conference, the RRI Tools Booklet a quick guide summarizing the main outputs of the project (“A practical guide to Responsible Research and Innovation - Key lessons from RRI Tools”) was published, explaining what responsible research and innovation means and why it is important for modern society. The guide explores RRI through the lens of the RRI Tools project and provides practical examples of its implementation through a number of case studies and an overview of the RRI Toolkit structure and main contents. A selection of ‘How To’ guidelines explain how to apply RRI to specific situations, including policy, research and business contexts. The guide also provides five recommendations that can help to make all types of research and innovation more responsible. The document explains how the RRI Tools project has laid the groundwork for more responsible, acceptable, and ethical science and technology development in Europe – in pursuit of a better, more sustainable and more equitable world.

List of Websites:
Mr. Ignasi López Verdeguer – Project Coordinator
Director of the Department of Science at “la Caixa” Banking Foundation
ilopez@fundaciolacaixa.es
Tel: +34 93 404 4989
www.rri-tools.eu

Final Report Summary - RRI TOOLS (RRI TOOLS, a project to foster Responsible Research and Innovation for society, with society.)

Related documents

Share this page

Download