Final Report Summary - COMPOSITES 4 IU (Grant to develop several composite indicators relevant for the monitoring of the progress towards objectives in the ERA and the Innovation Union) Executive Summary: The research project explores the possibility to develop a composite indicator measuring research interactions at the country level. The results show that constructing a composite indicator measuring research interactions is currently not feasible. As to research interactions in general, only a limited number of variables are available and those available correlate poorly. The same issues hold for measuring public-private research interactions and for measuring international research interactions. A conclusion can be drawn that interactions in research are multifaceted and need not go in the same direction. For example, higher overall mobility of researchers in science & technology need not go hand in hand with more collaboration as measured by co-publication and co-patent data. The research is based on a large scale data collection for 53 countries (all EU member countries; all potential, acceding, and candidate EU member countries; all non-EU OECD member countries; and all BRICS countries; spanning 12 years: 2000-2011). From this long list of variables we present a short list of 19 potential indicators that all have their merits by themselves but are more difficult to be summarized in a composite indicator. Concerning public-private research interactions, we see that mobility and collaboration do go hand in hand. In other words, the extent to which different kind of interactions (mobility versus collaboration) follow the same trend seems to depend on the dimension along which such interactions take place. Here, different kinds of cross-institutional interactions follow the same pattern. A composite indicator of research excellence is proposed. Having evaluated the quality profile of a large set of potential variables, we focus on four variables that measure the top-quality output of scientific and technological research activities at the national level. This composite indicator fills a gap in measuring research excellence at the country level and has an added value on top of other country-level performance indicators in the field of science and technology assessment. Measured against the most commonly used composite indicators of innovation and competitiveness it is shown that the proposed composite indicator on research excellence is akin to but also different from measurements dealing with the phenomena of innovation and competitiveness.A composite indicator on structural change is proposed at the country level, including indicators grouped in five pillars: R&D, skills, sectoral specialization, international specialization and internationalization. As such, this composite is a supply-oriented indicator that is largely based on past performance. All these indicators are related to the overall structure of the economy and are slow to change. The resulting composite scores of structural change are presented in two ways. First, the overall composite levels for three time points to show the relative size of the knowledge-based economy by country in all the five pillars considered. Then, we compare countries in terms of pillar level dynamics in order to better understand the strengths and weaknesses of countries. Another contribution of this project is the assessment of the efficiency of national research systems in achieving excellent research performances. The efficiency assessment is not only restricted to the production of research excellence in general, but is disentangled by type of research field, distinguishing between science and technology. This distinction provides a helpful tool for policy makers in assessing the discrepancy of efficiency in both science and technology excellence within and across countries. Efficiency analyses are conducted for the period 2004-2008 including 37 countries. We observe a positive relation between input measures of research and their respective output measures, indicating that countries employing more research resources in science (or technology) are in general recording higher levels of excellent scientific (or technological) research. We observe that most of the top ranking countries in terms of research inputs also classify highly on excellent research output. We also observe that most of the countries improved in their efficiency over time in the period of analysis (2004-2008). Note that efficiency is not the same as excellence as such. Countries that are generally considered as excellent scientific research performers might by virtue of investing a lot of public money turn out less efficient in the end. Project Context and Objectives:The EU2020 strategy contains a blueprint for transforming Europe into an ‘Innovation Union’ by 2020. The Innovation Union flagship initiative (October 6th, 2010) commits the EU to boosting investment in research andmaking Europe an attractive place to develop innovative products. According to this initiative, national governments will have to reform their innovation systems to boost cooperation between industry and universities, ensure a modernization of framework conditions for enterprises, and enact additional measures to enhance cross-border cooperation and to embrace joint programming.All these innovation aspects need to be carefully monitored by policy-makers in the European institutions, Member states and Regions. The focus of this project is on monitoring some key structural dimensions of the Innovation Union and the European Research Area (ERA).The study builds on the findings of a feasibility study called ERA_MONITORING carried out by the proponent in 2011 and financed by the 2010 Capacities work programme. The study addresses the development of four sets of indicators regrouped in relevant sub-dimensions and ultimately hinged into composite indicators to measure:1) progress towards a unified European Research Area (ERA), to attract talent and investment in order to create a genuine single market for knowledge, research and innovation.2) progress towards higher research excellence, meaning the vitality of the research environment and the quality of research outputs in both basic and applied research;3) structural change in the economy, to monitor the increase towards a more knowledge-intensive economy in Europe coherently with the orientations of the EU 2020 strategy and the Innovation Union initiative;4) diffusion of knowledge and technology between research institutions and industry to monitor the increase of knowledge transfer practices in view of optimizing market and societal uptakeProject Results:The highlights of the project can be summarized as follows:The research explores the possibility to develop a composite indicator measuring research interactions at the country level. To fulfill this aim, we follow three steps. First, we conceptualized research interactions within the broader framework of national research systems. From the notion of national research systems, research interactions are important as they involve the relationships research actors have among each other. Research interactions have different properties. In characterizing research interactions this research focuses on three such properties: (i) the mode or carrier of research interactions, (ii) the dimension along which research interactions take place, and (iii) the direction of research interaction. The results show that constructing a composite indicator measuring research interactions is currently not feasible. First, as to research interactions in general, only a limited number of variables are available and those available correlate poorly. Second, the same issues hold for measuring public-private research interactions. Again, data are limited and those variables available correlate poorly. Third, again, the same issues hold for measuring international research interactions. We conclude that for constructing a composite indicator on interactions in research and innovation it is important (i) to take the nature and direction of interactions in research and innovation duly into account and (ii) to acquire more data that is well-specified to measure research interactions (such as systematically collected data on mobile researchers at various levels; detailed data on publications; more reliable affiliation data for researchers in bibliometric sources; data on cross-border collaboration in funding joint research infrastructures, to mention but a few possible directions of improvement). From the analysis of the set of indicators a conclusion can be drawn that interactions in research are multifaceted and need not go in the same direction. For example, higher overall mobility of researchers in science & technology need not go hand in hand with more collaboration as measured by co-publication and co-patent data. The research is based on a large scale data collection for 53 countries (all EU member countries; all potential, acceding, and candidate EU member countries; all non-EU OECD member countries; and all BRICS countries; spanning 12 years: 2000-2011). From this long list of variables including data on funding, co-publication and co-patenting as well as researcher mobility based on bibliometric and survey data, we present a short list of 19 potential indicators that all have their merits by themselves but are more difficult to be summarized in a composite indicator. Concerning public-private research interactions, we see that mobility and collaboration do go hand in hand. In other words, the extent to which different kind of interactions (mobility versus collaboration) follow the same trend seems to depend on the dimension along which such interactions take place. Here, different kinds of cross-institutional interactions follow the same pattern. Although different kinds of international research interactions present a similar picture, we need to take into account that smaller (larger) national research systems are generally more (less) internationally oriented. A composite indicator of research excellence is proposed. Research excellence is defined as the top-end quality outcome of systematically performed creative work undertaken to increase the stock of knowledge and new applications. Having evaluated the quality profile of a large set of potential variables, we focus on four variables that measure the top-quality output of scientific and technological research activities at the national level:1) a field-normalized number of highly cited publications of a country as measured by the top 10% most cited publications (in all disciplines) per GDP;2) the number of high quality patents of a country as measured by the GDP normalized number of patents filed under the patent cooperation treaty (PCT);3) the number of world class universities and research institutes in a country as measured by the GDP-normalized excellence scores of the top 250 universities and research institutes; and4) the number of high prestige research grants received by a country as measured by the total value of European Research Council (ERC) grants received per GDP.This composite indicator fills a gap in measuring research excellence at the country level and has an added value on top of other country-level performance indicators in the field of science and technology assessment. Measured against the most commonly used composite indicators of innovation and competitiveness it is shown that the proposed composite indicator on research excellence is akin to but also different from measurements dealing with the phenomena of innovation and competitiveness.A composite indicator on structural change is proposed at the country level, including indicators grouped in five pillars: R&D, skills, sectoral specialization, international specialization and internationalization. As such, this composite is a supply-oriented indicator that is largely based on past performance (the outcomes of past efforts that are already measurable in terms of actual value added and employment levels in knowledge-based activities, revealed competitive advantages, supply of skilled human resources, etc.). All these indicators are related to the overall structure of the economy and are slow to change. The resulting composite scores of structural change are presented in two ways. First, we show the overall composite levels for three time points to show the relative size of the knowledge-based economy by country in all the five pillars considered, and present the level changes of the composite scores for each country, that is understood as an indicator of structural change. Then, we compare countries in terms of pillar level dynamics in order to better understand the strengths and weaknesses of countries. Country comparisons are also shown based on key indicators that could be considered as enablers of structural change.Another contribution of this project is the assessment of the efficiency of national research systems in achieving excellent research performances. The efficiency assessment is not only restricted to the production of research excellence in general, but is disentangled by type of research field, distinguishing between science and technology. This distinction provides a helpful tool for policy makers in assessing the discrepancy of efficiency in both science and technology excellence within and across countries. We address empirical issues concerning data requirements and mathematical methods used for efficiency analyses. We conducted efficiency analyses on three main model specifications in which we relate the amount of resource assets to the performance on excellent research. In a first type of model we relate public R&D capital investments to measures of excellent scientific output. Estimating this efficiency relationship is of particular interest for policymakers as the allocation of public investments in R&D can directly be influenced by them. Public R&D investments are measured by the R&D investments in the government sector and the higher education sector, while the excellence of scientific output is captured by the number of highly cited publications. In a second model specification private R&D investments (i.e. business enterprise expenditure on R&D) are related to an output measure capturing the technological research excellence. In this model specification, the number of PCT patents is used as proxy for the technological research excellence. Finally, a third type of model relates the total R&D investments to output measures capturing both scientific and technological research excellence. We use the gross R&D expenditures as measure for the total R&D investments and we proxy the scientific and technological research excellence by the number of highly cited publications and the number of PCT patents. Efficiency analyses are conducted for the period 2004-2008 including 37 countries, capturing the EU28, the candidate countries, most EFTA countries and some international benchmark countries (China, US, South-Korea and Japan). We observe a positive relation between input measures of research and their respective output measures, indicating that countries employing more research resources in science (or technology) are in general recording higher levels of excellent scientific (or technological) research. Second, we observe that most of the top ranking countries in terms of research inputs also classify highly on excellent research output. Countries with extensive research resources in terms of financial R&D expenditures do probably perform better on the underlying factors that influence research excellence (e.g. attracting and employing top scientists and having better (pre)conditions to encourage innovative entrepreneurship). Third, most of the countries improved in their efficiency over time in the period of analysis (2004-2008). The best performing countries in terms of efficient use of public research assets to achieve excellent scientific research are Belgium, Switzerland, Greece, Ireland and United Kingdom. The Republic of Korea, Japan and the Russian Federation are among the least performing ones. Efficiency scores and rankings for technological research show a different pattern. Top performing countries in this category are the Netherlands, Switzerland, Denmark, Sweden and Germany. Romania, Luxembourg and the Russian Federation are among the lowest in ranking. We draw several main conclusions and derive various recommendations from them. A first conclusion holds that some of the results of the analysis seem to be counter-intuitive at first sight. For example, while Greece ranks as highly efficient when it comes to public inputs and excellent scientific outputs, the US ranks low in efficiency when it comes to public inputs and excellent scientific outputs. Note however, that efficiency is not the same as excellence as such. In other words, countries that are generally considered as excellent scientific research performers might by virtue of investing a lot of public money turn out less efficient in the end. Second, countries that are efficient in the production of excellent scientific research need not necessarily also be efficient in the production of excellent research in technology or even in producing excellent research in general (i.e. including both excellent science and excellent technology outputs). As such, there seems to be room for most countries to either improve in efficiency in the production of scientific research excellence or to improve their efficiency in the production of technological research excellence. It remains for further research to address the underlying mechanisms that drive differences in efficiency scores across countries. Third, most European countries have improved over time in their use of research assets to produce excellent research in general. Disentangling efficiency in science from efficiency in technology, we notice that - except for Switzerland scoring well on both dimensions - the ranking and scores are quite heterogeneous. These results suggest that efficiency in science does not necessarily imply efficiency in technology. Moreover, empirical evidence shows that countries performing well on research excellence record relatively high efficiency scores, while this relationship is more scattered for countries with medium to poor research excellence performances. We note that for many countries then efficiency in the production of research excellence is less an issue than the production of research excellence itself. For sure, there are some countries that perform low in both excellence and efficiency. However, there are many more countries that despite their performance in efficiency perform relatively weak on excellence itself. This would seem to suggest that for most (or at least, these) countries (that are efficient already) emphasis should be placed more on excellence itself rather than efficiency.Potential Impact:The composite indicators are meant to become highly visible in the implementation of the Innovation Union and are expected to have a large expected impact, given that this activity is at the core of the Innovation Union flagship initiative and is part of the EU 2020 growth strategy. In this context, the availability of quantitative measures of research interactions, structural change and research excellence is of paramount importance. The results of the project have been published in the normal reporting activities of RTD. Specifically, in the report State of the Innovation Union 2012 and in the Innovation Union Competitiveness Report 2013.List of Websites:http://ipsc.jrc.ec.europa.eu/index.php/596/0/
