Measuring the impact of interdisciplinary research

Problem formulation and state-of-the-art:

Interdisciplinary research has often been praised for its potential to advance our scientific understanding by (re)-combining knowledge and resources that are otherwise latent within individual disciplines (e.g. Nelson and Winter, 1982; Fontana et al. 2020).

Yet to scientifically evaluate this statement a number of questions need to be iteratively addressed. First, how can interdisciplinarity and its impact be measured?

Over the last decades several concepts and measures of interdisciplinarity have been introduced, leveraging notions of diversity or atypicality of a paper’s content that can be computed by assessing the conceptual distance between keywords and tags used by journals (e.g. Brohmann et al. 2016) or by calculating the statistics or combinatorics of the forward citations of scientific papers (contributions referred to in a paper, Uzzi et al. 2013, Yergos-Yergos et al. 2016).

Is then interdisciplinary research more impactful?

Traditionally, the impact of different papers has been measured by counting their backward citations (contributions referring to a paper, Fortunato et al. 2018) and past work comparing the citation impact of interdisciplinary research with less interdisciplinary approaches has reported mixed results (Levitt and Thelwall, 2008; Lariviere et al. 2015).

The mere number of citations, however, might not fully reflect the quality and impact of research contributions, and especially interdisciplinary ones. Βut how else can we measure impact?

Recent approaches are going beyond citation counts suggesting to analyze not only the size but also the type of impact of different contributions. For example, Wu et al. (2019) suggest categorizing contributions as constructive vs. disruptive by measuring the overlap between the reference list of a target paper and the reference lists of papers citing it.

In our work, we aspire to contribute to the quantitative study of interdisciplinarity by analyzing the type of impact of interdisciplinary research and potentially proposing novel, more inclusive ways of measuring research impact.

Method:

We will use data from the Microsoft Academic Graph and analyze all papers published in specific target journals indexed by it (e.g. PNAS, Nature Communications etc.) or within a prespecified discipline (e.g. Economics). We will measure the interdisciplinarity of different contributions using measures of diversity (see Stirling, 2007; Yergos-Yergos et al.) and atypicality (Uzzi et al. 2013).

We will then assess the impact of different papers by reconstructing their citation networks, and by counting the backward citations, and we will contrast the impact of interdisciplinary projects with papers drawing from fewer disciplines. We will then study the structure of the citation network and use it to devise new measures of interdisciplinary impact (also see Foster et al. 2015).

For example, we will measure the extent to which a contribution reduces the average distance between other research papers (nodes) in the network. Further, we will apply the measure of disruptiveness introduced by Wu et al. (2019) to assess the extent to which interdisciplinary research tends to be more disruptive.

Last, we will examine how interdisciplinarity and impact type relate to a number of factors, such as the size of the team of researchers conducting the work, the diversity of academic backgrounds in it, and the status of the institutions of the involved researchers.

Outlook:

With this project we will capitalize on one of DIAS’ strategic advantages---the diverse and interdisciplinary backgrounds of the scientists working at DIAS. Science of science and the science of interdisciplinarity are emerging fields of research, and as an institution we are well-poised to contribute to their development.

Many of the methods that we will develop in the project are generic and can be later used to address further scientific questions, such as competition between different scientific paradigms, credit assignment when two studies on the same topic are published almost concurrently, the networks of knowledge in medieval scholarship, or the consolidation of knowledge in an emerging scientific discipline such as Genomics.

Initial results will be used to frame a more ambitious research proposal that will be submitted to Independent Research Fund Denmark or the Velux Foundation.

References

Analytis, PP., Barkoczi, D., Lorenz-Spreen, P., & Herzog, S. (2020). The structure of social influence in recommender networks. In Proceedings of The Web Conference 2020 (pp. 2655-2661).

Bromham, L., Dinnage, R., & Hua, X. (2016). Interdisciplinary research has consistently lower funding success. Nature, 534(7609), 684-687.

Fontana, M., Iori, M., Montobbio, F., & Sinatra, R. (2020). New and atypical combinations: An assessment of novelty and interdisciplinarity. Research Policy, 49(7), 104063.

Fortunato, S., Bergstrom, C. T., Börner, K., Evans, J. A., Helbing, D., Milojević, S., ... & Barabási, A. L. (2018). Science of science. Science, 359(6379).

Foster, J. G., Rzhetsky, A., & Evans, J. A. (2015). Tradition and innovation in scientists’ research strategies. American Sociological Review, 80(5), 875-908.

Larivière, V., Haustein, S., & Börner, K. (2015). Long-distance interdisciplinarity leads to higher scientific impact. Plos one, 10(3), e0122565.

Levitt, J. M., & Thelwall, M. (2008). Is multidisciplinary research more highly cited? A macrolevel study. Journal of the American Society for Information Science and Technology, 59(12), 1973-1984.

Nelson, R. R. and S. G. Winter (1982). An evolutionary theory of economic change. Harvard University Press.

Stirling, A. (2007). A general framework for analysing diversity in science, technology and society. Journal of the Royal Society Interface, 4(15), 707-719.

Uzzi, B., Mukherjee, S., Stringer, M., & Jones, B. (2013). Atypical combinations and scientific impact. Science, 342(6157), 468-472.

Wu, L., Wang, D., & Evans, J. A. (2019). Large teams develop and small teams disrupt science and technology. Nature, 566(7744), 378-382.

Yegros-Yegros, A., Rafols, I., & D’este, P. (2015). Does interdisciplinary research lead to higher citation impact? The different effect of proximal and distal interdisciplinarity. PloS One, 10(8), e0135095.