Purpose: Overton, a global policy index, provides new opportunities to study the interactions between science and policy. This study aims to characterize the presence of scholarly and policy references in Overton-indexed policy documents and examine their distribution across key bibliographic dimensions, thereby assessing Overton’s potential as a data source for policy metrics.
Design/methodology/approach: We analyze a dataset of approximately 17.5 million policy documents from Overton, incorporating metadata such as publication year, policy source, country, language, subject area, and policy topic. Descriptive statistics are employed to assess the presence and distribution of reference data across these dimensions.
Findings: Overton indexes a substantial volume of policy documents and identifies considerable reference data within them: 7.7% of documents contain scholarly references and 10.6% contain policy references. However, the presence of references varies significantly across publications years, source types, countries, languages, subject areas, and policy topics, indicating coverage biases that may affect interpretations of policy impact.
Research limitations: The analysis is based on the Overton database as of June 2025. As Overton is regularly updated, the distribution patterns of indexed documents and references may evolve over time.
Practical implications: The findings offer insights into the opportunities and constraints of using Overton for investigating evidence-based policymaking and for assessing the policy uptake of research outputs in the context of research evaluation.
Originality/Value: This is the first large-scale study to systematically examine the distribution of reference data in Overton. It contributes a foundational understanding of this emerging source for policy metrics, highlighting both its potential applications and limitations, and underlining the importance of addressing current coverage imbalances.

Purpose: Rank aggregation plays a crucial role in various academic and practical applications. However, accurately assessing the quality of ranking data remains a critical challenge. This study aims to propose methods for assessing the quality of ranking data from the perspective of its distribution.
Design/methodology/approach: This study adopts a network science perspective, transforming ranking data into a network and evaluating its quality using network structural entropy. In addition, we extended three commonly used ranking data generation models to produce ranking data with different distribution characteristics. Finally, the effectiveness of the proposed methods was validated using both synthetic and real-world data.
Findings: Through experiments, we validated the effectiveness of the proposed methods in assessing the quality of ranking data from the perspective of distribution. Additionally, the study revealed the following: (1) simply increasing the number of input rankings does not necessarily improve data quality; (2) when dealing with unevenly distributed ranking data, different aggregation methods exhibit significant differences in performance; and (3) increasing the length of input rankings can mitigate the decline in aggregation effectiveness caused by the uneven probability of each object being ranked.
Research limitations: (1) This study focuses on the impact of distribution characteristics on the quality of ranking data, without considering the effect of disagreements within the data; (2) although the proposed methods have been validated on synthetic and real-world datasets, their generalizability may still require further testing on more diverse datasets.
Practical implications: The methods proposed in this study enables researchers and information managers to more accurately assess the quality of input data before performing rank aggregation, thereby enhancing decision-making reliability.
Originality/value: This study proposes two novel methods from the perspective of network science to address the challenge of data quality assessment in rank aggregation, providing both theoretical support and practical insights for related fields.

Purpose: Prior Information Retrieval (IR) research synthesizes progress from individual studies, yet academia-industry collaboration dynamics remain unexplored. This study investigates: (1) productivity patterns and venues, (2) citations-downloads relationships, (3) topic evolution, and (4) collaboration trends.
PDesign/methodology/approach: We perform an analysis of 53,471 ACM IR papers (2000-2018) using bibliometrics and DistilBERT topic modeling.
Findings: We find that industry-involved papers preferred WWW/CIKM venues; collaborations dominated RecSys/CSCW. We see that academia-industry collaborations achieved the highest download-to-citation conversion rates. Academia focused on algorithms; industry on applications; collaborations bridged both with rising human-centered themes.
Research implications: This is a pioneering large-scale bibliometrics revealing collaboration’s impact on IR knowledge evolution and provides a methodological framework for cross-sector analysis.
Practical implications: The paper identifies optimal venues (RecSys/CSCW) for partnerships and guides joint initiatives (shared datasets, grants) to bridge academia-industry divides and enhance research translation.
Originality/value: This is the first large-scale bibliometric analysis of IR academia-industry collaboration. The paper finds many novel insights, including the fact that collaboration boosts citation efficiency, enables complementary specialization, and drives topic convergence.

Purpose: Examining the alignment (or “fit”) of China’s science and technology talent policies provides valuable insights into the challenges and shortcomings in supporting talent development, thereby offering a foundation for enhanced policy design and support.
Design/methodology/approach: This study introduces a policy fit analysis framework, which decomposes policy fit into three dimensions: consistency fit, embeddedness fit, and compensatory fit. By employing quantitative research methods, the study conducts a multidimensional analysis of China’s science and technology talent policies over the period from 2014 to 2023.
Findings: The findings indicate that, after a decade of evolution, China’s policy system for science and technology talent has largely matured into a relatively stable framework, with policy fit demonstrating an upward trend over time. However, several challenges persist. For instance, the policy system places a disproportionate emphasis on talent cultivation and development, while comparatively fewer policies address the introduction, aggregation, and strategic planning of talent. Additionally, there are observable gaps between policy objectives and actual outcomes, as well as a misalignment between policy supply and the demands of talent development.
Research limitations: The framework of policy fit analysis proposed by the study can only analyze policies at the same level, but it cannot conduct cross-level analysis. In the empirical analysis, the policy texts analyzed were limited to publicly available documents.
Practical implications: The findings provide new perspectives and methodologies for policy evaluation, expanding the scope of existing policy analysis, and also offer meaningful guidance for policymakers and relevant administrative personnel.
Originality/value: This paper introduces, for the first time, a policy fit analysis framework, addressing a gap in the study of policy alignment.

Purpose: Since peer review for funding decisions is crucial to the scientific system, we direct the reader towards new ideas related to research funding and the associated peer review process.
Design/methodology/approach: We describe the overall structure of the funding review system and explore the expectations of its various key stakeholders. An examination of testing across the review processes of different funding agencies revealed several issues in the current system. We then summarize the efforts to explore potential solutions. Before concluding, we also discuss recent initiatives, including partial lottery mechanisms, distributed peer review, and methods for identifying originality in proposals by examining areas of non-consensus among reviewers and applicants.
Findings: It is difficult to test whether the funding peer review system functions as expected. Moreover, when the peer-review process was replicated across different review groups, the inter-rater problem, where two or more well-intentioned reviewers reached divergent conclusions, was found to be widespread in funding evaluations. At its core, this issue stems from substantive disagreements among reviewers, which can introduce bias into the process. As a result, organizing a peer-review system that is fair, valid, and reliable for funding decisions is particularly challenging. The contemporary organization of the funding review system does not guarantee that it can fulfill its purpose. Consequently, scientists are looking to substantiate funding decisions with more scientific evidence. Some new initiatives have been proposed, which are either more interactive with a strictly organized procedure or are more random (or stochastic), leading to less bias.
Research limitations: For practical reasons, we were not able to discuss all, or at least the main, funders in the world.
Practical implications: Considering the various steps in peer review procedures for funding decisions may inspire the readers to suggest improvements to the existing system, resulting in reduced bias and greater equality among scientists.
Originality/value: Our work contributes to understanding peer review in funding contexts and to exploring possible reforms aimed at improving the existing system.