1 Introduction
Since the inception of modern science, the interplay between policy and science has been a critical area of focus for both scholars and practitioners (van den Hove, 2007). Despite the existing connections between policy and science, a significant gap remains between these two domains, as policies still frequently overlook science (Talbot & Talbot, 2015). When policy overlooks science, it results in a disconnect between scientific discoveries and policy decisions—a discrepancy that becomes especially pronounced during sudden public health emergencies like the COVID-19 pandemic (Hodges et al., 2022). During this unprecedented crisis, the scientific community responded swiftly, generating a substantial body of research on topics including viral transmission mechanisms, vaccine development, and therapeutic approaches. However, the process by which these scientific findings are translated into effective policy measures often lags behind, leading to severe consequences (Gentry et al., 2020). Consequently, the overlook of science by policy during the pandemic is highly destructive. For instance, it could have led to a delay in converting scientific advancements into timely and effective policy interventions, thus exacerbating the spread of the virus and causing excessive socio-economic damage.
Given these considerations, identifying and understanding the overlook of science by policy is of paramount importance. This insight can enhance the scientific rigor and effectiveness of policies while also promoting scientific research to better serve societal needs. However, exploring the overlook of science by policy presents numerous challenges, particularly from a quantitative research perspective. On one hand, policy and science operate within distinct discourse systems and knowledge domains. The data associated with each are markedly different in form, source, and structure, making it challenging to apply uniform quantitative analysis (Morlacchi & Martin, 2009). On the other hand, the interactions between policy and science are often complex and non-linear, influenced by a multitude of potential factors. This complexity significantly complicates the development of precise quantitative models (Cheng et al., 2021).
Against this backdrop, this study aims to address two critical research questions:
RQ1. Methodologically, how can we quantitatively explore the scientific content overlooked by policy?
RQ2. Practically, what scientific content was overlooked by policy during the COVID-19 pandemic?
To tackle these issues, the study proceeds as follows. First, from a methodological perspective, an effort is made to construct an effective quantitative analytical framework. This framework seeks to integrate textual information from policy documents and scientific papers, employing text analysis techniques to identify the scientific content overlooked by policy. Second, at a practical level, the study focuses on the unique context of the COVID-19 pandemic. We investigate the scientific content that has been overlooked in policy formulation, focusing on goal-setting, decision-making processes, and implementation outcomes during this period. Furthermore, this study provides empirical evidence and targeted recommendations aimed at bridging this divide.
The rest of this paper is organized as follows. The second part introduces the research related to the relation and gaps between policy and science; The third section of this paper addresses the data and procedures employed; The fourth section analyzes the scientific content overlooked by policy during COVID-19; In the final section, the discussions were presented.
2 Literature review
2.1 Theoretical foundations of the relationship between policy and science
The theoretical investigation of the relationship between policy and science has a long history. In previous studies, Funtowicz (2006) distinguished conceptual models regarding the relationship between science and policy in the decision-making process.
(1) Post-Normal Science, which aims to manage uncertainty in science and politics. It emphasizes scientific practice under conditions of high risk, uncertainty, and diversity or adversarial nature of decision values (Betz, 2006).
(2) Precautionary Model, which focuses on taking preventive measures when facing potential risks. It is based on the idea that action should be taken to prevent possible harm even in the presence of scientific uncertainty (Weinstein et al., 2020).
(3) Framework Model, which refers to promoting the development of machine learning applications through predefined structures. Although this definition is not entirely the same as the framework model in the policymaking process, the framework model in policymaking may refer to a series of principles or methods that provide structure and guidance for decision-making (Funtowicz, 2006).
(4) Demarcation Model, which involves defining and distinguishing the boundaries between different types of knowledge. In policymaking, this may relate to distinguishing scientific knowledge from other types of knowledge, such as local knowledge and traditional knowledge (Chen et al., 2021).
(5) Extended Peer Community Model, which emphasizes including a wider range of participants in the generation and evaluation of scientific knowledge, not just traditional scientific peers but also other stakeholders and the public (Funtowicz, 2006).
To put it more directly, four types of relationships exist between policy and science. Boswell and Smith (2017) proposed four distinct theoretical approaches to conceptualize the relationship between research and policy: (1) Knowledge shapes policy. This approach assumes that research and knowledge can have a direct impact on the policy-making process (Weiss, 2021); (2) Politics shapes knowledge. This method challenges the notion of research being independent of politics and policy, instead focusing on how political forces shape the development and direction of knowledge (Navarro, 2007); (3) Co-production. This approach posits that research knowledge and governance are mutually constitutive, meaning that knowledge and policy evolve through an interactive and co-evolutionary process (Jasanoff, 2004); (4) Autonomous spheres: This method views science and politics as separate domains with their own logics and systems of meaning (Caplan, 1979).
Based on this perspective, the four types of relationships between policy and science can be further specified as follows.
(1) Policy and science as disconnected domains. There has long been skepticism about the connection between science and policy, with both being viewed as highly unrelated fields (Luhmann, 1991). For instance, the two-communities theory highlights the significant gap between scientists and policymakers, effectively separating research from the policy-making process (Caplan, 1979).
(2) Policy significantly influences science. The COVID-19 policies have notably steered the direction of scientific research. Fields such as life medicine, clinical science, and certain areas of basic science have seen unprecedented development opportunities, profoundly impacting the historical trajectory of scientific research. The pandemic has thus had a deep and lasting influence on the course of scientific inquiry (Gao et al., 2021).
(3) Evidence-based decision-making enhanced by the pandemic. The COVID-19 crisis has brought evidence-based decision-making into sharper focus. Governments worldwide have sought to formulate the most rational policies based on scientific evidence, aiming to enhance policy effectiveness through science. This emphasis on evidence-based policymaking has become a crucial component of post-pandemic governance strategies in many countries (Myers et al., 2020).
(4) Co-evolution of policy and science. When scientific findings are rapidly translated into pandemic control policies within a short timeframe, it indicates that during the COVID-19 pandemic, policy and science have tended towards a state of rapid co-evolution (Cheng et al., 2021). Yin et al. (2021) utilizing the Overton database of policy documents and the Dimensions database of academic papers and citations to reveal the tight interaction between the political and academic spheres during the pandemic, exemplifying the co-evolutionary relationship between science and policy. Thus, it can be said that the pandemic has, to some extent, accelerated the evolution of the relationship between policy and science.
2.2 Research on the gap between policy and science
Despite the increasingly common interaction between policy and science, scholars generally agree that a gap still exists between these two domains. This gap can be summarized as follows.
(1) Cognitive and communication disparities. A notable divergence exists between scientists and policymakers in terms of cognitive frameworks and language. Scientists are accustomed to grappling with complexity and uncertainty, whereas policymakers seek certainty and clear directives (Bradshaw & Borchers, 2000). This disparity creates significant barriers to the communication and adoption of scientific evidence in policymaking (Sébastien et al., 2014).
(2) Evidence utilization and dynamism in the decision-making process. Policymakers may underutilize scientific evidence during decision-making, not only due to the inherent uncertainties within the evidence but also because of the urgency associated with policy decisions and political pressures (Brownson & Jones, 2009). Additionally, the dynamic nature of the policy environment contrasts with the relatively static progression of scientific research, leading to a lag in how scientific evidence informs policy (van der Arend, 2014).
(3) Misalignment of objectives and motivations. The pursuit of knowledge and understanding in scientific research diverges from the goal-oriented approach of policymaking, which aims to address specific issues and societal needs. This discrepancy in objectives and motivations can result in a disconnect between scientific research and policy goals (Borja et al., 2016).
(4) Uncertainty in scientific evidence and economic considerations. The uncertainty surrounding scientific evidence can lead to indecision among policymakers (Borja et al., 2017). Moreover, policymakers must consider cost-effectiveness under limited resources, while scientific research often demands substantial investment. This economic strain limits the application of scientific research in the policymaking process (Alazmi & Alazmi, 2023).
When exploring the gap between policy and science, existing studies employ a variety of research methodologies to analyze this intricate phenomenon. Firstly, many studies adopt an extensive literature review approach, systematically examining existing research findings to identify gaps between policy and science (Bradshaw & Borchers, 2000). This method allows for a comprehensive understanding of the historical context and prevailing issues in the field. Secondly, qualitative research methods, including semi-structured interviews, provide deeper insights into the communication and cognitive disparities between policymakers and scientists (Bradshaw & Borchers, 2000). These approaches facilitate a nuanced exploration of individual perspectives and experiences, highlighting the challenges faced in bridging the divide. Furthermore, theoretical models are utilized to dissect the formation and evolution of gaps within the policy process. For instance, Lasswell’s stages model and Kingdon’s multiple streams framework offer valuable lenses through which to understand how and why these gaps develop over time (Alazmi & Alazmi, 2023). Through case study analysis, researchers illustrate the concrete manifestations of these gaps within real-world policy environments, shedding light on the actual connections between scientific evidence and policy actions (Sébastien et al., 2014). This approach helps contextualize abstract theories and provides practical examples that can inform future policy interventions. Lastly, qualitative methods are employed to clarify and measure causal mechanisms within the policy process, elucidating how scientific evidence influences policy outcomes through various channels (Wellstead et al., 2018). Such analyses enhance our understanding of the complex pathways by which scientific knowledge impacts decision-making and ultimately shapes policy results.
Despite the in-depth analysis provided by the aforementioned literature on the gap between policy and science, several limitations are evident. Firstly, much of the literature tends to focus on analyzing gaps within a single disciplinary domain, lacking an interdisciplinary synthesis that bridges natural and social sciences. This narrow focus may constrain the understanding of the complexity inherent in these gaps, as the interaction between policy and science frequently requires the integration of knowledge from multiple disciplines. An interdisciplinary approach would provide a more holistic perspective, revealing the multifaceted nature of the challenges faced at the interface of policy and science. Secondly, the majority of studies rely predominantly on qualitative methods, with a relative dearth of quantitative data support. While qualitative approaches offer rich, detailed insights, incorporating quantitative research can provide a more objective basis for analysis. Quantitative methods enable researchers to measure the extent and impact of the gaps more precisely, offering empirical evidence that can strengthen the validity of findings and contribute to more robust policy recommendations.
Building upon qualitative research methods and incorporating quantitative approaches, especially text analysis methods, can address these shortcomings. Text analysis offers more precise measurements and deeper semantic relationships, providing a more comprehensive research perspective. Given that the COVID-19 pandemic has, to some extent, accelerated the evolution of the relationship between policy and science, and considering the vast amount of policy and research data related to the pandemic, this study attempts to use text analysis methods to explore the scientific content overlooked by policy during the COVID-19 period.
3 Data and methods
This paper employs text analysis methods to identify overlooked scientific contents in policy from textual topics and keywords separately. The research framework of this study is illustrated in Figure 1. In the framework illustrated in Figure 1, the core components consist of four parts: data acquisition and preprocessing, discovering the overlooked contents using topics (left branch), discovering the overlooked contents using keywords (right branch), and the overlooked contents analysis and discussion. This section focuses on detailing the first three parts, while the fourth part will be elaborated upon in conjunction with the case of COVID-19.
Figure 1. The research framework of this study. |
3.1 Data
The policy document data utilized in this study were sourced from the Overton database, recognized as one of the most comprehensive repositories of global policy documents. Given that the global COVID-19 pandemic had largely been brought under control by the second half of 2022, this study focuses on the policy landscape during the most critical phase of the pandemic. Consequently, we retrieved 247,580 metadata entries related to policies addressing COVID-19 from the Overton database. These entries encompass all policies published globally from January 1, 2020, to March 1, 2022, following the search criteria outlined by Yin et al. (2021). Each metadata entry includes details such as the policy release date, title, a link to the full-text PDF, policy type, classification, keywords, and citation information. Policy classification adheres to IPTC’s media topic taxonomy, ensuring standardized categorization. The extraction of keywords involves an advanced text analysis process where phrases and entities within the policy documents are identified. These are subsequently cross-referenced with Wikipedia’s extensive keyword list to pinpoint the most relevant and prevalent terms, which are then designated as the policy keywords.
The scientific paper data utilized in this study were sourced from the CORD-19 dataset and the OpenAlex database. The CORD-19 dataset is a comprehensive corpus of academic papers focusing on COVID-19 and related coronavirus research, curated and maintained by the Semantic Scholar team at the Allen Institute for AI to support text mining and natural language processing (NLP) research (Wang et al., 2020). Given its extensive collection of COVID-19-related scientific papers, the CORD-19 dataset is particularly suitable for this study. For our analysis, we downloaded version 109 of the CORD-19 dataset, which includes 992,921 papers related to COVID-19. However, only 599,773 of these papers had Digital Object Identifiers (DOIs), as the dataset also encompasses a significant number of unpublished papers that were not considered in this study. Due to the limited detailed metadata available in the CORD-19 dataset, we cross-referenced the DOIs with the OpenAlex database to enrich the metadata. This process yielded a final dataset of 500,526 scientific papers with comprehensive metadata. The OpenAlex database serves as an open-access alternative to the Microsoft Academic Graph (MAG), indexing over 200 million scholarly resources. The metadata obtained for each scientific paper include the title, publication date, journal information, author details, citations, references, and literature keywords (Priem et al., 2022).
This study utilized information from 247,580 policy documents and 500,526 scientific papers related to the COVID-19 pandemic. The keywords for policy documents were sourced from the Overton database, whereas those for scientific papers were obtained from the OpenAlex database. In both cases, the keywords are derived through text analysis of phrases and entities within the texts. These extracted terms are then compared against a comprehensive keyword list from Wikipedia, with the most significant and relevant phrases and entities ultimately selected as the keywords. By ensuring that both policy and scientific paper keywords are drawn from the same standardized Wikipedia keyword list, this approach enables a consistent and comparable analysis framework. This standardization facilitates a more rigorous comparison and analysis of the keywords across policy documents and scientific literature, aligning them under uniform criteria.
Based on the data mentioned above, this study will undertake two main areas of research: discovering the overlooked content using topic and discovering the overlooked content using keywords. Both sections of the text analysis are based on the keywords provided by Overton and OpenAlex, as previously described.
3.2 Discovering the overlooked content using topics
In this study, we identified the overlooked scientific contents in policy by examining the differences in topic attention between policy documents and scientific papers. To achieve this, we utilized the Word2Vec algorithm to extract topics from both types of documents. Word2Vec transforms words into numerical vectors by training on the contextual semantic information of feature words within the documents, thereby representing textual information with richer features in a K-dimensional vector space. The word vectors generated by the Word2Vec model are frequently employed for finding semantically similar words, clustering, and other natural language processing tasks. Semantic similarity between words is quantified using cosine similarity between their vectors. A key advantage of Word2Vec lies in its ability to effectively capture the contextual environment of words, ensuring that the vectors contain sufficient semantic information due to the consideration of the words’ contextual background (Church, 2017).
Specifically, let the set of policy documents be denoted as D={d1, d2,..., dn}, where each document di contains a set of m keywords {w1, w2,..., wn}. The sequence of these keywords from all documents is compiled into a corpus and input into the Word2Vec model for training. We chose the Continuous Bag-of-Words (CBOW) model to train this corpus, which maps each policy document into a corresponding keyword vector $\left\{v_{d_{1}}, v_{d_{2}}, \ldots, v_{d_{n}}\right\}$. Subsequently, the K-means clustering algorithm is applied to group the document vectors $\left\{v_{d_{1}}, v_{d_{2}}, \ldots, v_{d_{n}}\right\}$ into k clusters. Initially, k cluster centroids {μ1, μ2,...μk} are randomly selected. Each document di is then assigned to the cluster whose centroid has the highest cosine similarity with the document vector, i.e., cos< di, μj>. Following this initial assignment, the centroids are updated iteratively by recalculating them as the mean of the vectors in their respective clusters. Documents are reassigned to the nearest centroid until the centroids stabilize, indicating convergence. The optimal number of clusters k is determined using the sum of squared errors (SSE) metric, with the ideal k value typically identified at the elbow point of the K-SSE curve (Hamerly & Elkan, 2003). Upon completion of the clustering process, the 50 most frequently occurring keywords within each cluster are extracted. These keywords are then used to summarize the topics represented by each cluster.
The scientific papers were also processed using the same methodology, ultimately grouping both policy documents and scientific papers into several distinct topics. After identifying the topics of policy documents and scientific papers across different periods, this study further calculates the attention given to each topic over time. Let the variable Ciw (i∈[1, n]) denote the attention received by the i-th topic in policy documents. This attention is determined by the number of policy documents ti that belong to this topic and is calculated using the following equation.
Ciw = lg (ti+1) (i∈[1, n])
The variable Cil(i∈[1, n]) represents the attention received by the i-th topic in scientific papers. This attention is determined by the number of scientific papers pi associated with this topic and is calculated using the following formula:
Cil = lg (p+1) (i∈[1, n])
Finally, a scatter plot is generated for visualization, with the vertical axis representing the attention each topic receives in policy documents (Ciw) and the horizontal axis representing the attention each topic receives in scientific papers (Cil) Through this visualization, we can identify topics that do not receive sufficient attention in policy documents, thereby pinpointing the potential overlooked contents. Additionally, when the attention a keyword receives in policy documents is precisely equal to the attention it receives in scientific papers, Ciw = Cil. In the scatter plot, we plot a line where $y=\frac{C_{i w}}{C_{i l}} x$. This line serves as a reference:
(1) Above the Line: Keywords positioned above this line indicate topics that have received less attention in policy documents compared to scientific papers, suggesting these topics may be underrepresented in policy.
(2) On or Below the Line: Keywords located on or below the line indicate topics that have received adequate or more attention in policy documents relative to scientific papers.
3.3 Discovering the overlooked content using keywords
To identify the overlooked scientific contents in policy from keywords, this study conducts a detailed comparison of the keywords in policy documents and scientific papers across different time periods. The analysis divides policy documents and scientific papers into nine discrete three-month intervals (with the ninth interval covering only two months): January 2020 to March 2020, April 2020 to June 2020,..., October 2021 to December 2021, and January 2022 to February 2022.
The processing involves three key steps. Firstly, frequency Calculation. The frequency of each keyword in policy documents and scientific papers is calculated separately for each time period. Secondly, attention Calculation. The attention given to each keyword is then quantified using the formulas (1) and (2) presented earlier. Finally, comparison of Attention. We compare the attention levels of different keywords between policy documents and scientific papers for each respective period. To illustrate this process, we explore two specific cases using keyword A as an example. This approach allows us to highlight discrepancies in keyword attention over time, providing insights into the potential overlooked scientific content in policy.
Scenario 1: In the first scenario, keyword A appears in both policy documents and scientific papers during a specific time period. Specifically, keyword A occurs b times in policy documents and c times in scientific papers, where b < c. Consequently, the attention given to keyword A in policy documents is calculated as lg (b + 1), while in scientific papers it is lg (c + 1). Given that lg (b + 1) < lg (c + 1), this indicates that keyword A receives less attention in policy documents compared to scientific papers during that period. Thus, keyword A is likely to represent an overlooked scientific content in policy focus for that period, highlighting an area where policy attention may need to be increased to align with scientific emphasis.
Scenario 2: In a specific time period, keyword A appears in scientific papers but not in policy documents. This indicates that keyword A is exclusively addressed by scientific papers and receives no attention from policy documents during this period. Consequently, keyword A is identified as a overlooked scientific content in policy focus for that period.
Although other scenarios may arise—such as keyword A appearing exclusively in policy documents or not appearing in either policies or scientific papers—these cases do not contribute to detecting the overlooked scientific content in policy and are therefore not considered in this analysis. The identification of keywords as potential overlooked scientific content focuses primarily on the two aforementioned scenarios. Various visualizations are presented by calculating the attention given to different keywords in both policy documents and scientific papers, facilitating a clear comparison of keyword attention over time.
4 Research findings in the case of COVID -19
4.1 The overlooked topics
4.1.1 Topics extraction
In this study, we employ the method described above to derive the topics of policy documents and scientific papers. To determine the optimal number of clusters for the K-Means algorithm, we initially plot the K-SSE curves for both policy documents and research articles, as illustrated in Figure 2. The horizontal axis in Figure 2 represents the number of clusters k, while the vertical axis indicates the sum of squared errors (SSE) from the clustering results. According to the elbow rule, which identifies the point where adding another cluster does not significantly improve the model’s fit, we observe a clear inflection point at k = 9 for both policy documents and scientific papers. Therefore, we set the number of clusters k to 9 for the K-Means clustering of both document types. This choice ensures that the clustering reflects meaningful groupings without overfitting to the data.
Figure 2. Trend of SSE with k value in K-Means clustering. |
Following the clustering process, the top 50 keywords for each cluster were extracted, and the topics of each cluster were summarized based on these keywords. The identified topics from both policy documents and scientific papers, along with their associated information, are presented in Table 1.
Table 1. Topics of policy papers and scientific papers. |
| Policy topics | ||
|---|---|---|
| Topics | The most frequently keywords of the topic | The number of policies |
| Sustainability | sustainability, sustainable development goals | 26,795 |
| Economics and Finance | economy, finance, business, tax, bank, money, debt, macroeconomics, loan, recession, gross domestic product | 23,456 |
| Law and Public Affairs | government, law, politics, justice, public sphere, policy, social institutions, public law, issues in ethics, social issues | 31,813 |
| Education | education, learning research, teacher, technology, culture, child | 17,644 |
| Environment | natural environment, nature, low-carbon economy, transport, greenhouse gas, climate change, renewable energy, energy | 15,877 |
| Public Health and Nursing | health care, health sciences, public health medical specialties, hospital, health economics, disease | 23,315 |
| Infection and Immunity | epidemiology, infectious diseases, immunology, infection, vaccine | 25,478 |
| Biology and Medicine | microbiology, medicine, clinical medicine | 2,152 |
| Politics and Public Relations | Public Relations, global politics, united states, European Union, China, national security, united nations, international security, foreign policy, diplomacy | 12,174 |
| Policy topics | ||
| Topics | The most frequently keywords of the topic | The number of papers |
| Computer Science | computer science, artificial intelligence, data science, machine learning, mathematics, statistics | 35,650 |
| Psychology | psychology, clinical psychology, mental health, psychiatry, anxiety, social psychology | 56,202 |
| Politics and Public Relations | political science, public relations, geography, sociology | 56,197 |
| Biology and Medicine | medicine, biology, chemistry, computational biology, cell biology, pharmacology, genome, biochemistry | 41,560 |
| Virus Research | virology, betacoronavirus, coronavirus, virus | 81,904 |
| Disease Treatment | internal medicndine, pediatrics, disease, emergency medicine, surgery, pneumonia, cardiology, gastroenterology | 93,801 |
| Infection and Immunity | immunology, immune system, antibody, vaccination, inflammation | 41,021 |
| Public Health and Nursing | family medicine, health care, medical emergency, public health, nursing, intensive care medicine | 69,029 |
| Environment | environmental science, materials science, environmental health | 17,364 |
4.1.2 Topics as the overlooked content
In this study, we present a median scatterplot for visualization, with the horizontal axis representing the attention received by each topic in scientific papers (Cil) and the vertical axis representing the attention received by each topic in policy documents (Ciw). The results are illustrated in Figure 3. The dashed line in the figure represents y = Ciw/Cilx. Given that Ciw = log (500526) and Cil = log (247580), this results in the dashed line being y = 0.7x.
Figure 3. Comparison of the attention to topics in policies and papers. |
The median scatterplot in Figure 3 is divided into four quadrants by the two axes. The lower left quadrant indicates topics that receive moderate attention in both policy documents and scientific papers. The lower right quadrant highlights topics that receive significant attention in policy documents but insufficient attention in scientific papers. The upper right quadrant represents topics that receive substantial attention in both policy documents and scientific papers. The upper left quadrant identifies topics that receive low attention in policy documents but significant attention in scientific papers. Analysis reveals that the overlooked contents are primarily located in the upper left and lower right quadrants. These areas highlight topics where there is a notable discrepancy between policy focus and scientific emphasis, indicating potential areas for policy adjustment to better align with scientific priorities.
(1) The upper left quadrant
Four topics are located in the upper left quadrant: computer science, psychology, virus research, and disease treatment. According to our analysis, these four topics have garnered significant attention in COVID-19-related scientific papers but have not received adequate attention in policies aimed at combating the pandemic.
Virus research primarily focuses on understanding the nature and evolutionary pathways of novel coronaviruses, particularly in relation to pneumonia treatment. Disease treatment, meanwhile, centers on identifying medications for pneumonia and exploring its connections with other diseases. These two topics exhibit strong academic characteristics and are predominantly driven by scientific inquiry, which may not align directly with the practical objectives and content of policy documents. Consequently, they tend to receive less emphasis in policy formulations.
In contrast, computer science and psychology present distinct areas that warrant greater policy attention. The field of computer science explores the application of advanced technologies, such as artificial intelligence, to prevent the spread of the epidemic, accelerate drug development, manage public opinion, and optimize the deployment of medical supplies. In addition, the field of psychology examines the psychological impacts of the pandemic on individuals, including chronic stress, loneliness, depression, and anxiety. Addressing mental health concerns is crucial for maintaining societal resilience during prolonged crises. Despite their importance, these two topics have not yet received sufficient policy attention, highlighting the critical overlooked scientific contents in policy.
These issues demand greater policy attention but have thus far been overlooked. Given that computer science and psychology represent the significant overlooked scientific contents in policy addressing COVID-19, this article argues that the disparity in attention between these two topics and those of virus research and disease treatment stems from the differing objectives and content emphases between scientific research and policy documents.
(2) The upper right quadrant
In Figure 3, it is evident that five main topics are distributed in the upper right area: politics and public relations, biology and medicine, infection and immunity, public health and nursing, and environment. These topics have received significant attention in both policy documents and scientific papers. The dashed line in Figure 3 divides this quadrant into two parts: topics above the dashed line receive relatively less attention in policies compared to papers, while those below the line receive more attention in policies than in papers.
Our research reveals that the topics of environment, infection and immunity, public health and nursing, politics and public relations, biology, and medicine are all located above the dashed line, indicating that they receive proportionally less attention in policy documents. Among these, infection and immunity, as well as biology and medicine, exhibit stronger scientific attributes and focus less on immediate social concerns. Consequently, these two topics are not considered as the overlooked scientific contents in policy. In contrast, the remaining three topics—environment, public health and nursing, and politics and public relations—are more closely aligned with pertinent social issues. Despite their importance, these topics receive insufficient attention in policy documents, making them the potential overlooked scientific contents in policy. They better match the content and goals of policy-making but are underrepresented, highlighting a disconnect between policy priorities and societal needs.
Thus, from the perspective of scientific studies, topics such as environment, public health and nursing, and politics and public relations are identified as the overlooked scientific contents in policy aimed at combating COVID-19. This analysis underscores the need for policies to address these critical areas more comprehensively.
Among the policies and publications highlighted in this paper, the overlooked scientific contents in policy during COVID-19 regarding five key topics: computer science, psychology, environment, public health and nursing, and politics and public relations. Specifically, computer science and psychology are areas that should have been addressed but have not yet received adequate attention. In contrast, while the policy has already touched upon environment, public health and nursing, politics, and public relations, these topics still require significantly more focus and consideration.
4.2 Overlooked keyword analysis
To detect the overlooked scientific contents in policy during COVID-19 with greater precision, this study conducts a thorough analysis of the critical keywords from policy documents and scientific papers across various periods, as described above, and discusses them separately according to two distinct scenarios. The analysis reveals that scientific research findings were typically translated into COVID-19 policy within an average of 8.36 days after publication (Cheng et al., 2021). Both policy documents and scientific papers tended to evolve rapidly in a co-evolutionary manner. When comparing publication cycles across different periods, it was observed that scientific papers generally had a longer publication cycle compared to policy documents. To account for this discrepancy, this study adopts a three-month time frame and delays the inclusion of scientific papers by 10 days. For example, the median scatter plot for January 2020 to March 2020 (Figure 4(a)) is analyzed using policy documents dated from January 1, 2020, to March 31, 2020, and scientific papers published from November 20, 2019, to April 10, 2020. This approach effectively addresses the longer publication cycle of scientific papers, ensuring a more accurate alignment between research findings and policy implementation.
Figure 4. Distribution of the attention of each keyword in policies and papers in different periods. |
4.2.1 Overlooked keyword in scenario 1
To explore the overlooked topics in scenario 1, we utilized a median scatter plot for visualization. In this plot, the horizontal axis represents the attention each keyword received in scientific papers, while the vertical axis indicates the attention each keyword received in policy documents. The scatter plots for nine different periods are displayed in Figure 4, illustrating the distribution and temporal changes of the overlooked scientific contents in policy. Our analysis reveals that most keywords are consistently clustered in the lower-left area across all periods, indicating they receive limited attention in both policy documents and scientific papers. These keywords were deemed insufficient for detecting the overlooked scientific contents and were therefore excluded from further consideration in this study. Additionally, we identified that the overlooked scientific contents in policy predominantly occur in the upper-left and upper-right portions of the scatter plot, specifically above the dotted line. Consequently, this section focuses exclusively on keywords located in these two regions, as they highlight areas where there is a significant disparity between scientific emphasis and policy attention.
Figures (a) through (i) in Figure 4 depict the attention given to each keyword in policies and papers across nine distinct periods. This study focuses on two specific regions: the upper-left area and the region above the dotted line in the upper-right quadrant. Keywords distributed in these two areas are ranked based on the disparity in attention they received in policies versus papers, from the largest to the smallest difference. For each period, the keywords with the greatest disparity in attention within these areas were selected as representative keywords for analysis. Due to space limitations, this article does not provide an exhaustive interpretation of all key terms. Instead, it selectively analyzes the policy blind spots identified during different periods, highlighting the most significant discrepancies between scientific research and policy focus.
The evolution of the overlooked scientific contents in policy during the COVID-19 pandemic reflects the shifting priorities and challenges faced by policies as the crisis unfolded. Here is a refined summary of these changes over time.
(1) During the initial phase of the COVID-19 pandemic from January to March 2020, policies focused primarily on internal medicine, computer science, medical emergencies, and emergency medicine, as indicated by the concentration in the upper-left quadrant. This reflects a swift policy response to the pressures placed on the healthcare system by the pandemic, particularly in the areas of emergency and internal medicine. However, policies may not have fully leveraged the applications of computer science in medical emergencies, such as telemedicine and data analytics—technologies critical for alleviating strain on medical resources and enhancing the efficiency of healthcare services.
Simultaneously, disciplines such as virology, biology, disease studies, and microbiology garnered significant attention in academia, as evidenced by their prominence in the upper-right quadrant. Despite this, these foundational research areas may not have received adequate emphasis in policy discussions. Research in virology and microbiology is pivotal for understanding viral characteristics, transmission mechanisms, and developing effective therapeutic interventions. The disparity between policy focus and academic research during this period could have led to a delayed scientific foundation for pandemic understanding, underscoring the necessity of strengthening the link between basic research and policy formulation in pandemic response efforts.
(2) During the period from April to June 2020, the COVID-19 pandemic posed unprecedented challenges to global health systems. Despite showing higher policy attention in the upper-left quadrant for telemedicine, data science, and family medicine, policies may not have fully leveraged data science to optimize telehealth services. This suggests that while policies recognized the importance of these fields, there was a shortfall in implementing and integrating data science to enhance telemedicine capabilities.
Meanwhile, terms in the upper-right quadrant, such as political science, healthcare, economics, and microbiology, indicate that these areas received significantly more attention from academia than from policies. This discrepancy may reflect a deeper engagement with these topics within academic circles, but it also suggests that policies may not have adequately incorporated this research into their decision-making processes. Furthermore, interdisciplinary studies at the intersection of political science and healthcare could have provided crucial perspectives for pandemic response, but these insights may not have been sufficiently considered by policies.
(3) During the period from July to September 2020, global strategies for responding to the COVID-19 pandemic gradually took shape, but there remained significant disparities between policy and academic research priorities. While terms in the upper-left quadrant, such as internal medicine, computer science, and environmental health, received some policy attention, they may not have been fully leveraged, particularly concerning the recognition of environmental factors’ impact on the pandemic.
Simultaneously, disciplines in the upper-right quadrant, including psychology, political science, public health, and epidemiology, garnered substantial attention within academia. However, these research findings appear not to have been adequately translated into policy actions.
This disparity suggests that although policies likely recognized the importance of internal medicine and computer science in pandemic response, the integration of environmental health and mental health considerations into actual policy formulation was potentially insufficient. Moreover, interdisciplinary studies at the intersection of social sciences and public health, encompassing psychology and epidemiology, provided a multi-dimensional perspective on the pandemic. Yet, the incorporation of these perspectives into policy frameworks appears to have been inadequate.
(4) During the period from October to December 2020, the upper-left quadrant shows that internal medicine, computer science, and medical services received heightened policy attention. This likely reflects policies’ ongoing emphasis on direct healthcare services. However, the application of computer science in medical technology may not have been fully leveraged, thereby limiting the potential of technological innovation in pandemic response.
Simultaneously, disciplines in the upper-right quadrant, such as psychology, immunology, virology, and microbiology, garnered significant attention within academia. Yet, these academic research findings appear not to have been adequately translated into policy actions. Research in immunology and virology is crucial for vaccine development and pandemic control, but policies may not have fully utilized these insights to guide public health strategies. Additionally, psychology plays a vital role in understanding the impact of the pandemic on public mental health, yet this area may also have been overlooked.
(5) During the period from January to March 2021, the global response to the COVID-19 pandemic entered a new phase. Policies showed some attention to internal medicine, family medicine, and medical services in the upper-left quadrant, but there was room for increased focus. Meanwhile, disciplines in the upper-right quadrant, such as psychology, immunology, nursing, and epidemiology, received considerable attention within academia but may not have been adequately prioritized by policies.
The importance of family medicine and community healthcare in pandemic response appears to have been underappreciated, with this aspect not fully reflected in policy. Academic attention to mental health issues and immunological research is crucial for understanding the impact of the pandemic on individuals and society. However, these insights may not have been sufficiently integrated into policy frameworks.
This disparity could result in policies lacking comprehensiveness, particularly in terms of mental health support and the application of immunological research. Strengthening the integration of these critical areas into policy formulation would enhance the overall effectiveness of pandemic response strategies.
(6) During the period from April to June 2021, terms in the upper-left quadrant, such as internal medicine, computer science, and public relations, received some policy attention but may not have been fully leveraged to enhance pandemic communication effectiveness and healthcare service efficiency. Meanwhile, terms in the upper-right quadrant, including psychology, immunology, public health, and economic growth, reflect the academic community’s deep engagement with these areas. However, these research findings may not have been adequately prioritized by policies.
This disparity suggests that despite the extensive academic discussion on the critical role of public relations in pandemic communication and the importance of mental health during the pandemic, policies may not have effectively integrated these insights. Moreover, the balance between economic growth and public health—a topic extensively explored in academia—may also have been underrepresented in policy decisions.
(7) During the period from July to September 2021, terms in the upper-left quadrant, such as internal medicine, computer science, and odds ratios, indicate that policies emphasized traditional healthcare and data analysis. However, this emphasis may not have fully leveraged the critical role of statistics in pandemic analysis.
Meanwhile, the upper-right quadrant highlights that psychology, vaccination, business, and healthcare were focal points of academic research. Despite their significance, these areas may not have received adequate attention in policy discussions. Psychology plays a crucial role in understanding public behavior and mental health during the pandemic, while strategies for promoting vaccination and the influence of business factors on healthcare are equally important.
The failure of policies to fully integrate these academic insights could result in less comprehensive pandemic response measures. Specifically, the underutilization of psychological research might hinder efforts to address public behavior and mental health, while insufficient attention to vaccination strategies and business impacts on healthcare could limit the effectiveness of public health interventions.
(8) During the period from October to December 2021, internal medicine, computer science, and sociology in the upper-left quadrant received some policy attention but may not have been fully utilized to drive effective pandemic management. In particular, the sociological perspective in pandemic response appears to have been underappreciated, as was the application of technological innovation in healthcare.
Meanwhile, terms in the upper-right quadrant—such as psychology, immunology, mental health, and economic growth—reflect the academic community’s deep engagement with these areas. Research in psychology and immunology is crucial for understanding the impact of the pandemic on individuals and society. Additionally, the link between mental health and economic growth, a topic extensively explored in academia, may not have received adequate consideration from policies.
This disparity suggests that while policies recognized the importance of certain fields, they may have fallen short in integrating sociological insights and technological advancements into comprehensive pandemic management strategies. The underutilization of psychological and immunological research could hinder efforts to address both public health and socioeconomic challenges effectively.
(9) During the period from January to February 2022, terms in the upper-left quadrant, such as internal medicine, computer science, and public relations, received some policy attention but may not have fully reflected their actual importance in pandemic response. Specifically, the critical role of public relations in pandemic communication and the potential of computer science in data management and analysis were possibly underutilized by policies.
Meanwhile, terms in the upper-right quadrant—such as biology, psychology, immunology, and vaccination—highlighted areas of significant academic focus. These fields did not receive adequate emphasis in policy formulation despite their crucial contributions. Biological research is essential for understanding viral characteristics and pandemic progression, while psychological studies provide valuable insights into the impact of the pandemic on public mental health. Research in immunology and vaccination strategies is vital for guiding effective vaccine distribution and inoculation plans, but these findings may not have been fully translated into policy practice.
This disparity suggests that while policies recognized the importance of certain fields, they may have fallen short in leveraging the full potential of public relations, computer science, and academic research to enhance comprehensive pandemic response efforts.
The aforementioned results indicate that over time, policy responses to the pandemic have exhibited several key trends and shortcomings. Early 2020: Policies rapidly responded to the pressures on healthcare systems but underemphasized the application of computer science in medical emergencies. Mid-2020: There was increased attention to telemedicine and data science, yet implementation and integration remained inadequate. Additionally, interdisciplinary studies at the intersection of political science and healthcare may not have been fully considered. Late 2020 to Early 2021. Policy continued to focus on internal medicine and computer science but overlooked the application of psychological research and immunological studies, which are crucial for comprehensive pandemic response. Mid-2021 to Early 2022: Policies’ attention to public relations and sociology was insufficient, while academic research in psychology, immunology, and vaccination strategies received far greater emphasis than reflected in policy decisions.
Overall, policies have consistently underemphasized several critical areas, including: the application of computer science in healthcare, mental health issues, immunological and virological research. Additionally, less attention has been given to: environmental concerns, the integration of sociology and political science, the balance between economic growth and public health. The preceding analyses revealed that policy oversight often encompassed scientific insights from computer science, psychology, environmental studies, public health and nursing, as well as political and public relations domains. The findings presented in this section closely align with those earlier observations, underscoring a recurring gap between academic research and policy focus.
4.2.2 Overlooked keyword in scenario 2
In the context of scenario 2, certain keywords garnered attention solely within the realm of scholarly research during a specific timeframe, yet they were conspicuously absent from policy discourse. These keywords warrant meticulous examination, as they may illuminate overlooked aspects or reveal discrepancies between policy objectives and the thrust of academic literature. To elucidate this, the present study employs word cloud visualizations to represent keywords that are exclusive to scientific publications and do not feature in policy documents across nine distinct epochs. The outcomes of this analysis are graphically encapsulated in Figure 5, offering a visual synopsis of the thematic divergence between scholarly and policy spheres.
Figure 5. Word cloud of the overlooked topics in scenario 2. |
As illustrated in Figure 5, the scientific information overlooked by policies during the COVID-19 pandemic encompasses five principal categories: public psychological issues, biological research on the novel coronavirus, medical treatment of pneumonia, challenges in public relations, and environmental protection concerns.
Among these concerns, the psychological impact of the pandemic on the public has been a consistently evident area of scientific information overlooked by policy. This includes the critical need for guidance and intervention in addressing chronic stress, loneliness, depression, fear, and other emotional challenges experienced by individuals during the pandemic.
The biological research on COVID-19 has also been a significant area of scientific information that policies have often overlooked. Key areas such as computational biology, coronavirus studies, and cellular science have received inadequate attention, highlighting a gap between scientific inquiry and policy response.
Medical treatment for pneumonia has been another area frequently overlooked by policy throughout most periods. This primarily refers to emergency relief and treatment during the peak of the outbreak, with a shift towards pediatric and geriatric care in the mid-to-late stages of the pandemic, underscoring the evolving priorities in medical responses.
Public issues, including the development of policy science, government-population dynamics, and international public relations during the pandemic, were often overlooked in the pre-epidemic phase. This reflects an initial underestimation of the comprehensive impacts of the pandemic on societal structures and communication channels.
Environmental protection concerns have tended to be overlooked, particularly regarding changes in natural environmental conservation, greenhouse gas emissions, and resource utilization during the pandemic. These aspects have predominantly been ignored by policy for much of the crisis, indicating a broader disconnect between environmental sustainability and pandemic response strategies.
4.3 Result analysis
In the aforementioned analysis, we found five overlooked scientific contents by policy in the topics analysis: Computer Science, Psychology, Environment, Public Health and Nursing, Politics and Public Relations; six overlooked scientific contents in scenario 1 of the keyword analysis: the application of computer science in healthcare, mental health issues, immunological and virological research, environmental concerns, the integration of sociology and political science, the balance between economic growth and public health; five overlooked scientific contents in scenario 2 of the keyword analysis: psychological problems of the population, biological research of the new coronavirus, medical treatment of pneumonia, public relations, and environmental protection issues.
Based on the aforementioned results, we have further summarized four areas of scientific content that were overlooked by policies during the COVID-19. These overlooked scientific contents are discussed as follows.
(1) Insufficient Attention and Guidance to Public Psychological Well-being
The COVID-19 pandemic has profoundly impacted psychological behavior, disrupted daily routines and increased internal tension, which can lead to disorientation and impaired thinking and behavior. When these factors culminate, individuals may experience a psychological crisis (Hossain et al., 2020). Since the outbreak, the general public has faced a range of psychological challenges, including anxiety, depression, panic, and heightened suspicion of illness. The psychological state of the population during the pandemic has been a critical focus in scientific research at various stages. However, it remains underrepresented in policy documents, as evidenced by both topic analysis and keyword analysis in this paper. This oversight highlights one of the significant blind spots in policies addressing the COVID-19 pandemic: the insufficient attention and guidance provided to support public psychological well-being.
(2) Inadequate Attention to Environmental Issues
A comparison of keywords from policy documents and scientific papers relevant to COVID-19 response strategies reveals that policies have inadequately addressed environmental concerns, particularly during the middle and late stages of the pandemic. Initially, pollutant emissions from various industries were significantly reduced as many manufacturing sectors halted operations, regional traffic volumes decreased markedly, and service industries such as food services and labor-intensive light industries ceased operations. These changes temporarily alleviated environmental pressures. However, the pandemic also led to a surge in the production and disposal of medical waste and epidemic prevention materials, which increased pollution levels. Additionally, prolonged periods of home confinement resulted in spikes in household consumption, leading to rapid increases in domestic wastewater discharge, electricity and energy use, and waste generation. This heightened pressure on ecological and environmental protection efforts (Shakil et al., 2020).
(3) Underutilization of Computer Technology
This study highlights that policies have overlooked the significant contributions of computer technology during the COVID-19 pandemic. Despite its critical role in pandemic management, this area has not received adequate policy attention. Computer science and technology, encompassing artificial intelligence, computer simulation, automated robotics, and data science, have been instrumental in various aspects of prevention, control, and treatment.
Computer simulations have enabled researchers to model the spread of the virus, informing more effective public health strategies. Pattern recognition technologies have facilitated rapid facial recognition and body temperature measurements, enhancing surveillance capabilities. Automated robots have been deployed in virus-contaminated and quarantined areas, minimizing human exposure risks. Data science and network science have accelerated the discovery of effective treatments for pneumonia, while blockchain technology has supported economic development through secure and transparent transactions (Morselli Gysi et al., 2021).
(4) Inattention to Public Relations
During the COVID-19 pandemic, two critical areas of public relations between governments and populations have received insufficient attention in policy formulations. The pandemic has had far-reaching effects on international public relations, with countries frequently engaging in cooperative efforts to curb the virus’s spread. However, global governance, trust, peace, and development deficits have continued to widen. The balance of world power has accelerated its transformation, major power relationships have undergone significant adjustments, and the international order has been profoundly impacted.
Simultaneously, as the pandemic persisted and national policies evolved, public trust and satisfaction with governments have sharply declined in many countries. Government credibility has significantly dropped compared to pre-pandemic levels, leading to a noticeable crisis of confidence in social and interpersonal interactions. Previous social patterns have changed dramatically, resulting in a series of pressing social issues that urgently require governmental intervention (Tworzydlo et al., 2020).
5 Conclusion
This study applied a quantitative analysis framework to examine the extent to which scientific insights were integrated into policy during the COVID-19 pandemic. By conducting large-scale text analysis of policy documents and academic publications, the research identified four key scientific areas that received relatively less attention in policymaking: (1) insufficient attention and guidance to public psychological well-being; (2) inadequate attention to environmental issues; (3) underutilization of computer technology; (4) inattention to public relations. These findings do not suggest a deliberate neglect of scientific knowledge but rather underscore the complexity of policy decision-making, where factors such as time constraints, political considerations, and administrative feasibility influence the integration of scientific evidence. Recognizing these gaps offers a deeper understanding of the science-policy interface and underscores the need for systematic approaches to bridge this divide. The framework proposed in this study provides a structured method for identifying scientific insights that may be overlooked in policy, laying the groundwork for future research aimed at strengthening science-informed policymaking.
While this study offers valuable insights, it has certain limitations. First, the keyword frequency analysis used to infer policy priorities may not fully capture the significance of emerging fields, as low frequency could result from evolving terminology rather than a lack of importance. Second, the focus on a specific case limits the generalizability of the findings across diverse policy contexts. Future research could enhance this framework by integrating qualitative methods and applying it to a broader range of policy areas and public health emergencies.
Acknowledgements
Special thanks to the Overton for the data support in this paper.
Funding information
This paper is financially supported by the Ningbo University of Technology New Faculty Research Fund and the 2023 Interdisciplinary Innovation Research Cultivation Program of School of Interdisciplinary Studies, RUC, and Key Project of the National Social Science Foundation of China (21ATQ008).
Author contributions
Chao Ren (renchao@nbut.edu.cn, ORCID: 0009-0002-3989-9407): Formal analysis (Lead), Writing - original draft (Lead).
Menghui Yang (yangmenghui@ruc.edu.cn, ORCID: 0000-0001-6162-3373): Conceptualization (Lead), Methodology (Lead).
Data availability statement
The authors declare that the data in this paper will be made available upon reasonable request.