Research Papers

What factors influence research impact? An empirical study on the interplay of research, publications, researchers, institutions, and national conditions

  • Mudassar Hassan Arsalan , ,
  • Omar Mubin 1 ,
  • Abdullah Al Mahmud 2 ,
  • Sajida Perveen 3
Expand
  • 1School of Computer, Data and Mathematical Sciences, Western Sydney University, New South Wales 2116, Australia
  • 2Centre for Design Innovation, Swinburne University of Technology, Victoria 3122, Australia
  • 3School of Civil Engineering and Built Environment, Queensland University of Technology, Queensland 4001, Australia
†Mudassar Hassan Arsalan (Email: ; ).

Received date: 2024-06-29

  Revised date: 2024-09-30

  Accepted date: 2024-11-06

  Online published: 2024-12-09

Abstract

Purpose: This study investigates key factors contributing to research impact and their interactions with the Research Impact Quintuple Helix Model by Arsalan et al. (2024).

Design/methodology/approach: Using data from a global survey of 630 scientists across diverse disciplines, genders, regions, and experience levels, Structural Equation Modelling (SEM) was employed to assess the influence of 29 factors related to researcher characteristics, research attributes, publication strategies, institutional support, and national roles.

Findings: The study validated the Quintuple Helix Model, uncovering complex interdependencies. Institutional support significantly affects research impact by covering leadership, resources, recognition, and funding. Researcher attributes, including academic experience and domain knowledge, also play a crucial role. National socioeconomic conditions indirectly influence research impact by supporting institutions, underscoring the importance of conducive national frameworks.

Research limitations: While the study offers valuable insights, it has limitations. Although statistically sufficient, the response rate was below 10%, suggesting that the findings may not fully represent the entire global research community. The reliance on self-reported data may also introduce bias, as perceptions of impact can be subjective.

Practical implications: The findings have a significant impact on researchers aiming to enhance their work’s societal, economic, and cultural significance, institutions seeking supportive environments, and policymakers interested in creating favourable national conditions for impactful research. The study advocates for a strategic alignment among national policies, institutional practices, and individual researcher efforts to maximise research impact and effectively address global challenges.

Originality/value: By empirically validating the Research Impact Quintuple Helix Model, this study offers a holistic framework for understanding the synergy of factors that drive impactful research.

Cite this article

Mudassar Hassan Arsalan , Omar Mubin , Abdullah Al Mahmud , Sajida Perveen . What factors influence research impact? An empirical study on the interplay of research, publications, researchers, institutions, and national conditions[J]. Journal of Data and Information Science, 2025 , 10(1) : 188 -227 . DOI: 10.2478/jdis-2025-0001

1 Introduction

In today’s rapidly evolving global landscape, the significance of academic research extends far beyond the confines of scholarly journals and academic discourse. The concept of ‘research impact’ has undergone a profound transformation, shifting from traditional metrics of productivity and quality to a more holistic evaluation that encompasses the societal, economic, and cultural contributions of scholarly work (Robertson, 2016). This expanded perspective recognises that the actual value of research lies not only in advancing knowledge within academia but also in its capacity to drive innovation, inform policy, and enhance societal well-being (Gasparyan et al., 2018).
As nations increasingly acknowledge the critical role of universities and research organisations in promoting economic growth and addressing complex societal challenges, the pursuit of research impact has gained unprecedented importance. The emphasis is no longer solely on the quantity of publications or their perceived quality but on tangible outcomes that resonate beyond academic circles. For instance, the Australian Research Council’s Excellence in Research for Australia initiative rigorously evaluates the quality and impact of research conducted within the nation, aiming to enhance the contribution of Australian universities systematically (ARC, 2022). Similarly, in the UK, the Research Excellence Framework explicitly includes ‘impact’ as a criterion, evaluating research on its broader contributions to society and the economy (REF, 2021). In the United States, the National Science Foundation and other agencies have increasingly prioritised the societal impact in their funding decisions, recognizing that research should address societal needs and contribute to the public good (Roberts, 2009). Recently, China has also emphasised the importance of research impact, with its latest policies focusing on the translation of scientific research into societal and economic advancements, as seen in the growing number of patents and technology transfers (Jiang et al., 2022).
While funding, collaboration, and incentives are commonly recognised as crucial elements shaping research outcomes, their interplay and relative influence remain subjects of ongoing inquiry. Funding, for instance, has been identified as a significant driver of research productivity and impact. Coccia and Roshani (2024a) demonstrated that funded research tends to receive more citations than unfunded work, exhibiting super-linear growth in citation rates across basic and applied sciences. Similarly, Pao (1991) found that financial support enhances the quantity and quality of scientific publications, emphasising its role in boosting research visibility. Yet, studies on Nobel Laureates suggest that the impact of funding varies by discipline, with applied sciences benefiting more significantly compared to fields like physics, where intrinsic motivations often prevail (Coccia & Roshani, 2024b).
International collaboration has emerged as another pivotal factor in enhancing research impact. Adams (2013) highlights that international co-authorship can significantly boost citation impact, suggesting that global partnerships are crucial to producing high-quality research (Adams, 2013). This trend is further supported by (Coccia & Wang, 2016), who noted that increasing international collaborations are breaking down traditional barriers between basic and applied sciences, fostering innovation. Additionally, intrinsic and extrinsic incentives play a significant role in influencing research motivation and performance. Coccia (2019) argues that intrinsic incentives such as personal satisfaction, autonomy, and recognition are critical in enhancing research commitment and productivity, often surpassing the impact of financial rewards (Coccia, 2019). However, an overreliance on external rewards can undermine intrinsic motivation, a phenomenon known as the crowding-out effect, potentially reducing research outcomes’ quality and innovation potential.
Despite widespread recognition of the importance of research impact, not all scholarly work achieves substantial influence outside of academia (Dong et al., 2017; Fortunato et al., 2018; Wang & Barabási, 2021). Factors such as funding, collaboration, and incentives are commonly cited as crucial elements shaping research outcomes (Adams, 2013; Coccia & Roshani, 2024a). However, understanding these factors’ interplay and relative influence within the research ecosystem remains an ongoing challenge. To address this complexity, Arsalan et al. (2024) proposed the Research Impact Quintuple Helix Model (RIQHM), integrating five core elements—Research, Publications, Researchers, Institutions, and Countries—to provide a comprehensive framework for understanding research impact. While the RIQHM offers valuable theoretical insights, empirical validation is needed to assess its applicability and to know how these elements interact to influence research impact.
This study aims to fill the existing gap by empirically validating the Research Impact Quintuple Helix Model using Structural Equation Modelling (SEM) on data collected from a global survey of scientists and researchers. By analysing the complex interactions among the key elements influencing research impact, the research seeks to provide new insights into the synergistic relationships that drive impactful research.
The specific research questions guiding this study are:
● What are the significant factors that contribute to research impact?
● Are there differences in the perceived significance of these factors based on attributes such as gender, disciplinary field, and regional contexts?
● How do these factors interact within the Research Impact Quintuple Helix Model to influence the perceived impact of research?
By addressing these questions, the study contributes to a deeper understanding of the multifaceted nature of research impact. It evaluates the significance of several factors through empirical analysis, explores differences in perceptions based on demographic and professional attributes, and explains the influential role of each identified factor within the RIQHM framework.
Subsequent sections of this paper review the literature, detailing the theoretical underpinnings of the factors influencing research impact. This is followed by describing the methodology used to investigate the research hypotheses. The results section discusses the study’s findings, highlighting the significant factors and their interactions within the model. Finally, the implications of these findings are considered, providing insights for researchers, institutions, and policymakers on fostering impactful research.

2 Review of literature

2.1 Research impact contributing factors

Research impact is shaped by several critical factors that amplify its reach and significance across academic, societal, and policy-making domains. The quality of research is fundamental, as methodological rigour and innovation are essential for gaining recognition and citations (Coccia & Roshani, 2024a). Equally important are the relevance and timeliness of research topics, which capture broader interest by addressing urgent societal needs and emerging trends (Adams, 2013). Furthermore, collaborative efforts across disciplines enrich the research process, pooling diverse perspectives and expanding the applicability of findings (MacLean et al., 1998). Strategic decisions about publication channels, such as choosing high-impact journals or open access platforms, are crucial for enhancing visibility and facilitating wider dissemination (Coccia & Roshani, 2024b).
Influential impact factors are integral for strategic planning within research institutions and funding bodies, ensuring that investments are directed toward projects with substantial societal and economic returns. This strategic alignment is vital for supporting evidence-based policymaking and fostering cross-disciplinary collaborations, particularly in complex areas such as climate change (Stephan, 1996); (Larédo & Mustar, 2004). A deep understanding of these factors also improves stakeholder engagement, ensuring that research not only responds to but also drives user needs and integrates seamlessly into practical applications, crucial for achieving societal uptake and generating significant benefits (Mosleh et al., 2022).
By knowing research impact contributing factors, researchers and research institutions can maximise the impact of their work by developing tailored impact pathways and impactful research statements that specifically align with their project’s goals. This customised approach ensures that research activities not only meet academic standards but also address societal challenges, support ethical advancements, and promote sustainable practices (Coccia, 2022; Pao, 1991). Such strategic planning aids in reducing societal costs and improving outcomes, as seen in fields ranging from technological advancement to public health, underlining the profound benefits of well-aligned research endeavours (Stephan & Everhart, 1998).

2.2 Research Impact Quintuple Helix Model

Helix models have significantly evolved to incorporate a broader understanding of the research ecosystem, starting from the Triple Helix model introduced by (Etzkowitz & Leydesdorff, 2000), emphasising the dynamic interplay among universities, industry, and government. This model underlined their joint role in fostering innovation and a knowledge-based economy, highlighting the synergistic interactions crucial for enhancing innovation capacities and knowledge transfer. The model expanded through introducing the Quadruple Helix, which included the media and culture, recognising their substantial influence on the innovation ecosystem (Carayannis & Campbell, 2009). Each progressive model has broadened the conceptual scope, increasingly acknowledging the complex interdependencies among various stakeholders in driving beneficial innovations.
The Quintuple Helix Model significantly expands upon its predecessors by incorporating ecological sustainability as a central element, thus moving beyond the Triple Helix’s focus on university-industry-government interactions and the Quadruple Helix’s inclusion of media and culture. This evolution is marked by a shift towards integrating the natural environment into the innovation framework, highlighting the urgency of sustainable development. The Quintuple Helix Model advocates for a balanced synergy between ecological sustainability, societal development, and economic growth, addressing contemporary global challenges such as climate change more comprehensively (Carayannis & Campbell, 2010).
As proposed by Arsalan et al. (2024), the Quintuple Helix Model for Research Impact is defined by its comprehensive framework that emphasises the interactions among five principal components: research, publication, researchers, institutions, and countries (see Figure 1). This model integrates these elements within a multifaceted framework to explore how interconnected components and their dynamic interactions contribute to research impact. It aims to facilitate the effective dissemination and utilisation of knowledge, fostering a deeper integration of research outputs into broader societal, economic, and environmental contexts. In contrast to the standard Helix models, which primarily focus on fostering sectoral collaborations for economic development and innovation, the Research Impact Quintuple Helix Model provides a structured approach to assess and enhance the multifaceted impacts of research across societal, economic, and cultural dimensions. This model systematically organizes and analyses contributing factors, offering a comprehensive guide for researchers, policymakers, and institutions to maximise the benefits derived from research.
Figure 1. Interaction among research contribution factors.

Source: After Arsalan et al. (2024)

Research serves as the cornerstone of the Quintuple Helix Model, driving knowledge creation and innovation through its foundational role in the research ecosystem. Effective research is characterised by alignment with clear, strategic objectives, ensuring that studies address relevant questions and yield impactful results (Coombs & Meijer, 2021). The nature of research, whether fundamental or applied, significantly shapes its reach and applicability. Fundamental research, driven by the pursuit of basic understanding, expands the boundaries of scientific knowledge and influences broader cultural and policy landscapes (Calvert, 2006; Schauz, 2014). In contrast, applied research focuses on practical needs, often involving academia-industry collaborations that emphasise societal engagement and the translation of scientific discoveries into real-world applications (Bentley et al., 2015; Salvador et al., 2021). Both forms contribute to societal impact by fostering innovation and addressing complex challenges. Furthermore, the integration of monodisciplinary and interdisciplinary approaches enhances research impact; while monodisciplinary research provides deep, specialised insights within a single field, interdisciplinary research combines multiple perspectives to tackle complex issues holistically, fostering comprehensive solutions and innovative problem-solving (Bammer, 2013; Choi & Pak, 2006; Garcia Rodriguez et al., 2023). The adaptability of research methodologies across different contexts further strengthens its influence, making research a pivotal element in advancing knowledge and societal progress.
Publications are the primary avenue for disseminating research findings, directly shaping scholarly work’s visibility, reach, and overall impact. The choice of publication venues, including high-impact journals, books, patents, and policy briefs, significantly influences the audience and the perceived value of research outputs (Brown, 2017; Jaffe & De Rassenfosse, 2017; Karki, 1997). High-impact journals, while enhancing the visibility and prestige of researchers, often pose challenges such as limited accessibility and the pressure to publish in prestigious venues (Falagas et al., 2008; Garfield, 2006). Open-access platforms have revolutionised the field, enhancing visibility and accessibility and fostering broader community engagement and interdisciplinary collaboration (Bolick et al., 2017; Molloy, 2011). Digital media and preprints have further accelerated the dissemination process, enabling rapid sharing of findings and facilitating timely feedback and further research (Alfonso & Crea, 2023; Soderberg et al., 2020). Post-publication strategies, such as social media sharing and targeted research promotion, play a critical role in amplifying the reach and societal impact of scholarly work, demonstrating the need for effective communication to extend the influence of academic endeavours beyond traditional boundaries (Boyd et al., 2022; Rogers, 2019).
Researchers are pivotal to the research ecosystem, with their expertise, background, and collaborative networks influencing the direction, quality, and impact of research. Demographic factors such as age, gender, and geographical location shape researchers’ perspectives and approaches, affecting how research is developed and applied (Jung et al., 2017; Toledo-Pereyra, 2012). Academic experience, including formal education and hands-on research experience, significantly enhances researchers’ capabilities, while mentorship and the development of research self-efficacy are critical in boosting productivity and enabling researchers to tackle complex challenges (Lopatto, 2007; Wayment & Dickson, 2008). Researchers affiliated with prestigious institutions and active in scientific associations benefit from expanded professional networks and increased access to resources, which further elevate the quality and impact of their research (Stock et al., 2023; Zhang et al., 2022). Additionally, demographic factors such as age and experience can influence research output, with more seasoned researchers often achieving higher productivity, while gender disparities and cultural biases can affect publication success and visibility (Kyvik & Aksnes, 2015; Wahid et al., 2022)By fostering cross-disciplinary collaborations, researchers can bridge knowledge gaps and drive innovations contributing to scientific advancement and societal progress.
Institutions play a crucial role in supporting research by providing essential infrastructure, resources, and administrative assistance that enable high-quality research. This support includes access to laboratories, equipment, and funding, alongside fostering an environment of academic rigour and integrity through established research standards and practices (Kassim & Mwantimwa, 2022; Maharjan et al., 2022). Effective leadership within institutions is vital in aligning research efforts with institutional missions and societal needs, promoting innovation, and ensuring the maintenance of rigorous research standards (Askeland, 2020; Middlehurst et al., 2009). Additionally, institutions enhance research productivity and quality through organized training programs and interdisciplinary collaborations that develop researchers’ skills and foster professional growth (Ju, 2010; Kinney, 2007). Recognition and reward systems within institutions motivate researchers, aligning academic incentives with societal impact goals to ensure research contributes meaningfully to broader societal and environmental objectives. Ethical practices, fair authorship attribution, and transparent research processes further uphold the credibility and trustworthiness of research outputs (Grant, 2021; Wager, 2019).
Countries play a pivotal role in shaping research impact through national policies, economic support, and international collaborations. Government funding strategies, fiscal policies, and economic conditions directly influence the resources allocated to research and development, thereby affecting the scope, quality, and output of scientific work (Allareddy et al., 2015; Celeste et al., 2020). High-income countries often lead in global innovation due to their robust economies ancd extensive resources, yet they also face challenges in ensuring sustainability and maximizing social impact (Acharya & Pathak, 2019). National policies that support education, particularly in STEM fields, and encourage public-private partnerships are critical for developing a skilled research community and fostering international cooperation, which enhances the global reach and applicability of research (Martin, 2016; Trajtenberg, 2001). The political environment also significantly impacts research dynamics, with democratic nations generally providing more open and supportive settings for diverse research agendas and international collaborations compared to more restrictive political structures (Kim, 2011; Tavits, 2004). These factors collectively determine how effectively a country can respond to global and local challenges through research, underscoring the importance of strategic national policies in enhancing research impact.
Structural Equation Modelling (SEM) has emerged as a powerful analytical tool in the study of Helix Models, facilitating the examination of complex relationships between multiple variables and constructs within innovation ecosystems. The Helix Models, particularly the Triple Helix (university, industry, and government) and its extended forms, the Quadruple and Quintuple Helix models (adding society and the environment, respectively), benefit significantly from SEM due to their ability to manage intricate interdependencies and latent variables.
For instance, the study by Hamid et al. (2019) used SEM to evaluate the Triple Helix model’s impact on SMEs’ creativity and capabilities in Palopo City, Indonesia. They found that the interaction among intellectuals, government, and businesses significantly enhanced SMEs’ innovative environment. SEM enabled the researchers to rigorously evaluate the hypothesised relationships and confirm the model’s efficacy in predicting innovation outcomes within SMEs.
Similarly, the research conducted by Mineiro et al. (2023) applied SEM to analyse the practices and relationships of companies within Science and Technology Parks (STPs), focusing on the Quadruple and Quintuple Helix models. The study highlighted the positive relationship between integrating societal and environmental factors (the Quadruple and Quintuple Helix) and the future vision of STPs. SEM was instrumental in validating the hypothesis that collective actions and stakeholder engagements are crucial for fostering a sustainable and innovative environment within STPs.
These examples illustrate SEM’s versatility and robustness, highlighting its effectiveness in dissecting the dynamic interactions within Helix Models and providing valuable insights into the mechanisms driving innovation and collaboration across different sectors. Similarly, Structural Equation Modelling has been used to examine the Research Impact Helix Model, enabling a comprehensive analysis of the intricate relationships among research, researchers, publications, institutions, and national factors. This approach allows the theoretical model to be empirically validated, and critical determinants contributing to research impact are identified.

3 Methodology

Before describing the detailed methodology, it is imperative to clarify the conceptualisation of research impact within this study. Beyond traditional academic metrics such as citation counts and journal impact factors, research impact is envisioned through a wider influence lens. The Research Impact Quintuple Helix Model is employed, which incorporates a comprehensive framework involving research characteristics, researcher attributes, institutional support, publication strategies, and national socioeconomic conditions. This model characterises impactful research through academic contributions and delineates how such research engages with societal challenges, influences policy, and drives innovation. Notably, the term “impactful publication” is specifically used to denote outputs that significantly alter or contribute to these broader domains, reflecting the complex interplay of contextual factors as perceived by researchers. This term is thus defined as the output of research results in high-impact journals, books, or policy briefs and includes the entire process of publicising and disseminating these outcomes. This process involves active communication, sharing, and engagement with end users, including policymakers, practitioners, and the broader community, ensuring that the research effectively transfers knowledge and achieves tangible societal benefits. These comprehensive interactions are assessed through researchers’ perceptions, which help identify the pathways by which research activities lead to publications that significantly impact various stakeholders. Adopting this expansive view provides a deeper understanding of the multifaceted nature of research impact, guiding this study’s data collection and analysis phases.

3.1 Data collection

The study evaluated the significance of factors influencing research impact using a global survey targeting scientists and researchers using a questionnaire. The online questionnaire comprised two sections in English on the Qualtrics platform (see Appendix 1). The first section collected background information about participants through a series of questions. It began by categorising participants’ primary research areas into broad scientific domains, including basic sciences, social sciences, health sciences, applied sciences, engineering, and technology. Additional questions gathered details on participants’ gender, continent of residence, research experience, number of publications, and total citations to provide a comprehensive assessment of their research involvement.
The second section of the survey used a 1-5 Likert scale (1 - Not significant to 5 - Extremely significant) to assess participants’ perceptions of factors contributing to research impact. The 29 factors are selected based on the proposed Research Impact Quintuple Helix Model by Arsalan (2024) and are organised into the following categories:
Researcher Characteristics (8 items): This category captures elements related to individual researchers, including:
● Demographic characteristics (age, gender, region, and cultural background) shape perspectives and research impact.
● Academic experience, emphasising the influence of teaching and scholarly work on expertise and research quality.
● Domain knowledge refers to the depth of understanding in specific fields, enhancing the capacity to tackle complex issues.
● Research system understanding covers familiarity with methodologies, funding, and the research environment.
● Personality traits affect work style, collaboration, and overall research productivity.
● Affiliation and memberships, highlighting the value of professional connections and networks in supporting research.
● Involvement in multidisciplinary research, promoting innovation through diverse perspectives.
● Teamwork, collaboration, networking, and joint publications, enhancing research quality through collective efforts.
Research Attributes (5 items): Factors in this category focus on the nature and approach of the research itself:
● Monodisciplinary, multidisciplinary, and interdisciplinary research addresses research conducted within, across, or integrating multiple disciplines.
● Niche vs. popular research areas, exploring the impact of focusing on specialised versus widely studied topics.
● Quantitative and qualitative research, contrasting numerical data-driven approaches with in-depth, thematic analysis.
● Research target audiences, identifying and tailoring research to scholars, users, and policymakers to maximise relevance and application.
● Basic and applied research, distinguishing between fundamental scientific exploration and practical problem-solving efforts.
Publication Attributes (5 items): This category addresses the dissemination of research findings:
● Publication venues such as reports, journal articles, books, and conference papers serve distinct purposes and audiences.
● Ranking and impact factor of publication venues, reflecting the prestige and citation influence of where research is published.
● Open-access vs. subscription-based publication, examining the accessibility and reach of published work.
● Pre-publication presentations and discussions, such as conferences and preprints, which facilitate feedback and early dissemination.
● Post-publication research promotion, leveraging social media and showcases to enhance visibility and engagement.
Institutional Support (6 items): Factors here highlight the role of institutions in fostering research:
● Leadership, high values, and strict policies, emphasising the importance of ethical standards and support structures.
● Support facilities, resources, and libraries, ensuring access to necessary tools and environments for research.
● Appropriate time allocation support, balancing research with other responsibilities to optimise productivity.
● Recognition and rewards, incentivising research contributions through awards and promotions.
● Organized research training, workshops, and mentorship, enhancing skills and professional growth.
● Research funding, providing financial backing essential for conducting research.
National Role (5 items): This category considers broader national factors that impact research:
● Budgets, fiscal policies, and public-private financial resources dictate the level of investment in research and development.
● Demography and social structure, exploring how population characteristics influence research focus and outcomes.
● Economy, resources, industrialisation, and urbanisation, shaping the infrastructure and opportunities available for research.
● Government policies and interventions, including regulations and support programs that promote innovation.
● Political structure, where governance practices affect the stability and priorities of the research environment.
These categories provide a comprehensive framework for understanding the complex factors that influence research impact, aligning with the holistic perspective of the Research Impact Quintuple Helix Model. The study assesses research impact subjectively, focusing on contextual factors and participants’ perceptions rather than relying on objective and data-driven metrics. This approach allows for broader insights into the various elements shaping research outcomes.
Participants were identified using Scopus by searching for papers authored by researchers from various countries across Asia, North and South America, Europe, Africa, and Oceania. Detailed author information, including email addresses, was extracted through Scopus’s author search functionality to compile the participant list. An information letter was prepared to describe the study’s objectives and participation criteria, highlighting the requirement for involvement in research activities and at least one publication. The letter included a marked paragraph for consent, allowing researchers to opt out of the survey anytime. Ethics approval, the estimated time to complete the questionnaire, and a link to the participant information sheet and the online survey questionnaire were also included in the letter. Participants were assured of anonymity, and no personally identifiable information was collected.
Initially, 3,000 researchers from randomly selected universities across various global regions were invited, ensuring equal regional distribution (500 participants from each continent). Due to a lower-than-anticipated response rate, the invitation was expanded to 6,500 researchers using the same random selection pattern, and reminders were sent once to encourage participation. Despite these efforts, the overall response rate remained below 10%, yielding 630 complete responses. This sample size is considered statistically sufficient for the study population, which exceeds 100,000, achieving ± 5% precision at a 95% confidence level (Israel, 1992). A review of over 200 recent global surveys indicated that the sample size is within the range observed in similar studies, where sizes varied from as few as 20 to over 91,000 participants, with a median of 438 (Appendix 2). Examples include studies with larger scales, such as Lam et al. (2023) with 26,553 participants and Hashemi et al. (2023) with 91,056 participants, as well as studies conducted effectively with smaller samples, such as Zay Ya et al. (2024) with 352 participants and Adhiyaman et al. (2023) with 139 participants. This comparative analysis supports the adequacy of the sample size for drawing reliable conclusions about the global research community, consistent with established research norms.

3.2 Statistical analysis

The statistical analyses were divided into three types to evaluate hypotheses using SPSS Statistics, SPSS AMOS, and R. The hypotheses outlined below propose specific relationships between research characteristics, researcher attributes, institutional support, and national socioeconomic conditions, examining their combined effects on research impact.
Hypothesis 1: Factors identified by Arsalan (2024)—specifically related to the research, the researcher, the institution, and the country—significantly influence the production of impactful research publications.
● Hypothesis 1a: Research characteristics significantly contribute to impactful research publications.
● Hypothesis 1b: Researcher Attributes significantly contribute to impactful research publications.
● Hypothesis 1c: Institutional support significantly contributes to impactful research publications.
● Hypothesis 1d: National Socioeconomic Conditions significantly contribute to impactful research publications.
Hypothesis 2: The perceived significance of research impact factors varies significantly based on researchers’ demographic and professional attributes.
● Hypothesis 2a: Researchers’ gender significantly influences their perception of the importance of research impact factors.
● Hypothesis 2b: Researchers’ region significantly influences their perception of the importance of research impact factors.
● Hypothesis 2c: Researchers’ disciplinary field significantly influences their perception of the importance of research impact factors.
● Hypothesis 2d: Researchers’ experience level significantly influences their perception of the importance of research impact factors.
Hypothesis 3: The relationships among research characteristics, researcher attributes, institutional factors, and national socioeconomic conditions significantly influence the overall impact of research publications.
● Hypothesis 3a: The four factors (research characteristics, researcher attributes, institutional characteristics, and national socioeconomic conditions) are interrelated, forming synergistic constructs that collectively enhance the impact of research publications.
● Hypothesis 3b: Favourable national socioeconomic conditions positively influence institutional characteristics, enhancing researcher attributes and research characteristics, leading to more impactful research publications.
Factor Significance Assessment: The data were summarised, and each factor was ranked based on expected probability. To evaluate the first hypothesis, a one-tailed binomial test was used to determine the proportion of highly and extremely significant responses (1-Success) against a baseline proportion of 40%. The 1-5 Likert scale responses were converted into 0 - Fail (Not significant to moderately significant) and 1 - Success (highly significant to extremely significant). The null hypothesis was that the probability of 1-Success is less than or equal to 0.4 (H0: π ≤.4), and the alternative hypothesis was (Ha: π >.4). The one-tailed binomial test was selected due to its appropriateness for hypotheses predicting the direction of an effect in proportion data, which in this case relates to the recognition of impactful factors by researchers. This test provides a straightforward method for assessing whether the observed proportions of high-significance ratings significantly exceed the established baseline, reflecting a broader acknowledgment of each factor’s impact.
Comparative Analysis of Perceived Significance: The second hypothesis was evaluated by comparing means across different subgroups (disciplinary fields, regions, genders, research experience, number of publications, and citations) using F-tests and Mean Differences (MD) with a 95% Confidence Interval (CI). The Tukey HSD post-hoc test was used for mean differences. Quantitative parameters (research experience, number of published articles, citations) were converted into quantiles for comparison. Independent-sample t-tests were conducted for gender comparisons. The Levene test of homogeneity of variances led to merging less-represented regions into “Other Region” for meaningful analysis. Effect sizes were calculated to determine the strength of differences, following Cohen (1988) criteria, excluding insignificant differences.
Structural Analysis of Factor Interactions: Structural Equation Modelling (SEM) was employed to evaluate the third hypothesis. SEM, a multivariate analysis method, is widely used in social sciences to test theoretical models (Byrne, 2010; Kline, 2015). Figure 2 illustrates the conceptual Quintuple Helix Model for Research Impact. The model integrates five principal elements: Research, Researcher, Publication, Institution, and Nation (also called Country). “Research Impact” is a central sphere encompassing all five elements, signifying its overarching influence. Each component interacts with the others within this sphere, highlighting the complex pathways through which national policies, institutional support, and researcher characteristics collectively contribute to impactful research outputs. The publication is emphasised as the critical interface that transforms research into impact, showing the pathway starting from research, moving through publication, and leading to research impact. Bidirectional arrows indicate the dynamic and reciprocal relationships among the elements, reinforcing the holistic and iterative nature of the Quintuple Helix Model for Research Impact.
Figure 2. The conceptual Quintuple Helix Model for research impact.
Following the procedure by Naji et al. (2022) and Byrne (2010), data was prepared with no missing values and multivariate outliers were checked using Mahalanobis distance. A confirmatory factor analysis (CFA)-based measurement model with five latent variables (Research, Researcher, Publication, Institution, Nation) was designed (see Appendix 3: Figure A3-1). Model fitness was iterated by removing non-significant indicators and adding elements based on modification indices Abaci (2022); Ghaffari et al. (2022). Cut-off criteria for goodness-of-fit indices were X2 p >.05, CFI >.90, TLI >.9, RMSEA < 0.05, and SRMR < 0.08 (Amagasa & Nakayama, 2022; Oh et al., 2022; Pan et al., 2023). Multivariate data normality of the revised model was verified with and without bootstrap. Validity and reliability were evaluated, and non-contributing indicators were removed. The final measurement model was used to construct the path model (Appendix 3: Figure A3-4 and Table A3-9). Hypothetical paths were evaluated (see Appendix 3: Figure A3-4 and Table A3-12), and non-significant links (p >.05) were trimmed. The optimised path models (structural equation model) are shown in Appendix 3: Figure A3-5. The rationale behind model trimming is to focus on the most statistically significant relationships, ensuring that the final model is manageable and empirically strong. This process aids in maintaining a balance between complexity and clarity, allowing for a more interpretable model that aligns with the conceptual framework of the Research Impact Quintuple Helix Model.

4 Results

A total of 6,500 participants from all over the world were invited. Approximately 2.3% (n=162) of the email addresses bounced back or requested exclusion from the survey. Many requests were not responded to, as only 12.3% (n=802) of participants opened the questionnaire. Around 2.6% (n=172) of participants left the survey before completion, and only 9.7% (n=630) completed it. Table 1 shows the distribution of respondents with graphical representations. The proportion of participant gender is 62.4% male and 37.6% female. Half of the participants belong to the field of Social Sciences (n=315). Most respondents are young, having less than ten years of research experience (Mode (Q2)=10), but ranging from starter (<1 year) to 50 years. Their publications and citations are highly positively skewed towards lower values, with mean publications of 43.7 (SD=72.5) and mean citations of 2393 (SD=4557). The correlations between experience and publications (r=.66), experience and citations (r=.55), and publications and citations (r=.77) indicate strong positive relationships among these variables. The last column of Table 2 shows the geographical distribution of the participants. Most participants are from Asia (n=176), followed by North America, Africa, and other regions.
Table 1. Distribution of participants.
PRA Male Female Total Regions
Count PRA % Gender % Total % Count PRA % Gender % Total %
ASET 86 61.4% 21.9% 13.7% 54 38.6% 22.8% 8.6% 140
BS 68 60.7% 17.3% 10.8% 44 39.3% 18.6% 7.0% 112
HS 39 61.9% 9.9% 6.2% 24 38.1% 10.1% 3.8% 63
SS 200 63.5% 50.9% 31.7% 115 36.5% 48.5% 18.3% 315
Total 393 62.4% 100.0% 62.4% 237 37.6% 100.0% 37.6% 630

PRA = Primary research area, AEST = Applied sciences, engineering and technology, BS = Basic sciences, HS = Health sciences, and SS = Social sciences.

Figure 3 illustrates the spatial distribution of survey responses, highlighting significant variations across different regions and countries. Notably, the USA stands out as the country with the highest level of participation, emphasising its leading contribution to the overall survey results. Asia, with the highest engagement, includes countries such as China, India, and Japan, demonstrating strong participation from this continent. North America, represented by the USA, Canada, and Mexico, showed moderate response rates, reflecting solid but comparatively lower engagement than Asia. With key contributors like South Africa, Egypt, and Nigeria, Africa had the fewest respondents among the primary regions, suggesting potential barriers to participation. In contrast, South America (including Brazil and Argentina), Europe (with countries like the United Kingdom, Germany, and France), and Oceania (represented by Australia and New Zealand) collectively had lower response rates, leading to their consolidation into an “Other Region” category.
Figure 3. Spatial distribution of respondents.
Figure 4 provides an overview of researcher survey results, depicting experience distribution, number of publications, total citations, and their inter-correlations. Graph A illustrates the varying levels of research experience among participants; Graph B shows the number of publications; Graph C indicates the number of citations; and Graph D presents the correlation between experience, publications, and citations. Mean and standard deviation values for each metric are provided for clarity.
Figure 4. Researcher survey results: (A) Experience, (B) Publications, (C) Citations, and (D) Correlations between these metrics.
Table 2 presents the summary statistics, proportions, and results from the exact binomial test for all 29 factors influencing research impact. None of the factors passed the Kolmogorov-Smirnov or Shapiro-Wilk tests for normality, indicating significant deviations from a normal distribution (p <.001). The mean values for the factors range from 1.95 to 4.09, with standard deviations between 0.82 and 1.25.
Table 2. Summary statistics, proportions and one sample proportion test for individual factors.
Variable# Descriptive Summary Response Proportions Rank One sample Exact binomial test (n=630, One tailed successful proportion: π>.4)
Mean SD Median IQR EP Not significant Slightly significant Moderately significant Highly significant Extremely significant Highly significant and above count (N) Clopper-Pearson CI
(95%)
p-Val Successful Probability () H0: π≤.4)
R1 1.85 0.82 2 1 36.98 38.57 41.59 16.35 3.33 0.16 29 22 .024 - 1 .999 .035 Accepted
R2 2.39 0.94 2 1 47.84 16.03 43.17 27.62 11.9 1.27 24 83 .110 - 1 .999 .132 Accepted
R3 1.95 0.89 2 1 38.92 34.92 42.54 16.19 5.71 0.63 28 40 .048 - 1 .999 .063 Accepted
R4 2.01 0.93 2 1.75 40.19 32.86 42.06 17.62 6.19 1.27 26 47 .058 - 1 .999 .075 Accepted
R5 2.40 1.00 2 1 48.06 17.30 42.38 26.03 11.27 3.02 23 90 .120 - 1 .999 .143 Accepted
P1 2.01 0.86 2 1 40.19 30.16 44.92 18.89 5.87 0.16 27 38 .045 - 1 .999 .060 Accepted
P2 3.19 0.94 3 1 63.78 3.02 21.43 34.92 34.92 5.71 14 256 .374 - 1 .387 .406 Accepted
P3 2.55 0.90 3 1 51.05 10.95 38.89 35.08 14.13 0.95 21 95 .128 - 1 .999 .151 Accepted
P4 3.23 0.90 3 1 64.54 2.70 17.62 39.84 33.97 5.87 13 251 .366 - 1 .547 .398 Accepted
P5 2.93 0.96 3 2 58.54 6.03 28.1 36.51 25.87 3.49 18 185 .264 - 1 .999 .294 Accepted
RR1 3.68 0.94 4 1 73.56 3.33 6.67 25.56 47.78 16.67 8 406 .612 - 1 .000** .644 Rejected
RR2 3.90 0.88 4 2 77.97 1.11 4.92 22.7 45.56 25.71 3 449 .682 - 1 .000** .713 Rejected
RR3 2.69 1.25 3 3 53.78 26.67 12.22 33.33 21.11 6.67 19 175 .248 - 1 .999 .278 Accepted
RR4 3.50 0.92 4 1 69.94 1.59 12.86 31.9 41.59 12.06 11 338 .503 - 1 .000** .537 Rejected
RR5 2.13 0.88 2 2 42.57 25.24 44.29 23.33 6.67 0.48 25 45 .055 - 1 .999 .071 Accepted
RR6 2.60 0.93 3 1 52.10 10.32 38.25 33.97 15.56 1.9 20 110 .150 - 1 .999 .175 Accepted
RR7 3.80 0.96 4 1 75.97 2.22 7.78 21.43 45.08 23.49 4 432 .654 - 1 .000** .686 Rejected
RR8 3.69 1.10 4 1 73.78 5.56 8.89 21.11 40 24.44 6 406 .612 - 1 .000** .644 Rejected
I1 4.02 0.98 4 1 80.44 3.33 3.33 16.67 41.11 35.56 2 483 .737 - 1 .000** .767 Rejected
I2 4.09 0.87 4 1 81.71 0.48 4.29 18.25 40.16 36.83 1 485 .741 - 1 .000** .770 Rejected
I3 3.78 0.89 4 1 75.68 0.48 8.41 24.6 45.24 21.27 5 419 .633 - 1 .000** .665 Rejected
I4 3.69 0.90 4 1 73.75 0.95 9.21 27.62 44.6 17.62 7 392 .589 - 1 .000** .622 Rejected
I5 3.52 1.20 4 1 70.44 7.78 11.11 26.67 30 24.44 10 343 .511 - 1 .000** .544 Rejected
I6 3.54 0.96 4 1 70.89 3.02 10.79 29.05 43.02 14.13 9 360 .538 - 1 .000** .571 Rejected
N1 3.03 0.92 3 2 60.67 3.49 26.67 36.67 29.37 3.81 17 209 .301 - 1 .999 .332 Accepted
N2 2.41 0.91 2 1 48.25 14.13 44.13 29.21 11.43 1.11 22 79 .104 - 1 .999 .125 Accepted
N3 3.42 0.92 3 1 68.48 1.59 14.13 35.56 37.78 10.95 12 307 .454 - 1 .000** .487 Rejected
N4 3.05 0.90 3 2 61.05 2.22 26.83 38.89 27.62 4.44 16 202 .290 - 1 .999 .321 Accepted
N5 3.06 1.08 3 2 61.11 7.78 24.44 30 30 7.78 15 238 .346 - 1 .881 .378 Accepted

SD = Standard deviation, IQR = Inter-quartile range, CI = 95% confidence interval EP = Expected probability (Total response frequency for each variable / Maximum number of possible responses for the variable)

Null Hypothesis: The proportion of highly significant is less than or equal to 0.4 (H0: <=.4)

Alternate Hypothesis: The proportion of highly significant and above is more than 0.4 (Ha: >.4)

** Significant p-Val < 0.001

# See Table 4

Among these factors, the mode values indicate that eight factors are perceived as slightly significant (mode = 2), ten factors as moderately significant (mode = 3), and eleven factors as highly significant (mode = 4).
Proportional distribution analysis shows that ten factors (R1, R2, R3, R4, R5, RR5, RR6, P1, P3, and N2) are slightly significant, as the combined proportions of “Not significant” and “Slightly significant” responses exceed 40%. The exact binomial test for proportions determined the high significance level, considering the sum of “Highly significant” and “Extremely significant” responses. Twelve out of twenty-nine factors passed this test, demonstrating a high significance level. These highly significant factors are related to institutional support and researcher attributes, with one national factor (N3) included. The significant factors are appropriate time allocation, leadership and policies, support facilities, organised research training, recognition and rewards, research funding, academic experience, affiliations and memberships, domain knowledge, understanding of the research system, teamwork and collaboration, and national economic conditions.
The exact binomial test results indicate where the null hypothesis (that the proportion of “Highly significant” responses is less than or equal to 0.4) is rejected or accepted. Twelve factors (including institutional factors like I1, I2, I3, I4, I5, I6, and researcher attributes like RR1, RR2, RR4, RR7, RR8, and the national factor N3) rejected the null hypothesis (p <.001), showing that these factors are considered highly significant by the respondents. Conversely, factors like R1, R2, R3, R4, R5, RR5, RR6, P1, P3, and N2 did not reject the null hypothesis, indicating they are perceived as less significant (p >.05) (see Table 4 for variable names and groups). Overall, the results presented in Table 2 highlight the critical role of institutional support and researcher attributes in achieving high research impact, as evidenced by their high mean values and significant p-values. The confidence intervals and ranks further emphasise the importance of these factors, providing a clear understanding of the elements that most significantly contribute to research impact.
Figure 5 shows the cross-tabulated differences in the perceived significance of research impact factors (FGs) groups and their mean scores concerning gender, disciplinary fields, regions, and research experience as independent groups (IGs). Research-related factors are perceived as the lowest contributors (M = 2.12, SD =.77). The maximum subgroup-mean for research-related factors is 2.34 (Asia), which is lower than the minimum mean of any other factor group. However, this group of factors shows the highest difference of opinion based on regional variation. Participants from Africa rate research-related factors lower (M = 1.96, SD = 0.7) compared to Asian participants (M = 2.34, SD =.76). There are noticeable variations in the perception of research-related factors among different genders, groups of citations, publications, and disciplinary fields, although research experience shows minimal impact on these perceptions (Max. M = 2.17 for experience group Q2 [4 - 10 years], Min. M = 2.09 for experience group Q3 [10 - 18 years]).
Figure 5. Comparison of mean scores for research impact factors across different groups.
In contrast, the institution’s role is consistently considered highly significant across all regions, genders, disciplinary fields, research experiences, publications, and citations (M = 3.45, SD =.72). The highest subgroup mean difference for institutional factors is only 0.07, indicating broad agreement on their importance. North American females recognised the significance of these factors the highest (M = 3.49).
Intrinsic characteristics of researchers are also considered highly significant (M = 3.09, SD =.67), but there are regional differences. African participants consider these factors moderately significant (M = 2.98), whereas Asian participants rate them higher (M = 3.19). There is a negative correlation between the perception of researcher characteristics and research experience (r = -.182, p <.001), indicating that less experienced researchers value these characteristics more. Similar trends exist in publication (r = -.113, p =.004) and citation groups (r = -.093, p <.020). Publication (M = 2.78, SD =.71) and national factors (M = 2.87, SD =.72) show moderate variability, with disciplinary fields and regional differences influencing their perceived importance.
After applying the Levene test of homogeneity of variance (p > 0.05), rejecting the null hypothesis (p < 0.05), and considering effect size (η² ≥ 0.01), 42 reportable results are observed (see Table 3 and Appendix 3: Table A3-1). For further investigation, a post hoc Tukey HSD test was performed (see Appendix 3: Table A3-2 to A3-7). The primary research area (PRA) impacts two primary factors (P3 and RR2) and one mean score (P-A). The effect sizes are small (Max. 0.02), indicating that participants in applied sciences and technologies rate the significance of presenting and discussing research work in conferences (P3) higher than in other fields. Similar patterns exist for RR2, with significant differences between AEST vs BS and SS vs BS.
Table 3. Factors having significant mean differences (MD) within groups.
Group Var Statistics ES η2 Group Var Statistics ES η2 Group Var Statistics ES Cohen’s -d
PRA # P3 F(3, 626)=3.733, p=.011* .018 Citations# R1 F(3, 626)=3.542, p=.014* .017 Gender## R1 t(628)=-3.921, p=.000** -.323
RR2 F(3, 626)=4.193, p=.006* .020 RR5 F(3, 626)=2.922, p=.033* .014 R2 t(628)=-3.106, p=.002* -.255
P-A F(3, 626)=3.225, p=.022* .015 RR6 F(3, 626)=2.904, p=.034* .014 R3 t(628)=-3.044, p=.002* -.250
Publications# R1 F(3, 626)=2.793, p=.040* .013 R-A F(3, 626)=2.718, p=.044* .013 R4 t(628)=-3.088, p=.002* -.254
P4 F(3, 626)=4.044, p=.007* .019 RR-A F(3, 626)=5.631, p=.001* .026 R5 t(628)=-3.194, p=.001* -.263
RR-A F(3, 626)=4.524, p=.004* .021 P-A F(3, 626)=2.713, p=.044* .013 P2 t(628)=-2.934, p=.003* -.241
P-A F(3, 626)=3.520, p=.015* .017 T-A F(3, 626)=2.899, p=.034* .014 P3 t(628)=-2.582, p=.010* -.212
T-A F(3, 626)=3.801, p=.010* .018 Region# R1 F(3, 626)=4.356, p=.005* .020 P4 t(628)=-2.591, p=.010* -.213
Experience# RR2 F(3, 626)=3.627, p=.013* .017 R3 F(3, 626)=9.081, p=.000** .042 P5 t(628)=-3.316, p=.001* -.273
RR4 F(3, 626)=2.866, p=.036* .014 R4 F(3, 626)=4.104, p=.007* .019 R-A t(628)=-3.881, p=.000** -.319
RR6 F(3, 626)=4.859, p=.002* .023 R5 F(3, 626)=7.539, p=.000** .035 P-A t(628)=-3.417, p=.001* -.281
RR-A F(3, 626)=8.415, p=.000** .039 P5 F(3, 626)=2.683, p=.046* .013 T-A t(628)=-2.861, p=.004* -.235
T-A F(3, 626)=3.495, p=.015* .016 R-A F(3, 626)=8.123, p=.000** .037
P-A F(3, 626)=2.990, p=.030* .014
T-A F(3, 626)=4.597, p=.003* .022

ANOVA Effect Size: η2 = 0.01 => Small (S); η2 = 0.06 => Medium (M); η2 = 0.14 => Large (L)

T-test Effect Size: Cohen’s d = 0.20 => Small (S); Cohen’s D = 0.50 => Medium (M); Cohen’s D = 0.80 => Large (L)

Level of significance: ** Null hypothesis rejected with p <.001; * Null hypothesis rejected with p <.005

Null hypothesis (): There is no significant mean difference between groups

See Table 1

# One-way ANOVA test results

## Independent-Sample t-test results

Experience, publications, and citations are significantly correlated dimensions of academic experience (Res Exp vs Pub - r =.668, p <.001; Res Exp vs Cit - r =.558, p <.001; Pub vs Cit - r =.777, p <.001). The impact of experience, publications, and citations is observed in the mean scores of researchers (RR-A), publication (P-A), and total (T-A) factors. Five items (RR2, RR4, RR6, RR-A, and T-A) are affected by experience with small to medium effect sizes (η² =.014 to.039). The maximum mean differences between Q1 and Q4 (p <.05) indicate that younger researchers or those with fewer publications and citations perceive higher significance of these factors than senior researchers.
Spatial variation in the perceived significance of research impact factors is prominent. The highest mean differences are between Asians and Africans (e.g. Asia vs Africa R5 MD = 0.46, p <.001). Asians rate R1, R3, R4, R5, P5, R-A, P-A, and T-A significantly higher than Africans and participants from other regions. Participants from North America, Africa, and Other Regions think similarly. The effect size (η²) ranges from.014 to.042. Gender broadly impacts nine factors and three mean scores (R1, R2, R3, R4, R5, P2, P3, P4, P5, R-A, P-A, and T-A). Females rate the significance of these factors higher than males, with mean differences ranging from 0.19 (P3) to 0.262 (R1) and small effect sizes (Cohen’s d ranging from.212 to.323). The negative sign indicates that females rate these factors higher than males (see Table 1 for variable names and groups).
Overall, the results highlight significant differences in the perceived importance of numerous factors influencing research impact across diverse groups. Research factors show the most regional variability, while institutional factors are universally crucial. The perceptions of researcher characteristics are influenced by experience, with less experienced researchers rating them higher. Publication and national factors exhibit moderate significance, with disciplinary fields and regional differences influencing their perceived importance.
Table 4 provides an overview of different factor groups (Researcher, Research, Publication, Institutional, and National) along with their respective variables, each assigned a specific variable name, and reliability analysis is measured using Cronbach’s α. Reliability analysis was conducted to measure the internal consistency of the factors, with acceptable limits set at greater than 0.7. The Researcher Factors group, which includes variables such as academic experience (RR1), demographic characteristics (RR3), domain knowledge (RR4), and teamwork (RR8), has a reliability score of 0.588, indicating moderate internal consistency and falling below the acceptable limit. The Research Factors group, covering aspects like monodisciplinary vs. multidisciplinary research (R1), qualitative vs. quantitative approaches (R3), and target audiences (R4), has a reliability score of 0.896, showing high internal consistency and exceeding the acceptable limit. The Publication Factors group, encompassing elements like social media sharing (P1), open-access (P2), and the ranking of Cronbach’s α measures internal consistency and reliability to form a consistent group. Acceptable limit: >.7 publication venues (P4), has a reliability score of 0.846, indicating strong internal consistency and meeting the acceptable criteria. The Institutional Factors group, including time allocation (I1), leadership (I2), support facilities (I3), and research funding (I6), has a reliability score of 0.621, suggesting moderate internal consistency and falling below the acceptable limit. The National Factors group, considering the annual budget for R&D (N1), economy (N3), and government policies (N4), has a reliability score of 0.711, showing acceptable internal consistency. While the Research and Publication factors demonstrate high reliability, the Researcher and Institutional factors require improvement to meet the acceptable limit for internal consistency.
Table 4. Factor categories, variables and reliability.
Factor Groups (FGs) Description Variable Reliability
Researcher Academic experience RR1 Cronbach’s α = 0.588
After excluding RR1, RR3 & RR8
Cronbach’s α = 0.813
Affiliation and memberships RR2
Demographic characteristics RR3
Domain knowledge RR4
Involvement in multidisciplinary research RR5
Personality traits RR6
Understanding of the research system RR7
Teamwork, collaboration, networking, and joint publications RR8
Research Monodisciplinary, multidisciplinary and interdisciplinary research R1 Cronbach’s α =.896
Niche vs popular research areas R2
Qualitative and quantitative research R3
Research target audiences such as academicians, users, policymakers R4
Basic and applied research R5
Publication Post-publication Research Promotion, Showcasing and Social Media Sharing P1 Cronbach’s α =.846
Open Access vs Subscription-based Publication P2
Pre-publication Presentations and Discussions of Research Outcomes in Conferences, Seminars, Webinars, and Preprints P3
Ranking and Impact Factor of Publication Venues P4
Publication Venues such as Reports, Journal Articles, Books/Chapters, Patents, Conference Papers, Online Blog P5
Institutional Appropriate time allocation I1 Cronbach’s α =0.621
After excluding I1 & I5
Cronbach’s α = 0.847
Leadership, high values and strict policies I2
Support, facilities, resources and libraries I3
Organised Research Training, Workshops and Mentorship I4
Recognition and rewards I5
Research funding I6
National Budgets, Fiscal Policies, and Public-Private Financial Resources for R&D N1 Cronbach’s α = 0.711
After excluding N5
Cronbach’s α = 0.828
Demography and social structure N2
Economy, Resources, Industrialisation, and Level of Urbanisation N3
Government Policies and Interventions N4
Political structure N5
Mean scores Average of research factors group R-A
Average of researcher factors group RR-A
Average of publication factors group P-A
Average of institutional factors group I-A
Average of national factors group N-A
Average of all factors T-A
The results of the structural equation model are contained in Appendix 3. Figure A3-1 presents the standardised factor loadings for the conceptualised latent variables and the covariances between constructs. These loadings, ranging from.013 (low) to.863 (high), are crucial for understanding the reliability of the measurements. Although the initial measurement model failed due to low explained variances for some indicators, it laid the groundwork for subsequent analysis. The X² value was high (1,868.7, p <.001), and the poor goodness-of-fit values—CFI (.808), TFI (.787), and RMSEA (.081)—played a significant role in guiding the model optimisation. The degree of freedom (367) resulted from the 29 indicators, and SRMR (.076) remained within acceptable limits (0.08). After removing nonsignificant lower factor loadings, the goodness-of-fit indices improved significantly: X²(df) = 63.1(55), p =.210, CFI =.997, TFI =.996, RMSEA =.015, and SRMR =.022 (see optimised model in Figure A3-2). The multivariate normality values for both the conceptual (multivariate kurtosis = -.746, critical ratio = -.221) and optimised (multivariate kurtosis = -1.267, critical ratio = -.805) models were within acceptable limits as per Kline (2015). Additionally, the squared multiple correlations in the optimised model indicate acceptable multicollinearity (min: R² (N3) =.446, max: R² (I6) =.766).
Cronbach’s Alpha and Composite Reliability were used to assess construct reliability. Each construct’s Cronbach Alpha exceeded.7, and composite reliability ranged from.742 to.827, indicating good reliability. Convergent validity was established as all constructs’ AVEs were above.50, as suggested by Naji et al. (2022). Discriminant validity was confirmed using the Fornell and Larcker Criterion and the HTMT Ratios, with all constructs meeting the criteria in Table A3-8.
The fully optimised measurement model, which passed all tests for goodness-of-fit, construct validity, and reliability, shows the effectiveness of each factor and their contribution. Hypothesis 3a was evaluated with this model. The hypothesis that all factors are equally significant for effective constructs failed broadly. Figure A3-3, Tables 4-3 and 4-4 show the effectiveness and ranking of constructs and factors calculated using the methodology by Naji et al. (2022). A Combined Effect Model was created by redirecting all constructs to one secondary-level construct (see Figure A3-3 and Table A3-10). The contribution of each primary latent variable (standardised factor loading) was proportioned from the total (sum of all constructs’ factor loadings). This procedure was repeated to calculate each factor’s weight in their respective constructs. The overall weight of each factor was calculated by multiplying the factor weight by the construct weight (see Table A3-11).
The hypothetical path model analysis in Figure A3-4, Tables A3-12 and A3-13 show five out of ten pathways are statistically significant. The Nation path is insignificant for researcher (N-RR: β = -.110, p =.094), publication (N-P: β = -.013, p =.841), and research (N-R: β = -.010, p =.880). However, it is significant for the institute (N-I: β =.599, p <.001) with an explanatory power (R²) of 35.9%. The institute is significant for the researcher (I-RR: β =.196, p =.003) with an explanatory power (R²) of 2.9%. The researcher is significant for research (RR-R: β =.146, p =.005) and publication (RR-P: β =.285, p <.001). Research significantly links to publication (R-P: β =.379, p <.001), with explanatory powers (R²) of 2.6% and 27.5% for research and publication, respectively.
Figure 6 provides a comprehensive view of the direct, indirect, total, and split effects of significant paths. The nation, with its one direct path to the institution (β =.594, p <.001) and three indirect connections to researchers, research, and publication, plays a significant role in the model. It links to the researcher via the institution (N-I x I-RR), to research through the institution and then the researcher (N-I x I-RR x RR-R). Reaching publication is a split path for the researcher:
Figure 6. Optimised Path Model (OPM).
one is directly to publication, and the other is through research. All these paths are significant from nation to publication (see Table A3-14 to A3-16). While not fully confirming the hypothesis, the optimised path model provides valuable insights. The theoretical concept is valid, but not all hypothetical links are significant. The nation influences the institution, which in turn affects the researcher. An institutionally supported researcher can produce impactful research; the likelihood of impactful publication increases if research aspects are added.
Structural Equation Modelling (SEM) was employed in the study to evaluate the hypothesised relationships articulated in Hypotheses 3, 3a, and 3b, focusing on the dynamic interactions among research characteristics, researcher attributes, institutional factors, and national socioeconomic conditions. Given the exploratory nature of this study—aiming to first-time test these relationships—model trimming was strategically used to enhance the clarity and parsimony of the model. Non-significant links (p > 0.05) were systematically removed, resulting in an optimised path model depicted in Appendix 3: Figure A3-5. This process of model trimming was governed by the need to focus on the most statistically and theoretically significant pathways, adhering to principles of parsimony and theoretical coherence that are especially pertinent given the absence of existing theoretical frameworks for these specific interactions.
The trimmed model thus facilitates a more precise interpretation of the essential factors influencing the impact of research publications while ensuring the model remains manageable and empirically robust. The decision to exclude non-significant paths was driven by the objective of constructing a model that reflects observed data patterns and resonates with the conceptual underpinnings of the Research Impact Quintuple Helix Model. This approach helps refine the theoretical constructs by shedding light on the most influential relationships, thereby providing more precise insights into the mechanisms that drive research impact.

5 Discussion

5.1 Significance of research impact contributing factors

Consistent with prior findings such as Uwizeye et al. (2021), Scharnhorst et al. (2012) and Coccia (2020), institutional support—encompassing resources like funding and infrastructure—is highlighted as critical to achieving significant research impact. Similarly, pivotal roles are also played by researcher characteristics such as expertise and collaborative skills, confirming their influence on both research productivity and impact. This dual significance underscores the essential foundation provided by these factors in fostering an environment conducive to high-impact research outcomes. The findings also reinforce the importance of national economic conditions, which align with studies that link economic prosperity with enhanced research performance (Allareddy et al., 2015; Celeste et al., 2020). The capacity of a nation to support research through adequate funding and policy incentives is crucial for sustaining research productivity and enhancing its global impact, showcasing the interdependence between national resources and academic achievements (Coccia, 2019; Coccia & Roshani, 2024b). The significant impact of publication venues on research visibility and credibility is corroborated by the findings, which align with previous studies emphasising the role of high-impact journals (Morales et al., 2021). This highlights the critical importance of where research is published, underscoring the interconnectedness of publication strategies with broader institutional and national frameworks that support research innovation and dissemination.
Demographic and regional contexts significantly modulate the perception of what constitutes impactful research. Notably, the study reveals distinct perspectives across different regions, with unique insights provided by Asian researchers that challenge traditional divides in global research opinions (Jung et al., 2017; Raffer & Singer, 2002). This diversity of viewpoints highlights the importance of considering cultural and regional differences when designing policies to enhance research impact.
In assessing the effectiveness of numerous factors, we initially examined the individual statistical significance of 29 factors using the exact binomial proportion test on cumulative responses. This method allowed us to rank each factor according to its relevance, as shown in Table 2. However, this approach did not consider the interactions between factors, offering a limited perspective of their influence in realistic scenarios. To address this, we employed Structural Equation Modelling (SEM), which provided insights into the complex interactions between factors by assigning them effective weights based on their significance within the model. The SEM analysis revealed that more than half of the factors were statistically nonsignificant individually, indicating that their interactions might modify their influence.
Appendix 3: Table A3-17 compares the individual significance of factors with their effective significance as determined by SEM. Only two variables—Research Funding (I6) and National Economy (N3)—maintained consistent rankings across both individual and SEM analyses (ranks 9 and 12, respectively). Five factors, including Domain Knowledge (RR4), Institutional Leadership (I2), and Teamwork and Collaboration (RR7), showed significant influence in both analyses. Conversely, nine factors, such as Government Policies (N5) and Open Access Publishing (P1), were nonsignificant in both assessments.
This comparative analysis led to an important observation: factors that appeared less significant individually could exhibit substantial influence when considered within an interactive framework. For example, although low-ranked individually, Domain Knowledge (RR4) and Institutional Leadership (I2) demonstrated significant practical relevance in the SEM analysis. This discrepancy highlights the critical need to consider factor interactions to fully understand their impact on research outcomes, underscoring the complex nature of research dynamics.
The study challenges conventional wisdom by demonstrating that research impact is not solely the result of isolated factors but emerges from the intricate interplay among various elements within the research ecosystem. It reveals that interactions among institutional support, researcher attributes, and national conditions are more critical to research impact than the individual significance of each factor alone. This insight advances the theoretical discourse by highlighting the necessity of adopting a holistic perspective when analysing research impact rather than focusing on isolated factors. The findings suggest that institutional support is a pivotal mediator between national conditions and individual researcher attributes, influencing research characteristics and publication outcomes. By showing that some factors previously deemed less significant gain prominence within an interactive framework, this study underscores the importance of considering the synergistic effects of multiple factors. This offers a more comprehensive view of what drives impactful research, prompting a re-evaluation of how research impact is conceptualised and measured and encouraging scholars to develop more sophisticated models that accommodate the complexity of real-world research environments.

5.2 Structural insights from the Quintuple Helix Model

Structural Equation Modelling (SEM) was pivotal in explaining the complex relationships proposed within the Research Impact Quintuple Helix Model. This investigation leveraged SEM to rigorously test and validate hypothesised interactions among the five core elements of the model—research, researchers, publications, institutions, and nations. This methodological approach was instrumental in dissecting the solid and reciprocal influences that these components exert on the overarching concept of research impact, supporting foundational works on Helix models (Arsalan et al., 2024; Carayannis & Campbell, 2009).
Initial SEM analyses provided a foundational understanding of the theoretical interdependencies the Quintuple Helix framework posited. Through these models, several key theoretical pathways were quantitatively confirmed, affirming the hypotheses put forth by previous studies, while others were identified as statistically unsupported. The optimisation of the measurement model, prompted by suboptimal goodness-of-fit indices (e.g. Chi-square, CFI, RMSEA), was a critical phase of the study. This refinement process involved strategically removing nonsignificant indicators, significantly enhancing the model’s alignment with the empirical data. Such optimisations improved the model’s explanatory power and predictive accuracy regarding the factors most significantly influencing research impact.
Furthermore, the refined models robustly confirmed the proposed framework’s construct validity and reliability. Satisfactory levels in Cronbach’s Alpha and Composite Reliability were achieved, and the stringent criteria for convergent and discriminant validity were met, which aligns well with the reliability standards outlined in previous studies such as Hajjar (2018). These findings affirm that the constructs used were valid and reliable representations of complex real-world phenomena, providing a foundation for subsequent analyses.
A particularly compelling aspect of the study was the detailed examination of direct and indirect effects facilitated by SEM. This analysis revealed critical insights into how national policies and institutional frameworks serve as conduits, amplifying or mitigating the impact of research through institutional practices and individual researcher actions, as previously noted by Furman and Stern (2011). For instance, it was demonstrated that the direct influence of national innovation policies on institutional behaviours underscores the importance of policy alignment with research and development goals, confirming the findings of Bartels et al. (2012).
The implications of these findings extend beyond academic discourse, offering substantive insights for policymakers, institutional leaders, and stakeholders within the research ecosystem. By pinpointing the most influential factors within the Quintuple Helix, resources and strategic initiatives can be more effectively targeted to enhance the elements that most significantly boost research impact. This strategic focus is crucial in an era where research and innovation are increasingly called upon to address complex global challenges.
The Quintuple Helix Model for Research Impact, which integrates research, publication, researcher, institution, and nation as interconnected elements driving impact, is validated by empirical data through Structural Equation Modelling (SEM). The robustness of this model is evidenced by publications being identified as the most influential factor, followed by researcher attributes, institutional support, and national conditions, aligning with previous research on the pivotal role of high-quality publications in transforming research into impact. Moreover, this model’s concept of impactful publication is not confined to traditional academic outputs. Still, it encompasses a broader process that includes the active dissemination and public engagement necessary to translate research findings into substantial societal, policy, and innovation impacts. This redefines impactful publications as a dynamic process of engagement and influence, extending beyond mere publication metrics to include interactions that facilitate the real-world application and societal integration of research.

5.3 Implications of the study

This study presents a significant advancement in understanding the multifaceted nature of research impact by empirically validating the Research Impact Quintuple Helix Model proposed by Arsalan et al. (2024). Unlike previous models that primarily focused on linear relationships or isolated factors influencing research impact, this study employs Structural Equation Modelling (SEM) to explore the complex, interrelated dynamics among research characteristics, researcher attributes, publication strategies, institutional support, and national socioeconomic conditions.
The novelty of this study lies in its pioneering exploration of how numerous factors interact within the Research Impact Quintuple Helix Model to influence the perceived impact of research. It uniquely integrates subjective perceptions with objective statistical analyses through a global survey spanning multiple disciplines, genders, regions, and experience levels. By addressing key research questions that identify significant impact factors and exploring demographic and regional differences in these perceptions, the study offers new insights into their complex interrelationships within the proposed model. This approach captures the diversity of the global research community and contributes valuable insights for shaping future research policies and practices. The findings enhance the understanding of the variations in perceptions regarding the significance of research impact factors, thus improving the effectiveness of research activities across various scientific and cultural contexts.
This study contributes significantly to the theoretical understanding of research impact by empirically validating the Research Impact Quintuple Helix Model through Structural Equation Modelling. By confirming the model’s efficacy in capturing the complex, multidimensional relationships among research characteristics, researcher attributes, publication strategies, institutional support, and national socioeconomic conditions, the study not only introduces a new theoretical framework to the literature but also provides robust empirical evidence that underscores the complexity of achieving research impact. This validation sets the groundwork for future research to explore these interactions further and to develop strategies that can effectively enhance research impact on a global scale.
From a practical standpoint, the study offers valuable insights for researchers, institutions, and policymakers aiming to enhance research impact. Confirming that institutional support significantly influences researcher attributes and research outcomes suggests that universities and research organisations should invest in creating supportive environments. This includes providing adequate funding, infrastructure, mentorship programs, and recognition systems that motivate researchers to pursue impactful work. For researchers, the findings highlight the importance of leveraging institutional resources and engaging in collaborative efforts to amplify the impact of their research. Since publication strategies and dissemination efforts play a critical role, researchers are encouraged to consider open-access platforms, engage in pre- and post-publication promotions, and target their work toward relevant audiences, including policymakers and industry practitioners. Policymakers can draw on the study’s insights regarding the influence of national socioeconomic conditions on institutional support. Governments can indirectly enhance research impact by formulating policies that bolster funding for research and development, foster international collaborations, and create favourable economic conditions. Recognising the cascading effect of national policies on institutions and, subsequently, on researchers underscores the strategic importance of government intervention in the research ecosystem.

5.4 Limitations and future studies

Despite its methodological rigour and comprehensive scope, this investigation encounters several limitations that may influence the interpretation and generalizability of its findings. Firstly, a less than 10% response rate raises concerns regarding the sample’s representativeness. Although statistically adequate and more than the acceptable mode value of recent global surveys, such a low response rate might introduce a response bias, potentially skewing the perceptions of research impact factors toward those who participated. Additionally, the geographical and disciplinary spread of the sample, while extensive, required the consolidation of less-represented regions into an “Other Region” category, potentially masking regional insights into research impact.
The study’s reliance on researchers’ perceptions, measured via a Likert scale, introduces an element of subjectivity that may not necessarily align with more objective measures of research impact, such as quantifiable societal benefits or policy changes. This subjective measurement framework could limit the ability to fully capture the effectiveness of research outputs in real-world applications. Moreover, the survey’s administration in English may exclude non-native speakers, leading to cultural and linguistic biases that could affect the accuracy and universality of the responses.
Additionally, while using SEM provides a detailed analysis of variable interrelations and enhances model parsimony, it has limitations. The complexity of the SEM approach, including its assumptions about linearity and normality, may not fully capture the non-linear and complex causal relationships within the research impact ecosystem. Furthermore, the model’s reliance on modification indices for improving fit could lead to an over-fitted model that performs well on sample data but may not generalise well to other datasets or contexts. This necessitates a cautious interpretation of the SEM results, ensuring that improvements in model fit are not achieved at the expense of the model’s theoretical integrity and generalizability. In achieving an optimal model fit, a significant number of items (16 out of 29) had to be removed from the Structural Equation Modelling (SEM) analysis, resulting in a trimmed model that includes only 13 key factors. This reduction affected all constructs, such as the “Researcher” variable, where only RR4, RR5, and RR7 were retained. Consequently, the interpretation of the model is based solely on these selected items, limiting the comprehensiveness of each construct. This adjustment underscores the need for further research to identify a broader set of appropriate measures across all constructs to ensure a more robust representation of the variables impacting research impact.
Lastly, the current study measures research impact through a perception-based approach, reflecting a holistic conceptualisation without strict evidence-based metrics. This is consistent with models like the Research Excellence Framework (REF), which shares a broad, conceptual understanding of research impact but differs by implementing evidence-based monitoring and measurement. While both approaches start with a similar unlimited conceptualisation, REF and other impact assessment models proceed to evidence-based evaluations through specific indicators and metrics. This study, however, emphasises contextual factors influencing research impact, relying on researchers’ perceptions rather than quantitative measurements.
This perception-based approach represents a limitation of the study, as it lacks the evidence-based assessment seen in models like REF. Future studies could address this by transitioning from perception to evidence-based evaluations, employing indicators, structured scoring or weighting schemes to quantify research impact. Additionally, future research should aim to incorporate more diverse participant groups, use longitudinal designs, provide multilingual survey options, and explore data-driven impact metrics. These advancements would enable a more profound, evidence-based understanding of research impact, enhancing the comprehensiveness and applicability of the findings across different contexts.

6 Conclusion

This study explored the multifaceted factors contributing to research impact by empirically validating the Research Impact Quintuple Helix Model proposed by Arsalan et al. (2024) using Structural Equation Modelling. By conducting a global survey of 630 researchers across diverse disciplines, regions, genders, and experience levels, the research comprehensively analyses how various elements within the research ecosystem interact to enhance the significance and reach of scholarly work.
The study substantiates the crucial roles of institutional support and researcher attributes in driving impactful research outcomes. Key institutional factors, including the quality of leadership, availability of resources, recognition systems, and research funding, are shown to be highly significant. These elements cultivate an environment conducive to innovation, collaboration, and productivity among researchers. Equally important are researcher attributes such as academic experience, domain knowledge, familiarity with the research system, and teamwork. These attributes significantly enhance a researcher’s ability to produce high-quality work that appeals to academic and broader audiences. Moreover, the findings indicate that national socioeconomic conditions indirectly affect research impact by influencing institutional support. Although national policies and economic resources do not significantly impact researchers or research characteristics in the structural model, their influence on institutional support is substantial. This highlights the critical importance of robust national frameworks that equip institutions with the necessary infrastructure and resources to foster impactful research. Additionally, the study emphasises the significance of publication strategies, particularly the choice of publication venues and dissemination efforts, as analysed within an interactive framework using Structural Equation Modelling. While some publication factors may not seem highly significant individually, their importance is magnified when considering their interactions with other factors. This underscores the essential role of effective communication and dissemination in amplifying research impact.
The study also uncovers variations in the perceived significance of research impact factors based on demographic and professional attributes. For instance, female researchers and those from Asia rated certain factors higher than their male and African counterparts. Less experienced researchers tended to place more importance on researcher characteristics, suggesting that perceptions of what contributes to research impact evolve with experience. The study highlights the importance of considering demographic and regional differences when developing strategies to enhance research impact. Tailoring approaches to accommodate these differences can lead to more effective outcomes.
By validating the Research Impact Quintuple Helix Model, the study provides a comprehensive understanding of the complex interplay among several factors influencing research impact. The empirical evidence suggests that research impact is not solely the product of individual factors but arises from the synergistic interactions among national policies, institutional support, researcher attributes, research characteristics, and publication strategies. This holistic perspective challenges traditional models that consider these factors in isolation.
The findings have significant implications for researchers, institutions, and policymakers aiming to enhance the impact of scholarly work. Institutions are encouraged to invest in supportive leadership, provide adequate resources, and recognise and reward impactful research to motivate and enable researchers. Researchers should leverage institutional support and engage in collaborative and interdisciplinary efforts to amplify the reach and significance of their work. Policymakers should focus on creating favourable national conditions that empower institutions, recognising their impact on research, which is mediated through institutional support.
While the study offers valuable insights, it is not without limitations. Although statistically sufficient, the response rate was below 10%, suggesting that the findings may not fully represent the entire global research community. The reliance on self-reported data may also introduce bias, as perceptions of impact can be subjective. Future studies could incorporate objective measures of research impact and include qualitative methods to gain deeper insights. For future research, it would be beneficial to explore the longitudinal effects of these factors on research impact and to examine how emerging trends, such as digital transformation and open science practices, influence the dynamics within the Quintuple Helix Model. Additionally, expanding the sample size and including more participants from underrepresented regions could provide a more comprehensive understanding of global research impact factors.
This study contributes to the theoretical and practical understanding of research impact by demonstrating that it results from complex, interrelated factors within the research ecosystem. By empirically validating the Research Impact Quintuple Helix Model, the research provides a robust framework for stakeholders to develop strategies that enhance scholarly work’s societal, economic, and cultural significance. Recognising and fostering the synergistic relationships among national policies, institutional support, researcher attributes, research characteristics, and publication strategies is crucial for advancing impactful research that addresses global challenges.

Ethical statements

The study was approved by our institutional Research Ethics Committee (HREC Approval Number H13554).

Acknowledgements

We extend our deepest gratitude to the anonymous reviewers, whose insightful comments and suggestions were instrumental in refining this study’s presentation and substance. Their expertise has significantly enhanced the clarity and depth of our research.
We also wish to thank the members of the Western Sydney University Human Research Ethics Committee for their thorough review and valuable feedback on our study’s concept and methodology. Their guidance was crucial in ensuring our research processes’ ethical integrity and effectiveness.
Furthermore, we are immensely thankful to all the participants who generously contributed their time and perspectives to this research. Their engagement has been vital in shaping the study and enabling us to comprehensively explore the intricate factors impacting research effectiveness.

Author contributions

Mudassar Hassan Arsalan (Email: 19316071@student.westernsydney.edu.au; mharsalan@gmail.com; ORCID: 0000-0001-9622-5930): Conceptualization (Lead), Data curation (Equal), Formal analysis (Lead), Investigation (Lead), Methodology (Lead), Validation (Lead), Visualization (Lead), Writing - original draft (Lead).
Omar Mubin (Email: O.Mubin@westernsydney.edu.au; ORCID: 0000-0002-6435-6407): Conceptualization (Equal), Investigation (Supporting), Methodology (Supporting), Resources (Equal), Writing - review & editing (Equal).
Abdullah Al Mahmud (Email: aalmahmud@swin.edu.au; ORCID: 0000-0003-1059-8491): Conceptualization (Equal), Investigation (Supporting), Methodology (Supporting), Writing - review & editing (Supporting).
Sajida Perveen (Email: a.sajdaa@gmail.com; ORCID: 0000-0002-3887-3937): Data curation (Equal), Methodology (Supporting), Writing - review & editing (Equal).

Statements and declarations

The authors declare that they have no competing interests, including no conflicts of interest, no funding from any source, and no financial or non-financial interests that could be perceived as influencing the study or its outcomes.

Data availability statements

The data that support the findings of this study, including research data in Appendix 1-3, are openly available in [Science Data Bank] at [DOI:10.57760/sciencedb.16865; CSTR:31253.11.sciencedb.16865]. Please cite the source when using the data.

Appendix materials

-pdf file
The data can be accessed at: https://doi.org/10.57760/sciencedb.16865
[1]
Abaci N. I. (2022). Relationship between entrepreneurship perception and communication skill: A structural equation model. The International Journal of Management Education, 20(3), 100725.

[2]
Acharya K. P., & Pathak S. (2019). Applied research in low-income countries: Why and how? Frontiers in Research Metrics and Analytics, 4, 3.

DOI PMID

[3]
Adams J. (2013). The fourth age of research. Nature, 497(7451), 557-560.

[4]
Adhiyaman V., Chattopadhyay I., & Hobson P. (2023). Prescribing - a worldwide survey on governance. Medico-Legal Journal, 91(1), 50-53. https://doi.org/10.1177/00258172221145107

[5]
Alfonso F., & Crea F. (2023). Preprints: A game changer in scientific publications? US: Oxford University Press.

[6]
Allareddy V., Allareddy V., Rampa S., Nalliah R. P., & Elangovan S. (2015). Global dental research productivity and its association with human development, gross national income, and political stability. Journal of Evidence Based Dental Practice, 15(3), 90-96.

[7]
Amagasa T., & Nakayama T. (2022). Relationships Among Pay-for-Performance, Work Stressors, and Depression in Sales Workers: A Structural Equation Model Analysis. Journal of Occupational and Environmental Medicine, 64(12), 1018-1024.

[8]
ARC. (2022). Research Impact Principles and Framework. Australian Research Council. Retrieved October 2022 from https://www.arc.gov.au/about-arc/strategies/research-impact-principles-and-framework

[9]
Arsalan M., Mubin O., & Al Mahmud A. (2024). Dynamic interactions of research, publication, researchers, institutions, and countries: A Quintuple Helix model perspective on research impact. Malaysian Journal of Library & Information Science, 29(2), 19-59. https://doi.org/10.22452/mjlis.vol29no2.2

[10]
Askeland H. (2020). Institutional leadership: Maintaining and developing the ‘Good’organisation. Understanding values work: Institutional perspectives in organizations and leadership, 139-158.

[11]
Bammer G. (2013). Disciplining interdisciplinarity: Integration and implementation sciences for researching complex real-world problems. ANU Press.

[12]
Bartels F. L., Voss H., Lederer S., & Bachtrog C. (2012). Determinants of National Innovation Systems: Policy implications for developing countries. Innovation, 14(1), 2-18.

[13]
Bentley P. J., Gulbrandsen M., & Kyvik S. (2015). The relationship between basic and applied research in universities. Higher Education, 70, 689-709.

[14]
Bolick J., Emmett A., Greenberg M. L., Rosenblum B., & Peterson A. T. (2017). How open access is crucial to the future of science. The Journal of Wildlife Management, 81(4), 564-566.

[15]
Boyd C. J., Bekisz J. M., Salibian A. A., Karp N. S., & Choi M. (2022). Research promotion is associated with broader influence and higher impact of plastic surgery publications. Plastic and Reconstructive Surgery, 150(2), 466-472.

DOI PMID

[16]
Brown A. N. (2017). Why should practitioners publish their research in journals? Retrieved January, 10, 2020. https://researchforevidence.fhi360.org/practitioners-publish-research-journals

[17]
Byrne B. M. (2010). Structural equation modeling with AMOS: Basic concepts, applications, and programming (Vol. 396, pp. 7384). Taylor & Francis Group.

[18]
Calvert J. (2006). What’s special about basic research? Science, Technology, & Human Values, 31(2), 199-220.

[19]
Carayannis E. G., & Campbell D. F. J. (2009). “Mode 3” and “Quadruple Helix”: Toward a 21st century fractal innovation ecosystem. International Journal of Technology Management, 46(3/4), 201. https://doi.org/10.1504/IJTM.2009.023374

[20]
Carayannis E. G., & Campbell D. F. J. (2010). Triple Helix, Quadruple Helix and Quintuple Helix and how do knowledge, innovation and the environment relate to each other?: A proposed framework for a trans-disciplinary analysis of sustainable development and social ecology. International Journal of Social Ecology and Sustainable Development, 1(1), 41-69.

[21]
Celeste R. K., Muniz F. W. M. G., Ardenghi T. M., Collares F. M., & Rösing C. K. (2020). Brazilian dentistry research productivity: State level socioeconomic, educational and structural factors. Brazilian Journal of Oral Sciences. Piracicaba, 19, 1-12, e206977.

[22]
Choi B. C., & Pak A. W. (2006). Multidisciplinarity, interdisciplinarity and transdisciplinarity in health research, services, education and policy: 1. Definitions, objectives, and evidence of effectiveness. Clinical and Investigative Medicine, 29(6), 351.

[23]
Coccia M. (2019). Intrinsic and extrinsic incentives to support motivation and performance of public organizations. Journal of Economics Bibliography, 6(1), 20-29.

[24]
Coccia M. (2020). How does science advance? Theories of the evolution of science. Journal of Economic and Social Thought, 7(3), 153-180.

[25]
Coccia M. (2022). Probability of discoveries between research fields to explain scientific and technological change. Technology in Society, 68, 101874.

[26]
Coccia M., & Roshani S. (2024a). General laws of funding for scientific citations: How citations change in funded and unfunded research between basic and applied sciences. Journal of Data and Information Science, 9(4), 71-89.

[27]
Coccia M., & Roshani S. (2024b). Research funding and citations in papers of Nobel Laureates in Physics, Chemistry and Medicine, 2019-2020. Journal of Data and Information Science, 9(2), 56-80.

[28]
Coccia M., & Wang L. (2016). Evolution and convergence of the patterns of international scientific collaboration. Proceedings of the National Academy of Sciences, 113(8), 2057-2061.

[29]
Cohen J. (1988). Statistical Power Analysis for the Behavioral Sciences. NY: Lawrence Erlbaum Associates.

[30]
Coombs S. K., & Meijer I. (2021). Towards evaluating the research impact made by universities of applied sciences. Science and Public Policy, 48(2), 226-234.

[31]
Dong Y., Ma H., Shen Z., & Wang K. (2017). A century of science: Globalization of scientific collaborations, citations, and innovations. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining (pp. 1437-1446).

[32]
Etzkowitz H., & Leydesdorff L. (2000). The dynamics of innovation: From National Systems and “Mode 2” to a Triple Helix of university-industry-government relations. Research policy, 29(2), 109-123.

[33]
Falagas M. E., Kouranos V. D., Arencibia-Jorge R., & Karageorgopoulos D. E. (2008). Comparison of SCImago journal rank indicator with journal impact factor. The FASEB Journal, 22(8), 2623-2628.

[34]
Fortunato S., Bergstrom C. T., Börner K., Evans J. A., Helbing D., Milojević S.,... & Barabási A. L. (2018). Science of science. Science, 359(6379), eaao0185.

[35]
Furman J. L., & Stern S. (2011). Climbing atop the shoulders of giants: The impact of institutions on cumulative research. American Economic Review, 101(5), 1933-1963.

[36]
Garcia Rodriguez A., Barrio R. A., Govezensky T., Parga T., Pérez M. R., Calvet H. C., Jiménez Andrade J. L., Arencibia-Jorge R., & Kaski K. (2023). Impact of institutional organization on research productivity and multidisciplinarity. Frontiers in Physics, 11, 1161019.

[37]
Garfield E. (2006). The history and meaning of the journal impact factor. JAMA, 295(1), 90-93.

DOI PMID

[38]
Gasparyan A. Y., Yessirkepov M., Duisenova A., Trukhachev V. I., Kostyukova E. I., & Kitas G. D. (2018). Researcher and author impact metrics: variety, value, and context. Journal of Korean Medical Science, 33(18).

[39]
Ghaffari S. F., Elyasi F., Nikbakht R., & Shahhosseini Z. (2022). A structural equation model analysis of the relationship between maternal fear of childbirth and expectant fathers’ fear of childbirth: The mediating role of fathers’ depression, anxiety, and stress. Brain and Behavior, 12(12), e2802.

[40]
Grant J. (2021). Academic incentives and research impact: developing reward and recognition systems to better people’s lives. Paradigm Proj Acad Health, 22, 2021-2102.

[41]
Hajjar S. (2018). Statistical analysis: Internal-consistency reliability and construct validity. International Journal of Quantitative and Qualitative Research Methods, 6(1), 27-38.

[42]
Hamid R. S., Anwar S. M., & Lumoindong Y. (2019). Using the triple helix model to determine the creativity a capabilities of innovative environment. In IOP Conference Series: Earth and Environmental Science (Vol. 343, No. 1, p. 012144). IOP Publishing.

[43]
Hashemi F., Hoepner L., Hamidinejad F. S., Haluza D., Afrashteh S., Abbasi A.,... & Hoseini M. (2023). A comprehensive health effects assessment of the use of sanitizers and disinfectants during COVID-19 pandemic: A global survey. Environmental Science and Pollution Research, 30(28), 72368-72388. https://doi.org/10.1007/s11356-023-27197-6

[44]
Israel G. D. (1992). Determining sample size, Florida cooperative extension service. University of Florida PEOD-6, Florida.

[45]
Jaffe A. B., & De Rassenfosse G. (2017). Patent citation data in social science research: Overview and best practices. Journal of the Association for Information Science and Technology, 68(6), 1360-1374.

[46]
Jiang J., Zhao Y., & Feng J. (2022). University-Industry Technology Transfer: Empirical Findings from Chinese Industrial Firms. Sustainability, 14(15), 9582.

[47]
Ju M. (2010). The impact of institutional and peer support on faculty research productivity: A comparative analysis of research vs. non-research institutions. Seton Hall University.

[48]
Jung H., Seo I., Kim J., & Kim B.-K. (2017). Factors affecting government-funded research quality. Asian Journal of Technology Innovation, 25(3), 447-469.

[49]
Karki M. (1997). Patent citation analysis: A policy analysis tool. World Patent Information, 19(4), 269-272.

[50]
Kassim M., & Mwantimwa K. (2022). Impact of electronic resources utilization in research productivity in Tanzania Universities. Collection Management, 48(2), 127-149. https://doi.org/10.1080/01462679.2022.2114967

[51]
Kim J. (2011). Political institutions and public R&D expenditures in democratic countries. International Journal of Public Administration, 34(13), 843-857.

[52]
Kinney A. (2007). National scientific facilities and their science impact on nonbiomedical research. Proceedings of the National Academy of Sciences, 104(46), 17943-17947.

[53]
Kline R. B. (2015). Principles and practice of structural equation modeling. Guilford publications.

[54]
Kyvik S., & Aksnes D. W. (2015). Explaining the increase in publication productivity among academic staff: A generational perspective. Studies in Higher Education, 40(8), 1438-1453.

[55]
Lam C. S. P., Harding E., Bains M., Chin A., Kanumilli N., Petrie M. C., Pohja-Hutchison P., Yang J., & Butler J. (2023). Identification of urgent gaps in public and policymaker knowledge of heart failure: Results of a global survey. BMC Public Health, 23(1), Article 1023. https://doi.org/10.1186/s12889-023-15405-4

[56]
Larédo P., & Mustar P. (2004). Public sector research: A growing role in innovation systems. Minerva, 42(1), 11-27.

[57]
Lopatto D. (2007). Undergraduate research experiences support science career decisions and active learning. CBE—Life Sciences Education, 6(4), 297-306.

[58]
MacLean M., Davies C., Lewison G., & Anderson J. (1998). Evaluating the research activity and impact of funding agencies. Research Evaluation, 7(1), 7-16.

[59]
Maharjan M. P., Stoermer S., & Froese F. J. (2022). Research productivity of self-initiated expatriate academics: Influences of job demands, resources and cross-cultural adjustment. European Management Review, 19(2), 285-298. https://doi.org/10.1111/emre.12470

[60]
Martin B. R. (2016). R&D policy instruments-a critical review of what we do and don’t know. Industry and Innovation, 23(2), 157-176.

[61]
Middlehurst R., Goreham H., & Woodfield S. (2009). Why research leadership in higher education? Exploring contributions from the UK’s leadership foundation for higher education. Leadership, 5(3), 311-329.

[62]
Mineiro A. A. d. C., Arantes R. d. C., Vieira K. C., Castro C. C., Carvalho E. G., & Amaral M. G. d. (2023). Business practices for strengthening the quadruple and quintuple helix: A study using structural equation modeling. International Journal of Innovation Science, 15(1), 1-18.

[63]
Molloy J. C. (2011). The open knowledge foundation: Open data means better science. PLoS biology, 9(12), e1001195.

[64]
Morales E., McKiernan E. C., Niles M. T., Schimanski L., & Alperin J. P. (2021). How faculty define quality, prestige, and impact of academic journals. PLoS ONE, 16(10), e0257340.

[65]
Mosleh M., Roshani S., & Coccia M. (2022). Scientific laws of research funding to support citations and diffusion of knowledge in life science. Scientometrics, 127(4), 1931-1951.

DOI PMID

[66]
Naji K. K., Gunduz M., & Hamaidi M. F. (2022). Major factors affecting construction waste management in infrastructure projects using structural equation model. Journal of Construction Engineering and Management, 148(10), 04022101.

[67]
Oh W.-o., Yeom I., & Lim S.-H. (2022). Structural equation model based on salutogenesis theory for evaluating factors affecting health-related quality of life in adolescents with moyamoya disease. Scientific Reports, 12(1), 1-11.

[68]
Pan L., Zhang X., Wang S., Zhao N., Zhao R., Ding B., Li Y., Miao W., & Fan H. (2023). Determinants associated with self-management behavior among type 2 diabetes patients in China: A structural equation model based on the theory of planned behavior. International Journal of Clinical and Health Psychology, 23(1), 100332.

[69]
Pao M. L. (1991). On the relationship of funding and research publications. Scientometrics, 20(1), 257-281.

[70]
Raffer K., & Singer H. W. (2002). The economic North-South divide: Six decades of unequal development. Edward Elgar Publishing.

[71]
REF. (2021). REF 2021 Guidance on Submissions. Retrieved from https://2021.ref.ac.uk/media/1447/ref-2019_01-guidance-on-submissions.pdf

[72]
Roberts M. R. (2009). Realizing societal benefit from academic research: Analysis of the National Science Foundation’s broader impacts criterion. Social Epistemology, 23(3-4), 199-219.

[73]
Robertson J. (2016). Measuring research impact-does it matter?. Australian Journal of Forensic Sciences, 48(6), 613-614.

[74]
Rogers J. (2019). The use of social media and its impact for research. BioResources, 14(3), 5022-5024.

[75]
Salvador F., Martín-Retortillo T., Giarratana M. S., & Íñiguez S. (2021). What is applied research and why is it important? [Video]. IE Insights. https://www.ie.edu/insights/videos/what-is-applied-research-and-why-is-it-important/

[76]
Scharnhorst A., Börner K., & Van den Besselaar P. (Eds.) (2012). Models of science dynamics: Encounters between complexity theory and information sciences. Springer Science & Business Media.

[77]
Schauz D. (2014). What is basic research? Insights from historical semantics. Minerva, 52(3), 273-328.

[78]
Soderberg C. K., Errington T. M., & Nosek B. A. (2020). Credibility of preprints: An interdisciplinary survey of researchers. Royal Society Open Science, 7(10), 201520.

[79]
Stephan P. E. (1996). The economics of science. Journal of Economic Literature, 34(3), 1199-1235.

[80]
Stephan P. E., & Everhart S. S. (1998). The changing rewards to science: The case of biotechnology. Small Business Economics, 10, 141-151.

[81]
Stock W. G., Dorsch I., Reichmann G., & Schlögl C. (2023). Labor productivity, labor impact, and co-authorship of research institutions: Publications and citations per full-time equivalents. Scientometrics, 128(1), 363-377.

[82]
Tavits M. (2004). The size of government in majoritarian and consensus democracies. Comparative Political Studies, 37(3), 340-359.

[83]
Toledo-Pereyra L. H. (2012). Ten qualities of a good researcher. Journal of Investigative Surgery, 25(4), 201-202.

DOI PMID

[84]
Trajtenberg M. (2001). R&D policy in Israel. In M. P. Feldman & A. N. Link (Eds.), Innovation policy in the knowledge-based economy (Vol. 23). Springer. https://doi.org/10.1007/978-1-4615-1689-718

[85]
Uwizeye D., Karimi F., Thiong'o C., Syonguvi J., Ochieng V., Kiroro F., Gateri A., Khisa A. M., & Wao H. (2021). Factors associated with research productivity in higher education institutions in Africa: a systematic review. AAS Open Research, 4, 26.

DOI PMID

[86]
Wager E. (2019). Recognition, reward, and responsibility: Why the authorship of scientific papers matters. In A guide to the scientific career: virtues, communication, research and academic writing, 361-368.

[87]
Wahid N., Warraich N. F., & Tahira M. (2022). Factors influencing scholarly publication productivity: A systematic review. Information Discovery and Delivery, 50(1), 22-33.

[88]
Wang D., & Barabási A.-L. (2021). The science of science. Cambridge University Press.

[89]
Wayment H. A., & Dickson K. L. (2008). Increasing student participation in undergraduate research benefits students, faculty, and department. Teaching of Psychology, 35(3), 194-197.

[90]
Zay Ya K., Lambiris M. J., Levine G. A., Tediosi F., & Fink G. (2024). Coverage of policies to improve antimicrobial stewardship in human medicine in low and middle income countries: Results from the Global Survey of Experts on Antimicrobial Resistance. BMC Public Health, 24(1), Article 2297. https://doi.org/10.1186/s12889-024-19542-2

[91]
Zhang S., Wapman K. H., Larremore D. B., & Clauset A. (2022). Labor advantages drive the greater productivity of faculty at elite universities. Science Advances, 8(46), eabq7056.

Outlines

/

京ICP备05002861号-43

Copyright © 2023 All rights reserved Journal of Data and Information Science

E-mail: jdis@mail.las.ac.cn Add:No.33, Beisihuan Xilu, Haidian District, Beijing 100190, China

Support by Beijing Magtech Co.ltd E-mail: support@magtech.com.cn