Open Peer Review in Scientific Publishing: A Web Mining Study of PeerJ Authors and Reviewers

Peiling Wang; Sukjin You; Rath Manasa; Dietmar Wolfram

doi:10.20309/jdis.201625

Journal of Data and Information Science >

2016 , Vol. 1 >Issue 4: 60 - 80

DOI: https://doi.org/10.20309/jdis.201625

Research Paper

Open Peer Review in Scientific Publishing: A Web Mining Study of PeerJ Authors and Reviewers

Peiling Wang ,
Sukjin You ,
Rath Manasa ,
Dietmar Wolfram

Expand

1 School of Information Sciences, University of Tennessee, Knoxville, TN 37996-0332, USA;
2 School of Information Studies, University of Wisconsin-Milwaukee, Milwaukee, WI 53201, USA

Received date: 2016-07-30

Revised date: 2016-08-30

Online published: 2016-08-30

Fold

Abstract

Purpose: To understand how authors and reviewers are accepting and embracing Open Peer Review (OPR), one of the newest innovations in the Open Science movement.
Design/methodology/approach: This research collected and analyzed data from the Open Access journal PeerJ over its first three years (2013-2016). Web data were scraped, cleaned, and structured using several Web tools and programs. The structured data were imported into a relational database. Data analyses were conducted using analytical tools as well as programs developed by the researchers.
Findings: PeerJ, which supports optional OPR, has a broad international representation of authors and referees. Approximately 73.89% of articles provide full review histories. Of the articles with published review histories, 17.61% had identities of all reviewers and 52.57% had at least one signed reviewer. In total, 43.23% of all reviews were signed. The observed proportions of signed reviews have been relatively stable over the period since the Journal's inception.
Research limitations: This research is constrained by the availability of the peer review history data. Some peer reviews were not available when the authors opted out of publishing their review histories. The anonymity of reviewers made it impossible to give an accurate count of reviewers who contributed to the review process.
Practical implications: These findings shed light on the current characteristics of OPR. Given the policy that authors are encouraged to make their articles' review history public and referees are encouraged to sign their review reports, the three years of PeerJ review data demonstrate that there is still some reluctance by authors to make their reviews public and by reviewers to identify themselves.
Originality/value: This is the first study to closely examine PeerJ as an example of an OPR model journal. As Open Science moves further towards open research, OPR is a final and critical component. Research in this area must identify the best policies and paths towards a transparent and open peer review process for scientific communication.

Key words： Open Peer Review (OPR); Adoption of OPR; Open Access; Open Science; Open research; Scientific communication

Cite this article

Peiling Wang , Sukjin You , Rath Manasa , Dietmar Wolfram . Open Peer Review in Scientific Publishing: A Web Mining Study of PeerJ Authors and Reviewers[J]. Journal of Data and Information Science, 2016 , 1(4) : 60 -80 . DOI: 10.20309/jdis.201625

References

Baker, M. (2016). Statisticians issue warning over misuse of P values. Nature, 531(7593), 151.
Bartholomew, R.E. (2014). Science for sale:The rise of predatory journals. Journal of the Royal Society of Medicine, 107(10), 384-385.
Björk, B.C., & Solomon, D. (2013). The publishing delay in scholarly peer-reviewed journals. Journal of Informetrics, 7(4), 914-923.
Bohannon, J. (2013). Who's afraid of peer review? Science, 342(6154), 60-65.
Bornmann, L., Wolf, M., & Daniel, H.D. (2012). Closed versus open reviewing of journal manuscripts:How far do comments differ in language use? Scientometrics, 91, 843-856.
Council of the European Union. (2016). Outcome of the council meeting, 3470^th council meeting:Competitiveness (internal market, industry, research and space), Brussels, 26 and 27 May 2016. Retrieved on July 16, 2016, from http://www.consilium.europa.eu/en/meetings/compet/2016/05/st09357_en16_pdf/.
Enserink, M. (2016). In dramatic statement, European leaders call for‘immediate’open access to all scientific papers by 2020. Science, News, May 27, 2016. Retrieved on July 16, 2016, from http://www.sciencemag.org/news/2016/05/dramatic-statement-european-leaders-call-immediate-open-access-all-scientific-papers/.
Groves, T. (2010). Is open peer review the fairest system? Yes. BMJ, 341, c6424.
Himmelstein, D. (2015). Publication delays at PLOS and 3,475 other journals. Satoshi Village. Retrieved on April 16, 2016, from http://blog.dhimmel.com/plos-and-publishing-delays/.
Hunter, J. (2012). Post-publication peer review:Opening up scientific conversation. Frontiers in Computational Neuroscience, 6, 63.
Khan, K. (2010). Is open peer review the fairest system? No. BMJ, 341, c6425.
Kriegeskorte, N., Walther, A., & Deca, D. (2012). An emerging consensus for open evaluation:18 visions for the future of scientific publishing. Frontiers in Computational Neuroscience, 6, 94.
Laakso, M., & Björk, B.C. (2012). Anatomy of open access publishing:A study of longitudinal development and internal structure. BMC Medicine, 10, 124.
Lee, C.J., Sugimoto, C.R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2-17.
McCook, A. (2006). Is peer review broken? Submissions are up, reviewers are overtaxed, and authors are lodging complaint after complaint about the process at top-tier journals. What's wrong with peer review? The Scientist, 20(2), 26-35.
McNutt, R.A., Evans, A.T., Fletcher, R.H., & Fletcher, S.W. (1990). The effects of blinding on the quality of peer review. A randomized trial. Journal of the American Medical Association, 263(10), 1371-1376.
Nicholas, D., Watkinson, A., Jamali, H.R., Herman, E., Tenopir, C., Volentine, R., Allard, S., & Levine, K. (2015). Peer review:Still king in the digital age. Learned Publishing, 28, 15-21.
Pöschl, U., & Koop, T. (2008). Interactive open access publishing and collaborative peer review for improved scientific communication and quality assurance. Information Services & User, 28, 105-107.
Rennie, D. (2016). Make peer review scientific. Nature, 535 (July 7), 31-33.
Smith, R. (2006). Peer review:A flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99(4), 178-182.
Soergel, D., Saunders, A., & McCallum, A. (2013). Open scholarship and peer review:A time for experimentation. Retrieved on April 17, 2016, from http://tinyurl.com/h3jbkdz/.
Sumner, T., & Shum, S.B. (1996). Open peer review & argumentation:Loosening the paper chains on journals. Retrieved on July 17, 2016, from https://www.ariadne.ac.uk/issue5/jime/.
Taylor & Francis Group. (2015). Peer review in 2015:A global view. Retrieved on July 17, 2016, from http://authorservices.taylorandfrancis.com/wp-content/uploads/2015/10/Peer-Review-2015-white-paper.pdf/.
van Rooyen, S., Delamothe, T., & Evans, S.J.W. (2010). Effect on peer review of telling reviewers that their signed reviews might be posted on the Web:Randomised controlled trial. BMJ, 341, c5729.
van Rooyen, S., Godlee, F., Evans, S., Black, N., & Smith, R. (1999). Effect of open peer review on quality of reviews and on reviewers' recommendations:A randomised trial. British Medical Journal, 318(7175), 23-27.
Wang, P., Rath, R., Deike, M., & Wu, Q. (2016). Open post publication peer review:An innovation in scientific publishing. Retrieved on July 17, 2016, from https://www.ideals.illinois.edu/handle/2142/89432.
Walsh, E., Rooney, M., Appleby, L., & Wilkinson, G. (2000). Open peer review:A randomised controlled trial. The British Journal of Psychiatry, 176(1), 47-51.
Whither Science Publishing. (2012). As we stand on the brink of a new scientific age, how researchers should best communicate their findings and innovations is hotly debated in the publishing trenches. The Scientist, August 1. Retrieved on July 17, 2016, from http://www.the-scientist.com/?articles.view/articleNo/32378/title/Whither-Science-Publishing/.
Woosen, P. (2015). Journal publishers rethink a research mainstay:Peer review. The Chronicle of Higher Education, October 11. Retrieved on July 17, 2016, from http://www.chronicle.com/article/Academic-Publishing-Toward-a/236526.
Zielinska, E. (2013). Open-review journal launched. The Scientist, February 13. Retrieved on July 17, 2016, from http://www.the-scientist.com/?articles.view/articleNo/34367/title/Open-Review-Journal-Launched/.

Options

Outlines

模态框（Modal）标题

Abstract

Cite this article

References