Background, Motivation, and Purpose
In a survey of members of the Scientific Research Society, "only 8% agreed that 'peer review works well as it is'." (Chubin and Hackett, 1990; p.192).
This problem worsened, since then. Casadevall and Fang (2013) shows that "The number of retracted scientific publications has risen sharply." Richard Van Noorden (2014) affirmed, in a paper published by in Nature, that "The publishers Springer and IEEE are removing more than 120 papers from their subscription services after a French researcher discovered that the works were computer-generated nonsense." The IIIS had to cancel one conference presentation in 2005, because of a similar situation. These are just few wxapmaples of the increasing or retraction in scholarly publishing.
"A recent U.S. Supreme Court decision and an analysis of the peer review system substantiate complaints about this fundamental aspect of scientific research. Far from filtering out junk science, peer review may be blocking the flow of innovation and corrupting public support of science." (Horrobin, 2001)
Empirical studies have shown that assessments made by independent reviewers of papers submitted to journals and abstracts submitted to conferences generate a level of agreement among reviewers is about what is expected by chance alone. Rothwell and Martyn (2000), for example, analyzed the statistical correlations among reviewers' recommendations (made to two journals and two conferences) by analysis of variance and found out that for one journal "was not significantly greater than that expected by chance" and, in general, agreement between reviewers "was little greater than would be expected by chance alone."
These are just few examples of an increasing number of facts that are indicating that more research and reflections are urgently needed on research quality assurance and, specifically, on Peer Review. "Peer Review is one of the sacred pillars of the scientific edifice" (Goodstein, 2000). "Peer Review is central to the organization of modern science…why not apply scientific [and engineering] methods to the peer review process" (Horrobin, 2001). Why not apply peer review to current peer reviewing methodologies? To foster research and experience-based reflections with regard to peer review, the methodologies being used in it, and the potential solutions to this very important problem is the purpose of this event on "Knowledge Communication and Peer Review, as well as its related publications.
Objectives
More specifically, the objectives of KCPR 2025 Organizing Committee are:
- To extend this kind of reflections and research to diferent disciplines and to inter-and trans-disciplinary domains, so peer reviewing could be improved and/or complemented by other means of verification and validation of the knowledge to be communicated.
- To Encourage reflections and research of the subject from a systemic and cybernetic perspective, where Peer Review is seen as a means for Knowledge Communication, not as an end in itself; and potentially related though positive and negative feedback loops, as well as with, hopefully, feedforward ones.
- To encourage reflections on, and designs of, non-lineal models of publishing which are desirable and their implementation might be feasible by means of the new communication technologies and Information Systems. This is why KPCR 2025 is being held jointly with KGCM 2025 and in the context of IMCIC 2025.
Disciplinary, Inter-, and Trans-disciplinary Research in Peer-Reviewing Methodologies
Research and reflections on Peer Review have been mainly addressed by Bio-medical communities, and the results have been mostly shared via five International Congresses on "Peer Review in Biomedical Publication", the first of which was held in 1990. The sixth of these congresses will be held on September 2010 and is being organized by the Journal of The American Medical Association (JAMA) and the BMJ (British Medical Journal) Publishing Group.
We are convinced that reflections and research on Peer Reviewing is also needed in other scientific and engineering disciplines, as well as in multi-, inter- and trans-disciplinary research, technological projects, and Knowledge Management in Business and Government. Methodologies applied, and problems found, in peer reviewing in diverse academic disciplines, can synergistically cross-fertilize each other, and can contribute to knowledge quality assurance in the area of Knowledge Management, which would benefit the private and the public sectors and, in general, what has been called as "Knowledge Society". This is why we think that the multidisciplinary context of IMCIC 2025, and its collocated conferences, might fertilize the required cross-disciplinary opportunities.
Conceptual and methodological research and reflections on Peer Review are being increasingly desirable, important and even necessary in academic disciplines and interdisciplinary programs and projects. Peer Review is a research evaluation process which, in turn, requires to be researched and, in turn, peer-reviewed. Peer Review of Peer Review methodologies is urgently being required.
" 'Peer Review' is a name given to a principle that research should be evaluated by people bound by mutual trust and respect who are socially recognized as expert in a given field of knowledge." (Steve Fuller, 2002, Knowledge Management foundation p. 232; emphasis added) But, "peer review" is also a name given to the processes and/or the methodologies of implementing the mentioned principle and achieving the implied objective. In any case, "peer review" refers to knowledge quality control (as a principle, an end, or a mean). But the fact that only 8% of the members of the Scientific Research Society agreed that 'peer review works well as it is' means that peer review "as it is" needs to be, in turn, peer reviewed and, consequently, researched. Although we all agree on "peer review' as principle there is a solid disagreement regarding the effectiveness of the methodologies being applied into achieving the objectives implied by the commonly agreed principle. In the survey made to the members of the Scientific Research Society, 92% of its members disagreed with the actual implementations and methodologies applied in peer reviewing processes.
The almost unanimous agreement about peer reviewing as principle, and the huge disagreement about its current methods, are a clear sign that more efforts are needed in scientific and engineering research and development in order to identify more effective methodologies and support systems (especially with current Information and Communication Technologies) so the real purpose of peer review (based on its principle) is better fulfilled.
The Organizing Committee of the Special Track on Knowledge Communication and Peer Reviewing: KCPR 2025, thinks that the multi-disciplinary approach of IMCIC 2025 and the ICT orientation of many of their participants, might be a fertile context for academics and researchers who can help, through their experience and knowledge, their reflections, research, ideas and opinions, to identify solutions, innovations and support systems for more effective peer reviewing approaches, models, and methodologies.
The Organizing Committee of KCPR 2025 invites scholars, researchers, editors, publishers, authors, readers, professionals and, in general, any user or person affected by or affecting scientific and engineering peer review to submit articles related to their research reflections, ideas, hypothesis, models, etc. on peer review and how to improve it. Among the kinds of submissions accepted are the following:
- Research articles
- Reflection articles
- Literature research papers
- Experience-based Position papers
- Research proposals
- Engineering Design Proposals
- Decision Support Systems Engineering applied to editorial decisions
- New ICT-based peer reviewing models
References
Casadevall, A. and Fang, F. C., 2013, Correction: Why Has the Number of Scientific Retractions Increased?. PLOS ONE 8(7): 10.1371/annotation/0d28db18-e117-4804-b1bc-e2da285103ac. https://doi.org/10.1371/annotation/0d28db18-e117-4804-b1bc-e2da285103ac View correction
Chubin, D. R. and Hackett E. J., 1990, Peerless Science, Peer Review and U.S. Science Policy; New York, State University of New York Press.
Horrobin, D., 2001, "Something Rotten at the Core of Science?" Trends in Pharmacological Sciences, Vol. 22, No. 2, February 2001. Also at http://www.whale.to/vaccine/sci.html and http://post.queensu.ca/~forsdyke/peerrev4.htm (both pages were accessed on February 1, 2009)
Goodstein, D., 2000, "How Science Works", U.S. Federal Judiciary Reference Manual on Evidence, pp. 66-72 (referenced in Hoorobin, 2000)
Rothwell, P. M. and Martyn, C. N., 2000, "Reproducibility of peer review in clinical neuroscience Is agreement between reviewers any greater than would be expected by chance alone?" Brain, A Journal of Neurology, Vol. 123, No. 9, 1964-1969, September 2000, Oxford University Press
Van Noorden, R., 2014, Publishers withdraw more than 120 gibberish papers, Conference proceedings removed from subscription databases after scientist reveals that they were computer-generated, Springer Nature, © 2014 Macmillan Publishers Limited, part of Springer Nature, accessed on 12-11-19 at https://www.nature.com/news/publishers-withdraw-more-than-120-gibberish-papers-1.14763
