The self-review system
July 2, 2008
This post presents a substitute to the peer-review system: a method to regulate the publication of scholarly work in journals. For background see: Academia as the industry of self promotion and Taking the self promotion out of academia.
Notification of intent to publish, waiting period, publishing quota
A person who intends to engage in research in a field and may wish to publish results, or who already has material he wishes to publish, will submit an indication of intent to publish to a central repository tracking research activity. In that indication of intent, the person submitting will indicate a field, or a small number of fields (up to, say, 4) in which he will be active. Each field is associated with a set of journals – the sets may overlap.
A person may, at any time, revise the set of fields in which he is active – but leaving the number of fields of activity below or at the limit (i.e., 4).
Once an indication of intent to publish in a certain field is given, a waiting period is in effect – say, 18 months. After that period, the researcher may publish in the journals covering that field at any time. A researcher may not publish more than a fixed amount of papers, say, 3, in a fixed amount of time, say 10 years. A multiple-author paper counts toward the quotas of each of the authors as 1/2 of a paper.
Rationale: The rationale for the imposition of a quota is discussed in the background documents above. The imposition of a wait period is aimed at minimizing abuse of the system by people who are not committed to sustained contemplation of a field of research, such as activists of various sorts.
Submission, refereeing
Once an author or a group of authors decide to publish in a certain journal, and if all the authors are within their quotas and beyond their waiting period, they submit a manuscript to the editors of the journal. If the authors so wishes, they may choose to go through a process of refereeing. The authors can choose a set of referees (up to, say, 3) from a list provided by the journal. The referees’ must excuse themselves promptly or write their reports within a set period. The authors may introduce changes to the paper after receiving the referees’ reports. If both the authors and referees are willing, further communication may take place.
Referees’ reports and any communications between the author and the referees are published together with the paper. If these are too long, a prefix is published together with a link to a full version.
Rationale: In the suggested scheme refereeing is seen as a service rendered by the research community to the individual researchers active in the field. Its role is not to block unworthy papers but to help authors produce better papers. Publishing referee-author communications benefits readers by providing third party comments on the paper, and guarantees that referees receive appropriate credit for their work.
Form of publication, page limit, automatic redaction, link to full version
The refereed manuscript (or the unrefereed manuscript, if the authors declines to take advantage of the refereeing process) is truncated at a certain amount of pages (say, 20) and scanned automatically for discouraged patterns (say, profanities or unpleasant typesetting), which are redacted. The author can then resubmit a corrected version or accept the automatically edited version as is. This final version is published in the journal chosen by the authors. If the author so wishes, he may provide an opt-in link (i.e., a link which requires a certain amount of attention to follow) to a full version of the paper which he would host and which is not constrained by the page limit and the pattern prohibitions.
Rationale: The rationale for the guaranteed publication is discussed in the background documents above. The page limit in the centrally hosted version is meant to encourage authors to make their points succinctly.
November 29, 2013 at 4:13 am
How does this system address issues of quality control? I think your quota proposal has some interesting solutions to negative incentives in peer review, but what is the mechanism here for ensuring that the highest quality papers are published, not relative to what an individual is capable of, but rather relative to the quality of their peers? That is, if I have a quota on what I can publish, this may motivate me to be more selective about what I publish, and while this may help me achieve better work, my work does not necessarily deserve to be published when compared to other work. Why publish me when John H. Researcher does better work and there isn’t space for both of us?
November 29, 2013 at 12:18 pm
Hi – thanks for your comments.
I will admit upfront that the self-review system is inferior to some ideal mechanism which is somehow able to assess quality effectively and objectively across researchers. It is doubtful in my mind if such a mechanism could exist even in theory (since quality is to a non-negligible extent a subjective attribute), but even if such a mechanism is theoretically conceivable, it is clear that the peer-review system is very, very far from this ideal.
It is then a matter of what can be achieved in practice. For the reasons I discuss, I believe that the self-review system as I proposed it is likely to be much superior to the existing system. I am not aware of any other alternative that is promising. I’d certainly be interested in other proposals.
Regarding specifically the matter of outstanding researchers who have so much high-quality material to offer that they cannot fit it within their publication quota: presumably if this situation occurs then people would recognize that this is the case and would be willing to make the extra effort and visit the website of the author to read additional material.