Taking the self promotion out of academia

October 24, 2007

Academic science, as it is today (and, it seems to me, for centuries past) is the industry of self promotion. The way the system works, academics must promote themselves. In order to get hired, promoted or funding – or even just not to be let go – they must present their work as being of great importance or novelty. Every new thought or wrinkle or nuance must be presented as a scientific advance worth telling to others, over and over again.

Researchers are thus driven toward emphasizing quantity over quality, toward over-interpreting their theory or data and toward unfruitful complexity (since complexity is harder to criticize and easier to produce than simplicity).

Under current conditions, these trends, which are harmful to the community, are driven by the need to survive of the individual researchers. This is then, the tragedy of the scientific commons.

In addition, the system of peer review promotes the formation of elites and cliques and reduces innovation. The established researchers – being the referees on widely read publications – can prevent intrusion by newcomers and censor unusual ideas, and encourage the endless repetition of the conventional wisdom. This is the rule of the scientific oligarchy.

To resolve these difficulties the sacred cow of the peer review system must be slaughtered. It must be recognized the peer review system is an important part of the problem. The system relies on the idea that it is possible for a handful of referees to determine in a reasonable and fairly evenhanded way which articles are worth publishing and which are not. This idea is false. The decisions of referees are mostly political (i.e., influenced by the interests and personalities of the actors) rather than rational. Referees cannot be relied upon to effectively discriminate between good and bad papers.

Clearly, the authors of a paper, being most familiar with it, are those who are in the best position to estimate the quality of the work. It therefore makes sense to rely on their evaluation in the decision whether to publish it or not. Of course, the authors are biased in favor of their own work, but at least within the body of their own work, they should be able to do a good job of rating quality.

My proposed system is, thus, to give each author a quota of publications – say one article every 3 years. The author can choose what work to put in that article – which is guaranteed to be published. Preferably, the article will still be refereed, but it would be up to the author to decide whether to incorporate any suggestions by the referees into the work. A researcher would then choose carefully, picking the best of their work of the last 3 years (and incorporating any useful ideas of the referees) to produce the best paper he or she could come up with – with 3 years between publications it makes sense that there will be a lot of good work to choose from and a lot of time to write the paper well. The result will be more enjoyable and productive for the author, for his or her readers, and for the scientific community.


10 Responses to “Taking the self promotion out of academia”

  1. […] note the connections of this issue to those of the definition of free speech and of the function of peer review in academia. Posted in academia, democracy, elitism […]

  2. […] This is a nasty situation, but would you have things be any different? Would you rather have a non-self-promoting academia? Would you support science that is not based on “my work is better than yours”? […]

  3. SG Says:

    sortition, isn’t your recommendation here simply that academia should place more weight on technical reports? Essentially every academic can write one, they aren’t peer reviewed, etc.

    Maybe an alternative would be for the journals to publish an annual review of major work and themes that have been noticed in the last year, drawing on technical reports, published papers, what their mates know, etc. Then when academics apply for work, promotions etc. they can cite first and foremost their citations in those reviews as evidence of quality.

    Also, I’m not sure that self-promotion necessarily promotes quantity over quality – surely this is only the case where the majority of academics are crap to start with. And are you confident that peer review is conducted by a “handful” of elite scientists? I don’t think that is the case in my field (though I do think there are cliques in peer review, and a lot of very bad behaviour).

  4. yoramgat Says:

    Hi SG,

    The problem with technical reports is the volume: it is impossible to keep up since anyone can publish as many technical reports as they want. The role of controlled publication venues is to reduce the volume of publications to a manageable amount in a way that would tend to preserve the most interesting papers and dispense with the rest. I suggest that self-critique is the best way to achieve this goal.

    Regarding annual reviews: Who would author them? That system would probably be prone to even more extreme political bias than peer-review.

    Regarding peer-review promoting quantity over quality: I don’t think this is a matter of the academics being “crap”. Rather, it is a feature of the academic publishing system: an author has nothing to lose and a lot to win by submitting more. Modesty and being self-critical are mortal sins for the young academic – they make it unlikely you’ll survive. Of course, once you are tenured you can afford to be more relaxed.

    As for the “handful”: I may have been unclear. There are a handful of reviewers involved in any particular review – they are not the same handful every time. There is still an oligarchy of established authors the makes the pool from which referees are drawn, but this is likely more than a handful of people. Still, this group is an oligarchy since it is much smaller than the number of people active (readers and prospective authors) in the field and entry into the group is by cooptation by existing members.

  5. SG Says:

    how about random selection of peer reviewers, then? When you submit a paper to a journal you tick a box saying “I agree to review an article in the next [sensible time frame]” and you get sent one randomly. Then there’s no oligarchy, and new (but published) researchers get a chance to have input into the peer review process. Since new researchers often don’t know the bigwigs, and can’t recognise their work if it’s deidentified, it might actually encourage a bit of humility in reviewers, who will “grow up”, as it were, knowing they might be reviewing someone much more knowledgeable than them.

    (Plus authors might choose to focus on quality over quantity if they knew that increasing their publication list requires they do more reviewing).

    Plus of course, the quantity-over-quality thing would be much better addressed by universities using a more sensible system for assessing academic quality than the number of papers published (which my annual review idea would help to support, I might point out!)

  6. yoramgat Says:

    I am a great fan of random distribution of power, and your suggestion definitely goes some way toward reducing the elitism of academic publishing. One problem with it, however, is that you are still going to have groups with different levels of power (i.e., the published vs. the unpublished). The other problem is that the system you are suggesting, just like the existing system, does not really promote quality: quantity and arbitrariness will very likely still be the dominant characteristics of the system. (Handling more reviewing is a small price to pay for being better published, and can always be handled by putting very little effort into the review – a phenomenon that seems quite common today.)

    As for assessing quality over quantity: That is the essence of the problem here. Having such an assessment is a very difficult task intellectually, and an impossible one politically. This is exactly the task that peer-review is supposedly charged with, and it is this task that it is utterly failing at achieving.

  7. SG Says:

    good points all, I think. Good thing we’re statisticians then, isn’t it, so we straddle the professional and academic worlds, and aren’t so dependent upon the good esteem of our academic colleagues. My experience at least in Australia is that if you can do the more advanced technical details, and use the software well, people don’t give a rats arse how many publications you have. Provided you aren’t trying to be a theoretician, of course.

    Just as well for me, too…

  8. yoramgat Says:

    I too find the non-academic world much less stressful. Working in industry, it is enough that you are doing your part – you don’t have to prove that you are a star. Many academics, I think, are attracted to the notion of stardom and believe that academic competition is a good thing – I find it unpleasant, unscientific and counter-productive.

  9. […] the absence of the question of the objective, it is impossible to address fundamental questions – does refereeing serves the public (the public of researchers and the public at large) and whether th… – and refereeing, essentially in its present for function, is presented as an immutable natural […]

  10. […] July 2, 2008 This post presents a substitute to the peer-review system: a method to regulate the publication of scholarly work in journals. For background see: Academia as the industry of self promotion and Taking the self promotion out of academia. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s