Below is my page-by-page summary of Kuhn’s 1967 postscript to The Structure of Scientific Revolutions.

The following points seem to me to contain the essence of Kuhn’s thesis:

Exemplars:

  1. Exemplars (solved problems) are an important component of any specific scientific world-view. Without exemplars the laws and theories have little empirical content. The exemplars teach practitioners how to attach the relevant abstractions to elements of particular problems and how to see a variety of situations as being alike.
  2. The intuitive knowledge of which situations are alike is analyzable but not by specifying rules since it is perceptual rather than interpretive knowledge. It is similar to other knowledge of perception: e.g., identifying certain light patterns as all representing swans. Perceptual knowledge is selected for “success” – success in survival in the primitive case, success in puzzle solving in the scientific case.
  3. The situation may be similar in non-scientific schools (e.g., art).

Revolutions:

  1. Scientific revolutions – “a special sort of change in scientific thinking involving a certain sort of reconstruction of group commitments” – occur regularly on the smaller scale (within communities of dozens to hundreds).
  2. Crisis is the usual mechanism inducing revolutions. Crises supply a self-correcting mechanism which ensures that the rigidity of normal science will not forever go unchallenged.
  3. Judgments of scientific values – simplicity, consistency and compatibility – can vary greatly from individual to individual. Therefore, debates on theory-choice cannot be decided in a formal way. Still, those values constrain the dominant group view. This explains why competing world-views are rare in science but are common in other human activities.

Scientific progress:

  1. Because the commitment by scientists to puzzle-solving success shapes the long-term structure of scientific development (even if the application of this commitment in any particular case is ambiguous and subjective), scientific development, like biological development, is unidirectional and irreversible. When comparing scientific theories held by various scientific specialties to theories held by the specialty from which the former had their origins, “an uncommitted observer” would be able to consistently distinguish the newer theories from the older. The newer theories will enable more accurate, quantitative, predictions, in a wider variety of situations and they will be more esoteric. Differences in simplicity, scope and compatibility with other specialties are not as telling.
  2. Scientific progress is therefore well defined, but, like biological progress, it cannot be said that science comes closer and closer to a certain goal – “reality”. There is no objective reality which stands outside a specific scientific world-view.

Read the rest of this entry »

Advertisement

The latest issue of the IMS bulletin contains a letter in which a reader, Anirban DasGupta from Purdue University, lays out his thoughts regarding Ning-Zhong Shi’s MLE conjecture. Evidently, DasGupta’s comments were considered of higher relevance than my own letter to the bulletin’s editors regarding the conjecture, a letter which described the same counter-example to the conjecture that appears in the post linked above.

Read the rest of this entry »

The self-review system

July 2, 2008

This post presents a substitute to the peer-review system: a method to regulate the publication of scholarly work in journals. For background see: Academia as the industry of self promotion and Taking the self promotion out of academia.

Notification of intent to publish, waiting period, publishing quota

A person who intends to engage in research in a field and may wish to publish results, or who already has material he wishes to publish, will submit an indication of intent to publish to a central repository tracking research activity. In that indication of intent, the person submitting will indicate a field, or a small number of fields (up to, say, 4) in which he will be active. Each field is associated with a set of journals – the sets may overlap.

A person may, at any time, revise the set of fields in which he is active – but leaving the number of fields of activity below or at the limit (i.e., 4).

Once an indication of intent to publish in a certain field is given, a waiting period is in effect – say, 18 months. After that period, the researcher may publish in the journals covering that field at any time. A researcher may not publish more than a fixed amount of papers, say, 3, in a fixed amount of time, say 10 years. A multiple-author paper counts toward the quotas of each of the authors as 1/2 of a paper.

Rationale: The rationale for the imposition of a quota is discussed in the background documents above. The imposition of a wait period is aimed at minimizing abuse of the system by people who are not committed to sustained contemplation of a field of research, such as activists of various sorts.

Submission, refereeing

Once an author or a group of authors decide to publish in a certain journal, and if all the authors are within their quotas and beyond their waiting period, they submit a manuscript to the editors of the journal. If the authors so wishes, they may choose to go through a process of refereeing. The authors can choose a set of referees (up to, say, 3) from a list provided by the journal. The referees’ must excuse themselves promptly or write their reports within a set period. The authors may introduce changes to the paper after receiving the referees’ reports. If both the authors and referees are willing, further communication may take place.

Referees’ reports and any communications between the author and the referees are published together with the paper. If these are too long, a prefix is published together with a link to a full version.

Rationale: In the suggested scheme refereeing is seen as a service rendered by the research community to the individual researchers active in the field. Its role is not to block unworthy papers but to help authors produce better papers. Publishing referee-author communications benefits readers by providing third party comments on the paper, and guarantees that referees receive appropriate credit for their work.

Form of publication, page limit, automatic redaction, link to full version

The refereed manuscript (or the unrefereed manuscript, if the authors declines to take advantage of the refereeing process) is truncated at a certain amount of pages (say, 20) and scanned automatically for discouraged patterns (say, profanities or unpleasant typesetting), which are redacted. The author can then resubmit a corrected version or accept the automatically edited version as is. This final version is published in the journal chosen by the authors. If the author so wishes, he may provide an opt-in link (i.e., a link which requires a certain amount of attention to follow) to a full version of the paper which he would host and which is not constrained by the page limit and the pattern prohibitions.

Rationale: The rationale for the guaranteed publication is discussed in the background documents above. The page limit in the centrally hosted version is meant to encourage authors to make their points succinctly.

The March 2008 IMS (Institute of Mathematical Statistics) Bulletin issue has a special section discussing refereeing. The bulletin editor, Xuming He, introduces the matter on the front page of the issue while four present and past editors of statistics journals – John Marden, Michael Stein, Xiao-Li Meng and Rick Durrett – present their ideas about refereeing on the inside pages.

The discussion takes a predictable path – the writers describe and beseech what they perceive as good behavior on the side of referees (mostly focusing on promptness). Indeed, the choice of writers – established figures in the field – made it unlikely a-priori that a radical examination of the matter would be undertaken. The role of refereeing as gate-keeping is never questioned and the question of what is the objective of refereeing is not even raised by most writers (Marden is the exception here, see below). In view of the absence of the question of the objective, it is impossible to address fundamental questions – does refereeing serve the public (the public of researchers and the public at large) and whether there could be publication selection systems that are superior to refereeing – and refereeing, essentially in its present form and function, is presented as an immutable natural phenomenon.

Read the rest of this entry »

Epistemology omnibus

February 22, 2008

This omnibus post collects draft fragments that are associated with the economy of attention and the sociology of science and expertise.

Read the rest of this entry »

Looking back, looking ahead

February 6, 2008

This post is aimed at serving as something of a self evaluation of the first 6 months of this blog, and a statement of intent for its future.

Read the rest of this entry »

The IFHS study

January 17, 2008

The recent release of the IFHS study (via Deltoid) which put the number of Iraqis killed violently during the first 3 years and 4 months after the invasion at a mere 150,000 has generated the expected sigh of relief in the media (e.g., 1, 2, 3). Having previously been implicitly blamed for supporting an endeavor that generated 4 times as many violent deaths over the same period (Burnham et al., a.k.a. the second Lancet study), this new figure is celebrated as vindication.

Going over the pattern of media response to the IFHS study would be informative, but would produce unsurprising results. I therefore touch on only one point which also bears on the issue of any anti-IFHS bias by “Lancet supporters”.

Conveniently, the reports ignore the question of how many Iraqis died non-violently following the invasion as a result of the widespread devastation and breakdown of organization. This was helped to a large extent by the fact that the study itself, while giving an estimate for violent deaths, does not give an estimate for excess mortality – it merely gives pre-war and post-war mortality rate estimates. It is, however, quite easy to use those mortality rate estimates to generate an estimate for the excess mortality. Using those figures and applying the method used in the study to account for under-reporting, the estimate of excess deaths during the first 40 months after the invasion comes to around 400,000. This is not significant disagreement with the different from the Lancet figure.

The major disagreement between these two studies, therefore, is about what is the proportion of excess deaths were violent, rather than how many Iraqis died as a result of the invasion. The IFHS has it that only about 1/3 of the excess deaths are caused by violence – Burnham et al. put that figure at about 90%. In this disagreement, based on a-priori considerations, it seems that the IFHS findings are more reasonable. It would be a miracle if in a country of 30 million, there could be enough violence to cause hundreds of thousands of violent deaths, and yet non-violent mortality would barely budge.

In my mind, it is the total number of excess deaths that is of interest when trying to decide what is the cost in lives that is attributable to the invasion. It makes little difference to an Iraqi whether his child died when she was hit a bullet or when she was poisoned by contaminated drinking water. I also expressed the view that the ratio 10:1 violent to non-violent deaths estimated by Burnham et al. is problematic long before the IFHS was published.

I thus see no need to bash the IFHS study. It is however quite interesting to find that when examining the IFHS in a little detail (and it is really no more than a cursory examination that I undertook) several significant problematic points manifest themselves – I will enumerate them in an upcoming post (here). The fact that such problems exist is interesting for several reasons:

  1. The immediate interest is regarding the validity of the findings in the context of assessing the reality in Iraq and the impact of the decision to invade it.
  2. A second point of interest is the matter of how points of weakness are handled by various players (especially, powerful players, such as corporate media and the government). When do such points of weakness get to be played up and seen as undermining the credibility of a study and when do they get to be ignored or played down as mere nitpicking.
  3. An additional point is the fact that papers with such obvious weaknesses can pass the vaunted peer-review barrier – what does this imply about the process of peer-review and the politics of science?
  4. Finally, the broad epistemological issue – what can we know about what happens in other places? What do accounts, including scientific, establishment sanctioned accounts, teach us?

The Golem

November 1, 2007

Following some heretical comments by me regarding the nature of scientific activity, I was referred by commenter “dsquared” to a book called “The Golem” by two sociologists, Harry Collins and Trevor Pinch. This fairly reasonable book makes the point that, despite pretenses by scientists and non-scientists, science is not a formal system in which crisp models generate crisp predictions which can then be neatly confirmed or refuted by experiments with unambiguous results. This is a good point, but the book somehow still manages to miss, or at least to obscure, the main issue – as dsquared’s summary of the “the basic conclusions of the literature” shows.

Read the rest of this entry »

Academic science, as it is today (and, it seems to me, for centuries past) is the industry of self promotion. The way the system works, academics must promote themselves. In order to get hired, promoted or funding – or even just not to be let go – they must present their work as being of great importance or novelty. Every new thought or wrinkle or nuance must be presented as a scientific advance worth telling to others, over and over again.

Researchers are thus driven toward emphasizing quantity over quality, toward over-interpreting their theory or data and toward unfruitful complexity (since complexity is harder to criticize and easier to produce than simplicity).

Under current conditions, these trends, which are harmful to the community, are driven by the need to survive of the individual researchers. This is then, the tragedy of the scientific commons.

Read the rest of this entry »

The quality of an idea is an objective property, but it is one that we usually do not possess any direct way of measuring. Since any natural language argument is informal, it is impossible to make a definite objective determination of which ideas are good (or correct) and which are bad (incorrect). This does not mean that there aren’t good and bad ideas, and better and worse ways to think about issues and to reach conclusions. It does means that care must be taken when evaluating ideas, that every piece of an argument is open for dispute or reinterpretation and that all conclusions must be taken as tentative.

Read the rest of this entry »