Effective sample size

July 16, 2008

The notion of “effective sample size” is useful when comparing the information gathered using a certain sampling method to information gathered (or that would be gathered) with a different – reference – sampling method when applied to the same population. Given a known level of uncertainty of an estimate – the one achieved with the sampling method and sample size actually used, the “effective sample size” is the size of the sample that would be needed to achieve the same level of uncertainty if the reference sampling method were used instead.

Read the rest of this entry »

The latest issue of the IMS bulletin contains a letter in which a reader, Anirban DasGupta from Purdue University, lays out his thoughts regarding Ning-Zhong Shi’s MLE conjecture. Evidently, DasGupta’s comments were considered of higher relevance than my own letter to the bulletin’s editors regarding the conjecture, a letter which described the same counter-example to the conjecture that appears in the post linked above.

Read the rest of this entry »

The self-review system

July 2, 2008

This post presents a substitute to the peer-review system: a method to regulate the publication of scholarly work in journals. For background see: Academia as the industry of self promotion and Taking the self promotion out of academia.

Notification of intent to publish, waiting period, publishing quota

A person who intends to engage in research in a field and may wish to publish results, or who already has material he wishes to publish, will submit an indication of intent to publish to a central repository tracking research activity. In that indication of intent, the person submitting will indicate a field, or a small number of fields (up to, say, 4) in which he will be active. Each field is associated with a set of journals – the sets may overlap.

A person may, at any time, revise the set of fields in which he is active – but leaving the number of fields of activity below or at the limit (i.e., 4).

Once an indication of intent to publish in a certain field is given, a waiting period is in effect – say, 18 months. After that period, the researcher may publish in the journals covering that field at any time. A researcher may not publish more than a fixed amount of papers, say, 3, in a fixed amount of time, say 10 years. A multiple-author paper counts toward the quotas of each of the authors as 1/2 of a paper.

Rationale: The rationale for the imposition of a quota is discussed in the background documents above. The imposition of a wait period is aimed at minimizing abuse of the system by people who are not committed to sustained contemplation of a field of research, such as activists of various sorts.

Submission, refereeing

Once an author or a group of authors decide to publish in a certain journal, and if all the authors are within their quotas and beyond their waiting period, they submit a manuscript to the editors of the journal. If the authors so wishes, they may choose to go through a process of refereeing. The authors can choose a set of referees (up to, say, 3) from a list provided by the journal. The referees’ must excuse themselves promptly or write their reports within a set period. The authors may introduce changes to the paper after receiving the referees’ reports. If both the authors and referees are willing, further communication may take place.

Referees’ reports and any communications between the author and the referees are published together with the paper. If these are too long, a prefix is published together with a link to a full version.

Rationale: In the suggested scheme refereeing is seen as a service rendered by the research community to the individual researchers active in the field. Its role is not to block unworthy papers but to help authors produce better papers. Publishing referee-author communications benefits readers by providing third party comments on the paper, and guarantees that referees receive appropriate credit for their work.

Form of publication, page limit, automatic redaction, link to full version

The refereed manuscript (or the unrefereed manuscript, if the authors declines to take advantage of the refereeing process) is truncated at a certain amount of pages (say, 20) and scanned automatically for discouraged patterns (say, profanities or unpleasant typesetting), which are redacted. The author can then resubmit a corrected version or accept the automatically edited version as is. This final version is published in the journal chosen by the authors. If the author so wishes, he may provide an opt-in link (i.e., a link which requires a certain amount of attention to follow) to a full version of the paper which he would host and which is not constrained by the page limit and the pattern prohibitions.

Rationale: The rationale for the guaranteed publication is discussed in the background documents above. The page limit in the centrally hosted version is meant to encourage authors to make their points succinctly.