The IFHS study

January 17, 2008

The recent release of the IFHS study (via Deltoid) which put the number of Iraqis killed violently during the first 3 years and 4 months after the invasion at a mere 150,000 has generated the expected sigh of relief in the media (e.g., 1, 2, 3). Having previously been implicitly blamed for supporting an endeavor that generated 4 times as many violent deaths over the same period (Burnham et al., a.k.a. the second Lancet study), this new figure is celebrated as vindication.

Going over the pattern of media response to the IFHS study would be informative, but would produce unsurprising results. I therefore touch on only one point which also bears on the issue of any anti-IFHS bias by “Lancet supporters”.

Conveniently, the reports ignore the question of how many Iraqis died non-violently following the invasion as a result of the widespread devastation and breakdown of organization. This was helped to a large extent by the fact that the study itself, while giving an estimate for violent deaths, does not give an estimate for excess mortality – it merely gives pre-war and post-war mortality rate estimates. It is, however, quite easy to use those mortality rate estimates to generate an estimate for the excess mortality. Using those figures and applying the method used in the study to account for under-reporting, the estimate of excess deaths during the first 40 months after the invasion comes to around 400,000. This is not significant disagreement with the different from the Lancet figure.

The major disagreement between these two studies, therefore, is about what is the proportion of excess deaths were violent, rather than how many Iraqis died as a result of the invasion. The IFHS has it that only about 1/3 of the excess deaths are caused by violence – Burnham et al. put that figure at about 90%. In this disagreement, based on a-priori considerations, it seems that the IFHS findings are more reasonable. It would be a miracle if in a country of 30 million, there could be enough violence to cause hundreds of thousands of violent deaths, and yet non-violent mortality would barely budge.

In my mind, it is the total number of excess deaths that is of interest when trying to decide what is the cost in lives that is attributable to the invasion. It makes little difference to an Iraqi whether his child died when she was hit a bullet or when she was poisoned by contaminated drinking water. I also expressed the view that the ratio 10:1 violent to non-violent deaths estimated by Burnham et al. is problematic long before the IFHS was published.

I thus see no need to bash the IFHS study. It is however quite interesting to find that when examining the IFHS in a little detail (and it is really no more than a cursory examination that I undertook) several significant problematic points manifest themselves – I will enumerate them in an upcoming post (here). The fact that such problems exist is interesting for several reasons:

  1. The immediate interest is regarding the validity of the findings in the context of assessing the reality in Iraq and the impact of the decision to invade it.
  2. A second point of interest is the matter of how points of weakness are handled by various players (especially, powerful players, such as corporate media and the government). When do such points of weakness get to be played up and seen as undermining the credibility of a study and when do they get to be ignored or played down as mere nitpicking.
  3. An additional point is the fact that papers with such obvious weaknesses can pass the vaunted peer-review barrier – what does this imply about the process of peer-review and the politics of science?
  4. Finally, the broad epistemological issue – what can we know about what happens in other places? What do accounts, including scientific, establishment sanctioned accounts, teach us?

5 Responses to “The IFHS study”

  1. […] Yoram Gat at Pro Bono Statistics has an excellent post that analyzes the discrepancy between the Johns Hopkins study and the more recent WHO report in […]

  2. frankis Says:

    Well said.

  3. yoramgat Says:

    Thanks for visiting and for the compliment, frankis.

  4. […] 91 is Reuters. At the same time, searching for articles covering the three weeks old release of the IFHS study shows 295 hits (using the combination iraq+”new england journal of medicine”+151000 OR […]

  5. […] The series of posts on the studies of mortality in Iraq is within that category of topics, and I hope to continue to pursue this topic. Beyond the immediate practical interest in finding out the truth about reality in Iraq, this topic has other interesting ramifications (see the points at the end of this post). […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s