Freedom House - Systematic Evaluations

Systematic Evaluations

In several studies of the methodology used by Raymond D. Gastil and others to create Freedom in the World report, Kenneth A. Bollen found that "no criticisms ... have demonstrated a systematic bias in all the ratings. Most of the evidence consists of anecdotal evidence of relatively few cases. Whether there is a systematic or sporadic slant in Gastil's ratings is an open question". Bollen studied the question of ideological bias using multivariate statistics. Using their factor-analytic model for human-rights measurements, Bollen and Paxton estimate that Gastil's method produces a bias of 0.38 standard deviations (s.d.) against Marxist–Leninist countries and a larger bias, 0.5 s.d., favoring Christian countries. In contrast, another method by a critic of Freedom in the World produced a bias for Leftist countries during the 1980s of at least 0.8 s.d., a bias that is "consistent with the general finding that political scientists are more favorable to leftist politics than is the general population".

In 1990, Gastil discussed criticisms of Freedom in the World, stating that "generally such criticism is based on opinions about Freedom House rather than detailed examination of survey ratings".

The definition of freedom in Gastil (1982) and Freedom House (1990) emphasized liberties rather than the exercise of freedom, according to Adam Przeworski, who gave the following example: In the United States, citizens are free to form political parties and to vote, yet even in presidential elections only half of U.S. "citizens" vote; in the U.S., "the same two parties speak in a commercially sponsored unison".

Another study by Mainwaring, Brink, and Perez-Linanhe found the Freedom Index of Freedom in the World to have a strong positive correlation (at least 80%) with three other democracy indices. Mainwaring et al. wrote that Freedom House's index had "two systematic biases: scores for leftist were tainted by political considerations, and changes in scores are sometimes driven by changes in their criteria rather than changes in real conditions". Nonetheless, when evaluated on Latin American countries yearly, Freedom House's index were strongly and positively correlated with the index of Adam Przeworski and with the index of the authors themselves: They evaluated Pearson's coefficient of linear correlation between their index and Freedom House's index, which was 0.82; among these indices and the two others studied, the correlations were all between 0.80 and 0.86.

Freedom House maintains that its methodology is systematic and not culturally biased:

Freedom House does not maintain a culture-bound view of freedom. The methodology of the survey is grounded in basic standards of political rights and civil liberties, derived in large measure from relevant portions of the Universal Declaration of Human Rights. These standards apply to all countries and territories, irrespective of geographical location, ethnic or religious composition, or level of economic development.