What’s next for media independence and sustainability indices? A study that considers the role of informal information networks and forms of media such as popular songs? An assessment of the role of different forms of media on one local community? These are some of the more interesting comments to come out of a recent Center for International Media Assistance presentation “Evaluating the Evaluators.”

Earlier this week, I attended a presentation of a new report on the strengths and weaknesses of the three most well known international media climate studies. IREX’s Media and Sustainability Index, Freedom House’s Freedom in the World report, and Reporters Without Borders’ Press Freedom Index were discussed in detail. The discussion also focused on the importance of regional studies, for which the Afrobarometer has become the poster child, as well as fledgling and still developing newer types of media studies. The report, authored by former Washington Post editor John Burgess, can be read in its entirety on the Center for International Media Assistance website. The presentation touched upon some interesting findings and trends:

— Despite the frequent call for more local and context-specific media freedom studies, discussant Lee Becker cited a study he was involved in which found a high to medium correlation between Afrobarometer scores and comparable indictors from the “Big Three” (RSF, Freedom House and IREX). Lee and others pointed out that many of the indicators in the Afrobarometer are nearly identical to those fielded in the international indices.

Stemming from this study and from the industry-wide drive to dig deeper into questions of media sustainability, Lee recommended researchers explore the potential of more sub-national and media-specific studies. For example: a study comparing the independence of print news, radio stations and online media outlets in 5 cities in Indonesia.

— Leon Morse, the managing editor of IREX’s Media Sustainability Index, adamantly spoke out against the assertion that the index includes a “Western bias,” claiming this overarching statement does not help IREX refine its tool. Instead, Leon challenged critics to identify individual indicators or indicator clusters that project a Western bias so that IREX can improve upon its questionnaire. He argued that the use of local media practitioners to generate the research should “mediate” any Western bias.

— Another discussant, Libby Morgan of the Annenberg School at the University of Pennsylvania, made an interesting call for assessments to include other persuasive sources such as songs, lectures and religious sermons as sources of information for citizens. She also stated that NGOs should be considered as leading actors in the spread of information in some contexts.

You can watch the full presentation on the CIMA website. Overall, the report takes a pragmatic view of the “Big Three.” As John Burgees argued, despite all the critiques, these three indices do play an important role in understanding media landscape.

— Norah Mallaney