In a methodology reminiscent of the Failed States Index, rating firm Covalence SA assigns scores on corporate ethics, based on coding stories from a battery of news tickers. Their results appear to be real, but limited.

Covalence is measuring a consensus opinion — perceptions — as played out in newspapers reachable by Google News and similar services. They may be doing the coding reasonably well, but ultimately, their numbers are a reflection of the skill and reach of opposing PR teams: Wal-Mart versus Wal-Mart Watch. I strongly suspect their inputs are biased toward the English language press, a potentially significant issue: multinational PR teams speak English; multinational laborers speak Tagalog, Xhosa, Sinhalese… Likewise, PR firms put content on websites; local media often don’t.

This information is no doubt very useful for people with no background on a company — Widgets Anonymous India Inc — but it’s only the conventional wisdom, nothing more. The massive aggregation of small information (the Internet’s hive mind) produces results that are frequently ignorant or hysterical; I don’t trust it. Assigning quantified ratings to these decidedly qualitative inputs applies a veneer of empiricism that may not be justified. On the other hand, to investors making snap trading decisions, perhaps reputation is reality, regardless of the experience of actual factory workers in some forgotten country. In Global Integrity’s work, we set the bar a bit higher.

I will give Covalence points for a very nice write up of their methodology, very clear and useful. While we may differ on our methodological approaches, I can see they’ve done their homework.

Want more on ratings and metrics? Download A Users’ Guide to Measuring Corruption.

— Jonathan Eyler-Werve