A Minister of Transport sits drinking a beer on a restaurant patio.  Across from him is the vice president of the country’s leading private railway company. A week later, a multi-million dollar public contract to connect the country’s biggest metropolis to its countryside is awarded to the same vice president’s railway company.

Click. It is just one photo, but it represents an entire story. In Spain, dozens of reporters and citizen journalists are capturing images of this kind to expose hidden lobbying activities. CIVIO Foundation is one of our Testing 123 innovation fund investees that will use these photos as visual evidence to encourage the inclusion of public officials’ agendas in Spain’s freedom of information draft law.

The aim of their idea, “Who Rules?” is ambitious and the technique to achieving it is risky, but the well thought out theory of change framing this idea is persuasive.

In September 2012 we offered applicants $10,000 to propose and implement untested approaches to tackling problems of corruption.  We additionally asked them to identify their goals, steps to achieving them, and preconditions for success.  These blueprints constitute theories of change which we used to measure depth of thought around project requirements and aspirations to inform investee selection.

This has inspired us to experiment with creatively applying the same informational structure to the critical forthcoming task of monitoring and evaluating idea prototypes.  

It is an exciting opportunity to ease an ongoing tension between the desire to measure outcomes of efforts seeking more transparent and accountable governance — and the inability to do so. On the one hand, we are committed to measuring and communicating our impact because it keeps us accountable.  On the other, it is acutely difficult to track results and measure impact; a confluence of socio-political and economic factors generates outcomes that are not often quantifiable and can be interpreted in a number of ways.

Our strategy is to collaboratively revisit and build out theories of change to generate a more concrete understanding of which of the tested techniques work, don't work, and/or can travel. We want to be able to concisely and clearly communicate whether and why Testing 123 experiments work, and whether others should consider using them.

Operationally, this looks like a series of discussions with Testing 123 investees.  The discussions follow each iteration of hypotheses and anticipated results with the goal of ensuring detail and specificity.

Discussions are ongoing but they have already begun to yield interesting results:

Chats with CIVIO Foundation’s project team running “Who Rules?”, teased out a potentially key success factor: their preexisting relationships with relevant policymakers through an active role in Spain’s access to information movement.

Djordje Padejski pinpointed the central assumption and “secret sauce” (as he calls it!) to his real time corruption alert system in Serbia, Veritza.org – an algorithm to automate dataset crosschecks.

Visar Shehu dissected the “information” in Macedonia that his team seeks to collate and display through “Simply Visualizing Politics” by breaking it up into speech patterns and political trends.

The Ministry of Space team honed in on the channels they plan to use for retrieving Serbian urban development documents, including through Right to Information (RTI) requests and scraping online publically available data.

Eduardo Bohorquez and Transparencia Mexicana broke their “Police-Citizen Protocol” hypothesis into two: first, the development and design of the application together with Mexican law enforcement; second, the usage of the tool by citizens for mitigating the risk of corruption.

In jointly designing well-defined and detailed models, we hope to build a foundation around which our process will operate to encourage experiential learning for both our investees and ourselves. If it works, the iterative theory of change exercise could merit emulation to strengthen monitoring and evaluation of social impact work.

–Nicole Anand

–Photo Credit: Flickr/ London Permaculture