News

November 2021

A cheat sheet to “Evaluating the Results of Intermediary Organisations – A Paper for Intermediaries in Australia”, Mark Cabaj

Article by Liz Skelton, Director and Co-Founder of CFI

On the eve of COVID gripping the globe in February 2020, Collaboration for Impact and other Australian intermediaries were convened by the Paul Ramsay Foundation for exploration with social change evaluator Mark Cabaj. The convening sought to explore the role and impact of so-called ‘intermediaries’ in impacting social change work. A phrase more common in the global north, in Australia, Intermediaries play multiple roles in addressing disadvantage such as Capability Specialist; Evidence-Action Lab; Place-Based Backbone; Field Catalyst ((Hussein et al.2018).

Increasingly more intermediaries are emerging in the Australian social change field, responding to the complexity of the work and the need for multiple skills and capabilities. The role of intermediaries is often misunderstood or invisible. Measuring our impact is further challenged by the tension in claiming any part of the contribution to the success and impact of a collaboration in case it detracts from the collaborative work of communities themselves. This can result in intermediaries being hesitant to claim the valuable contribution they make, contributing to the misunderstanding and invisibility and opportunity to leverage.

“While the upfront case for strengthening eco-system-building intermediaries is strong, the work of understanding, measuring and demonstrating their impact is not. Investing in ecosystem-building intermediaries in Australia and internationally is relatively new, and the research, thinking and case studies are limited.”  Mark Cabaj, November 2021

The report uses Canadian and North American examples to describe the considerations in thinking about and measuring impact for intermediaries and provides important insights for philanthropy and government funders. As always Mark provides clear useful framing to help make sense of the complexity of the field. The exploration and report have further catalysed interest and enquiry not just on how to measure the impact of the work we do as ‘intermediaries’ but also the need for Australian intermediaries to work together more to deepen understanding and articulate clearly the impact of the work in Australia ourselves.

Intermediaries play a unique role in supporting the scaling of impact on the ground with communities and through leveraging networks across the ecosystems of social innovators, government and funders.  Understanding this role better is critical to accelerate the impact we can have collectively.

I’ve compiled a cheatsheet summary of the key insights in the report:

1. Intermediaries contribute to – rather than drive – results in breaking the complex cycle of disadvantage.

2. Intermediaries contribute three types of results that interact within ecosystems of social innovation to disrupt cycles of disadvantage:

• • Core capacity building activity, if valued and taken up.
  • “Strategic outcomes” that address the deeper systems that hold disadvantage in place (e.g., niche innovations, changes in policies, regulations and resource flows, cultural beliefs).
  • The impact of all these changes on breaking the cycle of disadvantage experienced by people and communities.

3. Intermediaries & funders need the ability to:

• • take a wide-angle lens to understand intended and unintended impacts including capturing what’s changed and ripple effects.
  • understand where progress has or hasn’t been made, rather than reductionist or simplistic measurements.
  • seek out their contribution – rather than attribution

4. To understand the impact of intermediaries, use a variety of “lenses.”:

1. • The unique theory of change, rather than a one-size-fits-all collection of indicators and methods;
   • See all activities and results as a portfolio
   • Evaluate all change from incremental to transformative
   • Use an equity lens to understand the effects on groups experiencing disadvantage
   • Understand how the impact fits the larger change context;
   • Get a diverse range of stakeholder views on the significance and merit of the impact

5. Intermediaries and funders need to invest in building their own capacity, relationships and understanding of each other and commit to ongoing learning and adaptation to design and implement useful evaluations.

> View the report on the Paul Ramsay Foundation site:  “Evaluating the Results of Intermediary Organisations – A Paper for Intermediaries in Australia”