By Katherine Nichol (Senior Technical Consultant, Kore Global) and Rebecca Calder (Director and Co-Founder, Kore Global)
In 2021, Grand Challenges Canada partnered with Kore Global on a journey to better equip social impact innovators with the know-how to measure impact related to violence against women and girls (VAWG). Through a process of reflection and review of existing literature, we created a broad theory of change to identify change pathways related to prevention and response to VAWG and developed comprehensive guidance on how to measure these change pathways in a safe, ethical and effective manner. The process—which involved a comprehensive review of existing Theories of Change on VAWG and rich reflection on ethical implications of measurement in this space—resulted in the identification of three key insights.
Insight One: You can’t measure what you can’t conceptualize
Rebecca Calder (Director and Co-Founder, Kore Global)
Creating a clear Theory of Change was a critical first step in identifying an approach to measurement. The Theory of Change provided a foundation from which we identified key outcomes to measure, and then appropriate measurement instruments. Reviewing a range of existing Theories of Change from UN agencies, INGOs and grassroots women’s rights organizations, it became clear to us that four critical and intersecting pathways were fundamental in effective programming on prevention and response to VAWG:
- Women and girls: In the medium term, we expect results in women and girls being empowered to express their equal worth, autonomy and agency across all areas of their lives, including within their intimate relationships, families and workplaces.
- Men and boys: In the medium term, changes related to men and boys demonstrating greater emotional awareness and resilience, and expressing themselves in ways that are non-violent are to be expected.
- Family and community: results of several short-term changes should ladder up to community members becoming active allies in promoting gender equality and ending VAWG, intervening when they witness violence and actively demonstrating a lack of tolerance for violence.
- Services and institutions: in the medium term, change is expected in survivor-centred reporting mechanisms and support services that are well resourced, accessible and consistently delivered to meet survivors’ needs. Alongside this, justice systems should operate appropriately and effectively.
Insight Two: Standard measurement approaches can be risky and problematic, but there are alternatives
Once we had an understanding of these pathways and expected outcomes in the short and medium term, we put our heads together to identify ways to measure impact.
Actors new to working on VAWG may be quick to look for data on prevalence of violence to assess the impact of their programs; that is, they expect to see a reduction in reports of violence once a program is implemented. While prevalence data can offer important insights and there is certainly a critical place for this approach in the sector, there are two key risks of using this approach:
- Risk 1: Collecting prevalence data on VAWG—and in fact, any data collection with survivors—requires expert skills. Specialist training and specific protocols are required. In the absence of these, use of any measurement tools can risk retraumatizing participants and can cause further harm.
- Risk 2: Using prevalence data can be problematic. Datasets that are generated by measures of the experience and perpetration of violence both suffer from severe underreporting. This can make interpretation of the data very difficult, particularly if measures are being used to track change over time. For example, data that shows an increase in women experiencing violent behaviours may reflect an actual increase in violence. However, the same data may actually be indicating that women are feeling better able to acknowledge and speak about the violence they have experienced. Nonetheless, secondary data from long-term studies (such as the Demographic and Health Surveys Program) can be useful for long-term trend analysis.
With these risks in mind, we assessed alternatives to prevalence data. We turned to the intermediate outcomes outlined in our Theory of Change and identified measurement instruments that look at 1) women’s agency (e.g. Northwestern University, QualAnalytics and Harvard University’s five-question validated survey module for women’s agency); 2) men’s attitudes and behaviours (e.g. Gender Equitable Men (GEM) Scale); 3) community attitudes and norms (e.g. Illinois Rape Myth Acceptance Scale (IRMA)) and 4) use and quality of services (e.g. drawing on questions from WHO’s multi-country study on women’s health and domestic violence).
Insight Three: Measurement approaches that are not ethical can undermine efforts to keep women and girls safe, and can compromise data quality
Katherine Nichol (Senior Technical Consultant, Kore Global)
As we have already established, collecting any data related to VAWG is inherently sensitive and potentially life-threatening. Discussing topics related to gender equality can unearth deeply entrenched gender norms, potentially threatening power structures, and thus can elicit backlash.
Given the complexity and sensitivity of this area of work, we felt that it was critical to arm practitioners with guidance on what to do—and indeed, what not to do—to ensure approaches to measurement support pathways to healing for survivors and generate robust evidence of change. We developed a checklist for practitioners that highlights three key principles:
- Ensure Monitoring & Evaluation teams have highly specialized skills and expertise to avoid re-traumatization of survivors, this includes training on psychosocial first aid and capacity to refer individuals to vetted medical, psychosocial, security and legal services.
- Develop thorough risk mitigation strategies to account for potential backlash and ensure researchers and participants are safe. For example, identify a safe space where data collection can take place that does not raise community suspicions (for example, consider using women’s safe spaces or a health center where groups of women tend to meet).
- As previously mentioned, question if collecting data on prevalence is needed and consider other areas of change to measure, such as women’s agency and determine if secondary data sources can be drawn upon.
Working on measurement of VAWG programs is complex but much needed. We hope that these three insights provide guidance to practitioners on how to navigate this challenging space and ultimately enable them to better deliver for women and girls.
