Measuring a 'moving target' - designing evaluations for HCD programs: About the webinar

Human-centered design (HCD) is increasingly being used as a complementary approach to traditional global health methods to design or optimize innovative solutions due to its ability to bring new ideas to entrenched problems, integrate multiple stakeholder perspectives, and bring in a strong human lens among other advantages.

However these tools also present some challenges when applying conventional evaluation approaches. There is an inherent difficulty in designing an evaluation for a project rooted in HCD and AI approaches that continues to adapt and improve its interventions over its project lifecycle.

Addressing these gaps and creating more effective ways to evaluate programs that utilize HCD and other continuous program improvement tools can provide tremendous value in scaling up evidence-based user-centered approaches.

Join us for a panel discussion with speakers from A360, CyberRwanda, HCDExchange’s Measurement & Evaluation Working Group to unpack these unique challenges for measuring a moving target and how we might address them moving forward!

Date: Wednesday, April 13th
Time: 4:30PM EAT / 7PM IST / 1:30PM GMT / 9:30AM EST
Zoom link: Webinar Registration - Zoom

We are very excited to introduce our wonderful panel of speakers for today’s call!

Fifi Oluwatoyin Ogbondeminu, A360 Deputy Project Director, PSI

Fifi is a highly enthusiastic, self-motivated, and creative Public Health Specialist with over 15 years of successful leadership experience in developing and leading health programs. She has a strong track record in applying contextual and evidence-based approaches across a variety of health areas including Adolescent Sexual and Reproductive Health. She exhibits an incredible work ethics with a proven record of aligning diverse, multi-level teams with project mission and vision, while ensuring policy compliance. Fifi holds a bachelor’s degree in Pharmacy, master’s degrees in public health and Business Administration, a fellow of the West African College of Pharmacist and currently working on a Doctorate in Public Health. Before joining PSI as the Deputy Project Director, A360, she was the Project Director on the A360 Amplify project in Nigeria, where she worked with a multidisciplinary team to design replicable, cost effective and adaptive solutions that empowers both married and unmarried adolescent girls less than 20 years to improve their sexual and reproductive health outcomes. She also has in her portfolio several awards, accomplishments, and publications. Fifi enjoys sports and plays Tennis. She also loves reading detective and mystery novels and volunteers at her local church working with the teens.

Matthew Wilson, A360 Project Director, PSI
Matthew is the Project Director for Adolescents 360, PSI’s flagship program for adolescent sexual and reproductive health and rights. Prior to joining the A360 team in late 2018, Matthew was a senior technical advisor for PSI’s sexual and reproductive health department. Before PSI, Matthew worked at Marie Stopes International (MSI) for eight years and in the child protection sector for five years. He has held a variety of field and HQ roles ranging from Country Director to Social Franchise Channel Director. Throughout his career in international development, he has managed grants and performance based contracts with budgets in excess of £90 million, working with donors such as DFID, USAID, EuropeAid, Unicef, UNHCR, Comic Relief and the Big Lottery Fund. Matthew holds a law degree from Durham University, a postgraduate diploma in management from Oxford University and a Masters degree in Development Management from the Open University.

Divya Datta, Director of Health Innovation, Vihara Innovation Network & Co-Chair of the Measurement & Evaluation Working Group at HCDExchange
Divya has more than a decade’s experience in investigating the human experience of disease, vulnerability, health and wellness as well as designing proven human centered solutions that help achieve better health in life stages of infancy, adolescence, motherhood and parenting. A specialist in human centered design and with work across 7 countries in the Global South, she is currently exploring last mile collaboration between system functionaries and citizens through participatory public health innovations to drive intractable health outcomes. Through her work approaches she intends to systematically dismantle dysfunctional, unfair and self-perpetuating structures of power within public health systems and place power to determine priorities and interventions to individuals and communities impacted by solutions.

Hanieh Khosroshahi, Senior Product Designer, YLabs
Hanieh is a UX researcher and designer who uses creative problem solving, storytelling, and systems thinking to design products and services that are innovative, impactful, and accessible. With a background in the arts and journalism, and a MSc in Human-Computer Interaction from the University of York in England, she approaches complex social challenges through an inter-disciplinary and equity-focused lens. She is also a UX instructor at Juno College of Technology and a design mentor at Springboard. She runs on coffee, enjoys climbing peaks, and reading a good book.

Laetitia Kayitesi, Research Manager, YLabs
Laetitia is a public health professional with experience in research and monitoring and evaluation of programs. She is passionate about using research and data to increase access and quality of youth sexual and reproductive health services and improve overall adolescent health. She holds a MSc in Global Health and Management. In her free time, she can be found watching sports, travelling, or spending quality time with her family and friends.

Thanks to everyone who came out and joined our event on Wednesday!

We’ll be sharing the recording in this channel, and a summary of the highlights from the discussion in the coming few days.

In the meantime, here are the speaker notes from the call in case you wanted to scroll through!

As promised, here is the recording for the webinar. Please feel free to watch if you missed it, or fast-forward to re-watch your favourite part if you attended.

We’re also written a summary piece to highlight the key points from the speakers. You can read it here!

Enjoy!

Key takeaways from our webinar on roadblocks in evaluating adaptive programs

Thank you for joining our webinar on designing evaluations for adaptive programs. We had an information- packed discussion with experts from PSI’s Adolescent 360 (A360), YLabs’s CyberRwanda and Vihara, where they shared their perspectives on how traditional program evaluation and measurement strategies can get turned on their head when working in a program that is influenced by Human-centered design (HCD) or other adaptive implementation approaches.

Here is a summary of key takeaways:

1. Key definitions, a shared understanding and more common language is an ongoing need: Human-centred design and adaptive implementation, and how they fit together. Click here for more information

2. Keeping the ‘essence’ of the intervention in mind: Through finding the balance between adaptation and staying true to the ‘essence’ of the intervention, drifting can be avoided, despite constant iteration. This requires a shift in the mindset of implementers, becoming systematic, and an investment in capabilities.

3. The qualities and characteristics of design-led thinking are rarely prioritized or translated into indicators for measurement: despite design-led thinking having emerged as critical to achieving service engagement and sustained uptake contributing to the achievement of public health goals.

4. Documenting the iterative processes, decision-making, testing along with way is often absent or can be incompatible with an acceptable or traditional measurement metric. Documenting change and adaptation made is important to be able to account for the change during the evaluation and data analysis process.

5. Differing learning priorities, styles of different disciplines and limited common language can be challenging: historically designers and design teams have had limited understanding and no incorporation of measurement priorities in the design process.

6. Programs need to promote collaboration between design and research teams so as to find the right balance . Such collaboration will help project teams to identify and align on key research objectives, using both quantitative and qualitative data, to measure the impact of the project and identify the current challenges.

7. Programs need to move toward agile, adaptive and utility focused evaluations. One can only measure what one designs for. As such grant structures and resources are needed to enable designers to work with program teams in shaping monitoring and evaluation. This will create more emphasis on evaluating how well a programme learns and adapts.

Read a short summary

1 Like