By Ricardo Romero (Program Impact Officer)
In addition to being a beloved alternative rock and roll band, REM (Ripple Effect Mapping) is a relatively new qualitative technique that we’re using to help understand and systematize the intended and unintended effects of our specific agricultural extension program (i.e., the ripples).
I know this may sound overly technical, but stay with me.
THE GREAT BEYOND
Sustainable Harvest International has been around since 1997 and we’ve done a lot of research to learn about the impacts of our work, but we wanted to find a way to dig deeper—to learn about how our program is perceived, both by those who have participated in it, as well as those who knew about it but didn’t participate.
Don’t get me wrong, quantitative data is great—we spend hours collecting it year round and rely heavily on it for reporting. Nevertheless, relying on quantitative data alone can fall short of giving a full, comprehensive sense of our impact.
As Abner Mendoza, our excellent EARTH University intern says, “If you only look at quantitative data, you are only looking at the results of the project. In the case that a project fails, quantitative data doesn’t really tell you why. If you don’t understand why a project failed, you can wrongly assume that everything within the project was wrong.”
Similarly, quantitative data alone can tell you that a project succeeded but not why.
Based on our eighteen years as a nonprofit, the quantitative data, and our observations, we know that our program is successful. But we wanted to know more.
Enter REM, the qualitative side of the equation.
An empty classroom in Tranquilla, Panama, awaits REM study participants - photo by Bailey McWilliams
WHAT’S THE FREQUENCY, KENNETH?
REM was developed by Scott Chazdon (Evaluation and Research Specialist) and a team of researchers at the University of Minnesota. It was appealing to us because it was low cost, not time consuming, and could help us detect both intended and unintended effects of our program.
When we originally got in touch with Scott and the REM researchers, the idea was met with some skepticism. We planned to adapt the tool to a rural community context with little access to electricity and with lower literacy rates than the researchers were used to. That is to say, we faced some important challenges.
Nevertheless, under the leadership of Scott and in partnership with Charlie French (our board member and Associate Extension Professor and Program Leader for Community and Economic Development) from the University of New Hampshire, we adapted the methodology to fit our context. It took three months of Skype calls, emails, and studying REM literature in depth, but eventually we were ready to pilot REM studies in two communities in Panama.
MAN ON THE MOON
As Program Impact Officer, a key part of my job is acting as a bridge between different worlds. This particular project was no exception. Conducting this project in Panama required attention to some special, contrasting details. To pull this study off, I had to ensure that two worlds (that of a Minnesota professor and that of rural Panamanian farming communities) could come together seamlessly.
First, I had to create a space for the Panamanian participants that was unlike anything they are accustomed to—an informal space with coffee and snacks where people are invited to speak openly about their feelings and experiences (both positive and negative) without fear of being labeled a complainer or ungrateful.
Participants in Tranquilla stretch and share their experiences - photo by Bailey McWilliams
At the same time, I had to brief a professor from Minnesota (Scott) for his first trip to Panama, suggesting that he avoid arriving in Indiana Jones attire but still pack comfortable clothes and shoes.
(Side note: When we got caught by heavy rain during a long walk in a national park the day after he arrived, I suspect Scott may have questioned why he had paid any attention to me whatsoever, as the Indiana Jones gear might have come in handy in those conditions!)
COUNTRY FEEDBACK
We performed the REM study in two graduate communities in Panama: El Cocal and Tranquilla. The idea was to have 10-12 people from families who participated in our program and an equal number of people who hadn’t participated were aware of the program.
Participants and non-participants paired up in teams and interviewed each other based on questions developed through the REM methodology. They then recorded their answers on post-it notes. In the case of illiterate participants, the Panama staff and EARTH University intern Abner were on hand to help with recording answers.
Each group then reported each other’s answers in an open discussion. During the course of these discussions they tried to identify different ripple effects (both positive and negative) of our program. (For a full explanation of REM methodology, see here.)
Mind mapping the effects of our agricultural extension program - photo by Bailey McWilliams
SHINY HAPPY PEOPLE
Some of the positive ripples reported in both El Cocal and Tranquilla were:
- Personal empowerment (e.g. having a voice, knowing how to organize people, and engage others in the community/region/country around key issues)
- Self-sufficiency (with respect to food security, economic situation, etc.)
- Skills gained and used to plant food in a sustainable manner that increases access to healthy fruits and vegetables while also sustaining ecosystem functions
- Community collaboration (program participants learned to work together leverage each other’s skills, increase efficiency, learn from each other, and share knowledge and resources with neighbors and others in the community)
Maruquel Sánchez shares a laugh with fellow participants - photo by Bailey McWilliams
YET…EVERYBODY HURTS (SOMETIMES)
The biggest challenge identified in Tranquilla was that of climate change. Specifically, more intense periods of rain and prolonged droughts were perceived as threats to their way of life. Though participants recognize the importance of protecting forests and other key ecosystem functions, they felt that they had little control over these functions and that they were still vulnerable.
One challenge that was identified in El Cocal is the lack of knowledge and skills to preserve fruits and vegetables, as well as convert them into value added products that can earn income (e.g. hot pepper salsa, fruit marmalades, etc.).
FIND THE RIVER
Everybody hurts sometimes...but hold on!
Carrying out these REM studies helped us to reflect on the way we implement our methodology and how we can further calibrate it. It helped us to understand both the intended and unintended “ripples” of our work. We’re able to look at these ripples and think about how we can improve our program. We can’t promise that no one will ever hurt but we can continue to partner with families to make sure they have the skills they need to carry on and be successful.
REM participants share their experiences - photo by Bailey McWilliams
A task force composed of staff and board members is working to improve the way we implement, monitor, and evaluate our methodology, both with participants who are "in” the program now and those who have already graduated. After completing these pilot studies it appears that REM has a lot of potential to become a standard monitoring and evaluation tool across all our programs—particularly with those in the post-graduate phase.
A full description of the methodology and results of these two pilot studies will be included in two different book chapters (one book by CATIE, Kansas State University, and Montana State University, to be published by Routledge, the second coordinated by the University of Minnesota and to be published by Digital Commons). Both will be released later this year—we’ll keep you updated on developments.
IT’S (NOT) THE END OF THE WORLD AS WE KNOW IT
So, how did these two worlds come together?
People in both El Cocal and Tranquilla liked the idea of being heard. They were enthusiastic about sharing what we could improve on and the positive aspects their participation in our program brought to their lives. Their sense of pride and achievement were palpable in the meeting atmosphere.
We all had a great time with a lot of sincere talk and meaningful feedback. Scott and Charlie surprised us with some good spoken Spanish (though their Spanish writing skills still need some improvement—let’s keep the sincere feedback alive here). We all had a lot of fresh fruit juice, rice pudding, and other local delicacies.
I enjoyed facilitating both meetings. We laughed, we ate, and we worked. What else could you want to have as part of your workday? The two worlds came together well—systematic evaluation of the program and the pride of those helping to develop our program from the inside out.
Scott (lead REM researcher/developer) and Charlie (UNH) enjoy some ice cream after a satisfying day of REM'ing - photo by Bailey McWilliams
Lastly, though Charlie loves Panama, Panamanian germs don’t really like him. He had a fever when he returned to New Hampshire. Despite the fever, I know that he would choose to REM again anytime. So would Scott. So would program participants. So would we! (And don’t worry—Charlie’s fever didn’t last long and he’s back to normal.)
The REM pilot project was made possible by a grant from the Bay and Paul Foundations, Inc.
All references to REM (the band) woefully inserted by Michele Christle, Communications Officer. Ricardo Romero, Program Impact Officer had nothing to do with it.
