UC Davis - SurveyCTO

SEEDS OF CHANGE

Impact Evaluation in Rural Kenya

CHALLENGE

Can farmers in Kenya increase their family income by switching to hybrid maize seeds? And are there financial and soil quality constraints that prevent them from doing so? These are the questions Emilia Tjernstrom and her team from the BASIS AMA Innovation Lab (hosted at the University of California Davis) set out to answer. To answer these questions, the team needed to conduct a sophisticated, multi-year agricultural study. And they needed to use computer assisted personal interview (CAPI) software that was up to the task. Specifically, the team sought a software platform that would:

Allow them to build and code a complex survey with ease.
Give them direct control over data collection.
Deliver robust functionality at a reasonable price.
Ensure quality control in a field setting.
Be flexible enough to support a range of possible uses.
Run on portable devices.
Keep data safe and secure.

SOLUTION

After evaluating the options, the team chose SurveyCTO. What cemented the decision? As Emilia explains, “It was clear that the SurveyCTO team had thought carefully about the practical realities of field surveys and what is actually needed on the ground.”

Here are some of the ways the impact evaluation team put SurveyCTO into action:

Developed a sophisticated survey – that was administered by 20 enumerators, across 1,800 farms in 108 villages.
Conducted the survey on Android tablets – which, compared with netbooks or laptops, reduced the barrier between enumerators and respondents and improved survey speed and flow.
Eliminated repeat questions – by setting up SurveyCTO to dynamically tally responses and govern survey logic. For example, if respondents mentioned they used hybrid seeds when talking about hybrid crops, later questions about hybrid seeds were automatically bypassed. Says Emilia, “People get annoyed if you ask them the same question again and again. We were able to build the survey to automatically adapt to respondents’ previous answers.”
Implemented auditing capabilities – “This feature jumped out at me right away,” says Emilia. “Luckily, the enumerators for this survey were really spectacular, but the auditing features SurveyCTO offers will be extremely valuable.” The team demonstrated SurveyCTO’s audio auditing features during training – letting enumerators know that they may be recorded at random for quality control purposes.
Created and linked household rosters – During the early part of the survey, enumerators used SurveyCTO to create rosters of household members. When asking about agricultural decision makers later in the survey, researchers were able to connect responses to the preexisting roster, saving time and data entry. “That was something our enumerators liked a lot,” says Emilia.

BENEFIT

With SurveyCTO in place, the BASIS AMA Innovation Lab has gained a host of benefits. The team is now able to:

Work faster. SurveyCTO reduces time and hassles. The team built and administered their survey in much less time than it would have taken using paper-and-pencil methods. In addition, as enumerators got more practice with the system, they cut survey administration time from three hours to two.
Boost quality and control. With SurveyCTO’s auditing capabilities, researchers gain a “fly on the wall” perspective that they can use to recalibrate surveys, correct errors, and improve data quality.
Improve efficiencies. For phone follow-up surveys, the team preloaded their surveys with household roster, agricultural decision-maker, and other details from the baseline survey, reducing follow-up interview time dramatically.
Get even more sophisticated. Thanks to SurveyCTO’s ability to adapt, the team is using the platform in new and innovative ways, including for building experimental games. They’re also collecting social network information more accurately and efficiently, by using participant photos to allow respondents to simply click on pictures rather than hunt for names in lists.

But perhaps the biggest benefit is the ongoing support that the team gets from SurveyCTO’s customer support team. “One of the reasons I recommend SurveyCTO is because, when there were issues, the support team was quick to respond. Not just respond with an answer, but by immediately getting to work. To me, that’s the mark of a good product.”

TIPS FROM THE FIELD

By building sophisticated agricultural surveys in SurveyCTO, Emilia has learned a lot. Some tips she would like to share:

Choose your multiple-choice value codes carefully. “For example,” says Emilia, “It’s easy to hinge survey logic on the crop code being greater than 102 and less than 118 vs. having to check for crop code 102, 105, 113, etc. individually.” Download sample…
Pre-load your multiple-choice options. “Rather than having thousands of multiple-choice options configured in the survey itself, load options dynamically from pre-loaded .csv files,” says Emilia. That significantly improves the survey’s performance in the field. Download sample…
Leverage your household roster. Ask basic information about all household members to create a roster early on in the survey. Then, when you ask subsequent questions about decision-making, you’ll have a ready-to-go list of names to choose from. “That makes it much easier to have the code link up with a person from the household roster,” explains Emilia. Download sample…
Use dynamic logic to avoid repetition. Have repeated questions dynamically tallied so that they can govern later logic in the survey. Says Emilia, “You eliminate duplicative questions by setting up the survey to adapt automatically, based on what the respondent has already said.” Download sample…

^* Photo credit: Emilia Tjernstrom