Bradlie Nabours, MPH, CPH
Sometimes evaluation capacity building looks like revising a survey so two copies fit on one sheet of paper.
Ahead of a community event, an organization I was working with raised concerns about whether its survey would work for the activities planned and the people attending. We worked through separate adult and youth versions. I preserved key questions in the adult survey so the responses could still contribute to the broader festival evaluation, and formatted the youth survey as a half-page to save paper. The conversation included everything from what to ask to how people would actually complete it.
That may not sound like the most exciting part of evaluation. But it reflects something I have come to appreciate through this work: a method is only useful if people can use it.
As Research and Evaluation Director at The Hypatia Collaborative, I work within a shared services model that connects nonprofits with support across evaluation, program management, technology, financial management, and other organizational needs. We start with an assessment of each organization rather than assuming everyone needs the same services.
Working in that setting has shaped how I think about evaluation capacity building. The work is not just helping an organization measure its programs. It is figuring out how evaluation can become a useful, manageable part of running the organization.
And one of the biggest lessons has been this: you cannot build evaluation capacity by simply giving an organization more evaluation to do.
Priorities are everything
One of the clearest lessons from this work was that evaluation needs looked different from one organization to the next. Traditional evaluation was not always the most useful or immediate place to start. For some organizations, the bigger need was simply having better systems for tracking participants, organizing information, or understanding what data they were already collecting. Those needs may seem more operational than evaluative, but they are often what make meaningful evaluation possible later. In practice, building evaluation capacity sometimes meant starting with the basics and helping organizations strengthen the systems that could support better learning and decision-making over time.
That distinction matters because a nonprofit leader may not come to us asking for “evaluation capacity building.” They may be trying to make monthly reporting less time-consuming, understand who is participating in a program, or pull together the information they need for a grant application. Those are not separate from evaluation. They are often the most practical place to begin.
In one recent client engagement, the initial goal was to make monthly reporting easier and less manual while also capturing information more consistently for grants. I started by reviewing the organization’s existing forms, trackers, reporting templates, and funder requirements. As I worked through those materials, I noticed that the intake form and reporting forms did not appear to capture the same information. Before recommending a solution, I first needed to understand where the reporting data was actually coming from.
That is not a problem another survey will automatically solve. It requires understanding the workflow: what information is collected, who records it, where it goes, and how it eventually becomes something the organization can use. It also means separating the information needed for routine reporting from the questions that can help an organization understand whether a program is working.
For me, this is an important part of the job. We should be able to connect evaluation to something the organization already cares about, not just explain why evaluation matters in general.
Start with what is already there
I do not think “building capacity” should imply that we are starting with nothing.
In a follow-up with another organization, I asked for existing surveys, registration and intake forms, participant homework and reflection activities, and qualitative material such as stories and examples of change. The starting point was to look at what was already working, then consider how those materials could better show what participants were learning, retaining, or gaining. The proposed next step was a simple evaluation plan that could support both internal use and future grants.
That is a different conversation from arriving with a standard package of tools.
A reflection activity may already offer insight into what participants are learning. A registration form may help describe who the program is reaching. Staff observations may point to an important question that a survey has missed.
None of that means every existing form is useful or every positive story demonstrates effectiveness. We still need to ask what the information can tell us, whose experiences are missing, and whether it is being collected consistently.
But reviewing what already exists gives us somewhere concrete to begin. It also respects the time an organization has already invested.
The goal should be to strengthen useful practices and address meaningful gaps, not replace everything simply because we would have designed it differently.
Adapt the method without losing the purpose
The community-event survey is a good example of the balance this work requires.
The organization brought knowledge of the event, its participants, and what would be manageable on the day. My contribution was to help adapt the tools while retaining questions needed for the larger evaluation. Neither perspective was sufficient on its own.
That is what working together should look like.
It does not mean abandoning rigor whenever something is inconvenient. It means being clear about what we need to learn and finding a credible way to learn it within the setting.
The same issue comes up with accessibility. In an evaluation plan for an arts organization serving adults with disabilities, the proposed methods included picture-based surveys, supported conversations, audio or video responses, and art-based activities. The plan also called for an accessible summary of findings for the artists themselves.
Those choices are part of the evaluation design, not extras to consider after the tool is finished.
If someone cannot meaningfully respond to our questions, we need to examine the method before concluding that they have nothing to contribute. And if participants help us understand the program, we should think about how they will receive the findings, not only how we will present them to a funder.
I still want clear questions, consistent documentation, and honest interpretation. But those standards do not require every organization to use the same format.
A completed evaluation is not the same as evaluation capacity
This is a distinction I think evaluators, including me, need to keep making.
After the festival, the work continued beyond designing the surveys. There were paper responses to gather alongside the online responses, follow-up conversations to arrange, and data to bring together. I eventually sent the organization both the evaluation report and the underlying data files.
That is useful evaluation support. But delivering a report does not, by itself, tell us whether an organization has greater capacity to evaluate its work.
For that, I think we need to ask different questions.
Does someone understand why the questions were selected? Can they find and update the information? Do they know what the findings support, and what they do not? Is there a realistic plan for reviewing the data and deciding what to do next?
These are the questions that move us from producing something for an organization toward building something with it.
They also keep us honest about our own work. A well-designed tool is a deliverable. Its continued, appropriate use is something we still need to support and assess.
At the same time, I do not think sustainability has to mean doing everything without outside help. In a shared services model, access to specialized support is part of the approach. A small nonprofit should not have to develop an entire evaluation department to ask good questions and make informed decisions. Hypatia’s model coordinates services around organizational needs rather than expecting every nonprofit to maintain all that expertise internally.
The goal, as I see it, is for organizations to understand and have ownership over their evaluation work, including knowing when additional expertise would help.
Fund the conditions that make evaluation possible
This also has implications for funders.
When we ask an organization to collect better data, we should be willing to support the work behind that request. That includes staff time, appropriate technology, practical guidance, and time to review what was collected.
It should also include room to adjust.
An evaluation plan may need to change after a conversation with program staff or after testing a tool with participants. That should not automatically be treated as a failure to follow the plan. Sometimes it is evidence that people are paying attention.
I want funders to ask what an organization needs to learn and what support would make that learning possible, not only what numbers it can submit at the end of a grant.
And I want us, as evaluators, to apply that same standard to ourselves.
The most useful contribution is not always a more sophisticated method. Sometimes it is helping connect an intake process to a reporting need. Sometimes it is strengthening a reflection activity that already exists. Sometimes it is revising a survey with the people who know the setting best.
Those steps are not the whole of evaluation capacity building. But they can be where it begins.
