Hiring an impact evaluation firm is one of the most critical decisions a nonprofit or social enterprise can make—get it wrong, and you're flying blind on the outcomes that matter most. The right firm can transform scattered data into credible, actionable insights; the wrong one wastes months and budget on framework misalignment or unusable findings. Before you sign a contract, you need to ask the hard questions about experience that separate seasoned practitioners from well-intentioned newcomers.
Why Experience Matters in Impact Evaluation
Impact evaluation isn't a commodity service. A firm that excels at outcome measurement for K-12 education may stumble with financial inclusion programs. One strong in randomized controlled trials (RCTs) might lack expertise in qualitative data synthesis or real-time monitoring systems. The stakes are higher than traditional consulting: your funder's trust, your board's confidence in strategy, and potentially millions in future funding decisions rest on evaluation credibility.
Experience isn't just about years in business—it's about depth in your sector, with your type of program, using methodologies that match your constraints and resources.
Questions About Sector and Program Expertise
Start by asking: Has your firm evaluated programs similar to ours in scope, beneficiary population, and theory of change? A one-sentence answer won't cut it. Push for specifics: How many education access initiatives have they evaluated? Were they rural, urban, or both? What was the typical program budget and timeline?
Request a list of 3–5 comparable projects and ask for permission to speak directly with past clients. Listen for whether they describe the firm as adaptable and sector-aware, or whether they felt their program was shoehorned into a generic evaluation framework.
Also ask about geographic and cultural context. An evaluation firm experienced in high-income country contexts may lack the methodological flexibility needed for low-resource settings where data collection is harder, baseline assumptions differ, and stakeholder engagement looks completely different. If you work in sub-Saharan Africa or rural Southeast Asia, find a firm with hands-on experience there.
Methodology and Technical Depth
Question the firm directly: What methodological approaches have you used in the last three years, and which are your core strengths? Listen for a nuanced answer. Strong firms can discuss trade-offs—when RCTs make sense versus when quasi-experimental or participatory evaluation methods are more appropriate. They should be able to explain why they'd not recommend an approach, not just what they offer.
Ask about their experience with:
- Randomized and quasi-experimental designs (if causality is critical for your program)
- Real-time monitoring and adaptive management (if you need mid-course feedback, not just end-of-year reports)
- Participatory and mixed-methods evaluation (if beneficiary voice and qualitative insight drive your strategy)
- Cost-effectiveness and return-on-investment analysis (if you need to justify program efficiency to funders)
- Data visualization and storytelling (if evaluation findings need to resonate with boards and donors, not just researchers)
Red flag: A firm that claims equal mastery across all methods. Reality is narrower—and that's fine if they're transparent about their niche.
Questions About Team and Capacity
Who will actually conduct the evaluation, and what are their credentials? This matters enormously. A firm's senior leadership may have PhDs and stellar reputations, but if a junior analyst with minimal field experience manages your day-to-day work, quality suffers. Ask about:
- The lead evaluator's specific experience with your program type
- How many concurrent projects the team carries (overcommitted firms produce weaker work)
- Whether they have in-country or in-region staff if you work globally
- Their turnover rate in the last two years
A trustworthy firm will give you a named team and explain their bench depth. If they hedge or offer vague answers, that's a signal to dig deeper.
Understanding Costs and Timeline
Impact evaluation budgets range wildly. A basic outcome survey for a mid-sized nonprofit might run $15,000–$40,000. A rigorous quasi-experimental evaluation with 18 months of data collection could cost $150,000–$400,000+. Costs scale with:
- Sample size and geographic spread
- Depth of data collection (survey only vs. interviews, focus groups, observational data)
- Methodological rigor (basic reporting vs. experimental design)
- Timeline (compressed timelines cost more)
Ask the firm to itemize costs by component and explain what each price tier includes. Vague, all-in quotes often hide scope creep later.
Frequently Asked Questions
Q: Should I always hire the biggest, most prestigious firm? No. Larger firms excel at complex, multi-country evaluations but may deprioritize smaller programs; a mid-sized or boutique firm with deep sector expertise often delivers more attentive service and better understanding of your constraints.
Q: How long does a typical impact evaluation take? Most rigorous evaluations span 12–24 months minimum (baseline, implementation period, endline), though rapid assessments or monitoring-focused work can move faster; timeline depends heavily on your program's operational timeline and the rigor level you need.
Q: Can I compare firms based on price alone? Absolutely not. Two firms quoting different prices may be proposing vastly different methodologies, team experience, or rigor levels; always compare scope, methodology, and team credentials alongside cost.
Find and compare vetted impact evaluation firms that match your needs—Mercoly makes it easy to filter by sector experience, methodology, and budget in one place.