When you're hiring an impact consultant, their portfolio tells you far more than a polished pitch deck ever will. The quality, depth, and relevance of their past work—and how they explain it—separates consultants who understand rigorous evaluation from those who've merely read the literature.
Start by Requesting Real Examples, Not Case Study Brochures
Ask for 3–5 concrete projects where they've designed or managed impact measurement systems. These should include the raw outputs: logic models, evaluation frameworks, data collection tools, or sample dashboards they've actually built. Generic case studies with vague metrics ("improved outcomes by 30%") without showing methodology are red flags.
Look specifically for work within your sector or a closely related one. A consultant who designed evaluation systems for youth employment programs can transfer those skills to workforce development, but asking them to suddenly evaluate environmental conservation requires new expertise they may not have demonstrably practiced.
Examine the Methodology Behind the Numbers
The sample work should clearly show their logic. Dig into:
- How did they define success? Are their outcome metrics SMART (Specific, Measurable, Achievable, Relevant, Time-bound), or are they wishy-washy? "Improved well-being" isn't a metric; "80% of participants report increased financial stability within 12 months" is.
- What data did they actually collect? Did they use surveys, interviews, administrative records, or randomized comparisons? Understand their reasoning for each choice—cost constraints, population size, and timeline all matter.
- Did they address attribution? Strong evaluators acknowledge whether they can claim the program caused the change or only correlate with it. Weak ones ignore confounding factors entirely.
Look for evidence they've grappled with real constraints: limited budgets, hard-to-reach populations, or missing baseline data. Consultants who've solved these problems messily are often more useful than those with pristine projects under ideal conditions.
Check for Actionability and Learning
The best sample work isn't just a final report—it's evidence of a consultant who helped clients use the findings. Review whether their deliverables included:
- Simplified dashboards or scorecards designed for board-level decision-making
- Specific recommendations, not just observations ("Consider testing a shorter intake form to improve completion rates" versus "Engagement could be improved")
- Breakdowns by participant segment so leadership could see where programs worked best
- Clear timelines showing how findings were shared during the evaluation, not only in a final report months later
If their samples are dense academic reports that sit on shelves, that's a telling pattern.
Assess Their Evaluation Sophistication Level
Different projects require different evaluation rigor. Be realistic about what you need:
Light touch ($5K–$15K): Annual surveys, basic data cleanup, simple trend reporting. Look for samples showing they can deliver focused insights without overcomplicating things.
Moderate depth ($15K–$50K): Multi-year tracking, comparison groups, mixed-methods (surveys plus interviews), dashboard development. Samples should show experience with longitudinal data management and synthesis across quantitative and qualitative sources.
Research-grade ($50K+): Randomized controlled trials, sophisticated statistical modeling, peer-reviewed publication. Only pursue this if funders actually require it or your program is at scale.
Their portfolio should roughly match your budget level. Consultants comfortable doing $500K evaluations may treat your $20K project as tedious; those exclusively doing boutique work may lack systems thinking for larger initiatives.
Ask About Tools and Replicability
Modern impact consultants use platforms like SurveySparrow, Tableaus, or Salesforce for data collection and analysis. Look at their samples: Do dashboards auto-update? Are instruments stored in a system that survives staff turnover? Did they hand off clean, documented datasets and codebooks so you (or another consultant) could analyze further?
Avoid anyone whose best work would vanish if they left—that's a sign they built bespoke solutions without thinking about your long-term capacity.
Talk to Their References About Actual Implementation
After reviewing samples, call 2–3 past clients and ask: Did the consultant deliver on time? Did they respond to pushback on findings? When your messy real-world data showed something unexpected, could they adapt? How much hand-holding did you need?
Samples look polished; references reveal whether the consultant can handle chaos.
Frequently Asked Questions
Q: How recent should sample work be to be relevant? Projects from the past 2–3 years are ideal since evaluation standards, data tools, and funder expectations shift regularly; anything older than 5 years should raise questions about whether they've kept current.
Q: What's a reasonable timeline for a consultant to deliver an evaluation design? A solid evaluation framework typically takes 6–10 weeks for a mid-sized program, accounting for stakeholder interviews, literature review, and refinement; anything promised in 2 weeks suggests they're copying templates rather than customizing.
Q: Should I prioritize a consultant with evaluations published in peer-reviewed journals? Not necessarily—academic publication takes time and funding; strong consultants who've published alongside strong ones who haven't both exist, so judge the rigor of their methods, not their CV.
Find the right impact measurement partner by comparing real credentials and sample work side by side—Mercoly helps you browse and evaluate trusted consultants in this field all at once, saving you the legwork of background research.