When you outsource data annotation work, annotation quality directly impacts your model's performance—yet revision policies vary wildly between vendors, leaving you vulnerable to hidden costs and delivery delays. Understanding what guarantees matter before you sign prevents painful rework cycles and ensures your training datasets stay on schedule and within budget. This guide breaks down the revision commitments worth demanding.
Why Revision Policies Matter More Than You Think
Data annotation isn't a one-shot deliverable. Labeling mistakes compound through your pipeline: a mislabeled bounding box in 500 images means retraining your object detection model weeks later. Most annotation vendors offer some form of rework, but the scope, timeline, and conditions attached to those guarantees differ drastically. A vendor offering "unlimited revisions" with a 60-day turnaround isn't the same as one guaranteeing 48-hour corrections on critical labeling errors.
Your revision policy directly affects downstream costs. If rework requests drag on or carry per-revision fees, a 5% error rate suddenly costs 15–20% more than advertised.
What to Look For in a Revision Guarantee
Error threshold and scope
Ask vendors to define which errors trigger rework obligations. Legitimate categories include:
- Misclassification (label applied to wrong class)
- Incomplete annotations (missing objects in images)
- Bounding box misalignment exceeding agreed tolerance (e.g., IoU < 0.85)
- Polygon or keypoint coordinate errors beyond specified pixel ranges
Red flag: vendors who exclude "subjective interpretation" errors without a clear definition. Ambiguous labels do exist, but the threshold should be documented before work begins.
Rework windows and turnaround
Standard practice is 10–14 calendar days for correction batches under 1,000 items. For mission-critical datasets, negotiate faster windows—some vendors offer 3–5 day turnarounds for expedited correction at a modest premium (typically 10–15% of the base annotation cost). Confirm whether the rework timeline resets if corrections fail QA a second time.
Quality acceptance criteria
Insist on written precision/recall thresholds or accuracy targets tied to rework obligations. Common benchmarks:
- 98–99% accuracy for classification tasks
- 95–97% accuracy for bounding box annotation
- 90–95% for segmentation masks (varies by use case)
If your vendor can't commit to specific metrics, they're deferring responsibility. Request sample QA reports from previous projects to see how they measure compliance.
Cost structure for revisions
Most reputable vendors include one revision cycle at no charge for errors attributable to their team. Beyond that, costs vary:
- Per-image rework: $0.02–$0.10 per corrected item
- Batch rework: 20–40% of the original batch cost
- Unlimited revisions (first 30 days): included in base price, then per-image rates apply
Avoid vendors with vague "rework available upon request" language—you need a price list before problems occur.
Practical Checkpoints Before Signing
1. Request a pilot batch
Annotate 100–500 items first. Measure actual accuracy against your ground truth, then assess how long true rework takes. This reveals whether vendor timelines hold and whether their quality baseline matches your standards.
2. Define edge cases in writing
Annotation ambiguity happens. Before full-scale work, document 10–15 tricky examples and get vendor sign-off on correct handling. This prevents disputes over whether revision requests are legitimate.
3. Establish escalation paths
If a rework request isn't addressed within the agreed window, who do you contact? A named project manager beats generic support queues by weeks. Some vendors include escalation-to-leadership clauses for delayed corrections—request this for datasets over $10,000.
4. Check their QA process
Ask vendors how they validate annotations before delivery. Multi-pass QA (initial annotation → internal review → final check) reduces your rework burden. Some use AI pre-screening to catch obvious errors, then humans validate borderline cases—this hybrid approach often yields better accuracy per dollar spent.
Bundling Revisions Into Your RFP
When soliciting quotes, include revision expectations explicitly:
- "Accuracy target: 97% for bounding boxes, measured via IOU ≥ 0.85"
- "Included revisions: one full correction cycle for vendor-attributable errors within 10 business days"
- "Rework beyond first cycle: $0.05 per image, 5-day turnaround"
This prevents surprise invoices and ensures apples-to-apples pricing comparisons. Platforms like Mercoly help you compare multiple annotation providers' revision policies side-by-side, saving you the legwork of individual negotiations.
Frequently Asked Questions
Q: What's the difference between "unlimited revisions" and a standard rework guarantee? Unlimited revisions usually mean any number of correction rounds within a defined period (typically 30 days) at no extra cost, while standard guarantees cover one revision cycle covering vendor errors only.
Q: Should I pay extra for faster rework turnaround? Yes, if it's mission-critical. Expedited rework (3–5 days vs. 10–14) typically costs 10–15% more but can be worth it for time-sensitive model training.
Q: How do I prove an error was vendor-caused vs. ambiguous instruction? Request a shared annotation guide or style document signed before work begins, then reference specific violations; video walkthroughs with your vendor before scaling also help prevent disputes.
Start comparing annotation vendors with clear revision expectations in mind—it'll save you thousands in unexpected rework costs.