Batch Failure Handling & Constraints

Batch processing introduces unique failure modes that require specific handling strategies.

Batch Failure Handling

When a batch completes, some requests may fail while others succeed. Handle failures by:

1. Identify failures via custom_id: Each request has a unique custom_id that correlates to the response
2. Resubmit only failed documents: Don't reprocess the entire batch
3. Modify failing requests: Chunk documents that exceeded context limits, simplify prompts that hit edge cases
4. Track failure patterns: If many documents fail for the same reason, fix the root cause before resubmitting

Critical Constraint: No Multi-Turn Tool Calling

The batch API does not support multi-turn tool calling within a single request. Since processing is asynchronous, there is no mechanism to:

• Intercept a tool call mid-request
• Execute the tool
• Return results for Claude to continue

This fundamentally breaks iterative tool-calling workflows. Batch is single-turn only.

Sample-First Strategy

Before batch-processing large volumes:

1. Run the prompt on a representative sample (50-100 documents)
2. Analyze success rates and failure modes
3. Refine the prompt based on failures
4. Only then submit the full batch

This maximizes first-pass success rates and avoids expensive iterative resubmission.