The Message Batches API, Realtime vs. Batch & Third-Party Vendors

The Message Batches API

For large, latency-tolerant workloads, submit many requests as a single asynchronous batch. Anthropic processes the batch within a 24-hour window at a substantial per-token discount (roughly half the standard rate). This is the correct choice for overnight bulk jobs where cost matters more than immediacy -- not a matter of running the same synchronous calls faster in parallel.

The Exact Mechanics and Limits

A single batch call accepts up to 100,000 requests or 256 MB, whichever limit is hit first. Submitting the batch returns a batch_id immediately -- the batch itself has not been processed yet, only accepted. Your application then polls using that ID until the batch reports completion, at which point you download the results.

batch = client.messages.batches.create(requests=[
    {"custom_id": "ticket-001", "params": {"model": "claude-sonnet-4-5", "max_tokens": 256,
                                            "messages": [{"role": "user", "content": ticket_1}]}},
    {"custom_id": "ticket-002", "params": {"model": "claude-sonnet-4-5", "max_tokens": 256,
                                            "messages": [{"role": "user", "content": ticket_2}]}},
    # ... up to 100,000 requests, or 256 MB total, whichever comes first
])
print(batch.id)  # poll client.messages.batches.retrieve(batch.id) until processing ends

The order results come back in has no relationship to the order you submitted them. That's not a bug you're working around -- it's how the API is documented to behave. Because of that, every request in the batch needs its own custom_id, since that's the one dependable way to tie a given result back to the request that generated it:

results_by_id = {}
for result in client.messages.batches.results(batch.id):
    results_by_id[result.custom_id] = result  # never assume result order == submission order

Any code that pairs results with the original request list purely by index, assuming the two lists line up, will silently mismatch inputs and outputs the instant the API returns results in a different order than submitted -- which it's entitled to do on any given batch.

Realtime vs. Batch -- The Decision

The decision follows directly from the requirement-extraction questions (latency sensitivity, volume, cost ceiling): a requirement that names an overnight or next-day turnaround and calls out cost as the primary concern is describing a batch workload, full stop.

Third-Party Vendors

The same Claude models are available through Amazon Bedrock and Google Vertex AI, in addition to Anthropic's direct API. The message format is largely consistent across all three; what differs is authentication, endpoint/region, and model-ID naming. Choose a vendor for reasons like existing cloud footprint, data-residency requirements, and procurement constraints -- not because the underlying model behaves differently.

Common exam traps

• "Batch is just parallel realtime calls." It is a distinct asynchronous API with its own 24-hour SLA and a real per-token discount -- firing many synchronous requests concurrently is not the same thing and does not capture the discount.
• Assuming batch results preserve submission order -- they don't. A custom_id on each request is required to match results back to inputs; positional zipping breaks silently.
• Forgetting the batch is capped by whichever limit is hit first -- 100,000 requests or 256 MB -- not just a flat request count.
• Assuming vendor choice changes model behavior -- Bedrock and Vertex AI host the same Claude models; the differences are integration mechanics (auth, region, model-ID naming), not model quality.
• Picking realtime for a workload whose actual requirement (cost-sensitive, next-morning turnaround, high volume) squarely describes batch.