Batch API Guide

​

Overview

The Batch API enables you to efficiently run and manage multiple task executions in a single request. It also allows you to export results in bulk, such as in a CSV.

This can be useful for various use cases, including processing lists of companies or transactions, running multiple variations of research queries, analyzing data sets in parallel, and periodic bulk data updates.

​

Create Batch Run

A batch run is scoped to a task. Runners can be specified globally for the whole batch and individually for each task run within the batch as well. The endpoint for batch runs look like this:

POST /v0/tasks/{task_id}/batches # runners are specified in the batch run request

Here is an example request:

curl -X POST https://api.parallel.ai/v0/tasks/{task_id}/batches \
 -H "x-api-key: $PARALLEL_API_KEY" \
 -H "Content-Type: application/json" \
 -d '
 {
   "task_run_requests": [
     {
       "arguments": {
         // Input arguments matching task's input schema
       }
     },
     {
       "arguments": {"company": "OpenAI"}
     }
     // Additional runs...
   ],
   "runner": "neon"
 }'

This creates the following response:

{
  "task_id": "uuid",
  "batch_run_id": "uuid",
  "run_ids": ["uuid1", "uuid2"],
  "created_at": "2025-01-08T11:30:20.258784"
}

​

Get Batch Status

Track the progress of your batch execution:

GET /v0/tasks/{task_id}/batches/{batch_run_id}

Returns a status map grouping run IDs by their current status:

• queued: Waiting to start

• running: In progress

• awaiting_tool_calls: Waiting for tool responses

• complete: Finished successfully

• failed: Error encountered

This creates the following response:

{
  "status_map": {
    "queued": ["run-id-1", "run-id-2"],
    "complete": ["run-id-3", "run-id-4"]
  }
}

​

Export Batch Results

Retrieve results for all completed runs in your preferred format, either as a JSON or CSV.

GET /v0/tasks/{task_id}/batches/{batch_run_id}/export

Query parameters include format: json (default) or csv

These are the CSV headers created for each run: task_id, batch_run_id, run_id, status, runner, input.arguments.keys(), output.keys()

This returns a streaming response containing all completed run results.

​

Get a single run result

In order to see a single run result, use the following endpoint:

GET /v0/task/{task_id}/run/{run_id}

​

Using callbacks with batch endpoint

Each run within a batch request can have its own callback configuration. When a run completes, its associated callback is triggered automatically.

For each run in your batch request, you must provide:

1. An arguments field containing your run parameters
2. A callback_webhook field specifying where to send completion notifications

Example request data:

{
 "task_run_requests": [
   {
    "arguments": {
     // Input arguments matching task's input schema
    },
    "callback_webhook": {"url": https://webhook.com}
   },
   {
    "arguments": {"company": "OpenAI"}
    "callback_webhook": {"url": https://webhook2.com}
    // can be the same or different as the first webhook
   }
   // Additional runs...
 ]
}

The callback system processes runs independently, sending notifications to each run’s specified webhook as they complete. This allows you to track the progress of individual runs within your batch.

For the specific callback payload format and additional configuration options, see our async documentation.