Rate limits and batching

Two kinds of limit

An indexing API caps how often you may call it and how much work a single call may contain. The first protects the service from bursts; the second keeps individual jobs to a size that can be processed and reported on. Clients hit the first when they poll too eagerly and the second when they pass a whole export file in one request. Both are easy to design around once the numbers are known. This page uses the IndexChex API, published by the same company as this handbook, as the worked case.

The numbers

The current figures are maintained on the API reference; credit costs per URL are listed on IndexChex pricing.

What happens at each limit

Exceeding the request rate returns HTTP 429. On v1 the body is:

{"message": "Too many requests. Please wait and retry."}

On v2 it keeps the SpeedyIndex envelope:

{"code": 2, "message": "Too many requests. Please retry shortly."}

The response includes standard rate-limit headers (Retry-After and X-RateLimit-*), which a client can read instead of guessing a delay.

Exceeding the job size returns HTTP 422 before any credits are taken. On v1 the body is a validation error with the message "Too many URLs. Maximum allowed: 10000" (or 1000 when instantIndex is true) under errors.urls. On v2 it is {"code": 3, "message": "Invalid request payload.", "errors": {...}} with the same text inside errors. Both shapes are covered in handling API errors.

Batching a large list

A 35,000-URL backlink export becomes four check jobs: three of 10,000 and one of 5,000. Before splitting, normalise and deduplicate, because every array entry is charged.

import os, time, requests

API = "https://indexchex.com/api/v1"
H = {"Authorization": os.environ["INDEXCHEX_API_KEY"], "Accept": "application/json"}

def chunks(items, size):
    for i in range(0, len(items), size):
        yield items[i:i + size]

def create_jobs(urls, kind="index-check", instant=False, label="batch"):
    urls = list(dict.fromkeys(u.strip() for u in urls if u.strip()))
    size = 1000 if instant else 10000
    job_ids = []
    for n, part in enumerate(chunks(urls, size), start=1):
        body = {"urls": part, "name": f"{label} {n}"}
        if kind == "index-submit":
            body["instantIndex"] = instant
        while True:
            r = requests.post(f"{API}/{kind}/jobs", json=body, headers=H, timeout=60)
            if r.status_code == 429:
                time.sleep(int(r.headers.get("Retry-After", 30)))
                continue
            r.raise_for_status()
            break
        job_ids.append(r.json()["job_id"])
    return job_ids

Name each part so jobs can be matched up in the dashboard and in reports. Check the balance with GET /v1/balance first: a job that would overdraw the account is refused with HTTP 422, and the loop above would stop part way through the list.

curl https://indexchex.com/api/v1/balance -H "Authorization: $INDEXCHEX_API_KEY"
{"balance": 41250}

Pacing the poll loop

Create calls are few; polling is where the budget goes. Some planning arithmetic:

Run one timer that walks the list of open jobs, not one thread per job. On v2, the status endpoint accepts task_ids so several tasks can be read in one request. The interval guidance per job type is on polling job status.

Sharing the limit across tools

The allowance is per user, so a Google Sheet, a cron script and a CMS publish hook on the same account all draw from the same 120 requests. If several integrations run, give the busiest one most of the headroom and slow the others. A Python indexing script that runs nightly is a good place to concentrate bulk work.

Batching submissions by priority

Splitting is also a chance to separate URLs by value. Fresh, expensive placements can go into a small instant job; the long tail goes into standard jobs of 10,000. The submission guide covers the payload for each, and checking index status covers the verify step.

Related

How limits compare across vendors is part of the backlink indexer API overview, and they are one item on any evaluation of backlink indexer software. The IndexChex entry summarises the account-level limits.