Python indexing script

What the script does

The script below drives the v1 API of IndexChex, publisher of this handbook. It has two commands. submit reads URLs from a text file and creates a standard submission with a scheduled index re-check. report takes the submission's job ID, waits until the re-check has run, and writes a CSV with one row per URL. It is intentionally short: no framework, one dependency, and every request visible in a few lines.

The IndexChex software and API page lists what the API covers. For a comparison with other providers' interfaces, see backlink indexer APIs.

Setup

python3 -m pip install requests
export INDEXCHEX_API_KEY='paste-your-key-here'

The key comes from the API settings page of your account and is sent as the full Authorization header value. Keep it in the environment or a secrets manager, never in the script file or the repository; API key security covers the rest.

The script

#!/usr/bin/env python3
"""Submit URLs with a scheduled re-check, then export the re-check report to CSV.

Usage:
  indexchex.py submit urls.txt [days]
  indexchex.py report SUBMISSION_JOB_ID out.csv
"""
import csv
import os
import sys
import time

import requests

BASE = "https://indexchex.com/api/v1"
HEADERS = {"Authorization": os.environ["INDEXCHEX_API_KEY"], "Accept": "application/json"}


def call(method, path, **kwargs):
    while True:
        r = requests.request(method, BASE + path, headers=HEADERS, timeout=60, **kwargs)
        if r.status_code == 429:
            time.sleep(int(r.headers.get("Retry-After", 60)))
            continue
        if r.status_code >= 400:
            sys.exit(f"{method} {path}: HTTP {r.status_code} {r.text}")
        return r.json()


def wait_for(path, done, interval):
    while True:
        data = call("GET", path)
        if done(data):
            return data
        time.sleep(interval)


def submit(url_file, days=3):
    with open(url_file) as f:
        urls = [line.strip() for line in f if line.strip()]
    job = call("POST", "/index-submit/jobs",
               json={"urls": urls, "name": os.path.basename(url_file), "checker": int(days)})
    print(f"submission {job['job_id']}: {job['total_urls']} URLs, "
          f"re-check scheduled for {job['scheduled_check_at']}")


def report(submission_id, out_csv):
    sub = wait_for(f"/index-submit/jobs/{submission_id}",
                   lambda d: d["scheduled_check_run_id"], 3600)
    check_id = sub["scheduled_check_run_id"]
    wait_for(f"/index-check/jobs/{check_id}", lambda d: d["status"] == "completed", 300)
    rep = call("GET", f"/index-check/jobs/{check_id}/report")

    with open(out_csv, "w", newline="") as f:
        writer = csv.writer(f)
        writer.writerow(["url", "result"])
        for key, label in (("indexed_links", "indexed"),
                           ("unindexed_links", "not indexed"),
                           ("failed_links", "check failed")):
            writer.writerows([url, label] for url in rep[key])
    print(f"{rep['indexed_count']} indexed, {rep['not_indexed_count']} not indexed, "
          f"{rep['failed_count']} failed -> {out_csv}")


if __name__ == "__main__":
    commands = {"submit": submit, "report": report}
    if len(sys.argv) < 3 or sys.argv[1] not in commands:
        sys.exit(__doc__)
    commands[sys.argv[1]](*sys.argv[2:])

How each step behaves

Submit. POST /index-submit/jobs takes urls (an array), an optional name, and checker, the number of days (1 to 5) before the automatic re-check; 0 or an absent value means no re-check. A successful call returns HTTP 202 with job_id, total_urls, scheduled_check_days and scheduled_check_at. Standard submission costs 1 credit per URL, up to 10,000 URLs per job, and the re-check reserves another 1 credit per URL up front. A request the API rejects, such as too many URLs or too few credits, comes back as HTTP 422 with a message.

Poll the submission. GET /index-submit/jobs/{id} reports status, progress_percent, success_percent and the re-check fields. scheduled_check_run_id is empty until the re-check has started, then holds the ID of a normal index-check job. Hourly polling is plenty for an event days away.

Poll the check. GET /index-check/jobs/{id} returns status (pending, processing or completed) with running counts. The script polls every five minutes. The polling job status page discusses intervals and back-off.

Fetch the report. GET /index-check/jobs/{id}/report returns three URL arrays: indexed_links, unindexed_links and failed_links, plus totals. Asked too early, it answers 422 "Report not ready", which is why the script waits for completed first.

Running it on a schedule

Run submit when a batch of new links goes live, note the printed job ID, and schedule report for the day after the re-check date:

python3 indexchex.py submit new-links.txt 3
# submission 4812: 250 URLs, re-check scheduled for ...
python3 indexchex.py report 4812 links-4812.csv

If report starts before the re-check, it waits rather than failing, so a cron entry or CI job can call it daily without extra logic. The 60-second request timeout and the 429 handling keep it inside the per-minute rate limit; rate limits and batching explains why one large job beats many small ones.

Reading the CSV

A row marked indexed was found in Google's live results at check time. not indexed means the check completed and the URL was absent; Google may still be processing it, or may have declined it. IndexChex guarantees Googlebot crawling, not indexing, so a not indexed row is a prompt to inspect the page, not a failed purchase. check failed means the lookup did not complete and says nothing about index status; re-check those URLs. Turning this file into something a client can read is covered in client reporting.

Extending the script

• Read URLs from a sitemap or database instead of a text file.
• Resubmit the unindexed_links list automatically, with a cap so the same URL is not paid for repeatedly.
• Post the summary line to a chat channel.

People who prefer not to run code can get the same loop from a spreadsheet or an automation platform. Exact field definitions and error bodies are in the API reference, and the broader category of tools that do this work is described in what backlink indexer software is.