API Documentation

Integrate Pay Per Crawl data access into your applications with our REST API. Get instant quotes, process payments, and access web data programmatically.

Quick Start API Reference

REST API

JSON-based

<50ms

Response Time

99.9%

Uptime SLA

OpenAPI

3.0 Spec

Quick Start

1. Get Your API Key

Get API Key

2. Make Your First Request

Get a quote for crawling specific URLs:

# cURL Example
curl -X POST https://aipayercrawl.com/api/quote \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "urls": [
      "https://example.com/page1",
      "https://example.com/page2"
    ]
  }'

Authentication

Authenticate your API requests by including your API key in the Authorization header:

Authorization: Bearer YOUR_API_KEY

Security: Keep your API key secure and never expose it in client-side code. All API requests must be made over HTTPS.

API Reference

Base URL

https://aipayercrawl.com/api

POST /quote

Get a quote for crawling specific URLs. Returns itemized pricing and a payment URL.

Request Body

{
  "urls": [
    "https://example.com/page1",
    "https://example.com/page2"
  ],
  "callbackUrl": "https://your-app.com/webhook",
  "metadata": {
    "userId": "user123"
  }
}

urls (required) - Array of URLs to crawl (max 1000)

callbackUrl (optional) - Webhook URL for notifications

metadata (optional) - Custom metadata object

Response

{
  "quoteId": "quote_1234567890abcdef",
  "totalCost": 0.003,
  "currency": "USD",
  "items": [
    {
      "url": "https://example.com/page1",
      "domain": "example.com",
      "pricePerRequest": 0.001,
      "currency": "USD",
      "available": true
    },
    {
      "url": "https://example.com/page2",
      "domain": "example.com", 
      "pricePerRequest": 0.001,
      "currency": "USD",
      "available": true
    }
  ],
  "expiresAt": "2024-01-02T10:00:00Z",
  "createdAt": "2024-01-01T10:00:00Z",
  "paymentUrl": "https://aipayercrawl.com/api/checkout?quoteId=quote_1234567890abcdef"
}

Error Handling

The API uses conventional HTTP response codes to indicate success or failure:

2xx - Success

Request completed successfully

4xx - Client Error

Invalid request data or authentication

5xx - Server Error

Internal server error

429 - Rate Limited

Too many requests, retry later

Code Examples

Node.js

// Node.js Example
const response = await fetch('https://aipayercrawl.com/api/quote', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer YOUR_API_KEY'
  },
  body: JSON.stringify({
    urls: [
      'https://example.com/page1',
      'https://example.com/page2'
    ]
  })
});

const quote = await response.json();
console.log('Total cost:', quote.totalCost);

python

# Python Example
import requests

url = "https://aipayercrawl.com/api/quote"
headers = {
    "Content-Type": "application/json",
    "Authorization": "Bearer YOUR_API_KEY"
}
data = {
    "urls": [
        "https://example.com/page1",
        "https://example.com/page2"
    ]
}

response = requests.post(url, headers=headers, json=data)
quote = response.json()
print(f"Total cost: {quote['totalCost']}")

curl

# cURL Example
curl -X POST https://aipayercrawl.com/api/quote \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "urls": [
      "https://example.com/page1",
      "https://example.com/page2"
    ]
  }'

SDKs & Libraries

Official SDKs and community libraries to help you integrate faster:

Node.js SDK

Official Node.js/TypeScript SDK with full type support

Python SDK

Official Python SDK with async support