> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sideshift.app/llms.txt
> Use this file to discover all available pages before exploring further.

# Quickstart

> Create a scraper key, make your first request, and fetch multiple pages.

The Scraper API is plain HTTP. You do not need an SDK.

<Steps>
  <Step title="Create a scraper key">
    Open the [Scraper dashboard](https://app.sideshift.app/scraper), choose **API Keys**,
    and click **Create key**.

    Copy the key immediately. The full value is shown only once; the dashboard keeps only
    a truncated preview. Scraper keys begin with `scrape_live_`.
  </Step>

  <Step title="Store the key on your server">
    Keep the key in a secret manager or server-side environment variable. Do not put it
    in browser code, a mobile app, or a repository.

    ```bash theme={"system"}
    export SIDESHIFT_SCRAPER_KEY="scrape_live_YOUR_KEY"
    ```

    <Note>
      Scraper keys are separate from SideShift Platform and Connect credentials. Other
      SideShift key types are rejected on scraper routes.
    </Note>
  </Step>

  <Step title="Request a profile">
    Send a `POST` request with JSON and the `x-api-key` header.

    <CodeGroup>
      ```bash cURL theme={"system"}
      curl -X POST https://app.sideshift.app/api/v1/scrape/tiktok/profile \
        -H "x-api-key: $SIDESHIFT_SCRAPER_KEY" \
        -H "Content-Type: application/json" \
        -d '{ "username": "mrbeast" }'
      ```

      ```js Node.js theme={"system"}
      const response = await fetch(
        "https://app.sideshift.app/api/v1/scrape/tiktok/profile",
        {
          method: "POST",
          headers: {
            "x-api-key": process.env.SIDESHIFT_SCRAPER_KEY,
            "Content-Type": "application/json",
          },
          body: JSON.stringify({ username: "mrbeast" }),
        },
      );

      const result = await response.json();
      if (!response.ok) throw new Error(`${result.error}: ${result.message}`);
      console.log(result.data);
      ```

      ```python Python theme={"system"}
      import os
      import requests

      response = requests.post(
          "https://app.sideshift.app/api/v1/scrape/tiktok/profile",
          headers={"x-api-key": os.environ["SIDESHIFT_SCRAPER_KEY"]},
          json={"username": "mrbeast"},
          timeout=90,
      )
      response.raise_for_status()
      result = response.json()
      print(result["data"])
      ```
    </CodeGroup>
  </Step>

  <Step title="Read the envelope">
    The resource payload is in `data`. Request tracing and billing metadata are always
    outside it.

    ```json theme={"system"}
    {
      "data": {
        "username": "mrbeast",
        "display_name": "MrBeast",
        "follower_count": 120000000
      },
      "request_id": "req_8f3c9a2b1d4e6f70",
      "upstream_calls": 1,
      "meta": {
        "credits_charged": 1,
        "credits_remaining": 9999
      }
    }
    ```

    Log `request_id`, `meta.credits_charged`, and `meta.credits_remaining` for every
    request.
  </Step>
</Steps>

## Fetch a page of posts

Change the resource to `posts`. The normalized listing response contains `posts`,
`profile_pictures`, and `next_cursor` when another page is available.

```bash theme={"system"}
curl -X POST https://app.sideshift.app/api/v1/scrape/instagram/posts \
  -H "x-api-key: $SIDESHIFT_SCRAPER_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "username": "nasa", "include_carousels": true }'
```

Pass a returned `next_cursor` back as `cursor`. Never parse, edit, or construct a cursor.

```js theme={"system"}
const endpoint = "https://app.sideshift.app/api/v1/scrape/tiktok/posts";
const posts = [];
let cursor;

do {
  const response = await fetch(endpoint, {
    method: "POST",
    headers: {
      "x-api-key": process.env.SIDESHIFT_SCRAPER_KEY,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ username: "mrbeast", ...(cursor ? { cursor } : {}) }),
  });

  const result = await response.json();
  if (!response.ok) throw new Error(`${result.error}: ${result.message}`);

  posts.push(...result.data.posts);
  cursor = result.data.next_cursor || undefined;
} while (cursor);
```

Each page is one separately billed request. Snapchat and LinkedIn listings are single-page
only and never return a cursor.

## Production checklist

* Use a background job for scraping rather than blocking an interactive page request.
* Set a request timeout of at least 60 seconds; use 90 seconds for YouTube and Facebook.
* Store platform post `id` plus `platform` as your deduplication key.
* Download short-lived media URLs instead of saving the URL.
* Retry only transient, refunded failures and honor `Retry-After` on `429` responses.

<CardGroup cols={2}>
  <Card title="Understand responses" icon="brackets-curly" href="/scraper/responses">
    Learn the normalized and platform-specific data contracts.
  </Card>

  <Card title="Add safe retries" icon="rotate-cw" href="/scraper/billing-and-limits">
    Copy the retry wrapper and review billing rules before going live.
  </Card>
</CardGroup>