> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sideshift.app/llms.txt
> Use this file to discover all available pages before exploring further.

# Scraper API

> Profiles, posts, audience data, search, comments, transcripts, and more from seven social platforms.

The Scraper API gives you social data from TikTok, Instagram, YouTube, Facebook,
Snapchat, X, and LinkedIn through one authenticated HTTP API.

> [card] [Make your first request](https://docs.sideshift.app/scraper/quickstart.md)
>
> Create a key and run a working request in cURL, Node.js, or Python.

> [card] [Download the OpenAPI reference](https://app.sideshift.app/openapi/sideshift-scraper.yaml)
>
> Use the complete schema in your API client or code generator.

## Choose the right endpoint

There are two response styles. Both use the same base URL, authentication, billing,
errors, and outer response envelope.

| Style | Use it when | Response inside `data` |
| - | - | - |
| **Normalized** | You need profiles or posts in one stable schema across platforms | SideShift's consistent `profile`, `posts`, `post`, or `audience` shape |
| **Platform-specific** | You need a TikTok- or Instagram-only resource or the platform's detailed fields | The documented platform field names and nesting |

Start with the normalized endpoints. Move to a
[TikTok or Instagram operation](https://docs.sideshift.app/scraper/platform-operations.md) only when the normalized
resource does not contain the data you need.

### Normalized resources

| Resource | Path | Availability | Cost |
| - | - | - | - |
| Profile | `POST /scrape/{platform}/profile` | All seven platforms | 1 credit |
| Posts | `POST /scrape/{platform}/posts` | All seven platforms | 1 credit per page |
| Post | `POST /scrape/{platform}/post` | All seven platforms | 1 credit |
| Audience | `POST /scrape/tiktok/audience` | TikTok | 25 credits |

Supported platform path values are `tiktok`, `instagram`, `youtube`, `facebook`,
`snapchat`, `twitter` for X, and `linkedin`.

### Platform-specific operations

The focused TikTok and Instagram surface adds 44 operations for resources such as
favorites, reposts, stories, followers, sounds, hashtags, playlists, search, trends,
comments, transcripts, ads, reels, audio, and highlights. Every operation has a clean,
descriptive path under the same API:

```text theme={"system"}
POST https://app.sideshift.app/api/v1/scrape/{platform}/{resource}
```

Rich resources have explicit names such as `profile-details` and `post-details`. They
are separate because their response schemas are intentionally more detailed than the
normalized schemas. TikTok audience sampling stays on the single normalized `audience`
route because its extra request controls do not require a second response contract.

## One request model

Every endpoint is a `POST` with a JSON body and a scraper key in `x-api-key`:

```bash theme={"system"}
curl -X POST https://app.sideshift.app/api/v1/scrape/tiktok/posts \
  -H "x-api-key: $SIDESHIFT_SCRAPER_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "username": "mrbeast" }'
```

Every successful request uses the same outer envelope:

```json theme={"system"}
{
  "data": {},
  "request_id": "req_8f3c9a2b1d4e6f70",
  "upstream_calls": 1,
  "meta": {
    "credits_charged": 1,
    "credits_remaining": 9999
  }
}
```

Keep `request_id` in your logs. It is the identifier support uses to trace a request.

## Core behavior

* A completed lookup is billed even when the platform confirms that the resource does
  not exist. Validation, authentication, balance, payload-size, and account-rate-limit
  rejections are free.
* Each listing page is a separate request and charge. Treat cursors as opaque and pass
  them back exactly as returned.
* Media URLs can expire within hours. Download media you need to retain.
* The default account limit is 120 requests per minute across all endpoints. Successful
  responses and `429` errors include rate-limit headers.
* Scraper keys begin with `scrape_live_` and are separate from SideShift Platform and
  Connect credentials.

## Status

Live health, 90-day uptime and incidents for the Scraper API are published at
[app.sideshift.app/status/scraper](https://app.sideshift.app/status/scraper), with the
same data as public JSON at
[app.sideshift.app/api/status/scraper](https://app.sideshift.app/api/status/scraper).
The page tracks the Scraper API itself, the upstream data provider, and each platform:
TikTok, Instagram, YouTube and Facebook each have a row, and X, Snapchat and LinkedIn share
one. Platform rows come from real API traffic over the last 15 minutes. A platform reads as
degraded or unavailable when its requests fail well above their normal rate across more than
one caller. A failure that affects only your own targets, such as a deleted post, does not
change the page. When there are too few requests to measure, a row keeps its last state.
Checks run every two minutes. Check the page before you file a ticket, and poll the JSON from
your own monitoring if you want alerts.

## Continue

> [card] [Quickstart](https://docs.sideshift.app/scraper/quickstart.md)
>
> Authentication, cURL, Node.js, Python, and a pagination loop.

> [card] [Responses and data](https://docs.sideshift.app/scraper/responses.md)
>
> Envelopes, field behavior, media URLs, transcripts, comments, and cursors.

> [card] [Billing, errors, and limits](https://docs.sideshift.app/scraper/billing-and-limits.md)
>
> Credit costs, purchase rates, refunds, retries, and rate-limit handling.

> [card] [Platforms and optional parameters](https://docs.sideshift.app/scraper/platforms.md)
>
> Platform behavior and the exact flags supported by normalized endpoints.

> [card] [Download the OpenAPI document](https://app.sideshift.app/openapi/sideshift-scraper.yaml)
>
> Use the complete OpenAPI 3.0 document for code generation or your own API
> tooling.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.