# Push API: Monitor Data You Send Us

Source: PageCrawl.io Help Center
URL: https://pagecrawl.io/help/integrations/article/push-api-data-sources

---

Most PageCrawl monitors watch a public web page. Data sources flip the direction: you push values into PageCrawl over a simple REST endpoint (or by email), and PageCrawl treats each value exactly like a check on a monitored page. You get the same change comparison, history, charts, multi-channel notifications, and AI summaries, but for data that only you can see.

  <strong>Note:</strong> Data sources are available on every plan, including Free.

### When to Use a Data Source

Use a data source whenever the value you care about is not on a public page that a monitor can reach:

- **Internal KPIs** - Signup counts, error rates, queue depth, revenue figures from your own systems
- **Values behind authentication** - Numbers from an admin panel, a private dashboard, or an intranet page that your own script can read
- **Partner or third-party API responses** - Poll an API yourself on your own schedule and push the result, so PageCrawl tracks it over time
- **Anything a crawler cannot reach** - Values computed on a device, inside a private network, or output by a batch job

If the value is on a public web page, a regular monitor is usually simpler. Data sources are for everything else.

### Creating a Data Source

#### In the app

Go to **Settings > API > Data Sources** and click **Create Data Source**. Give it a name, optionally define named fields (for example `price` and `stock`), and save. The secret ingest URL and the private inbound email address are shown right after creation, and again on the monitor's **Send data** panel whenever you need them later.

#### Via the API

All requests use a Bearer token from **Settings > API** (see the [API and Webhooks guide](/help/features/article/api-webhooks-for-custom-integrations.md) for token setup).

```bash
curl -X POST "https://pagecrawl.io/api/data-sources" \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Competitor API price",
    "fields": [
      {"label": "price", "type": "number"},
      {"label": "stock", "type": "number"}
    ],
    "ai_prompt": "Flag pricing or plan changes."
  }'
```

- **`fields`** is optional. Leave it out to track a single value.
- **`ai_prompt`** (optional) guides the AI summaries: what the value means and what to flag. AI summaries are on by default, so recorded values get a readable summary and a priority score. Set **`ai_summaries_enabled`** to `false` for high-frequency numeric metrics where you only want charts and thresholds.
- **`email_sender_filter`** (optional) restricts who can email values in, either an exact address or an exact domain.

The response is `201` with the monitor object: `id`, `slug`, `elements`, `ingest_url` (the secret push URL for this source), and `ingest_email` (the private inbound email address).

### Pushing Values

Every data source has its own secret ingest URL, shown when you create the data source and on the monitor's **Send data** panel. The URL is the whole credential: no API token, no headers, no client library. POST a value to it and the value is recorded.

**Single value, one line:**

```bash
curl -d value=42 https://pagecrawl.io/api/ingest/data-a1b2c3d4e5f6g7h8i9j0k1l2
```

The same call from code:

```javascript
fetch('https://pagecrawl.io/api/ingest/data-a1b2c3d4e5f6g7h8i9j0k1l2', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ value: 42 }) });
```

```php
file_get_contents('https://pagecrawl.io/api/ingest/data-a1b2c3d4e5f6g7h8i9j0k1l2', false, stream_context_create(['http' => ['method' => 'POST', 'header' => 'Content-Type: application/json', 'content' => json_encode(['value' => 42])]]));
```

```python
import requests

requests.post('https://pagecrawl.io/api/ingest/data-a1b2c3d4e5f6g7h8i9j0k1l2', json={'value': 42})
```

**Multiple named fields** work with plain form encoding:

```bash
curl -d 'values[price]=9.99' -d 'values[stock]=12' https://pagecrawl.io/api/ingest/data-a1b2c3d4e5f6g7h8i9j0k1l2
```

or as JSON:

```bash
curl -H 'Content-Type: application/json' \
  -d '{"values": {"price": 9.99, "stock": 12}}' \
  https://pagecrawl.io/api/ingest/data-a1b2c3d4e5f6g7h8i9j0k1l2
```

  <strong>Note:</strong> The ingest URL and the ingest email address contain the secret key. Treat them like a password and share them only with systems that should be able to record values.

A successful push returns `200`:

```json
{
  "changed": true,
  "check_id": 123,
  "monitor_id": 55,
  "values": {"price": "9.99", "stock": "12"},
  "created_fields": []
}
```

The `changed` field tells you whether the value differed from the previous one, so a single push both records the value and reports whether it moved. Each named field becomes its own tracked element with its own chart and history, so a data source with `price` and `stock` gives you two separate value timelines on one monitor. New field names in `values` are created automatically: numeric-looking values become number fields, everything else becomes a text field, and any fields created this way are listed in `created_fields`.

Pushing the same value again returns `"changed": false` and stores nothing new, so it is safe to push on a schedule even when the value rarely moves.

Add `"return_summary": true` (JSON body) or `-d return_summary=1` (form) to also get the AI summary and priority score of the change back in the same response:

```bash
curl -d value=42 -d return_summary=1 https://pagecrawl.io/api/ingest/data-a1b2c3d4e5f6g7h8i9j0k1l2
```

```json
{"changed": true, "check_id": 124, "summary": "Signups rose to 42.", "score": 55, "values": {"Value": "42"}, "created_fields": []}
```

`summary` and `score` are included only when you ask for them, and are null when no change was detected or AI summaries are off for the data source.

#### Authenticated endpoint

You can also push to `/api/ingest/{slug}` with a Bearer token from **Settings > API**. Prefer this for workspace-scoped scripts that already hold an API token, or when you do not want the secret URL stored in the script:

```bash
curl -X POST "https://pagecrawl.io/api/ingest/my-kpi-source" \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"value": 42}'
```

Both endpoints accept the same bodies and return the same responses.

### Sending Values by Email

Every data source has a private inbound address like `data-a1b2c3d4e5f6g7h8i9j0k1l2@ingest.pagecrawl.io`. It is shown in the app and returned as `ingest_email` when you create the source via the API. Recording a value is one line:

```bash
echo "42" | mail data-a1b2c3d4e5f6g7h8i9j0k1l2@ingest.pagecrawl.io
```

Email a value in the message body:

- Lines like `price: 9.99` that match existing field names map to those fields.
- Otherwise the whole body is stored as the single value.
- If the body is empty, the subject line is used instead.

This is useful for cron jobs, no-code tools, and systems that can only send email: pipe a value into your mail command, point a form or automation at the address, and the value lands in PageCrawl like any other push.

  <strong>Note:</strong> The address contains the same secret key as the ingest URL, so keep it out of shared documents. For extra protection, set a sender filter (an exact address or an exact domain) so mail from anyone else is ignored.

### Reading Values Back

A `GET` on the same secret URL returns the current value of each field, so you can read data back with the exact URL you push to (no API token, no client library):

```bash
curl https://pagecrawl.io/api/ingest/data-a1b2c3d4e5f6g7h8i9j0k1l2
```

```json
{
  "monitor_id": 55,
  "name": "Competitor API price",
  "last_received_at": "2026-07-27T09:14:00+00:00",
  "fields": [
    {"label": "price", "type": "number", "value": "9.99", "changed": true},
    {"label": "stock", "type": "number", "value": "12", "changed": false}
  ],
  "values": {"price": "9.99", "stock": "12"}
}
```

Use `values` for a quick flat map, or `fields` for the type of each value. Add `?history=<n>` (up to 100) to also get the most recent points, newest first:

```bash
curl "https://pagecrawl.io/api/ingest/data-a1b2c3d4e5f6g7h8i9j0k1l2?history=20"
```

Reading does not count against your check allowance. Because the same token both reads and writes, treat the URL as a full credential.

  <strong>Note:</strong> If you connect an AI assistant over MCP, you do not need this endpoint: ask it for the latest values or history of the data source and it uses the built-in read tools.

### How Changes and Notifications Work

A data source behaves like any other monitor once a value arrives:

1. **Each changed push is stored as a check.** The value appears in the monitor's history with a timestamp, and number fields plot on the value chart.
2. **The new value is compared with the previous one.** If nothing changed, the push is acknowledged with `"changed": false` and no new history entry is created.
3. **When a value changes, your notification rules fire.** Email, Slack, Discord, Teams, Telegram, and webhooks all work, and alerts arrive shortly after the value is received.
4. **AI features apply the same way.** When enabled, AI summaries and priority scoring run on data source changes just like on page changes.

Because pushes flow through the normal change pipeline, everything built on top of monitors (dashboards, reports, RSS feeds, webhook payloads, MCP read tools) works on data sources with no extra setup.

### Errors and Limits

| Status | Meaning |
|--------|---------|
| `403` | The data source is disabled. |
| `404` | Unknown ingest URL or data source (check the secret or the `slug`) |
| `422` | Validation error, for example a non-numeric value pushed into a number field, a value over the size ceiling, or the per-source field cap reached |
| `429` | Rate limited; honor the `Retry-After` header before retrying |
| `401` | Invalid or missing API token (authenticated endpoint only) |

Value size: a single pushed value can be up to 500,000 characters. Data sources store the first 65,535 characters of each value (10,000 on the Free plan); longer values also keep a checksum of the full content, so a change anywhere in the value is still detected and recorded even when it falls beyond the stored portion. Truncated values end with a marker line that includes the checksum.

Every accepted push counts toward your plan's check allowance, the same as a scheduled check on a monitored page, whether or not the value changed. Pushes that return `"changed": false` deduplicate the stored history entry (no new history row is written), so frequent same-value pushes do not flood your history.

### Using Data Sources with AI Assistants

The PageCrawl MCP server includes two data source tools, available on all plans:

| Tool | What It Does |
|------|-------------|
| **create-data-source** | Create a new data source, optionally with named fields |
| **record-data** | Push one or more values into a data source |

Existing read tools (get latest values, get monitor history) work on data sources like any other monitor, so you can ask your assistant things like "record today's signup count in my KPI data source" or "chart the last 30 days of the price field". See the [AI Assistants (MCP Server) guide](/help/integrations/article/mcp-server-ai-tools.md) for setup.

### Related Articles

- [API and Webhooks for Custom Integrations](/help/features/article/api-webhooks-for-custom-integrations.md) - Token setup, monitor endpoints, and webhook quick start
- [Webhook Integration](/help/integrations/article/webhook-integration.md) - Receive change payloads from data sources and monitors alike
- [AI Assistants (MCP Server)](/help/integrations/article/mcp-server-ai-tools.md) - Connect Claude, ChatGPT, and other MCP clients
- [Full API Reference](/developers) - Interactive OpenAPI reference with every endpoint and schema

---

Need more? The complete PageCrawl.io help center, with every article, is available as a single document at https://pagecrawl.io/llms-full.txt. Read it for context on anything this page does not cover.