Some pages load perfectly in your browser and refuse PageCrawl. You see a 403, a challenge page, or a stripped-down version with none of the content you wanted to track. PageCrawl Relay fixes that class of page by running the check through a computer you already own, so the site sees an ordinary connection rather than ours.
What is PageCrawl Relay?
PageCrawl Relay is a small program you run on a computer you own. When a monitor is bound to it, PageCrawl still opens and reads the page as usual, but the network traffic leaves from your machine instead of ours. It is available on every plan, including Free, and costs nothing beyond your own bandwidth.
Almost any machine will do. It uses about 15 MB of memory and no measurable processor time, because the browser work stays on PageCrawl's side and your computer only carries the traffic. A Raspberry Pi, a NAS, a Home Assistant box or an old laptop left plugged in are all more than enough, and something that stays on beats something fast.
It is not a proxy server. It opens no port, accepts no incoming connection, and carries only your own monitors.
When do I actually need it?
You need it when a page treats our servers differently from a normal connection, or when the page is only reachable from your network at all. If a page loads fine for you but returns a 403 or a challenge in PageCrawl, that is the signal.
Three common cases:
- A site that refuses our servers. Retail, ticketing and travel sites often do this while serving your browser normally.
- A page behind your network. An intranet, a staging site, or a supplier portal that only allows your office address.
- Content that varies by country in a way our locations do not cover.
You are less likely to need it on Enterprise and Ultimate, where the Location dropdown already includes the residential option and per-country choices. On Free and Standard the built-in selection is narrower, which is where a relay earns its place.
Why not just run a proxy server at home?
Because a proxy server at home has to accept incoming connections, which means opening a port on your router and publishing your home address. Open proxies get found by scanners within hours and used by strangers, and the traffic looks like it came from you.
A relay inverts that. Your computer makes one outgoing connection to PageCrawl and holds it open, so:
- there is no port to forward and nothing for a scanner to find;
- nothing breaks when your home address changes, which it will;
- there are no proxy credentials to rotate or leak;
- if the machine sleeps, checks fall back to PageCrawl's own locations on their own rather than failing.
You can still bring your own proxies if you prefer. See Custom Proxies for Proxy Pools, which work on every plan too.
Why not just buy residential proxy bandwidth?
Sometimes you should. They solve different problems, and the deciding factor is usually whether you need somebody else's address or simply a normal one.
Buying bandwidth is the better answer when you need pages checked from several countries, when the page serves a market you are not in, when you would rather your own address stayed out of it entirely, or when you do not want to keep a machine running. PageCrawl's residential option covers all of that, on Enterprise and Ultimate plans.
A relay is the better answer when the page simply wants to see a normal connection and you already have a computer that stays on. Two reasons:
- It is metered, and pages are heavy. PageCrawl's residential bandwidth is $10 per gigabyte on Ultimate and Enterprise, and a standard page is around 2 MB once images and scripts are counted. One monitor checking that page hourly uses about 1.4 GB a month, so roughly $14 for the one page, and twenty of them is a bill worth thinking about. The cost calculator prices your own pages. A relay uses the connection you already pay for instead, on any plan including Free.
- A bought address is a shared address. Residential pools are rented to many customers at once, so a given address has been used for a great deal of traffic before it reaches you, and some sites have already formed an opinion about it. Your own connection is one a site has no particular reason to distrust.
Note: a relay only gives you the one location you are actually in. If you need a page checked from three countries, that is what the built-in locations and the residential option are for, and you can mix both: use a relay for the pages that need your network, and PageCrawl's locations for the rest.
How does it keep my network private?
Your machine never chooses where to connect. PageCrawl asks it to reach one site for one of your monitors, and before connecting, your machine checks the address is a real public one. Anything on your own network is refused outright.
That refusal list covers your router and any device on it, the private ranges
described in RFC 1918, carrier-grade
NAT ranges, the
link-local addresses
that cloud servers use for their own configuration, and names ending in .local or
.lan. The check happens after the address is looked up and before the connection is
made, so a web address that secretly points at your router is refused as well.
The relay also cannot read the pages it carries. Encryption runs between PageCrawl and the site, so your machine moves sealed traffic it cannot open, and so can anyone watching your network.
How do I set it up?
- Go to Settings → Relays and choose Add machine. Copy the token it shows, which appears once.
- Download and open the program on the computer you want to use. A settings page opens in your browser.
- Paste the token and press Connect.
- On any monitor, set Location to that machine, or to Any of my machines if you run more than one.
The settings page shows whether it is connected, how much data it has carried, and every site it has been asked to reach. A Run a check button reports in plain language if something is wrong.
For a computer that stays on, such as a home server, a NAS or a Raspberry Pi, there are Docker, systemd and Home Assistant packages. A Home Assistant box is often the best machine for this, because it is already running all the time.
What happens when my computer is off?
By default the check runs from PageCrawl's own locations instead, so nothing stops being monitored, and your machine is used again the moment it returns. If a result from a different address would be wrong rather than merely different, you can tell PageCrawl to wait instead, or to flag the monitor.
The choice is made once for your whole account, under Settings → Relays:
| When no machine is reachable | What happens | Who it is for |
|---|---|---|
| Check from PageCrawl's locations (the default) | The check runs as it would without a relay | Almost everyone. A check that still happens beats no check |
| Wait until a machine is back | The check is postponed and nothing is recorded | Pages that answer our addresses with a block page or different content, where recording that would replace a good history with a wrong one |
| Wait, and mark the monitor as failed | The same, and the monitor shows Relay offline and follows your usual alert rules | Pages only your own network can reach, where the machine being off is itself the thing you want to be told about |
There is no interval to choose, because waiting ends on the event rather than on a clock: the moment one of your machines is usable again, its postponed checks are due. PageCrawl also looks once a day regardless, so a monitor is never left waiting on a signal that went missing.
This is worth planning around either way. A laptop that sleeps relays only while it is awake, so if a page genuinely needs your connection, run the relay on something that stays on. You can also set a monthly data limit per machine, and once it is reached, checks go back to PageCrawl's locations on their own.
Can I see what it is doing?
Yes. The settings page lists every destination the machine was asked to reach, newest first, marking each one as reached or refused with the reason. Refusals are the guard protecting your own network, and seeing them is the point.
The program is also open source, so you can read exactly what it does, build it yourself, and check that the binary you downloaded matches. Everything that runs on your computer is published; the part that stays private is PageCrawl's own infrastructure, which cannot change what the program does on your machine.