PageCrawl provides built-in proxy locations and supports custom proxy servers for pages that require specific geographic access or have IP-based restrictions.
For business monitoring, we recommend Enterprise or Ultimate, which include managed Residential connection options at no extra cost. Start with those before buying a separate proxy service.
For personal monitoring on a budget, try PageCrawl Relay if the public page works through your own connection. Relay has no PageCrawl fee on any plan, including Free. You supply the bandwidth and a computer that stays online, and normal monitoring limits apply.
Relay works with public pages only and does not share your browser's signed-in session. Pages that require authentication still need a login configured in PageCrawl. It may help with access errors, but it does not guarantee access to every website.
Built-in Proxy Locations
Start with Auto in the monitor's Location dropdown. The picker shows the other connections available to your account, including your relay machines and proxy pools. In the simple editor, expand Page adjustments to find Location. You can also apply a location to multiple pages via Bulk Edit.
Proxy Pools (Bring Your Own Proxies)
When the built-in locations are not enough, bring your own proxies as a Proxy Pool: a reusable, named set of proxy servers you manage in one place and reference from any page or template.
1. Create a pool. Go to Settings → More → Proxies, find Custom Proxy Pools, and click Add New. Give the pool a name and paste your proxy servers, one per line:
host:port
username:password@host:port2. Use it. Open the Location dropdown on a page (or a template, or via Bulk Edit) and pick your pool. Save the changes and review the next check's captured content to confirm the pool works for that page.
3. Keep it healthy. Open a pool to see health charts and a plain-language status for the pool, plus each proxy's recent success rate. You can disable or remove individual proxies that are failing; removed proxies are kept for reference so you have a record of what you took out. Add proxies at any time and every monitor using that pool picks them up immediately. You can also enable alerts (email, Slack, Discord, or webhook) so you are notified when a pool starts failing and when it recovers.
| Apply to | How |
|---|---|
| A single page | Edit the page → set Location to your pool |
| Multiple pages | Select pages → Bulk Edit → Location → your pool |
| A template | Set the template's Location to your pool; pages using the template inherit it |
If you set the same proxy list on several monitors, PageCrawl links them to a single shared pool automatically, so you maintain the list once.
If a Page Still Does Not Load
Keep the default Engine while trying different connections. PageCrawl automatically retries some loading and access errors. See Monitoring Pages Behind Bot Protection for troubleshooting steps before purchasing another service.
Included Residential Options and Web Unblocker
Enterprise and Ultimate include managed Residential options at no extra cost, making these plans our recommendation for business monitoring. The separate Web Unblocker option uses purchased bandwidth at $10/GB. Try the included options before buying bandwidth. See Residential Proxies and Web Unblocker for the differences and setup instructions.
Choosing a Proxy Provider
Start with Auto. For business monitoring, try the included Residential options on Enterprise or Ultimate. For personal monitoring, Relay may help if the page works through your own connection. Consider a custom proxy service if these options do not work or you need another connection.
If the built-in proxy locations are not enough for your needs, you can use a third-party proxy provider. Here is what to look for and some popular options.
Understanding bandwidth usage:
Each page check downloads the full page without caching, so bandwidth adds up quickly. An average web page uses 2-3 MB per check. Heavier pages (news sites, e-commerce, image-heavy pages) can use 5-10 MB or more. For example, monitoring 50 pages every 30 minutes at 3 MB each would use roughly 7 GB per day, or around 216 GB per month. Because of this, avoid proxy providers that charge per GB of traffic. Those plans are designed for one-off scraping, not ongoing monitoring.
What to look for:
- Unlimited bandwidth - This is the most important factor. Look for plans priced per proxy/port or as a flat monthly rate, not per GB.
- Username/password authentication - PageCrawl connects to proxies dynamically, so IP-based allowlists will not work. Choose a provider that supports
username:password@host:portauthentication. - Rotating IPs - Providers that rotate IPs automatically reduce the chance of being blocked over time.
- Geographic coverage - Pick a provider with servers in the regions your monitored pages target.
- HTTP/HTTPS support - PageCrawl requires standard HTTP proxies. SOCKS proxies are not supported.
Datacenter vs. residential proxies:
Datacenter proxies with unlimited bandwidth are the most cost-effective option for monitoring. They work well for most websites. Residential proxies (using real ISP addresses) are only needed for sites with strict bot detection that blocks datacenter IPs. If you need residential proxies, look for providers that offer them with unlimited bandwidth or per-IP pricing rather than per-GB billing.
Popular proxy providers that work with PageCrawl:
| Provider | Type | Pricing Model | Notes |
|---|---|---|---|
| Webshare | Datacenter, Residential | Per proxy, unlimited bandwidth | Free tier available, good for testing. Paid datacenter plans include unlimited bandwidth. |
| IPRoyal | Datacenter, Static residential | Per proxy (datacenter) | Datacenter proxies with unlimited traffic. Static residential proxies available per IP. |
| Proxy-Cheap | Datacenter, Static residential | Per proxy, unlimited bandwidth | Budget-friendly static residential and datacenter proxies with no traffic limits. |
| ProxyRack | Datacenter, Residential | Flat monthly rate | Unlimited bandwidth on most plans. Rotating and geo-targeted options. |
These are independent providers and not affiliated with PageCrawl. Prices and features may change.
Not every provider works for every website. A proxy that works perfectly for one site may get blocked on another. This depends on the website's bot detection, the proxy provider's IP reputation, and the type of proxies used. Always test a provider against your specific pages before committing to a long-term plan. Most providers offer short trial periods or small starter plans for this purpose.
Country-specific access: Some websites restrict content to visitors from a specific country (geo-blocking). Government portals, local news sites, and region-locked services often require an IP address from that country to load correctly. If you are monitoring pages like these, make sure the proxy provider offers proxies in the required country. Check the provider's location list before purchasing, as coverage varies significantly between providers, especially for smaller countries.
username:password@host:port format. Paste it into a Custom Proxy Pool under Settings → More → Proxies. If the provider gives you a single endpoint, you only need to add one line.
Avoiding Free Proxies
Free public proxy servers can be unreliable and may stop working without notice. For business monitoring, we recommend the managed connection options included with Enterprise or Ultimate. For personal monitoring, PageCrawl Relay can use a computer you control, provided it stays online and the public page accepts your connection.
Related Articles
- PageCrawl Relay - Use your own connection on any plan, including Free
- Real Browser Mode - Engine selection including Stealth mode
- Monitoring Pages Behind Bot Protection - Handling bot-protected pages
- Bulk Edit - Apply proxy settings to multiple pages at once