Some websites use Cloudflare or similar services (DataDome, PerimeterX) to block automated access. PageCrawl provides several tools to handle these protections.
Automatic Detection
When PageCrawl detects that a page is blocked by Cloudflare (timeout, 403, or 401 response), it automatically switches the page to Stealth mode on the next check. No manual action is required for most cases.
Manual Solutions
If automatic detection does not resolve the issue, try these options in order:
| Solution | How to Enable | When to Use |
|---|---|---|
| Stealth mode | Edit page > Power User > Engine: Stealth | First thing to try for any blocked page |
| Proxy rotation | Edit page > Proxy: Random | When a specific IP is blocked |
| Residential proxy | Edit page > Proxy: Residential | When datacenter IPs are blocked |
| Custom proxy | Edit page > Power User > Custom Proxy | When you need a specific location or provider |
| Custom User Agent | Edit page > Power User > User Agent | When the default user agent is flagged |
Configuration Steps
- Open the page that is being blocked
- Click Edit
- Enable Power User mode
- Set Engine to Stealth
- Optionally change the Proxy Location to Random or Residential
- Save and trigger a manual check to verify
Still Blocked?
If Stealth mode and proxy changes do not resolve the issue:
- Try a custom proxy via third-party provider.
- Contact support@pagecrawl.io for assistance. We can review the specific page and suggest the best configuration.
Related Articles
- Real Browser Mode - Engine selection including Stealth mode
- Custom Proxies - Configure proxy servers
- Page Loading Issues - Other common loading problems