Complete technical guide for integrating AI features with your own API keys for enhanced page monitoring capabilities.

This guide provides detailed instructions for setting up AI-powered features in PageCrawl.io using your own API keys from supported providers. The BYOK (Bring Your Own Key) approach gives you full control over costs, data privacy, and service levels.

Overview

PageCrawl.io's AI integration supports:

• Multiple AI Providers: OpenAI, Google Gemini, Anthropic Claude
• Flexible Model Selection: Choose models based on your needs and budget
• Real-time Usage Tracking: Monitor costs and token consumption
• Enterprise-grade Security: Your keys and data remain under your control

Supported Providers and Models

OpenAI

• GPT-3.5-turbo: Cost-effective for basic summarization
• GPT-4: Enhanced accuracy for complex analysis
• GPT-4-turbo: Latest model with improved performance
• Pricing: Pay-per-token directly to OpenAI

Google Gemini

• Gemini Pro: General-purpose model for text analysis
• Gemini Pro Vision: Enhanced model with vision capabilities
• Pricing: Pay-per-request directly to Google

Anthropic Claude

• Claude 3 Haiku: Fast and cost-effective
• Claude 3 Sonnet: Balanced performance and cost
• Claude 3 Opus: Highest accuracy for critical applications
• Pricing: Pay-per-token directly to Anthropic

Step-by-Step Setup

1. Obtain API Keys

OpenAI Setup

# Visit: https://platform.openai.com/api-keys
# 1. Create account or log in
# 2. Navigate to API Keys section
# 3. Click "Create new secret key"
# 4. Copy the key (starts with sk-...)
# 5. Add billing information and credits

Google Gemini Setup

# Visit: https://makersuite.google.com/app/apikey
# 1. Sign in with Google account
# 2. Create new project (if needed)
# 3. Enable Gemini API
# 4. Generate API key
# 5. Copy the key for use in PageCrawl.io

Anthropic Claude Setup

# Visit: https://console.anthropic.com/
# 1. Create account or sign in
# 2. Navigate to API Keys
# 3. Create new key
# 4. Copy the key (starts with sk-ant-...)
# 5. Add credits to your account

2. Configure in PageCrawl.io

Access AI Settings

1. Log into your PageCrawl.io account
2. Navigate to Settings → Integrations
3. Scroll to AI Features section
4. Click Configure AI Integration

Add Your API Key

1. Select Provider: Choose from OpenAI, Google Gemini, or Anthropic
2. Enter API Key: Paste your API key securely
3. Choose Model: Select the AI model for your use case
4. Test Connection: Verify the key works with a test request
5. Save Configuration: Store your settings securely

Enable Features

Toggle the features you want to use:

• ✅ AI-Powered Summarization: Intelligent change summaries
• ✅ Signal Prioritization: Automatic importance ranking
• ⚙️ Custom Settings: Per-page feature control

3. Page-Level Configuration

Global Settings

Apply AI features to all monitored pages:

1. Go to Settings → Workspace → AI Features
2. Set default behavior for new pages
3. Choose which features are enabled by default

Individual Page Settings

Configure AI features for specific pages:

1. Navigate to your Pages list
2. Click on a page to edit its settings
3. Find the AI Analysis section
4. Toggle features on/off per page:
   • Summarization: Get AI summaries for this page
   • Prioritization: Apply smart prioritization
   • Custom Instructions: Add context for better analysis

Advanced Configuration

Model Selection Guidelines

For Cost Optimization:

• High Volume Pages: Use GPT-3.5-turbo or Claude Haiku
• Simple Changes: Gemini Pro for basic summarization
• Budget Conscious: Start with the most affordable models

For Accuracy:

• Critical Pages: Use GPT-4 or Claude Opus
• Complex Content: Claude Sonnet for nuanced analysis
• Technical Content: GPT-4-turbo for detailed summaries

For Speed:

• Real-time Alerts: Claude Haiku or GPT-3.5-turbo
• High Frequency: Gemini Pro for quick processing
• Immediate Response: Fastest available models

Usage Optimization

Token Management

• Monitor Usage: Check daily/weekly consumption
• Set Limits: Configure monthly usage caps
• Cost Alerts: Get notified when spending thresholds are reached

Troubleshooting

Common Issues and Solutions

Authentication Errors

# Error: "Invalid API key"
Solution:
1. Verify key is copied correctly (no extra spaces)
2. Check key hasn't expired
3. Ensure sufficient credits/quota
4. Confirm key has required permissions

Rate Limiting

# Error: "Rate limit exceeded"
Solution:
1. Upgrade your provider plan
2. Implement usage throttling
3. Switch to higher-tier models
4. Contact provider support

High Costs

# Issue: Unexpected high usage
Solution:
1. Review which pages have AI enabled
2. Check for very large page changes
3. Consider using cheaper models
4. Implement usage caps

Inaccurate Results

# Issue: Poor AI summaries/prioritization
Solution:
1. Try different AI models
2. Provide more page context
3. Adjust feature settings
4. Contact support for prompt optimization

Provider-Specific Issues

OpenAI

• Quota Issues: Add more credits to your account
• Model Access: Ensure you have access to chosen models
• API Changes: Monitor OpenAI's changelog for updates

Google Gemini

• Quota Limits: Check Google Cloud quotas and limits
• Regional Availability: Verify service availability in your region
• Authentication: Ensure proper Google Cloud project setup

Anthropic Claude

• Credit Balance: Monitor your Anthropic credit usage
• Model Availability: Check which models are available in your region
• Usage Policies: Review Anthropic's usage guidelines

Support and Resources

Getting Help

1. Documentation: Review this guide and related articles
2. Support Tickets: Contact PageCrawl.io support with specific issues
3. Community Forum: Join discussions with other users
4. Provider Support: Contact AI provider support for API-specific issues

Additional Resources

• API Documentation: Provider-specific API documentation
• Cost Calculators: Estimate usage costs before implementation
• Best Practices: Industry guides for AI integration
• Compliance Guides: Data privacy and security requirements

Updates and Maintenance

• Feature Updates: Regular enhancements to AI capabilities
• Provider Updates: New models and features from AI providers
• Security Updates: Regular security patches and improvements
• Performance Optimizations: Ongoing improvements to speed and efficiency

The AI BYOK integration provides powerful capabilities while maintaining your control over costs, data, and service levels. Start with a small test deployment and gradually scale based on your results and requirements.