How to Easily Set Up Proxies in Octoparse for Smooth Web Scraping

Octoparse is a popular, no-code web scraping tool that helps you extract data from websites effortlessly. To avoid IP bans and manage web traffic limits, using proxies is essential—especially when working with large-scale projects. Luckily, configuring proxies in Octoparse is straightforward and enhances your scraping workflow by enabling automatic IP rotation and extended session handling.

In this guide, we'll walk you through the step-by-step process of integrating proxies with Octoparse, using DataImpulse as a reliable proxy provider.

Why Use Proxies with Octoparse?

Octoparse can handle complex website structures using machine learning, extracting various data formats like text, image URLs, links, and HTML content. However, many websites impose restrictions based on IP addresses, limiting how frequently you can access data.

Proxies allow you to:

• Rotate IP addresses to avoid IP blocks
• Maintain sessions without being flagged
• Manage scraping speed and concurrency safely

By adding your IP to a whitelist and configuring proxies in Octoparse, you can scrape data more efficiently and securely.

Step 1: Add Your IP Address to the Proxy Whitelist

Before connecting Octoparse to proxies, your IP must be whitelisted, so you won’t need to enter login credentials every time.

How to whitelist your IP:

1. Choose a proxy plan that suits your needs on DataImpulse (or another provider).
2. Go to the Manage Whitelist IPs section on your proxy dashboard.
3. Click Detect my IP or enter your IP address manually.
4. Hit Add new IP to whitelist it.

Once your IP is added to the whitelist, you're ready to configure proxies in Octoparse.

Step 2: Install and Launch Octoparse

• Download and install Octoparse from its official website.
• Open the application after installation.

Step 3: Create a New Custom Task in Octoparse

• Click the +New button in the top-left corner.
• Select Custom Task to start configuring your scraping job.

Step 4: Enter the Target Website

• In the URL input field, type the webpage URL you want to scrape.
• For demonstration, enter: http://books.toscrape.com
• Click Save to proceed.

Step 5: Access Anti-blocking Settings

• Once the webpage loads in Octoparse, click the Settings button at the top-right.
• Scroll down to locate the Anti-blocking Settings section.

Step 6: Enable Proxy Usage

• Check the box labeled Access websites via proxies.
• This action reveals proxy configuration options and a Configure button.

Step 7: Enter Your Proxy Details

• Click Configure to open the proxy setup window.
• Paste your DataImpulse proxy IP addresses in the format IP:PORT.

Example:

148.251.5.30:823

Step 8: Customize Proxy Rotation Settings

• Decide on your IP switching interval.
• This depends on whether you're using rotating proxies (which regularly change IPs) or sticky sessions (which maintain the same IP for longer).

Step 9: Save Your Proxy Configurations

• Click Confirm to save your proxy settings.
• Back in the Anti-blocking Settings, ensure there’s a checkmark beside the Configure button.
• Then, click Save to finalize the changes.

Step 10: Build Your Scraping Workflow

• Return to the main scraping screen.
• Click the lightbulb icon to expand options for pagination or page scrolling.
• Choose the method to navigate through pages.
• Click Create Workflow to start designing your extraction steps.

Step 11: Select Elements to Extract

• Click on the item you want to scrape (e.g., “Mystery” category text).
• Choose Extract text of the selected element.
• A popup will appear—press Save at the top-right.

Step 12: Run Your Scraping Task

• Click Run to execute the job.
• You’ll see options like running locally or cloud-based (some may require additional payment).
• For the example, select Run on your device and Standard mode.

Step 13: Monitor and Control the Scraping Process

• A new window opens and the scraping process starts.
• You can pause or resume as needed.
• When done testing, stop the run.

Step 14: Export Your Data

• After stopping, you’ll see task statistics.
• Choose when to export your data—now or later.
• Select your preferred data format (CSV, Excel, JSON, etc.) in the final popup.

Wrapping Up

Integrating proxies into your Octoparse workflow adds a vital layer of flexibility and stability to your web scraping projects. Using DataImpulse residential proxies helps you avoid tracking and IP blocks while maintaining fast and reliable data extraction. Follow this guide to set up proxies quickly and unleash the full potential of Octoparse in your next scraping task.

Happy scraping!