Web scraping tools: 10 APIs, proxies, and frameworks ranked for 2026

Best fit

Enterprise data teams running high-volume, compliance-sensitive scraping across hundreds of target sites. A poor fit for solo developers on a tight budget.

Bright Data runs 400M+ residential IPs, 1.3M+ ISP proxies, and 1.3M+ datacenter proxies across 195 countries (brightdata.com, accessed August 24, 2026), the largest published proxy footprint in this ranking.

Its Web Scraper APIs, SERP API, Web Unlocker, and Browser API cover structured extraction across 800+ sites, backed by a Dataset Marketplace of 600+ pre-collected domains and more than 5 billion regularly refreshed records.

Key features

• Web Scraper APIs from $0.75 per 1,000 records.
• SERP API from $1 per 1,000 requests across multiple search engines.
• Web Unlocker API from $1 per 1,000 requests for CAPTCHA and block bypass.
• Browser API from $5/GB for remote stealth browser sessions.
• Dataset Marketplace covering 600+ pre-collected domains from $250 per 100,000 records.
• 5,000 free monthly credits on new accounts across Web Unlocker, SERP API, and Web Scraper API.

Pricing

Residential proxies start at $2.50/GB, datacenter proxies from $0.90/IP, and ISP proxies from $1.30/IP (brightdata.com/pricing, accessed August 24, 2026). Zone bandwidth limits recalculate only every 15 minutes, so usage can run over a configured cap for up to 15 minutes before enforcement catches up (docs.brightdata.com).

Pros

• The largest proxy footprint in this ranking, at 400M+ residential IPs across 195 countries.
• One vendor covers proxies, scraper APIs, unlocking, browser automation, and datasets, cutting tool sprawl.
• A 99.99% uptime guarantee and a claimed 99.95% success rate.

Cons

• Entry-level accounts pay meaningfully higher per-unit rates than enterprise accounts, the top complaint on G2.
• Accounts can be suspended mid-job on usage triggers with no advance warning, per user reports.
• Corporate-tier discounts require serious volume commitments most small teams won't reach.

"A bit tricky to understand pricing as we are being provided with different information. From 0$ to 250$ or even 200 000$."

Monika V., IT Project Manager, G2, May 2026

The largest network in this ranking

Get started with Bright Data

One vendor for proxies, scraper APIs, unlocking, browser automation, and datasets, from $0.75 per 1,000 records.

Visit Bright Data →

May earn a commission if you sign up through this link, at no extra cost to you. Full affiliate disclosure.

Why it's ranked #1. Bright Data's 400M+ residential IPs, 195-country reach, and one vendor covering proxies, scraper APIs, unlocking, browser automation, and datasets beat every narrower specialist ranked below it, the breadth that puts it ahead of TinyFish's smaller, agent-only platform at #2. Its entry-level per-unit pricing still runs the highest of any tool in the top five, the tradeoff for owning the largest published footprint in the category.

Best fit

Data and engineering teams scraping authenticated, multi-step pages (logins, forms, dashboards) who want a managed, pay-per-step agent over building browser automation and anti-bot handling in house.

TinyFish sells four APIs: Search and Fetch are free on every plan, Browser runs $0.002 a minute, and Agent runs $0.016 a step, all funded from a wallet with no monthly minimum (tinyfish.ai/pricing, accessed August 24, 2026).

Its homepage claims 89.9% accuracy on the Mind2Web benchmark, a figure partly checked by a third party: analyst Mersault ran the separate WebVoyager benchmark, 641 tasks across 15 sites, and got 91.1% in May 2026 (tinyfish.ai/blog/most-accurate-ai-web-agent).

Key features

• Search API: free, 30 requests a minute on the self-serve tier.
• Fetch API: free, 150 URLs a minute, renders the page first and returns markdown, JSON, or HTML, never raw HTML.
• Browser API: a native stealth Chromium profile with residential proxy routing across seven countries bundled into every session.
• Agent API: turns a plain-language goal into a multi-step workflow, two concurrent runs on the self-serve tier.
• MCP-native support for Claude and Cursor, plus prebuilt connectors for n8n, Dify, and ChatGPT.
• Publishes all 300 execution traces behind its benchmark as a public spreadsheet for outside checking.

Pricing

Search and Fetch cost nothing; Browser runs $0.002 a minute and Agent $0.016 a step, wallet-funded with an $8 starting credit and no monthly minimum (tinyfish.ai/pricing). Enterprise pricing is custom.

Pros

• A free, permanent Search and Fetch tier with no card required.
• Publishes its benchmark's failure traces for outside review, unusual in this category.
• A $47M Series A led by ICONIQ Capital, with named production customers including Google, DoorDash, and Amazon.

Cons

• Zero submitted reviews on G2 or Gartner Peer Insights as of August 2026, so there's no independent buyer record to check a purchase against.
• Its own homepage (89.9%) and its own blog post (81%) currently disagree on the same Mind2Web benchmark by nine points.
• 12 of the 40 failures in its own benchmark were anti-bot blocks, eight of them concentrated on one site.

"The failure traces being public is a nice touch. Looked through a few and they're actual failures, not cherry-picked easy ones. Most companies in this space wouldn't do that."

ivywho, Hacker News, February 2026

"I assume you used your own product to hype up this post? I was excited about it till I saw this behavior. Please operate ethically next time."

pants2, Hacker News, February 2026

Why it's ranked #2. TinyFish is the only tool here with native MCP support on every plan and a free, uncapped Search and Fetch tier alongside its Agent API, an agent-integration edge nothing else in the top five matches. It ranks below Bright Data because a 400M+ IP network and five years more operating history outweigh TinyFish's zero G2 reviews and single year since founding, the proof gap that keeps an AI-native platform at #2, not #1.

Best fit

Enterprise and mid-market teams running large, recurring scraping jobs (price monitoring, SERP tracking, AI training data) who can commit to real monthly volume.

Oxylabs runs 175M+ residential IPs and 20M+ mobile IPs across 195+ countries, with a claimed 99.95% average success rate on residential proxies and roughly a 0.6-second response time (oxylabs.io, accessed August 24, 2026).

Its Web Scraper API adds pre-built endpoints for Amazon, Google, ChatGPT, Gemini, Walmart, eBay, Target, and 100+ other named targets, and its OxyCopilot layer turns a URL plus a plain-language prompt into working scraping and XPath parsing code (oxylabs.io/oxycopilot-story).

Key features

• Web Scraper API from $0.25 per 1,000 results, varying by target.
• A 175M+ residential proxy pool with city-level targeting and sticky sessions up to 24 hours.
• Automatic CAPTCHA handling and JavaScript rendering, billed separately at $1.35 per 1,000 results.
• 30+ integrations including Cursor, LangChain, CrewAI, Zapier, and n8n.
• Pre-built scrapers for 100+ named sites, cutting custom parser work.
• 100 patents filed as of 2024, underpinning its XPath pattern-recognition layer.

Pricing

The Web Scraper API starts at $0.25 to $1.15 per 1,000 results with a $49/month minimum plan; residential proxies tier from $6/GB at a 5GB minimum down to $2.50/GB at a 1TB minimum (oxylabs.io/pricing, accessed August 24, 2026).

Pros

• 175M+ residential IPs and 20M+ mobile IPs across 195+ countries.
• Pre-built scrapers for 100+ named sites cut custom development work.
• A 99.95% average success rate on residential proxies, per the vendor's own published figure.

Cons

• JS rendering runs as a separate line item at $1.35 per 1,000 results, stacking on top of base pricing.
• The best corporate-rate discount needs a 1TB/month minimum to reach $2.50/GB.
• A G2 reviewer called support "one of the worst experiences I've ever encountered" (December 2025).

"It's definitely on the higher end cost-wise if you're not committing to decent volume."

Scott W., Media Specialist, G2, July 2026

Why it's ranked #3. Oxylabs' 100+ named pre-built scrapers beat ScrapingBee's smaller target list below it, but its 175M+ residential pool trails Bright Data's 400M+ above it, and its JS-rendering surcharge stacks on top of per-result pricing most rivals bundle in.

Best fit

Developers who need one API to render JavaScript-heavy pages and pull e-commerce or SERP data without maintaining headless-browser or proxy infrastructure.

ScrapingBee renders pages through headless Chrome by default for 5 credits a request, adds rotating or premium proxies for 10 to 25 credits, and runs a stealth proxy pool for the hardest sites at 75 credits a request, with geotargeting across 100+ country codes (scrapingbee.com/documentation, accessed August 24, 2026).

It also ships dedicated Google Search and Amazon APIs, the Google endpoint claiming a 99.9% success rate.

Key features

• Headless Chrome rendering with JS execution at 5 credits per request.
• A Google Search API returning organic results, knowledge graph, and ads in one call.
• An Amazon API with separate Product, Pricing, and Search endpoints.
• A stealth proxy pool at 75 credits per request for the toughest anti-bot sites.
• AI-powered extraction via ai_extract_rules and ai_query parameters.
• Native n8n, Make, Zapier, and MCP Server integrations for Claude and Cursor.

Pricing

Plans start at $49/month for 250,000 credits and 50 concurrent requests, scaling to $599/month for 8,000,000 credits (scrapingbee.com/pricing, accessed August 24, 2026), plus a 1,000-credit free trial with no card required. Oxylabs' company group acquired ScrapingBee in June 2025.

Pros

• A 1,000-credit free trial with no card required.
• A 4.9/5 rating across 137 verified reviews on Capterra.
• Dedicated Google and Amazon endpoints beyond a generic HTML fetch.

Cons

• JS rendering costs 5x a plain request, and premium proxy jumps from 10 to 25 credits once JS turns on.
• The stealth proxy needed for the hardest sites runs 75 credits a request.
• The entry plan caps at 250,000 credits a month, around 50,000 JS-rendered scrapes before an upgrade.

"Credits are consumed quickly when using JavaScript rendering or advanced features, making it harder to justify for smaller projects."

Nick S., Manager, Capterra, January 2026

Why it's ranked #4. ScrapingBee bundles headless rendering and a stealth proxy pool that Proxy-Seller doesn't offer below it, but its credit costs stack fastest under load of any API here, and it lists fewer pre-built targets than Oxylabs above it.

Best fit

Teams running multi-account or multi-geo scraping workflows (SERP tracking, price monitoring, ad verification) who want raw proxy infrastructure to plug into their own scraper. A managed extraction API is the wrong shape for this buyer.

Proxy-Seller runs a 47M+ IP residential pool across 220+ locations with roughly a 0.35-second average response time, backed by ISO/IEC 27001:2022 certification and GDPR, CCPA, and DPA/SCC compliance documentation (proxy-seller.com/about-us, accessed August 24, 2026).

It sells five proxy types under one account (residential, mobile, IPv4, ISP, IPv6), from $0.02 per IPv6 address to $10 per mobile IP, and reports 185,000+ users with 99.7% uptime.

Key features

• Five proxy types on one account: residential, mobile (Vodafone, T-Mobile, Orange, 20+ countries), IPv4, ISP (AT&T, Windstream, Frontier), and IPv6.
• HTTP, HTTPS, and SOCKS5 protocol support across IPv4, ISP, and IPv6.
• Rotation by time interval, by request, or sticky session, with session TTL control via API.
• An API with SDKs for PHP, Node.js, Python, Java, and GoLang.
• ISO/IEC 27001:2022 and ISO 9001:2015 certification, with GDPR, CCPA, and DPA/SCC compliance documentation.
• Unlimited bandwidth on IPv4 and ISP plans with channels up to 1 Gbps.

Pricing

IPv4 starts at $0.49/IP, IPv6 at $0.02/IP, ISP at $0.98/IP, and mobile at $10/IP (proxy-seller.com, accessed August 24, 2026). Residential proxies list a $1.30/GB figure in the page's meta description, but the visible pricing table runs $3.50/GB down to $2.20/GB at the 100GB tier, with a $1.99 three-day trial (proxy-seller.com/residential-proxies, accessed August 24, 2026).

Pros

• Five proxy types under one account and API, matching proxy type to target site without switching vendors.
• ISO/IEC 27001:2022 certification with published GDPR, CCPA, and DPA/SCC compliance documentation.
• A 4.8/5 rating across 154 reviews on G2.

Cons

• No bundled scraping API, parser, or unblocker layer; it sells proxy infrastructure only.
• The advertised $1.30/GB residential rate isn't the price a buyer sees on the live table; the real floor is $2.20/GB.
• Its Fair Use Policy threshold is undisclosed and dynamic, per a support exchange logged on G2.

"Their staff is live 24/7, not a bunch of BOTs. So I had to set up mobile proxies in an anti-detect browser. I do not have any technical experience but the customer support staff assisted me into a real quick setup."

David C., Paid Advertising Manager, G2, August 2026

Why it's ranked #5. Proxy-Seller's $0.02 IPv6 entry price undercuts every metered API ranked below it, but it ships no bundled rendering or unblocking layer the way ScrapingBee does above it, so a buyer still has to pair it with a scraper of their own.

Best fit

Developers and small-to-mid teams who want one API call to handle proxy rotation, CAPTCHAs, and JS rendering without running their own scraping infrastructure.

ScraperAPI routes requests through 40M+ proxies across 50+ countries and bundles JS rendering, premium proxies, and automatic bypass of Cloudflare, Datadome, and PerimeterX into every plan (scraperapi.com, accessed August 24, 2026). The company reports over 11 billion requests served in the trailing 30 days for 10,000+ customer companies, with a 99.9% uptime guarantee.

Key features

• Structured data endpoints for Amazon Product, Google Search, and Walmart Search.
• Bot-protection bypass for Cloudflare, Datadome, and PerimeterX at +10 credits when triggered.
• JS rendering through a single render=true parameter at +10 credits.
• Geotargeting by country code, though full global targeting needs the Business tier and above.
• An async API, a proxy port, and a DataPipeline service alongside the standard endpoint.
• Unlimited bandwidth on every tier, with no separate bandwidth bill.

Pricing

The Hobby plan starts at $49/month for 100,000 credits and 20 concurrent threads, scaling to $1,975/month for 21.5 million credits (scraperapi.com/pricing, accessed August 24, 2026). A 7-day trial includes 5,000 free credits with no card required. Credit multipliers stack: Google and Bing searches run 25 credits, LinkedIn 30, and bot-protection bypass adds 10 more on top of the base cost.

Pros

• A flat entry price with unlimited bandwidth included on every tier.
• 40M+ proxies across 50+ countries with no separate bandwidth bill.
• Over 11 billion requests served across 10,000+ customer companies in the trailing 30 days.

Cons

• Credit multipliers on SERP and social targets can turn a 1-credit scrape into 20 or more once JS and bypass both trigger.
• Full global geotargeting is locked behind the $299/month Business tier.
• A Trustpilot reviewer reported an undisclosed 5-credit multiplier that shrank a 60-million-request plan to roughly 12 million usable requests.

"After paying, they applied a 5-credit multiplier without ever disclosing this upfront, which meant our 60M plan was only worth 12M requests, a massive shortfall."

verified user, Trustpilot, August 2025

Why it's ranked #6. ScraperAPI's 99.9% uptime guarantee and bundled CAPTCHA and JS handling beat Scrapy's fully self-managed setup below it, but its $49/month floor and stacking credit multipliers cost more at low volume than Proxy-Seller's $0.02-per-IP entry above it.

Best fit

Engineering teams comfortable writing Python who need a self-hosted, code-first framework for large or recurring crawls over a point-and-click tool.

Scrapy is a free, open-source Python crawling framework first released in August 2008, now carrying 62,100 GitHub stars, 11,600 forks, and 55,200 dependent repositories (github.com/scrapy/scrapy, accessed August 24, 2026). Its latest stable release, v2.16.0, shipped May 19, 2026 with official Python 3.14 support, and an experimental reactor-less mode landed the month before.

Key features

• Spiders: self-contained crawler classes defining how a site gets followed and parsed.
• Parsel selectors chaining CSS, XPath, and regex extraction in a single call.
• Item pipelines for post-extraction validation, cleaning, and storage hooks.
• A middleware, extension, and signal system for pluggable request and response processing.
• Built-in feed exports to JSON, CSV, and S3.
• An async engine on Twisted, with an experimental httpx-based download handler added in 2026.

Pricing

Free, under a BSD-3-Clause license (github.com/scrapy/scrapy). There's no vendor fee, but no bundled hosting or proxies either. Teams supply their own servers, their own proxy or anti-block layer such as scrapy-zyte-api, and their own JS-rendering add-on for any target that needs one.

Pros

• 62,100 GitHub stars and 55,200 dependent repositories, the largest install base of any tool in this ranking.
• 18 years of continuous development, with three releases shipped in the two months before August 2026.
• Parsel selectors chain CSS, XPath, and regex extraction in a single call, without a second parsing library.

Cons

• No native JavaScript rendering; JS-heavy targets need the separate scrapy-playwright add-on.
• No built-in proxy or anti-bot handling, so IP rotation and CAPTCHA evasion are the team's own infrastructure to build.
• 437 open issues on the main repo at time of writing, the maintenance load of a widely used volunteer project.

"1,000 pages in ~8 minutes vs 2+ hours... ~$1-3 in tokens per website with Sonnet 4.5, not per page."

iranu, Hacker News, March 2026

Why it's ranked #7. Scrapy's 62,100 GitHub stars and 55,200 dependent repos beat Crawlee's smaller, younger install base below it, but it ships with no managed proxies or uptime guarantee, the gap that keeps ScraperAPI's bundled infrastructure ranked above it.

Best fit

Node.js and TypeScript engineering teams who want a code-first framework with built-in queueing, proxy rotation, and headless-browser support over hand-built Puppeteer or Playwright plumbing.

Crawlee is a free, open-source Node.js and TypeScript library maintained by Apify, carrying 25,200 GitHub stars, 1,600 forks, and 165,462 weekly npm downloads for the week of August 16-22, 2026 (github.com/apify/crawlee, accessed August 24, 2026).

It ships three unified crawler classes: CheerioCrawler, PuppeteerCrawler, and PlaywrightCrawler, the last supporting Chromium, Chrome, Firefox, and WebKit in one interface.

Key features

• RequestQueue: a persistent, disk-backed URL queue supporting breadth- or depth-first crawls.
• SessionPool: rotates proxy IPs bound to cookies and headers, with retire and mark-good lifecycle controls.
• AutoscaledPool: scales concurrency automatically to available system resources.
• Three crawler classes covering plain HTTP parsing through full multi-browser automation.
• HTTP2 support with browser-like request headers via got-scraping.
• Full TypeScript typing with generics and a CLI project bootstrapper.

Pricing

Free, under the Apache License 2.0 (github.com/apify/crawlee). As with Scrapy, hosting and headless-browser compute are the user's own cost; Apify separately sells managed cloud hosting that can run Crawlee code, a fact worth knowing even though this ranking doesn't score the Apify platform itself.

Pros

• One unified API spanning plain HTTP scraping and full headless-browser automation.
• 25,200 GitHub stars and 165,462 weekly npm downloads, evidence of an active, non-abandoned project.
• Random session rotation built to avoid burning out a small proxy pool, per its own session-management docs.

Cons

• Requires real Node.js and TypeScript proficiency; there's no mature Python-native path in the original package.
• A documentation gap was flagged directly by a Hacker News user the same week as the quote below.
• Its own docs concede: "Crawlee won't fix broken selectors for you (yet)"; layout changes still break scrapers same as any other tool.

"You'll want to prioritize documenting the existing features, since it's no good having a super awesome full stack web scraping platform if only you can use it."

mdaniel, Hacker News, July 2024

Why it's ranked #8. Crawlee's single API for HTTP and full multi-browser automation beats Firecrawl's hosted-first credit model below it on self-hosted flexibility, but its 25,200 GitHub stars trail Scrapy's 62,100 and 18-year track record above it.

Best fit

Teams building LLM or RAG pipelines that need public pages converted into clean, structured Markdown or JSON, never raw HTML.

Firecrawl turns URLs into LLM-ready Markdown or structured JSON through /scrape, /crawl, /map, and /extract endpoints, with a core engine licensed AGPL-3.0 and MIT-licensed SDKs, carrying 166,200 GitHub stars (github.com/firecrawl/firecrawl, accessed August 24, 2026).

Hosted pricing starts at $16/month for 5,000 credits and runs to $599/month for 1,000,000 credits, with a free tier of 1,000 credits (firecrawl.dev/pricing).

Key features

• LLM-ready output in Markdown, HTML, structured JSON, and screenshots.
• An /extract endpoint for LLM-driven structured extraction from a prompt or JSON schema.
• A self-hostable core engine plus a hosted cloud version on proprietary infrastructure.
• SDKs across eight languages plus a native MCP server for Claude- and Cursor-style agent tooling.
• An /interact endpoint for click, fill, and navigate actions inside a scrape.
• Credit pricing at 1 credit per page for scrape, crawl, and map calls.

Pricing

The free plan gives 1,000 credits a month at 2 concurrent requests; Standard runs $83/month for 100,000 credits, Growth $333/month for 500,000, and Scale $599/month for 1,000,000 (firecrawl.dev/pricing, accessed August 24, 2026).

Pros

• Fully open source under AGPL-3.0 and self-hostable, avoiding lock-in for teams that want to run it themselves.
• 166,200 GitHub stars and 9,300 forks, among the largest open-source projects in this ranking.
• Eight-language SDK coverage plus a native MCP server for direct agent use.

Cons

• Developer and API-first, with no visual, no-code workflow builder.
• A TinyFish-run head-to-head test (a competitor's own claim, not an independent measurement) put Firecrawl at 1 of 6 successes against actively bot-protected sites.
• Its own G2 review to date flags a real limit on anti-bot depth.

"Sometimes it's hard to get the depth right or it still hits anti bot mechanisms."

verified user, Consulting, G2, May 2026

Why it's ranked #9. Firecrawl's AGPL-3.0 self-hostable core and 166,200 GitHub stars beat Browse AI's closed, non-technical platform below it on developer control, but its /interact automation runs hosted and credit-billed over self-hosted free, the depth that keeps Crawlee ranked above it.

Best fit

Non-technical teams and solo operators who need point-and-click website monitoring and extraction with no engineering resource to spend.

Browse AI is a no-code scraping and monitoring platform built around a visual "robot" recorder that trains by clicking through a live page, no script required (browse.ai, accessed August 24, 2026). It reports 770,000+ users and a marketplace of 250+ prebuilt robots for sites like Amazon, eBay, Airbnb, and LinkedIn, plus a Chrome extension with 100,000+ installs and a 3.9/5 rating from 44 raters on the Chrome Web Store.

Key features

• A visual, no-code robot recorder trained by clicking through the target page.
• 250+ prebuilt robot templates for common sites.
• Scheduled monitoring down to a 5-minute interval on the Professional plan, with change-detection alerts.
• Google Sheets sync plus Airtable, Zapier, Make, and 7,000+ connected apps in total.
• A REST API, webhooks, and CSV/JSON export, including Amazon S3 storage.
• A credit system where 1 credit covers 10 extracted rows or one screenshot.

Pricing

The free plan gives 50 credits a month across 2 sites; Personal runs $48/month ($19 billed annually) for 2,000 credits, Professional $87/month ($69 annually) for 5,000 credits, and Premium starts at $500/month for 600,000+ credits a year (browse.ai/pricing, accessed August 24, 2026). Harder, bot-protected sites carry a 2-10 credit minimum per run regardless of row count.

Pros

• A genuine no-code setup: robots train by clicking through the page, with no scripting required.
• A usable free tier (50 credits, 2 sites, 3 users) before any card is required.
• 770,000+ users and 250+ prebuilt robot templates for sites like Amazon and LinkedIn.

Cons

• Heavy JavaScript sites and bot protection need extra manual tweaking, per multiple reviewers.
• No support for two-factor-authentication-gated pages.
• Credit-based pricing scales up fast once a workflow needs more than the free tier's 50 credits.

"Websites protected by hCAPTCHA and reCAPTCHA and IFRAME doesn't work."

Feiga M., Manager, Capterra, March 2023

Why it's ranked #10. Browse AI's genuine no-code setup and 770,000+ users make it the clearest pick here for non-technical buyers, but it can't self-host, script, or convert pages to LLM-ready output the way Firecrawl does above it, the developer-facing coverage gap that puts it last.