Search APIs are good at telling you where information lives. They’re less helpful when you need the information itself. If you’ve built anything that depends on web data, such as a research tool, a price tracker, or an AI agent that answers questions, you’ll know a list of links is only the starting point. Something still has to fetch each page, strip out the navigation and ads, and pull out the parts you care about.
SerpApi, best known for its search engine results APIs, is aiming to close that gap with its Extract API. Launched on October 13, it takes the URL of a public web page and returns the contents in Markdown format, with additional metadata available in JSON. It can also list the pages on a website and run a limited crawl, using the same account and API that developers already use for SerpApi’s search engines.
- Key highlights
-
Turns public web pages into structured JSON or Markdown; scrape, map and crawl modes; crawls up to 100 pages and three links deep per reques
- Brand
-
SerpApi
- Developer(s)
-
SerpApi, LLC
- Price model
-
Choice of 4 month-to-month contracts
- Platform
-
Platform independent
SerpApi’s Extract API fetches a public web page and returns its content as structured JSON or Markdown, ready for apps, research tools and AI agents. It parses metadata, headings, links, tables, media and full text. It can also list a website’s URLs or crawl up to 100 pages per request. Extract API works alongside SerpApi’s search APIs, using the same account, libraries and search credits.
At its simplest, you give Extract API a URL, and it returns a breakdown of the page. The request goes to the same endpoint as SerpApi’s other engines, with the engine set to “extract” and the page address passed as the URL parameter.
The response depends on the requested format. The Markdown output mode provides a converted version of the page, complete with headers, inline images, links, and other data in standard Markdown format. The JSON output mode includes the converted Markdown, with additional metadata like Open Graph tags, JSON-LD data, lists of all links and embedded media, and other information. Sections that don’t apply to a page are left out of the response instead of being returned empty.
Every link in the response includes a ready-made Extract API request for that URL, so following a link to the next page is one more call. Pages that sell something can also return a price offers section, which normalizes product pricing where the parser can find it.
If you only want part of a page, you can say so up front. The include_tags and exclude_tags parameters accept CSS selectors, so you can keep just the main article body or remove the menu, footer, and cookie banner before the page is parsed. That keeps irrelevant text out of whatever you feed the results into.
Three ways to work with a website
Extract API has three modes. Scrape mode is the default and handles a single page. Map mode lists a website’s URLs, drawing on the links in the starting page and on the site’s sitemaps. And crawl mode follows links on the same website and extracts each page it visits.
Crawling has firm limits. You can set a maximum number of pages, up to 100, and how many links deep the crawler goes from the starting page, up to three. By default, it stays on the same host, although you can choose to include subdomains. SerpApi calls this bounded crawling. It’s built to collect a defined set of pages from one site, not to index large parts of the web.
Map and crawl modes work well together. In one of SerpApi’s own examples, a developer maps an online store’s Wi-Fi product category, filters the results down to product pages, then extracts each one to read product names and prices from the product data available in the JSON response.
Crawl usage is billed per successfully extracted page, including the starting page. Pages that fail to extract and responses served from cache aren’t counted.
JSON, Markdown, or raw HTML
Markdown is the Extract API’s primary output because it works well with large language models (LLMs). The JSON output mode includes the converted Markdown, while adding page metadata like Open Graph cards and a list of all inline links. Finally, the page’s original HTML is also available, allowing developers to build their own parser on top of SerpApi’s infrastructure if needed.
Search and extraction in one workflow
The more interesting part of the launch is how Extract API fits with the rest of SerpApi. The company already offers more than 100 specialized APIs covering sources like Google, Bing, and YouTube. Pairing those with Extract API means an application can run a search, take the URLs from the results, and read the pages behind them, all with one service. Developers often handle search, extraction, and crawling with separate tools, so keeping everything under one account, one set of libraries, and one pool of search credits removes some of the glue code.
SerpApi gives an SEO example. A short script fetches the top Google results for a keyword, sends each URL to Extract API, and prints every page’s external link. That gives you a quick look at how competing articles are organized without opening each one in a browser.
Extract API also routes requests sensibly. If you pass it a URL that one of SerpApi’s dedicated engines already covers, such as a YouTube video, an Amazon product, or a Google search page, the request goes to that engine and returns its schema. You get the more detailed, purpose-built output without having to check which engine applies first.
Who it’s for
AI agents are the most obvious fit. SerpApi’s MCP server, command-line tool, and other integrations already let agents search the web and read the results. With Extract API, they can open the pages behind those results, too, so an agent can check a claim against the source or pull supporting evidence instead of relying on a two-line snippet.
SerpApi also points to research platforms that gather information from public websites, product and market monitoring, and SEO work. Internal knowledge and retrieval systems that need fresh material from the web are another fit. What these have in common is a need for page content in a predictable shape, without someone writing and maintaining a custom scraper for every site.
Limitations worth knowing
Extract API uses a general-purpose parser instead of site-specific rules, and SerpApi is upfront that support varies from site to site. A clean page with good structured data usually returns more useful output than a messy one.
It also doesn’t run JavaScript. Pages that load content in the browser after the initial HTML arrives may return less than you see on screen, and the documentation notes that the optional pre-fetch delay isn’t a rendering delay. If your project depends heavily on JavaScript-rendered pages, or you need to crawl far more than 100 pages at a time, a dedicated scraping or browser automation tool may still be a better fit.
Finally, Extract API follows each site’s robots.txt rules using the “serpapi” user agent. If a site blocks crawlers from a page, the request returns an error. Map mode leaves out disallowed URLs, and crawl mode won’t follow disallowed links. Site owners who want to block Extract API specifically, while still allowing other crawlers, can add a rule for that user agent. That’s a sensible default for a tool that reads other people’s websites, though it does mean some pages will be off limits.
Getting started
Extract API is available now on all SerpApi accounts. You need an account and an API key, and you can try requests in SerpApi’s browser-based Playground before writing any code. The Extract API documentation covers every parameter and output section in detail, and SerpApi’s announcement post at serpapi.com/blog/introducing-extract-api/ walks through the SEO and product-mapping examples step-by-step.
