Skip to content
GPTSheetsPlus

Web scraping

The scraping functions fetch a web page through Bright Data Web Unlocker, which handles anti-bot blocks, CAPTCHAs and JavaScript rendering. You get 10 free scrapes in the trial. After that you need a Bright Data account: create a Web Unlocker API zone, copy the key from Account settings → API keys, and add it via Set API Keys → Insert Bright Data API Key. Bright Data offers a free tier for Web Unlocker.

Functions

FunctionReturns
=SCRAPE(url, [maxAge], [waitFor])page content as markdown
=SCRAPE_HTML(url, [maxAge], [waitFor])raw HTML
=SCRAPE_TITLE(url, [maxAge], [waitFor])the <title> of the page
=SCRAPE_DESCRIPTION(url, [maxAge], [waitFor])the meta description
=SCRAPE_KEYWORDS(url, [maxAge], [waitFor])the meta keywords
=SCRAPE_LANGUAGE(url, [maxAge], [waitFor])the page language, for example en
=SCRAPE_LINKS(url, [maxAge], [waitFor])every link on the page as a JSON array string
=SCRAPE_STATUS(url, [maxAge], [waitFor])the HTTP status code, for example 200 or 404
=BRIGHTDATA(url, [output], [render], [country])markdown or HTML with full control over rendering and proxy location
=Firecrawl(url, [maxAge], [waitFor])markdown via Firecrawl, using your own Firecrawl key

Parameters

  • url (required): a full address starting with http:// or https://.
  • maxAge: kept for compatibility with older sheets and ignored. Scrapes are cached for 6 hours regardless.
  • waitFor: kept for compatibility. Any value greater than 0 forces browser (JavaScript) rendering, which is slower but handles single-page apps.
  • output (=BRIGHTDATA()): "markdown" (default) or "html".
  • render (=BRIGHTDATA()): TRUE forces browser rendering, which takes roughly 15 to 25 seconds per page. Default FALSE.
  • country (=BRIGHTDATA()): two-letter country code for the proxy location, for example "us" or "de". Useful for region-specific pricing pages.

Examples

=SCRAPE("https://example.com/pricing")
=SCRAPE("https://app.example.com/dashboard", 0, 1)
=SCRAPE_TITLE(A2)
=SCRAPE_DESCRIPTION(A2)
=SCRAPE_STATUS(A2)
=BRIGHTDATA("https://shop.example.com", "markdown", TRUE, "us")
=EXTRACT_INFO(SCRAPE(A2), "price per month", "number")
=GPT(SCRAPE(A2), "List the product names on this page, comma separated")

Behaviour worth knowing

  • Cell size limit. Google Sheets cells hold at most 50,000 characters. Markdown and HTML output is truncated to 49,000 characters with a note at the end when a page is longer.
  • One request per URL for metadata. =SCRAPE_TITLE(), =SCRAPE_DESCRIPTION(), =SCRAPE_KEYWORDS(), =SCRAPE_LANGUAGE(), =SCRAPE_LINKS() and =SCRAPE_STATUS() share a single fetch per URL. Using all six on the same row costs one scrape, not six.
  • 6-hour cache. Every scrape is cached for 6 hours by URL, output type, rendering and country. The cache is checked before credentials, so recalculating never spends a trial use or a credit.
  • Links come back as JSON. =SCRAPE_LINKS() returns up to 300 links as a JSON array string such as ["https://a.com/x","https://a.com/y"]. To spill them into rows: =TRANSPOSE(SPLIT(REGEXREPLACE(B2,"[\[\]""]",""),",")).
  • Rendering. Plain fetches are fast and usually enough. If a page comes back empty or as a loading shell, pass waitFor greater than 0 or use =BRIGHTDATA(url, "markdown", TRUE).

Bright Data zone

Bright Data needs a zone name in addition to the key. GPTSheetsPlus looks up your account’s active zones and picks a Web Unlocker zone automatically the first time you scrape. If you have several zones or auto-detection fails, set one explicitly with Set API Keys → Insert Bright Data Zone (optional). Leave the field empty to go back to auto-detection.

Firecrawl

=Firecrawl(url, [maxAge], [waitFor]) scrapes with your own Firecrawl key instead of Bright Data. It requires a license and the key from Set API Keys → Insert Firecrawl API Key. It returns the main content of the page as markdown. maxAge (milliseconds, default 7 days) and waitFor (milliseconds) are passed straight to Firecrawl.

Errors

Cell showsMeaning
Error: Free trial for Bright Data used up (10/10) ...Activate your license and add your Bright Data key.
Error: Bright Data API key not set ...Add your key via Set API Keys → Insert Bright Data API Key.
Error: Invalid URL: ...The value is not a full URL. Include https://.
Error: Bright Data returned HTTP 401 ...Your key was rejected. Re-copy it from Account settings → API keys.
Error: Bright Data returned HTTP 403 ... or a message mentioning zonesThe zone is wrong or missing. Create a Web Unlocker API zone, or set it via Insert Bright Data Zone.
Error: Bright Data returned HTTP 429 ...Rate limit. Wait a moment and try again.
No title found, No description found, No links foundThe page loaded but has no such element. Try forcing rendering.