Apify action

Crawl Website Content (Apify)

Crawls website pages and extracts structured content including text, metadata, and URLs.

Run in Studio →All Apify actions
Apify logoApify
apify-website-content-crawler
enrichmentReturns in secondsStandard tier
Input

What it needs.

Map each field to a column in your table. Optional fields can stay empty.

FieldType
  • url
    Website URL
    text

When to use this.

The "Crawl Website Content (Apify)" action is designed to efficiently crawl specified website pages and extract structured content, including text, metadata, and URLs. By providing a single input parameter, the website URL, users can initiate the crawling process. The action outputs essential data fields such as page text, title, description, canonical URL, language code, loaded URL, and referrer URL, all formatted as text or URL types. This action is particularly useful for data enrichment, enabling businesses to gather insights from web content for SEO analysis, competitive research, and content aggregation. With its ability to deliver structured data, it supports various applications in digital marketing, content management, and web analytics.

Output

What it returns.

Every field this action writes back to the row. Click a line to expand nested objects.

FieldType
  • page_text
    Page Text
    text
  • page_title
    Page Title
    text
  • page_description
    Page Description
    text
  • page_canonical_url
    Page Canonical Url
    nullabletext
  • page_language_code
    Page Language Code
    nullabletext
  • crawl_loaded_url
    Crawl Loaded Url
    text
  • crawl_referrer_url
    Crawl Referrer Url
    nullabletext
Workflow

How this fits a workflow.

Trigger → Crawl Website Content (Apify) → enrich → push to your CRM. You set it up once; after that it runs without you.

  1. Step 1TriggerNew row, form submit, or schedule.
  2. Step 2Crawl Website Content (Apify)The Apify action runs.
  3. Step 3EnrichWaterfall fills missing fields.
  4. Step 4Push to CRMHubSpot / Pipedrive / Zoho CRM / GoHighLevel.

Two actions you can chain after this one:

Crawl Website Content (Apify)Search LinkedIn Sales Navigator People (Apify)Search LinkedIn Profiles (Apify)
Reliability

Throughput & reliability.

How fast it drains a table, and what happens when the upstream provider has a bad day.

  • Per-minute limit30 / min
  • Per-day limit5,000 / day
  • RetriesAutomatic w/ backoff
  • Failed lookupsZero credits

Who runs this

GTM EngineerRevOpsAgency

Standard tier. See pricing; per-action credit cost is private.

Related

More Apify actions.

FAQ.

Mostly. Fast actions fill the cell in a few seconds. Long-running ones run in the background and write back to the row when they finish, so you can close the tab.

No. A failed lookup costs zero credits. When a provider misses or times out, the waterfall moves to the next one and you only pay for the match you actually got.

No. Run it on one cell, on a selection, on the whole table, or on a schedule. Same action either way.

HubSpot, Pipedrive, Zoho, and GoHighLevel are bidirectional, with Salesforce coming soon. Smartlead, Instantly, Lemlist, HeyReach, Apollo Sequences, and Reply.io for outbound.

Run Crawl Website Content (Apify) in 60 seconds.

Connect the account, drop the action on a column, run it on your rows.

Start free →All Apify actions