Do I need a community node to use Scrapernode in n8n?

No. Scrapernode works with n8n's built-in HTTP Request node. Set the method, URL, headers (API key), and body — that's it. No community node installation required. We're also building a dedicated n8n node for even simpler setup.

How do I handle the async wait in n8n?

Use n8n's Wait node after creating a job. Set it to 30-60 seconds for most scrapers. Then use an HTTP Request node to fetch results. For large batches, add a simple loop: Wait → Check Status → IF complete → Fetch Results, ELSE → loop back to Wait.

Can I use Scrapernode with self-hosted n8n?

Yes. Scrapernode is a standard REST API. It works with n8n Cloud, self-hosted n8n on Docker, and n8n Desktop. Your n8n instance just needs outbound HTTPS access to call the Scrapernode API.

What's the best way to handle errors in n8n?

Use n8n's Error Trigger node to catch failed HTTP requests. Scrapernode returns clear error codes — 400 for bad input, 402 for insufficient credits, 404 for job not found. Route errors to Slack or email for monitoring.

How many URLs can I send per job?

You can submit hundreds of URLs in a single job. For large lists, batch into groups of 100-200 URLs per job. This is more efficient than creating one job per URL and reduces the number of HTTP Request nodes in your workflow.

Use Case

The scraping API built for n8n workflows

Drop a single HTTP Request node into any n8n workflow to scrape LinkedIn, Instagram, TikTok, Twitter/X, YouTube, and more. Per-result credits, structured JSON output, zero scraper maintenance.

Build your first n8n workflow View pricing

The Problem

Challenges teams face without Scrapernode

Complex HTTP node configs

Setting up Brightdata, Apify, or PhantomBuster in n8n means configuring OAuth, pagination, webhook callbacks, and custom headers. Each tool needs a different HTTP Request node pattern.

Code nodes that break

You wrote a Puppeteer script inside n8n's Code node to scrape profiles. It worked for a week. Then the site updated its markup and now you're debugging JavaScript at midnight.

No native social scraping nodes

n8n has nodes for Slack, Google Sheets, and Airtable — but nothing for extracting structured data from LinkedIn, Instagram, or TikTok. You're stuck with raw HTTP or Code nodes.

Async polling is tedious

Scraping APIs return results asynchronously. In n8n, that means building Wait → HTTP → IF → Loop patterns to poll for job completion. Every workflow has the same boilerplate.

The Solution

How Scrapernode solves this

Two HTTP Request nodes, any platform

POST to create a job, GET to fetch results. Same node config for LinkedIn, Instagram, TikTok, Twitter/X, YouTube, Facebook, Indeed, Glassdoor, Yelp, GitHub, and Crunchbase. Copy-paste between workflows.

Clean JSON for Set and IF nodes

Scrapernode returns flat, structured JSON — names, titles, follower counts, post content. Use n8n's Set node to extract fields, IF node to filter, and merge directly into your downstream tools.

Scrapers that never break

When platforms change their markup, Scrapernode updates the scraper. Your n8n workflow keeps running. No more midnight debugging sessions for broken Code nodes.

Example Workflows

Copy these n8n patterns to get started in minutes

Basic scrape-and-store workflow

The simplest Scrapernode + n8n pattern. Trigger, scrape, store. Works for any platform — just change the scraper slug.

1
Manual or schedule trigger
Kick off manually or run on a schedule (daily, hourly, etc.)
2
HTTP Request: create job
POST /api/jobs/create with scraper slug, URLs, and API key
3
Wait node
Pause for 30-60 seconds while Scrapernode processes the job
4
HTTP Request: get results
GET /api/jobs/results returns structured JSON array
5
Set node: extract fields
Pull out the fields you need — name, title, bio, followers, etc.
6
Google Sheets / Airtable
Append rows to your tracking sheet or base

Multi-platform enrichment with merge

Enrich the same list of people across LinkedIn, Twitter, and Instagram, then merge results per person.

1
Trigger
Webhook receives a list of people with social URLs
2
Split by platform
Code node groups URLs by platform (LinkedIn, Twitter, Instagram)
3
Parallel Scrapernode jobs
Three HTTP Request nodes fire simultaneously — one per platform
4
Wait and collect
Wait for all three jobs to complete, fetch results from each
5
Merge node
Combine results per person using name or URL as the join key
6
Output
Write enriched profiles to Airtable, CRM, or send via webhook response

Conditional scraping with IF nodes

Scrape LinkedIn profiles, then conditionally scrape additional platforms based on the results.

1
Scrape LinkedIn profiles
Initial enrichment with linkedin-profiles scraper
2
IF: has Twitter URL?
Check if the LinkedIn profile includes a Twitter/X handle
3
Scrape Twitter profiles
For leads with Twitter handles, run twitter-profiles scraper
4
IF: follower count > 1000?
Filter for leads with significant social presence
5
Tag as high-value
Mark qualifying leads as high-priority in your CRM with the social context attached

Recommended Scrapers

The scrapers most relevant to this use case

Profiles

LinkedIn · 2 credits / row

Companies

LinkedIn · 2 credits / row

Posts

LinkedIn · 1 credits / row

Profiles

Instagram · 1 credits / row

Posts

Instagram · 1 credits / row

Profiles

TikTok · 1 credits / row

Videos

TikTok · 1 credits / row

𝕏

Profiles

Twitter/X · 1 credits / row

𝕏

Posts

Twitter/X · 1 credits / row

Channels

YouTube · 1 credits / row

Profiles/Posts

Facebook · 1 credits / row

Companies

Crunchbase · 2 credits / row

Job Listings

Indeed · 1 credits / row

Integrations

Connect your scraped data to your favorite tools

Google Sheets

Auto-sync results to spreadsheets

Webhooks

Real-time delivery to any endpoint

REST API

Programmatic access for developers

n8n

Connect to 1000+ apps

CSV/JSON

Download in standard formats

Frequently Asked Questions

Common questions about n8n

The Problem

Challenges teams face without Scrapernode

Complex HTTP node configs

Setting up Brightdata, Apify, or PhantomBuster in n8n means configuring OAuth, pagination, webhook callbacks, and custom headers. Each tool needs a different HTTP Request node pattern.

Code nodes that break

You wrote a Puppeteer script inside n8n's Code node to scrape profiles. It worked for a week. Then the site updated its markup and now you're debugging JavaScript at midnight.

No native social scraping nodes

n8n has nodes for Slack, Google Sheets, and Airtable — but nothing for extracting structured data from LinkedIn, Instagram, or TikTok. You're stuck with raw HTTP or Code nodes.

Async polling is tedious

Scraping APIs return results asynchronously. In n8n, that means building Wait → HTTP → IF → Loop patterns to poll for job completion. Every workflow has the same boilerplate.

The Solution

How Scrapernode solves this

Two HTTP Request nodes, any platform

Clean JSON for Set and IF nodes

Scrapernode returns flat, structured JSON — names, titles, follower counts, post content. Use n8n's Set node to extract fields, IF node to filter, and merge directly into your downstream tools.

Scrapers that never break

When platforms change their markup, Scrapernode updates the scraper. Your n8n workflow keeps running. No more midnight debugging sessions for broken Code nodes.

Example Workflows

Copy these n8n patterns to get started in minutes

Basic scrape-and-store workflow

The simplest Scrapernode + n8n pattern. Trigger, scrape, store. Works for any platform — just change the scraper slug.

1
Manual or schedule trigger
Kick off manually or run on a schedule (daily, hourly, etc.)
2
HTTP Request: create job
POST /api/jobs/create with scraper slug, URLs, and API key
3
Wait node
Pause for 30-60 seconds while Scrapernode processes the job
4
HTTP Request: get results
GET /api/jobs/results returns structured JSON array
5
Set node: extract fields
Pull out the fields you need — name, title, bio, followers, etc.
6
Google Sheets / Airtable
Append rows to your tracking sheet or base

Multi-platform enrichment with merge

Enrich the same list of people across LinkedIn, Twitter, and Instagram, then merge results per person.

1
Trigger
Webhook receives a list of people with social URLs
2
Split by platform
Code node groups URLs by platform (LinkedIn, Twitter, Instagram)
3
Parallel Scrapernode jobs
Three HTTP Request nodes fire simultaneously — one per platform
4
Wait and collect
Wait for all three jobs to complete, fetch results from each
5
Merge node
Combine results per person using name or URL as the join key
6
Output
Write enriched profiles to Airtable, CRM, or send via webhook response

Conditional scraping with IF nodes

Scrape LinkedIn profiles, then conditionally scrape additional platforms based on the results.

1
Scrape LinkedIn profiles
Initial enrichment with linkedin-profiles scraper
2
IF: has Twitter URL?
Check if the LinkedIn profile includes a Twitter/X handle
3
Scrape Twitter profiles
For leads with Twitter handles, run twitter-profiles scraper
4
IF: follower count > 1000?
Filter for leads with significant social presence
5
Tag as high-value
Mark qualifying leads as high-priority in your CRM with the social context attached