How do I connect Webhooks to Scrapernode?

Add your endpoint URL in Scrapernode's webhook settings. Scrapernode sends an HTTP POST with the full JSON payload when jobs complete. Verify delivery with a test scrape first.

What GitHub repositories data fields are sent to Webhooks?

All 20 output fields from the GitHub Repositories scraper are included: url, id, code_language, code, num_lines, user_name, and more. The full payload is sent as structured JSON — no fields are filtered out.

Can I schedule recurring GitHub scrapes to Webhooks?

Yes. Use the Scrapernode REST API to create scraping jobs programmatically on any schedule. Results are automatically delivered to Webhooks when each job completes.

How much does it cost to scrape GitHub repositories into Webhooks?

Each repositories record costs 2 credits. Credits start at $9 for 3,000 credits (Starter plan). The Webhooks integration itself is included free — no additional charges for data delivery.

What happens if the Webhooks delivery fails?

Scrapernode retries webhook deliveries with exponential backoff. Failed deliveries are logged and can be manually re-triggered from the job details page. Your scraped data is always available for download even if webhook delivery fails.

GitHubWebhooks

How to Scrape GitHub Repositories into Webhooks

Extract repositories data from GitHub and deliver via Webhooks automatically

Step-by-step guide

Choose the GitHub Repositories scraper

Navigate to the GitHub Repositories scraper in Scrapernode. Select "Fresh Scrape" for real-time data or "Quick Lookup" for pre-collected records. Each record costs 2 credits.

Set up your Webhooks connection

Add your endpoint URL in the Scrapernode webhook settings. Each completed job sends a POST request with the full JSON payload, including all output fields and metadata.

Provide your GitHub input URLs

Paste the GitHub URLs you want to scrape — one per line, or upload a CSV. Scrapernode accepts direct profile links, search result URLs, and content pages.

Launch the scraping job

Click "Start Extraction" to begin. Scrapernode handles proxy rotation, rate limiting, and anti-bot detection automatically. Jobs typically complete in under 60 seconds per batch.

Receive data in Webhooks

When the job completes, Scrapernode delivers the full JSON payload to your endpoint. Each record includes 20 structured fields like url, id, code_language, code. Parse the payload in your application and process the data as needed.

Cost per record

2 credits

Output fields

20 fields

Destination

Webhooks

Sample Output

Preview the data you'll receive — 5 sample records

Record 1 of 5

Url

sample_url

ACoAABcXYZ123

Code Language

sample_code_language

Code

sample_code

Num Lines

1,000

User Name

James K.

User Url

sample_user_url

Size

51-200

Size Unit

sample_size_unit

Size Num

1,000

Breadcrumbs

sample_breadcrumbs

Num Issues

1,000

Num Pull Requests

1,000

Num Projects

1,000

Num Fork

1,000

Num Stared

1,000

Last Feature

sample_last_feature

Latest Update

sample_latest_update

Website Url

sample_website_url

License

sample_license

Data Dictionary

20 fields returned per record

Repository web address (100.00% fill rate)

Unique repository ID (100.00% fill rate)

Main programming language used in the repository (79.55% fill rate)

Repository source code files (86.22% fill rate)

Sub-fields

file_nameTextName of the source code file

file_pathTextPath to the file in the repository

file_contentTextContent of the source code file

Total lines of code in the repository (100.00% fill rate)

Repository owner's username (100.00% fill rate)

Owner's GitHub profile URL (100.00% fill rate)

Repository size with units (100.00% fill rate)

Repository size measurement units (KB, MB, GB) (100.00% fill rate)

Repository size as a numeric value (100.00% fill rate)

Repository navigation path and hierarchy (100.00% fill rate)

Sub-fields

nameTextBreadcrumb navigation element name

urlTextURL of the breadcrumb navigation element

Total count of issues in the repository (100.00% fill rate)

Total count of pull requests (100.00% fill rate)

Number of associated GitHub projects (100.00% fill rate)

Number of times the repository has been forked (100.00% fill rate)

Number of stars the repository has received (100.00% fill rate)

Description of the latest feature or change (99.98% fill rate)

Date of the most recent repository update (99.99% fill rate)

Repository website URL from the About section (72.75% fill rate)

Repository license information (99.83% fill rate)

Sub-fields

nameTextLicense name

urlTextURL to the license details

Frequently Asked Questions

Common questions about How to Scrape GitHub Repositories into Webhooks

Ready to connect GitHub data to Webhooks?

Start extracting github repositories data and deliver via Webhooks in minutes.

Go to GitHub Repositories scraper Browse all integration guides

No code requiredAuto-delivery to Webhooks20 data fields