Build: Web Scraping & Lead Gen
A scheduled pipeline that scrapes fresh leads from Google Maps every day, writes them to a spreadsheet without duplicating existing rows, and — because not everything you'll ever need has a native n8n node — how to call literally any API on the internet with the HTTP Request node.
What we're building, and why
Manually pulling a thousand rows of business listings off Google Maps is a full day of copy-paste. Apify's pre-built scrapers turn that into a scheduled n8n workflow that runs unattended and drops clean rows into a spreadsheet — the backbone of any outbound lead-gen offer you'd sell to a client or run for your own pipeline.
Trigger: schedule
Add trigger → Schedule → interval of 1 day. Unlike the polling triggers in Layer 00, this one is deliberately time-based rather than event-based — you're deciding when fresh data gets pulled, not waiting for an app to tell you something changed.
Scrape with Apify
Sign up at apify.com (five free scraping credits monthly, roughly 1,000 results). In the Apify Store, search "Google Maps" and pick a scraper maintained directly by Apify — third-party scrapers come and go, but the store always has an actively maintained option for any major platform (Maps, TikTok, Instagram, Amazon…).
- In n8n, add the Apify node (install it if prompted — it's community-maintained, not built into core n8n) → Run Actor and Get Dataset Items.
- Connect your Apify account, then select the saved actor from the dropdown (you must run and save it at least once inside Apify's own console before it appears in n8n's list).
- Switch the actor's input from Manual to JSON, and paste in the JSON input generated by Apify's own console (search term, location, and a conservative result cap — start at 10 while testing, not 10,000).
- Execute the step. Scraping takes 30–60 seconds; the node returns each result as a separate item automatically, with no separate Split Out step required.
Write results to Sheets without duplicates
Add Google Sheets → Add or Update Row (a three-in-one: search for a match, update it if found, create it if not). Set Column to match on to the business's website URL — the only field in a listing that's reliably unique. Map title, category, website, phone, and review score/count from the Apify output.
+ will throw a type error when written to certain
Sheets columns. Strip it with an expression, or just ask ChatGPT: "give me an n8n expression that
removes any + character from this field."
Looping & rate limits
Writing ten rows at once is fine. Writing a thousand rows at once risks tripping the target app's rate limit — Google Sheets, for example, throttles both reads and writes per minute per project. Wrap the write step in a Loop Over Items node with a short Wait (3–5 seconds) between iterations. At one item every 5 seconds you're sending roughly 12 requests a minute — comfortably under virtually every API's limit — and the whole batch survives even at scale.
The HTTP Request node: calling anything
Every native n8n integration — Gmail, Sheets, Slack — is an HTTP request under the hood; the n8n team simply pre-built the wiring. For the thousands of apps without a dedicated node, the HTTP Request node lets you build that wiring yourself, provided the app publishes a public API. Four pieces, every time:
| HTTP method | Plain English | Example |
|---|---|---|
POST | Create something | Start a new scrape run, create a lead |
GET | Retrieve something | Fetch a webpage, look up a record |
PUT | Update something | Edit an existing record |
DELETE | Remove something | Delete a record |
Search "[app name] API documentation" first — if the app has no public API docs, there's no back door, full stop. If it does, most documentation pages include a ready-made curl example; paste that directly into n8n's Import cURL button and the method, URL, headers, and body populate themselves.
Worked example: scraping a website with Firecrawl
Say you've scraped listings from Google Maps and now want to visit each business's actual website to pull more detail. Firecrawl is a scraping API built for exactly this and handles most anti-bot protection automatically.
- Filter out any listing missing a website (condition:
websiteexists) before looping, so you're not wasting calls on empty fields. - In the HTTP Request node: method
POST, URLhttps://api.firecrawl.dev/v1/scrape. - Headers:
Authorization: Bearer <your API key>andContent-Type: application/json. - Body (JSON):
{ "url": "{{ $json.website }}", "timeout": 10000 }— set a generous timeout in milliseconds so a slow site doesn't kill the run. - Under node Settings, enable On Error → Continue so one unreachable site doesn't stop the other nine from completing.
Debugging HTTP errors
When an HTTP Request node throws an error, copy the exact error text into ChatGPT along with the JSON body you sent — "I have an error with this n8n HTTP request: [paste]." Two errors show up constantly enough to name in advance: a field the API considers mutually exclusive with another field you also sent, and a timeout set too low for the target site to respond. Full status-code reference and the rest of n8n's common failure modes are in Layer 09.
Variations worth selling
- Score each scraped lead with AI (1–5) against a client's ideal-customer profile before it ever reaches a spreadsheet.
- Auto-draft a personalized outreach message per lead using the scraped business details.
- Swap Google Maps for Upwork, Apollo, or Yellow Pages actors — the pattern (schedule → scrape → dedupe-write → rate-limit) is identical across all of them.
Sources
| Source | Used for |
|---|---|
| Jono Catliff (Automatable), "n8n 8 Hour Masterclass — Build & Sell AI Agents (Automate Anything)" | Primary source for this build's workflow steps |
| Apify docs, "Actors" | Running & scheduling the Google Maps scraper |
| Firecrawl docs, Introduction | Scraping individual business websites |
| n8n docs, HTTP Request node | Calling any API directly, incl. Firecrawl |
| n8n docs, Google Sheets node | Dedupe-safe row writes |