DevilScrapes

Turn any site's sitemap.xml, sitemap index, or robots.txt Sitemap directive into a clean URL dataset with loc, lastmod, changefreq and priority per row. Recursively expands sitemap indexes (bounded depth), follows redirects, and decompresses .xml.gz automatically.

Query the YC company directory by batch, industry, or status via the same Algolia index the website uses. Returns name, batch, status, industries, team size, website, one-liner, and YC profile URL as clean structured rows.

Pull Yahoo Finance quote snapshots and historical OHLCV candles for any list of tickers — equities, ETFs, indices, crypto, FX. We handle the retries, fingerprint rotation, and proxy routing so the data lands. One typed row per symbol, JSON or CSV export.

Search Yellow Pages business listings by keyword and US city — name, phone, address, categories, rating, review count, and website for every result. We rotate fingerprints and proxies so blocks don't stop your dataset. Local lead generation.

Detect brand-deal signals in a YouTube channel's recent Shorts — spoken sponsor mentions in the transcript, disclosure hashtags, @brand/domain mentions — and score each Short's sponsorship likelihood (0.0–1.0).

Search Zenodo's open research repository and export records — DOI, title, authors, resource type, publication date, license, access rights, file count and download stats — as clean JSON, CSV or Excel.

Scrape licensed daycare and childcare facility registries across 5 US states (NY, CT, CO, DE, TX) via official state open-data APIs. Get facility name, license number, capacity, address, phone, and status — clean B2B leads, ready to export.

Scrape every job posting from any Recruitee-hosted employer career site via Recruitee's own public, unauthenticated JSON API, no login or browser required. One request per company returns titles, locations, salary, and full HTML descriptions.

Scrape SocialBlade creator/channel stats across YouTube, Twitch, Instagram, and TikTok — subscriber/follower counts, view counts, SocialBlade's letter grade, platform + category ranks, and 15-day daily growth history, normalized per handle.

Pull fresh building and construction permit records from any US city or county Socrata open-data portal. We normalize permit ID, type, status, address, valuation, and contact info into one typed row per permit — ready to feed your contractor CRM or lead pipeline.

Scrape UK property listings from Zoopla (for-sale or to-rent) — price, address, bedrooms, agent phone, floor plan, EPC, photos, full description. Export to JSON or CSV. We handle the blocks so your dataset stays clean.

Bulk YouTube transcript downloader and extractor — pull captions (manual or auto-generated) from one video or a whole channel, in any language. Returns plain-text transcript plus timed segments, export to JSON or CSV. We retry and rotate so the captions land. Subtitles, SRT, VTT and playlist data.

Scrape any Reddit user's submissions or comments by username — no login — and export to JSON or CSV. A Reddit user downloader returning title, body, subreddit, score, upvotes, timestamp, and permalink per post. We rotate fingerprints and retry on Reddit's 429s.

Batch-verify US company names against official state Secretary-of-State business registries (NY, CO, CT, OR) — get status, incorporation date, entity type, registered agent, and officer data (CT) as structured KYB / lead-gen rows. No API key, no login. Bulk business entity search + company lookup.

Scrape Discogs marketplace listings — asking price, seller, condition, ships-from — plus public marketplace stats for any release ID or search query, export to JSON or CSV. A Discogs API + HTML scraper that handles the rate limits and retries for you. No login. Vinyl record price data.

Scrape used-car listings from polovniautomobili.com — Serbia's #1 car marketplace. Returns make, model, year, price (RSD/EUR), mileage, fuel, transmission, engine power, body type, color, seller, location and photos. Enriches each ad from its detail page. Export to JSON or CSV.

Scrape for-sale listings from Tori.fi — Finland's dominant classifieds marketplace (Schibsted recommerce). Search by free-text keyword or category and get title, price in EUR, location, seller type (private/dealer), brand, photos, condition and description. Export to JSON or CSV.

Scrape which domains Google's AI Overview cites for your target queries — one row per (query × cited source) with position, snippet, and selector telemetry — export to JSON or CSV. AEO / generative-SEO data for 2026, with captcha-aware retry and Pydantic-validated output.

Search US issued patents and published applications via Google Patents' public search index — patent number, title, abstract, inventors, assignee, classifications, priority, filing, and grant dates — bulk export to JSON or CSV. Free, no key, no login.

Scrape used-car listings from Autoplius.lt, Lithuania's #1 car marketplace — make, model, year, price in EUR, mileage, fuel type, gearbox, engine power, body type, color, seller, and photos. Export to JSON or CSV; enrich each listing from its detail page.

Scrape used-car listings from AUTO.RIA (auto.ria.com), Ukraine's #1 car marketplace with ~300k live ads. Get price (USD/UAH), make, model, year, mileage, fuel, transmission, engine, body, color, VIN, location, seller type, photos, and full description — export to JSON or CSV.

Scrape used-car listings from Marktplaats.nl, the Netherlands' #1 marketplace (auto's) — price, make, model, year, mileage (km), fuel, transmission, body type, drivetrain, seller (dealer/private), city, and photos. Export to JSON or CSV; optionally enrich with the full Dutch description.

Search the Open Library API (the Internet Archive's open book catalogue) and export structured book metadata — title, authors, ISBNs, subjects, publish year, cover URL, edition count, OpenLibrary ID — to JSON or CSV. We handle pagination and retries across 30M+ works.

A BuiltWith & Wappalyzer alternative: detect any company's real tech stack (Postgres, Django, AWS, Kubernetes, Snowflake…) from Greenhouse, Lever & Ashby job posts. 110+ frameworks, databases and cloud tools, export to JSON or CSV. B2B sales & recruiter intel — pay-per-result, no SaaS bill.