Discoverability
Store classification
Categories
No public values available.
Store labels
No public values available.
01
Tools & integrations
02
Core features
03
How it works
04
MCP server
05
Plans & pricing
06
Get started
07
The builder's digest
Catalog ID sT6eG72iaB6oMM4XP
Public Apify Actor
hipersoft/robots-txt-checker
Test whether URLs are allowed or blocked by a site's robots.txt for any crawler user-agent. Returns allowed/blocked, the matched rule, crawl-delay and the site's sitemaps. Great for SEO audits and crawler compliance. One row per URL as JSON, CSV or Excel.
Active users / 30d
0
Total users
3
Runs / 30d
56
Total runs
61
Rating
—
No reviews
Bookmarks
0
Usage history
Latest captured Active users / 30d
0
-1 since 4 Oct
28 of 30 UTC days captured
Latest captured Runs / 30d
56
+55 since 4 Oct
28 of 30 UTC days captured
0
Users / 7d
Discoverability
Categories
No public values available.
Store labels
No public values available.
Monetization
Pay per event
2 charge events configured
URL checked
url-checked
Charged for each URL tested against robots.txt.
$0.0003
Actor Start
0
Users / 30d
0
Users / 90d
apify-actor-start
Charged when the Actor starts running. Number of events charged depends on Actor memory (one event per GB, minimum one event).
$0.00005
Loading public Actor documentation
Reading the current public Store definition and reviews.
More from this builder
YouTube Video Downloader — MP4 & MP3
28 active users · 124 runs / 30d
Est. monthly creator revenue
Bulk Image Downloader — URLs to Stored Files
26 active users · 5.5K runs / 30d
Est. monthly creator revenue
YouTube Scraper — Videos, Transcripts, Comments, Channels
17 active users · 120 runs / 30d
Est. monthly creator revenue
🇷🇴 eJobs.ro Jobs Scraper
11 active users · 80 runs / 30d
Est. monthly creator revenue