Discoverability
Store classification
Categories
AI
DEVELOPER_TOOLS
SEO_TOOLS
Store labels
No public values available.
Catalog ID 6QG3JfFNAakFTlpeO
Public Apify Actor
stive.wg/robots-audit
Parse robots.txt for up to 100 sites into one JSON record each: rule groups, sitemaps and crawl delays, plus an AI crawler policy audit across 17 bots like GPTBot, ClaudeBot and PerplexityBot. Each bot rated allowed, partial or blocked, with an open, selective or closed summary per site.
Active users / 30d
1
Total users
2
Runs / 30d
21
Total runs
23
Rating
—
No reviews
Bookmarks
0
Usage history
Latest captured Active users / 30d
1
0 since 25 Aug
22 of 30 UTC days captured
Latest captured Runs / 30d
21
+20 since 25 Aug
22 of 30 UTC days captured
1
Users / 7d
Discoverability
Categories
Store labels
No public values available.
Monetization
Pay per event
2 charge events configured
Actor Start
apify-actor-start
Charged when the Actor starts running. Number of events charged depends on Actor memory (one event per GB, minimum one event).
$0.0005
1
Users / 30d
1
Users / 90d
result
apify-default-dataset-item
Single result in the default dataset.
$0.002
Loading public Actor documentation
Reading the current public Store definition and reviews.