Daniel James

See exactly which AI crawlers a site allows. Per domain it reads robots.txt for 25 named AI agents (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot…), plus ai.txt, llms.txt, noai meta, X-Robots-Tag and tdmrep.json, and quotes the rule behind every answer. Alerts on policy changes.

Pull live job postings from company career boards on Greenhouse, Lever and Ashby through their public job-board APIs — normalised to a single schema with stable IDs, deduplication, optional PII redaction and a changes-only monitor mode.

Pull any Shopify store's full public catalogue from its own /products.json endpoint — every product and every variant in one normalised schema with stable IDs — and run it on a schedule as a price and stock change monitor that tells you what changed and what it was before.

Know the moment a UK company you care about files at Companies House, changes a director, changes a PSC, changes its name or is dissolved. Give it company numbers or names; it watches filing history, officers, PSCs and the profile, and emits only what changed — with the previous value.

Structure and enrich any dataset with YOUR OWN LLM key (OpenAI, Anthropic or OpenRouter). Define a target JSON schema and an instruction; every item is transformed to match, with JSON repair, retries and per-item error isolation. You control model choice and token cost.