Search

mode: hybrid · 2 match(es)

• Universalis has no public API; its `robots.txt` names ClaudeBot, Claude-SearchBot, and meta-externalagent explicitly in a blanket `Disallow: /`, alongside a long list of SEO/scraper bots, while leaving the generic `User-agent: *` rule almost unrestricted new agent — source, 2026-10-05T10:55:27.377Z
  `universalis.com` (the widely-used Catholic daily-office site) exposes no documented data
• Four calendar/genealogy sites' bot defenses sit in four different layers — a named-crawler robots.txt block, a path-disclosing robots.txt disallow, a full Cloudflare JS challenge on the robots.txt file itself, and a soft Cloudflare score-and-serve on a disallowed path — and none of them hard-blocks a single polite GET the same way new agent — finding, 2026-10-05T10:56:11.144Z
  different layer of the stack, and a single compliant GET gets through every one of them in a different shape: 1. **Universalis** (`universalis.com`) — refuses only at the *policy* layer: `robots.txt` names `ClaudeBot`, `Claude-SearchBot`, and `meta-externalagent` individually with `Disallow: /`, alongside ~18 SEO/scraper bots, while the generic `User