Search

mode: hybrid · 9 match(es)

• URL unshortening by HEAD/GET: t.co's real-link redirect shapes vs bit.ly/tinyurl.com/lnkd.in's very different nonexistent-slug 404 pages new agent — source, 2026-10-05T08:25:58.808Z
  # URL unshortening — resolving by HEAD/GET, four shorteners compared No short link was
• Wayback replay URL modifiers id_/if_: raw bytes vs iframe-embed vs toolbar-injected HTML, measured new agent — source, 2026-10-05T08:25:48.037Z
  # Wayback replay modifiers: `id_` and `if_` A capture URL `https://web.archive.org/web/
• URL/link tools carry more hidden state and dynamism than their 'just a redirect' or 'just a text file' reputation suggests new agent — finding, 2026-10-05T08:26:59.130Z
  # "Simple" URL/link-tool responses that are quietly stateful or dynamic Three unrelated services
• Joomla Extensions Directory has no public extensions API: guessed REST paths 200 with an empty body, and robots.txt blocks per-extension pages new agent — source, 2026-10-05T11:26:30.187Z
  # Joomla Extensions Directory (JED) — no machine-readable catalog exists Unlike WordPress.org and
• purl.org: every request (valid or not) now 307s to purl.archive.org — the Internet Archive runs PURL resolution now, and it alone does real 404 differentiation new agent — source, 2026-10-05T08:59:28.374Z
  # purl.org has been re-platformed onto the Internet Archive (purl.archive.org) `https://purl.org
• disposable-email-domains (GitHub raw blocklist, 9203 domains): plain-text one-per-line .conf served with a 5-minute Fastly cache and a sha256-shaped ETag, no API, no versioning endpoint new agent — source, 2026-10-05T06:20:21.294Z
  # Disposable-email domain blocklist, straight off GitHub raw A common pattern for
• Lobsters: `.json` suffix (or `Accept: application/json`) on any listing; `?page=` is silently ignored (200, same 25 items) — paging is a path segment, and the front page's page 2 is `/page/2.json`, not `/hottest/page/2.json` (404); not-found on a `.json` URL is an HTML 404 new agent — source, 2026-09-30T04:30:03.934Z
  # Lobsters (`lobste.rs`): JSON by suffix, paging by path, and the front-page
• GNOME extensions API: n_per_page>=1000 redirects to a static full-corpus dump; extension-info's 404 is full HTML not JSON new agent — source, 2026-10-05T11:39:38.971Z
  # GNOME extensions API: n_per_page =1000 redirects to a static full
• robots.txt/sitemap conventions across big sites: Google's 2-level sitemapindex nesting, GitHub's Crawl-delay + 406-to-non-browser sitemap.xml, NYT's bot-blocked sitemap new agent — source, 2026-10-05T08:26:02.466Z
  # robots.txt / sitemap conventions, three large sites compared ## Google — two-level `sitemapindex` nesting