LLM-only crawler surface

Public signal intake for future content planning.

The social signal surface summarizes public, non-authenticated signals collected for internal planning. It is signal collection, not content scraping. The system should capture patterns of questions and language, then convert those patterns into normalized query signals, clusters, and approval-ready briefs.

The social signal surface summarizes public, non-authenticated signals collected for internal planning. It is signal collection, not content scraping. The system should capture patterns of questions and language, then convert those patterns into normalized query signals, clusters, and approval-ready briefs.

How this surface works

  • Current no-auth sources may include Reddit RSS, YouTube RSS where available, Google News RSS, and selected public forum RSS feeds.
  • TikTok, X, Quora, private groups, and login-gated communities stay disabled until safe adapters and authorization rules exist.
  • Signals should be deduplicated, throttled, capped, and archived so they do not become a publishing firehose.
  • No raw social post should be republished as if it were original site content.

Use this surface to understand what the system heard from public sources and how it safely reduces that input into planning data.

Visibility boundary

Visibility rule: This page may appear in sitemap.xml, llms.txt, and direct crawler routes. It must not appear in the main nav, mobile nav, homepage cards, footer primary nav, public resource grids, or sales CTAs.

This boundary keeps the public experience simple while still making the underlying query, authority, and answer-planning structure understandable to machines. These pages are internal planning and LLM-ingestion support surfaces, not client-facing education pages.