Query Fan-Out Expander
AGENT ROLE: Autonomous query-discovery agent. Expand seeds breadth-first across autocomplete + SERP question sources, dedupe, cluster, and emit the JSON in references/output.schema.json.
OBJECTIVE
From a few seeds, reconstruct the broad "fan-out" of real user queries and sub-questions around a topic — the same decomposition modern AI search performs — so content can cover the whole intent surface.
INPUTS
seeds (REQUIRED string[]).
depth (OPTIONAL, default 2): expansion hops (depth 1 = direct suggestions only).
max_queries (OPTIONAL, default 500): hard cap.
sources (OPTIONAL): subset of ["autocomplete","paa","related"]. Default all.
hl/gl (OPTIONAL): default "en"/"us".
AUTHENTICATION
autocomplete: Google Suggest is keyless — GET https://suggestqueries.google.com/complete/search?client=firefox&hl={hl}&q={q} returns [query,[suggestions...]]. Use UA seoskills-fanout/1.0; it is unofficial, so treat failures as soft.
paa/related: REQUIRE env SERP_API_KEY (SerpApi related_questions + related_searches). IF unset AND those sources requested THEN drop them, keep autocomplete, and note sources_used.
- IF NEITHER autocomplete nor a SERP key is available THEN STOP
error.code="NO_SOURCES_AVAILABLE".
EXPECTED TOOL CALLS
- Run
scripts/fanout.py --seeds seeds.json --depth 2.
- BFS: expand each frontier query via the enabled sources; enqueue new normalized queries until
depth or max_queries.
PROCEDURE (deterministic)
STEP 1 — INIT frontier = seeds (depth 0). Maintain a seen set (normalized: lowercased, whitespace-collapsed).
STEP 2 — For each frontier query at depth d < depth: collect suggestions/questions from the enabled sources; add unseen to seen and to the next frontier; record each query's source and parent.
STEP 3 — STOP expanding when depth reached OR len(seen) >= max_queries (set hit_cap=true).
STEP 4 — CLUSTER the full set by shared head-term + question-word into intent buckets (question/comparison/commercial/transactional/other) using the same modifier rules as the intent clusterer.
STEP 5 — EMIT the deduped query list with depth, source, parent, plus cluster rollups sorted by size.
RATE LIMITS & ERROR HANDLING
- Autocomplete: throttle ≤ 5 req/s with 150ms jitter; on repeated
429/block, stop autocomplete for the run and mark autocomplete_status="throttled" (partial results still returned).
- SERP API:
429/quota → backoff 2^attempt (max 5), then stop SERP sources and continue with whatever is collected; set serp_status="rate_limited".
- Every source failure is soft — the run always returns the queries gathered so far.
MISSING / INSUFFICIENT DATA
- IF a query yields no expansions THEN it is a leaf (kept, not errored).
- Dedupe is mandatory; never emit the same normalized query twice.
- Autocomplete reflects locale — always pass
hl/gl; note them in output.
OUTPUT
One JSON object per references/output.schema.json. No prose.
FILES
scripts/fanout.py — BFS expander over autocomplete + SERP questions, dedupe + clustering.
references/output.schema.json — output contract.