AI Answer Probes
Usage scenarios for measuring whether models mention, recommend, or cite you: model setup, D/K probes, frozen watchsets, calibration, and Agent/MCP
Page GEO (Schema, llms.txt, heuristic scores) answers whether models can read you. Answer probes actually ask the models: after you name the domain, do they recognize it? On a brand-free keyword, do they mention, recommend, or cite you? Every run keeps the original answer.
Do not merge recognition, mention, recommendation, and citation into one “AI visibility” score. Definitions: Probe methodology.
Scenario map
| What you want | Where | What you get |
|---|---|---|
| Connect models that can be queried | Settings → AI Models | Probes can send requests |
| See the latest three rates | Site overview AI Answer Probe card | Mention / recommend / cite (not the page score) |
| Named recognition | GEO Optimization → Answer Probe → domain recognition | Recognition (protocol D) |
| Unprompted discovery | Same tab, brand-free keyword | Mention / recommend / cite (protocol K) |
| Compare after a content fix | Save watchset → retest | Percentage-point change on the same denominator |
| Consumer-UI check | Calibrate on Perplexity | Public-page calibration; ChatGPT needs login (unsupported) |
| Let Cursor / Claude run it | Quickstart MCP tools | History, watchset, retest |
Path: /en/dashboard/sites/{id}/geo-optimization?tab=answer-probe
Scenario 1: Configure models used for probes
Probes use enabled models on your account. They do not fall back to an env DeepSeek key when the session is missing.
- Open Dashboard → Settings and open AI Models
- Enable at least one provider your plan allows (OpenAI / Qwen / DeepSeek)
- To bill usage to your own account: enable Vercel AI Gateway (BYOK) and paste an AI Gateway API key. Default base URL:
https://ai-gateway.vercel.sh/v1.
One Gateway key covers OpenAI / Anthropic / Gemini / DeepSeek — you do not need a separate OpenRouter account. Probe tokens bill to Vercel, not a SoloKit plan gate. Env:
AI_GATEWAY_API_KEY.
If no model is enabled, D/K runs fail. Configure models before probing a site.
Scenario 2: Open the Answer Probe tab
Either:
- On the site overview, click the AI Answer Probe card (subtitle: mention / recommend / cite)
- Or open GEO Optimization and switch to the Answer Probe tab
The four pills:
| Rate | Comes from | Question |
|---|---|---|
| Recognition | D: domain recognition (named) | Domain-only prompt: can the model describe this site? |
| Mention | K: keyword discovery (unnamed) | Does the brand or official host appear? |
| Recommendation | K | Is it an explicit preferred / suggested option? |
| Citation | K | Do answer URLs hit the official domain? |
GEO AI visibility on the same site is input-side readiness. It is not these four rates.
Scenario 3: Domain recognition (D)
Use when you launched, rewrote the brand blurb, or want to know if models can describe you once named.
- Domain recognition (named) is prefilled with the site hostname (no
www.) - Change it if needed, e.g.
seo.solokit.run - Click Run recognition probe (about 30–90 seconds)
- Expand history and check each model’s recognition label plus the raw answer
The prompt contains only the domain. Being able to recap a named domain is not the same as bringing it up on a neutral query.
Scenario 4: Keyword discovery (K)
Use when you want to know if a query like “GEO optimization platform” mentions, recommends, or cites you.
- In Keyword discovery (unnamed), enter a brand-free keyword, e.g.
GEO optimization platform(not SoloKit) - Click Run keyword probe
- Expand the run: mention / recommendation / citation, plus URLs in the original answer
Keywords that contain the brand are excluded from the watchset denominator. Named without a recommendation must keep recommendation at 0.
Protocol example (not a leaderboard): GEO Insights uses K terms like “GEO 优化平台” / “AI 搜索可见性监测”, not the brand name.
Scenario 5: Freeze a watchset and retest
Use after you change Schema, FAQ, or body copy and need an apples-to-apples comparison.
- Under Watch keywords (frozen retests), one item per line:
- Brand aliases: official name, short name, English name
- Watch keywords: brand-free; max 8
- Competitors:
Name | domain, e.g.NiubiGEO | niubigeo.com
- Optionally enable Daily scheduled retest (about 04:00 UTC / 12:00 Beijing)
- Click Save watchset
- After publishing, click Retest watchset: one D on the frozen domain, then K for each enabled keyword
Compare in percentage points (pp) and keep the previous original answers. A new keyword list is not comparable to the old one.
Max 12 aliases and 12 competitors.
Scenario 6: Perplexity calibration (optional)
Use when you want a consumer search page, not only provider APIs.
- Enter a K keyword, or keep one in the watchset
- Click Calibrate on Perplexity
- History
sourcemarks the calibration run
ChatGPT’s consumer UI requires login; the API returns login_required and the tab does not expose that button. Consumer answer-share products (Profound / Peec) are not this API probe.
Scenario 7: History, original answers, published-content hits
- Open a run under History
- Read that run’s four rates and each model’s mention / recommendation / citation flags
- Read the original answer (truncated around 4000 characters)
- Published content hits appear when a cited URL matches a distribution
published_url(exact / canonical)
source may be manual, watchset retest, or the daily cron. Citation rate checks official-host URLs in the answer; content writeback is what shows that this published URL was used.
Scenario 8: Run via Agent / MCP
Setup: Quickstart. Probe tools:
| Tool | Permission | Scenario |
|---|---|---|
list_geo_probes | read | History and latest four rates |
run_geo_probe | write | One D or K run |
get_geo_watchset | read | Frozen watchset |
retest_geo_watchset | write | Retest the watchset |
Example:
List GEO answer probes for site 2. Report recognition and recommendation separately — do not merge them.
Run keyword_discovery on the brand-free term "GEO optimization platform".
Retest site 2 against the frozen watchset.
REST: GET/POST /api/public/geo-probes. Watchset and retest: OpenAPI GEO Probes tag.
Do not
- Weight page GEO and probe rates into one score
- Put brand names in K denominators
- Treat “they recognize you” as “they recommend you”
- Treat public cases as a live leaderboard
Next: Probe methodology, then fix the input-side page and retest the output side.