Measuring AI-search visibility across markets without mixing the numbers
Build a report that distinguishes sampled visibility, accurate citations and attributable customer actions. Keep the raw observations so another person can reproduce the calculation.
In this article
Keep each country denominator separate
A useful visibility dataset preserves the market conditions that make a retail answer meaningful. Define cohorts by country, language, product family and buyer intent, then retain the exact question, surface, date, answer and cited pages. A single blended score cannot show whether an answer recommends a product that is actually sold in the tested market.
Publish valid-observation counts before rates and keep failed or unavailable runs visible. If one country has more questions, repeated runs or better platform coverage, an overall percentage will give it more influence. Compare stable cohorts first; only then offer a weighted summary with the weighting stated.
Separate visibility from commercial evidence. Track mentions, recommendations, citations, factual accuracy, referral visits and orders as different fields. A correction request is not a corrected answer, and an AI referral is not by itself an attributable sale.
Do not average away the market mix
Suppose one country has more questions and repeated runs than another. An overall mention rate will give that country more influence even if nobody deliberately chose such a weighting. Publish the count of valid observations for each country and language. Compare stable cohorts before deciding whether to calculate a combined number. If central management needs one summary, explain the weighting and keep the underlying regional results visible in the same report.
Label product coverage inside the sample
Tag observations by product family and whether the product is sold in the tested market. Keep a separate count of accurate recommendations, unsupported availability claims and citations that do not substantiate the answer. These categories should come from the retailer’s operational needs, not a generic dashboard preset. Have a product owner inspect a sample of classifications. Save disagreements and corrections so the next reporting cycle uses the same interpretation.
An illustrative visibility comparison
- 90 valid observations
- 18 mentions = 20%
- 27 mentions = 30%
- Difference: +10 percentage points
- Illustrative comparison, not a growth promise
Illustrative counts from comparable samples. This is not a forecast, causal estimate or reported client result.
Use results to allocate correction work
Review recurring factual errors before celebrating a change in the headline rate. If several answers confuse the countries you serve, improve the relevant public explanation and observe the same questions again. Keep referral visits and completed purchases separate from the answer sample. A change in product stock, promotion or shipping coverage may also affect behaviour. The report should support a choice about the next correction or market investigation without presenting the sampled answers as a sales experiment.
What to put on the comparison sheet
On narrow screens, scroll the table to see all columns.
| Measurement control | Evidence to retain | Acceptance record |
|---|---|---|
| Country samples | Separate answer counts for every market | Do not pool unequal country samples silently |
| Commercial accuracy | Delivery claim, cited URL and current policy | Operations validates the claim in that market |
| Customer actions | Attributed sessions and orders by country | Report observed referrals separately from visibility |
Use counts that can be audited
An illustrative panel contains 90 valid answer observations. If 18 mention the brand, the observed mention rate is 20%. If a comparable later panel has 27 mentions out of 90, it is 30%, a difference of 10 percentage points. These are example numbers, not a forecast or a reported client result.
The calculation is simple; the interpretation is not. Repeated answers to related questions are not necessarily independent observations. A small panel does not represent the entire audience. Show the counts, failed runs and question composition alongside the rate so the reviewer can see what actually changed.
Separate mentions, sources and accuracy
A brand can be mentioned without a link, linked without being recommended, or recommended with an incorrect claim. Define these outcomes before scoring. Preserve the source URL and relevant answer passage so a reviewer can check whether the cited page supports the statement.
Track important factual errors as a separate work queue. Correct the public information you control and record requests concerning other sources. Do not count a correction request as a completed correction. The report should distinguish what the agency changed from what a third-party platform later displayed.
