Your AI Visibility Score Is Probably Lying to You
Ask an AI the same question twice and you'll get two different answers. Most tools ask once.
Here's an experiment you can run right now. Ask ChatGPT for the best plumber in your town. Note the names. Open a new chat, ask the identical question again.
Different list, isn't it?
That's not a glitch. It's how these systems work. Every answer is generated fresh, and the generation has randomness in it. Ask once and you've taken one spin of a wheel.
Why that breaks the dashboards
The AI-visibility tools popping up everywhere run your questions once and hand you a score. Next month they run once more, and the score moves.
On a fifteen-question set, a business's score can swing about twelve points between runs with absolutely nothing changing. No website updates, no new reviews, no algorithm shift. Just the wheel spinning again.
So the dashboard says you went from 34% to 24% and you panic, or 24% to 34% and your agency takes credit. Both times, nothing happened. A single-run tool cannot tell the difference, and it will never tell you that, because “this number might be noise” is a terrible thing to print above a renewal button.
What honest measurement looks like
Ask every question multiple times. Report the rate and the range it moved across the passes. “You were named in 30% of answers, and across repeated passes that sat between 24% and 36%”. Now a move to 27% next month means nothing, and a move to 8% means something. You can finally tell signal from noise.
One more thing to watch for: anyone selling you a rank position in AI: “you're #3 in ChatGPT.” There is no such thing as a rank in a system that gives different answers every time it's asked. That number is invented.
The takeaway
Before you trust any AI visibility number, ours included, ask the vendor one question: how many times did you ask?
If the answer is once, you're not looking at a measurement. You're looking at a screenshot of a slot machine.
Based on Kensume's variance study: the same question sets run repeatedly across five AI platforms over three independent runs, June to August 2026.
Want to see where your business stands?
We'll run the questions your customers ask across all five AI platforms and show you exactly where you're missing.
Run my free scanNo credit card. Takes 2 minutes.