Ep 31 Found in AI: How Should Brands Measure Visibility in AI Search?

Do AI search rankings actually matter?

In this episode of Found in AI, Cassie breaks down new research from Rand Fishkin and SparkToro that challenges one of the biggest assumptions marketers are making about AI-powered search: that rankings still matter.

She walks through what the data actually shows about how AI engines like ChatGPT and Google AI generate brand recommendations, why rankings are wildly unstable, and why visibility — not position — is the metric brands should be paying attention to instead.

Using both Rand’s research and her own Search Engine Journal displacement case study, Cassie explains how smaller brands can compete with legacy publishers in AI-generated answers by optimizing for visibility, probability, and share of voice.

In this episode, you’ll learn:

• Why AI search engines don’t produce stable rankings — and never will
• What Rand Fishkin’s research reveals about how brands appear in AI answers
• The difference between visibility frequency and AI Share of Voice
• How to correctly calculate AI Share of Voice (and what it actually tells you)
• Why rankings have nothing to do with appearing in AI-generated recommendations
• How a GEO strategy helped a smaller brand displace Search Engine Journal in AI answers
• Why AI search rewards consistency and pattern recognition over domain size
• What marketers should understand before spending money on AI visibility tools

Transcript: How Should Brands Measure Visibility in AI Search?

AI-powered search engines do not behave like traditional search engines. They are probabilistic, highly variable, and fundamentally inconsistent in how they generate answers. This means many of the concepts marketers rely on in classic SEO—especially ranking position—do not apply in the same way to AI systems like ChatGPT, Google AI Overviews, Gemini, or Perplexity.

If you are still thinking in terms of “ranking number one in ChatGPT,” that mental model is going to cause confusion and wasted effort. AI systems do not maintain stable rankings, and they are not designed to.

This distinction matters because companies are already spending real money on AI visibility tracking. As Rand Fishkin points out in his recent research, an estimated $100 million per year is already flowing into this category. That raises an obvious question: are AI engines consistent enough for rank tracking to be a meaningful metric at all?

What Rand Fishkin’s research actually tested

To answer that question, Rand Fishkin and a research partner at Gumshoe AI ran a large-scale experiment. This was not a casual test with a few prompts and anecdotal observations.

The study involved 600 volunteers running 12 prompts across ChatGPT, Claude, and Google’s AI systems, for a total of 2,961 AI responses. The results were normalized into ordered lists so that the outputs could be compared consistently.

The headline finding is simple and uncomfortable: if you ask the same AI engine the same “best brands” or “best products” question 100 times, there is less than a 1 in 100 chance you will receive the same list of brands twice. If you are looking for the same list in the same order, the odds drop closer to 1 in 1,000.

The length of those lists also changes. Sometimes the AI returns two or three recommendations. Other times it returns ten or more, using the exact same prompt.

This means that dashboards claiming you moved from position four to position two in an AI engine are not providing meaningful insight. That kind of movement is more like astrology with a user interface than actionable data.

The important finding: brand presence is more stable than rank

Where Rand’s research becomes especially valuable is that it does not stop at criticizing rankings.

While ranking order was extremely unstable, the set of brands appearing over time was often much more consistent. In one example, when Google’s AI was asked to recommend digital marketing consultants for e-commerce, one agency appeared in 85 out of 95 responses, even though its position varied from run to run.

This shifts the real question marketers should be asking. Instead of asking whether a brand ranks first, the more meaningful question is whether the brand appears at all.

That is the fundamental difference between traditional SEO thinking and AI search optimization. For AI systems, probability of inclusion matters more than position.

People do not prompt AI the same way they search Google

Another important part of the research reflects real human behavior.

The study measured the semantic similarity of human-written prompts and found the similarity score to be extremely low—around 0.081. This means people can have the same intent but phrase their prompts in dramatically different ways.

Despite this variability, AI systems often returned a relatively consistent core set of brands within certain categories. In one test, across 142 human-crafted prompts and 994 AI responses, headphone brands like Bose, Sony, Apple, and Sennheiser appeared between 55 and 77 percent of the time.

Even when prompts are messy and inconsistent, AI systems are effective at intent recognition. This is why visibility percentage holds up as a meaningful metric when ranking position does not.

Visibility frequency and AI Share of Voice are not the same thing

When discussing AI visibility, it is important to distinguish between two related but different measurements. They answer different questions, and confusing them leads to poor analysis.

Visibility frequency answers a basic question: does your brand appear at all? This is measured by running the same prompt multiple times, or running closely related prompts, and observing how often your brand shows up. Because AI answers are probabilistic, this number will fluctuate, but patterns emerge over time.

AI Share of Voice answers a more specific question: when your brand appears, how much of the answer does it occupy?

AI Share of Voice is calculated per prompt, not across a batch of prompts. The math is simple. You take the number of times your brand is mentioned in a single AI response and divide it by the total number of brand mentions in that same response.

For example, if you ask “Who are the best AI search optimization consultants?” and the AI lists four companies total, including yours once, your AI Share of Voice for that prompt is 25 percent.

This metric does not care whether you were listed first or last. It tells you how much of the recommendation space you occupy in that moment, which is far more meaningful than rank in a system that reshuffles constantly.

Why consistent share of voice changes AI behavior

This is where Rand Fishkin’s research connects directly to work I have been doing independently.

Because rankings reshuffle constantly, traditional positional thinking does not apply. However, when a brand appears consistently and earns a meaningful share of voice within AI-generated answers, it becomes part of the model’s default consideration set.

That consideration set is what brands should be targeting.

A real-world case study: displacing Search Engine Journal

In December, I published a case study titled The Displacement of Legacy Authority, based on 125 hours of AI Share of Voice analysis.

After a targeted GEO update, my brand began appearing more frequently in AI-generated answers. When it appeared, it occupied a larger portion of the answer. Over time, this resulted in my brand appearing in responses where Search Engine Journal had previously dominated.

This was not because my brand outranked Search Engine Journal. There is no stable ranking to beat in AI systems. Instead, the probability of my brand being mentioned increased because the content was more aligned with user intent and reinforced consistently across channels.

Different runs of the same prompt still produced different citations, but the likelihood of inclusion shifted. That shift is what matters.

How smaller brands can compete in AI search

AI search engines do not care about traditional domain authority or website size in the way classic search engines do. They respond to consistent, reinforced patterns across the surfaces they learn from, including websites, landing pages, social media, and other public content.

If an AI system learns that a brand is relevant, credible, and contextually appropriate, it will include that brand regardless of how large or well-known it is.

This is how smaller brands can compete with legacy publishers in AI-generated answers.

A necessary warning about AI visibility tools

At this point, it is important to be clear about tools.

Tools are not the enemy. There are intelligent teams building thoughtful products in this space, and measurement will matter as AI search continues to evolve. I have spoken with many of the builders behind these tools and even hosted several of them on the podcast.

However, tools are amplifiers, not teachers.

If you do not understand how AI systems generate answers, why rankings fluctuate, what visibility actually represents, or which prompts matter most to your business, a dashboard will not help you. It will simply give you numbers without context.

Before spending money—especially money that stretches your budget—it is critical to understand the mechanics first. You need to decide what visibility means in your category and what success actually looks like. Only then should you choose a tool that aligns with those goals and provides the measurements you actually need.

The real takeaway from Rand’s research

Rand Fishkin’s research did not prove that AI visibility cannot be measured. It proved that rankings are the wrong metric.

Visibility in AI search is a probability game.

Brands that understand this, measure the right signals, and optimize for consistent inclusion rather than position will be better equipped to succeed as AI-powered search continues to reshape discovery.

If you want help understanding how AI engines evaluate brands, measuring real AI Share of Voice, or building a GEO strategy that increases your chances of being included, that is exactly the work I do.

You do not need to chase rankings. You need to increase the odds.

If we haven’t met yet….

Hi, I’m Cassie, a fractional content strategist for early-stage startups who focuses on AI search optimization.

If AI-generated answers are already influencing your buyers, your content strategy needs to account for that. I build content programs that connect strategy with execution: clear positioning,  systems, and content that actually drive revenue. If you’re ready to stop guessing and start growing, here’s how we can work together.