A wrong number rarely gets caught the first time it’s said out loud in a meeting. It gets caught the third time, once it has already shaped a decision or two, when someone finally asks where it came from.
That delay is the actual challenge with shadow AI. It doesn’t announce itself. There’s no login screen, no request ticket, no line item anywhere that says “business question answered outside governed systems.” The evidence is scattered across normal-looking behavior that only reads as a pattern once someone is looking for it.
Here’s what to actually look for, and what it usually means when you find it.
Analysts talk about double-checking more than they used to
Listen for a specific phrase: “let me just double-check that.” If it’s coming up more often, especially attached to numbers the analyst didn’t personally produce, that’s usually a sign someone upstream got an answer from somewhere other than the governed system and brought it into a conversation as if it were settled.
The double-checking itself isn’t the problem. It’s a reasonable response to uncertainty. The problem is what created the uncertainty in the first place: a number entered circulation without going through anything the data team controls.
A number shows up that nobody on the data team recognizes
This is usually the first hard signal, and it typically arrives sideways. Someone references a figure in a meeting, a data team member hears it, and has a moment of “wait, where did that come from?” If that moment is happening even occasionally, it’s worth treating as a signal rather than a one-off.
The specific tell is not that the number is wrong. Sometimes it’s approximately right. The tell is that nobody can trace it back to a query, a report, or a person who built it deliberately.
People describe getting answers “faster” from somewhere else
If you hear language like “I just asked it directly” or “it was quicker to check myself,” pay attention to what “it” refers to. Often this is completely legitimate, someone found a faster path through an approved tool. Sometimes it isn’t, and the faster path is a public AI assistant that has no connection to certified definitions, no access controls, and no record of what was asked or answered.
The distinguishing question is simple: can the data team see what was asked and what came back? If the honest answer is no, the speed came from somewhere ungoverned.
Reconciliation requests are increasing
A rising number of “why doesn’t this match” questions is one of the more measurable signals available, because it usually leaves a trail in tickets, Slack threads, or email. If reconciliation requests are trending up without a clear cause like a new data source or a metric redefinition, ungoverned answers entering circulation is a reasonable hypothesis to test.
The pattern to watch for specifically is reconciliation requests where the second number has no clear origin. A mismatch between two known systems is a data quality issue. A mismatch where one side traces to nothing is usually shadow AI.
Nobody can say with confidence where a specific number came from
This is the clearest version of the problem, and also the hardest one to notice, because the absence of an answer doesn’t feel like an event. Try it as a direct exercise: pick a number that appeared in a recent leadership deck and ask whoever presented it to trace its exact origin, the source table, the definition used, the person or system that produced it.
If that trace takes more than a few minutes, or comes back with genuine uncertainty, that’s not a one-time gap. It’s evidence of how numbers are currently entering the organization, and it will keep happening the same way until something changes about the path itself.
The data team has started asking “did this come from us?” first
This is a late-stage signal, and it means the organization has already been through enough incidents that the data team has adapted its default posture. When “did this come from us” becomes a routine first question rather than an unusual one, the shadow AI problem has been present long enough to change behavior, not just create occasional friction.
What to do with this list
None of these signals alone proves anything. A single moment of double-checking, a single unfamiliar number, a single fast answer, all of those happen in healthy organizations too. What matters is whether several of these are showing up together and trending in the same direction.
If they are, the next step is not a policy memo banning specific tools. That tends to move the behavior somewhere less visible rather than removing it, since the underlying need for a fast answer to a business question doesn’t disappear because the tool got blocked.
The more durable fix is making the governed path at least as fast as the ungoverned one: conversational access to data that resolves through certified definitions, with every question and answer logged automatically. When the sanctioned path is no slower than the alternative, there’s no longer a reason for the alternative to exist quietly in the background.