Binding verification
The capture layer checks on each page load that its hooks are attached to the live composer. Installed and binding are different facts and should be different fields.
The core problem
A console showing no violations means one of two things.
One: nothing sensitive was submitted. Excellent.
Two: the capture layer stopped working three weeks ago, when the AI tool shipped a redesign and a selector stopped matching.
A dashboard that cannot distinguish these is worse than no dashboard, because it converts an outage into confidence.
Why AI monitoring is prone to it
Most capture layers depend on the structure of someone else’s web application.
When a vendor redesigns their composer, hooks unbind. No error is raised — there is simply nothing to report, from a component that is still running, still reporting healthy, still installed.
Deploy-and-forget is not viable here. The upstream surface changes without notice and without a changelog.
Making gaps loud
Four mechanisms, roughly in order of value.
Synthetic canaries
Scheduled benign submissions with known test patterns, confirming the whole path is alive end to end. A canary that stops arriving is an alert.
Published gap lists
A gap a customer knows about is a risk they can size. A gap they do not is an incident waiting for someone else.
Degradation as a metric
Where a control fails open under load, the rate of fail-open events is itself a service level. Silent fail-open is how a system reports perfect health while protecting nothing.
What to ask a vendor
Four questions, and the first is disqualifying.
- 01
What surfaces do you not cover?
If the answer is “none”, end the evaluation.
- 02
How do I tell working-and-quiet from broken-and-quiet?
If the console cannot, neither can you.
- 03
What happens when a monitored site redesigns?
Specifically: how fast do you know, and how do I find out?
- 04
Does the console distinguish “nothing found” from “we ran out of time”?
Those are different events that look identical in most products.
Related terms