The five questions
Each one has a wrong answer you can act on.
- 01
Where does classification physically execute?
Endpoint, your cloud, or the vendor’s? This single answer determines whether the tool creates a second copy of the data you are protecting, whether you need another DPA or BAA, and whether the vendor’s breach becomes your breach.
- 02
Is the stored snippet masked or reversible?
Masking destroys the characters. Tokenisation replaces them with something a key reverses. If it is reversible, who holds the key, and who can request it?
- 03
What does it not cover?
Any vendor answering “everything” has ended their own evaluation. Ask specifically about file uploads, pasted images, coding assistants, desktop apps, mobile and unmanaged browsers. Then ask how you would know if coverage broke.
- 04
Does the console report intent, or outcome?
A policy set to block that fell back to warn under load should report warn. Systems reporting policy intent rather than enforcement outcome systematically overstate your protection.
- 05
What happens at the latency deadline?
Does the product hold the submit indefinitely, fail open silently, or fail open visibly with a flag and a metric? The third is the only acceptable answer, and it is the least common.
Verify the privacy claim yourself
Don’t take anyone’s word, ours included.
A vendor confident in their claim will help you run this test. One that deflects has told you something useful.
- 01
Install the agent in a lab
On a device you control.
- 02
Capture outbound traffic
At the network layer, not from the product’s own logs.
- 03
Submit a known unique string
Through a covered tool.
- 04
Search the capture
For that string, and for any encoding of it.
- 05
Repeat with debug logging on
Debug builds are where real leaks live.
Why there are no named comparisons here yet
Because sourcing them properly is slow.
A comparison page that characterises a competitor from memory, a sales deck or a conference conversation is how vendors end up in legal correspondence — and, more to the point, how buyers end up with a wrong picture. Every claim on a named comparison has to come from that vendor’s own public documentation, dated and linked.
Until we can do that, the framework above is more useful than a table we cannot defend.