The Two Room Thought Experiment
What happens when we try to separate expertise from access? The answer is surprisingly asymmetric: hiding a person's identity is easy. Hiding a company's? Nearly impossible.
The Setup
Picture two rooms, two people, no direct line of sight but can talk.
Room 1 has all the sensitive stuff: names, addresses, credit cards, confidential data. But the person in there? Not an expert.
Room 2 has the expert. They give instructions, make decisions, solve problems. But they only see what Room 1 tells them, stripped of identifying details.
So here's the question: for what kinds of tasks can this actually work?
Level 1: Simple Task
Someone needs dinner ordered. Room 1 has DoorDash open with saved payment and address. Room 2 knows how food delivery apps work.
The food gets ordered. The expert never learns who they're ordering for, where it's going, or how it's paid for. Task complete, privacy intact.
Level 2: Medium Complexity
A customer service expert needs to handle a complaint. Room 1 can see the customer's account, order history, personal info.
Sound business decision made. The expert saw patterns and history, not names or addresses.
For most cases, this works fine. But consider the edge: a customer who ordered a custom-engraved laptop with accessibility modifications, mentioned they're a professor at a local university, and previously bought textbooks on quantum physics? That combination of details starts to become identifying on its own. These cases are rare, though. Individual anonymization holds up pretty well.
Level 3: High Complexity
A VC firm needs to evaluate a startup. Strict NDA. Room 1 has the pitch deck and financials. Room 2 is a senior investor, 25 years in the game.
The abstraction just collapsed. Every piece of information the expert needs to make a real decision (the technical approach, the competitors, the market segment, the team) is effectively the identity. Even without the company name, Room 2 can figure out who they're evaluating.
The Complexity Threshold
A pattern becomes visible across these levels:
- Simple tasks work perfectly. Generic instructions, no specific context needed.
- Individual PII can usually stay hidden. Edge cases exist, but they're uncommon.
- Organizational identity fails fast. The details needed to evaluate a company are what identify it.
A person's name doesn't change how you order their food. But a company's technology, its competitors, its market position? That is the evaluation. You can't separate the two.
Theoretical Foundations
Claude Shannon's information theory and decades of privacy research formalize what the thought experiment reveals intuitively.
Shannon's Information Entropy
Shannon formalized a way to measure information content. Entropy quantifies the average "surprise" in a random variable — roughly, how many yes/no questions you'd need to pin down an unknown value. When a combination of attributes is rare, each attribute carries high self-information, and the set becomes identifying.
Common things have low information content. Rare things become identifying.
k-Anonymity
Privacy researchers use k-anonymity: data is k-anonymous if each record is indistinguishable from at least k-1 others. In plain terms: how many people (or companies) match this description?
For individuals, k is typically in the thousands or millions. For organizations with specific market positions, k drops to single digits almost immediately.
Differential privacy (Dwork, 2006) takes this further: rather than trying to hide identities after the fact, it mathematically bounds how much any single record can influence a query result. But even differential privacy can't help when the query itself is the identity — which is exactly the organizational case.
Real-World Applications
How This Plays Out by Domain
| Domain | Individual PII | Organizational Identity |
|---|---|---|
| Healthcare | Patient processing works | Hospital review? Can't hide it |
| Legal | Client comms mostly work | M&A due diligence fails |
| Finance | Transaction processing works | Investment eval? Impossible |
| HR | Resume screening works | Executive search breaks down |
Implications
The Limits of Anonymization
You can strip a person's name from a document and still get useful work done. Try that with a company, and you've removed the very thing that makes evaluation possible.
What Expertise Actually Is
It's not just following procedures. It's pattern recognition across specific contexts. Simple tasks reduce to generic steps. Complex decisions require context that, for organizations, is inseparable from identity.
AI and Automation
- Simple tasks automate easily with minimal context
- Medium complexity requires careful trade-offs between privacy and capability
- High complexity may resist both anonymization and full automation for the same reason
The Trust Question
For complex tasks, the solution isn't better anonymization. It's better trust frameworks: contractual, technical, and legal structures that enable necessary information sharing with appropriate protections.
Where does the threshold lie in your domain?
References
- Shannon, C.E. (1948). "A Mathematical Theory of Communication." Bell System Technical Journal.
- Sweeney, L. (2002). "k-Anonymity: A Model for Protecting Privacy." Int'l Journal of Uncertainty, Fuzziness and Knowledge-Based Systems.
- Narayanan, A. & Shmatikov, V. (2008). "Robust De-anonymization of Large Sparse Datasets." IEEE S&P.
- IBM Security. (2023). "Cost of a Data Breach Report 2023."
- Dwork, C. (2006). "Differential Privacy." ICALP.