1. Shared by Default: How Commercial AI Actually Works
The Apartment Building and the House
Think of a standard enterprise AI plan as an apartment in a large apartment building. Your apartment has its own lock, and your neighbors cannot walk in. But everyone shares the same foundation, plumbing, and wiring, and the management company holds a master key to every door. If a pipe bursts two floors up, your apartment can flood no matter how good your lock is.
In technical terms, that is multi-tenancy: many customers share the same computers, and software keeps them apart. Each customer's data is tagged with an account number, and the software is supposed to show you only the data with your number on it. This is how nearly all cloud software works, and most of the time it works well.
In contrast, a single-tenant environment is a house on its own lot. The computers, network, and storage are set aside for one organization. Nobody else's data is processed there, and you decide who gets a key. Security engineers call this kind of setup an enclave, which NIST defines as "a collection of information systems connected by one or more internal networks under the control of a single authority and security policy" (NIST CSRC Glossary, Enclave).
When Shared Systems Fail
Software-based separation is only as good as the software. Three examples show how it can break, and they differ in how real the threat is:
- A real incident: one customer's data shown to another. In March 2023, a bug in a widely used open-source component caused ChatGPT to show some users the titles of other users' conversations, and in some cases partial payment details. OpenAI took the service offline and published a full account (OpenAI, March 20 ChatGPT Outage: Here's What Happened). No one attacked anything. A routine bug in shared infrastructure was enough.
- A real incident: prompts sitting in a log file. In January 2025, security researchers found a DeepSeek database open to the public internet. It held more than a million lines of system logs, including users' chat histories in plain text (Wiz Research, Wiz Research Uncovers Exposed DeepSeek Database Leaking Sensitive Information). Systems record what passes through them, and those records are easy to overlook.
- A research demonstration: reading another user's AI output from shared hardware. In January 2024, researchers at Trail of Bits showed that on certain graphics chips (from Apple, AMD, Qualcomm, and Imagination), one program could read leftover data from another program's AI session, enough to reconstruct the AI's responses (Trail of Bits, LeftoverLocals). To be clear, this was a laboratory demonstration, not a known attack on an AI vendor, and the affected manufacturers have issued fixes. But it shows that sharing hardware carries real, if remote, risk.
For a retailer, risks like these are acceptable costs of doing business. For a firm holding a client's merger terms or a witness outline, a single leak can mean a malpractice claim, a lost client, and a fight over privilege.
Haven't Lawyers Dealt With This Before?
Yes, they have. Bar regulators have addressed cloud computing for more than a decade. New York approved cloud storage of client files in 2010, as long as the lawyer takes reasonable care to protect confidentiality (NYSBA Ethics Opinion 842). The ABA later endorsed a fact-specific, risk-based approach to securing client information (ABA Formal Opinion 477R (2017)). The major cloud providers already sell every control described in this article.
What is new is how AI is used. Lawyers do not just store files in AI tools. They paste in strategy memos, draft arguments, and deposition outlines, often the most sensitive material a firm has. A single AI product may also involve several companies: the app vendor, the model provider, and the cloud host, each with its own terms. And the default settings tend to favor the vendor, not the firm.
So the problem is not that no one has solved this. The tools exist. The hard part is choosing the right ones, setting them up correctly, and confirming they work as promised.
2. Who Holds the Key?
Every vendor will tell you that your data is encrypted. That is the bare minimum. The question that matters is who holds the key.
The Hotel Safe and the Safe-Deposit Box
With a standard plan, encryption works like a hotel room safe. Your valuables are locked up, but the hotel can open the safe whenever it decides to, and you will not necessarily know.
Customer-Managed Encryption Keys (CMEK) work more like a safe-deposit box that needs your key to open. The vendor stores your locked data, but every time the system needs to read it, it has to ask permission to use your key. Every request is recorded in a log that you can review. If you destroy the key, the data becomes inaccessible to everyone, the vendor included. At the end of a matter, destroying the key effectively destroys every copy of the data, including old backups.
There is one caveat. With standard CMEK, your key lives in the cloud provider's key service, inside your own account. The provider cannot casually use it, and every use is logged, but it is still in the provider's building. For the most sensitive work, external key management keeps the key with a separate key manager outside the cloud provider entirely (Google Cloud, Cloud External Key Manager; AWS, External Key Stores). Which option fits depends on the matter.
Why This Matters If Your Vendor Is Subpoenaed
Under the Stored Communications Act, the government can demand customer records directly from a cloud or AI provider (18 U.S.C. § 2703). A court can also order the provider not to tell you (18 U.S.C. § 2705(b)). The Department of Justice tells its prosecutors to seek business data from the business itself when practical, but it allows them to go straight to the provider when, for example, notice would put an investigation at risk (DOJ, Seeking Enterprise Customer Data Held by Cloud Service Providers (Dec. 2017)).
If the vendor holds the key, it can be ordered to hand over readable copies of your client's information, possibly without your knowledge. If you hold the key, and especially if it sits outside the provider, the vendor has only scrambled data to give. The practical path to a readable copy runs through you, which gives you the chance to assert privilege before anything is disclosed. That is not a guarantee, since courts can order a great deal. But it puts you back in the conversation.
3. A Private Road Instead of the Public Highway
Encryption protects data at rest, but you also need to control how data moves.
A Virtual Private Cloud (VPC) is a walled-off section of the cloud that only your organization uses. In a dedicated setup, your attorneys reach the AI system through private connections from your firm's network, rather than over the public internet. Think of it as a private road between your office and the system, instead of a public highway.
Just as important, the AI system itself cannot make outbound connections to the internet. That matters more than you might think. AI tools can be tricked by instructions hidden inside a document, a risk the security community calls prompt injection, which ranks first on OWASP's list of AI application risks (OWASP, LLM01: Prompt Injection). If a booby-trapped document tells the AI to send your files somewhere, a system with no outbound connections has nowhere to send them.
4. "No Training" Is Not the Same as "No Retention"
This is the most common point of confusion in vendor conversations. A promise not to train on your data says nothing about whether the vendor keeps it.
By default, OpenAI keeps API prompts and outputs for up to 30 days to monitor for abuse. Customers who qualify can apply for Zero Data Retention (ZDR), under which that data is not stored (OpenAI, Data Controls in the OpenAI Platform).
Why does 30 days matter? Because anything a vendor keeps can be swept into someone else's lawsuit. In May 2025, in the New York Times' copyright case against OpenAI, a federal court ordered OpenAI to preserve output logs it would otherwise have deleted, including conversations users had deleted themselves. OpenAI said customers using Zero Data Retention were not affected, because there was nothing stored to preserve (OpenAI, How We're Responding to The New York Times' Data Demands). If your client had nothing to do with that case, their data could have been caught up in it anyway.
What Zero Retention Does and Does Not Mean
Zero Data Retention means the AI model provider keeps nothing after it answers. It does not mean nothing is stored anywhere. Your attorneys need to come back to yesterday's research and pick up last week's draft, so your application has to save chat histories and outputs, and it should.
The difference is where those records live. In a dedicated environment, they sit in your own database, encrypted with your keys, governed by your retention policy, and covered by your litigation holds. Meanwhile, the external AI's working memory is wiped after each request, on hardware no one else uses. The goal is not zero records. It is that the records remain solely in your control.
5. What This Means for Privilege
Let's start with a clear distinction. Dedicated infrastructure is first a confidentiality measure. Privilege protection follows from being able to show you took reasonable precautions. No ethics rule requires single-tenant AI. It is a risk-based choice, and for routine work a well-configured enterprise plan might be reasonable.
The Ethics Standard: Reasonable Efforts
ABA Model Rule 1.6(c) requires "reasonable efforts" to prevent unauthorized access to client information. The comments ask how sensitive the information is, how likely a disclosure is without more safeguards, and what those safeguards would cost. ABA Formal Opinion 512 (2024) adds that lawyers must understand how an AI tool uses and stores what they put into it. In practice, that means reading the terms of use and privacy policy, or consulting someone who has.
What Courts Are Starting to Say
As discussed in my last article, United States v. Kovel, 296 F.2d 918 (2d Cir. 1961), extends privilege to outside parties who help a lawyer give legal advice. An AI vendor whose contract describes it as an independent contractor, not your agent, is an awkward fit for that doctrine (OpenAI, Services Agreement).
Courts are beginning to look at exactly these terms. In February 2026, Judge Rakoff held in United States v. Heppner, No. 25-cr-00503-JSR, 2026 WL 436579 (S.D.N.Y. Feb. 17, 2026), that documents a defendant created with a consumer AI tool were not privileged. Among other things, the court pointed to the tool's terms, which allowed the provider to review user data, as undermining any expectation of confidentiality. Heppner involved a free consumer product used on the defendant's own initiative, not under his lawyers' direction, so it does not decide how an enterprise or dedicated deployment would fare, but it confirms that courts will read your vendor's terms when deciding whether a communication was confidential.
Harleysville Insurance Co. v. Holding Funeral Home, Inc. shows how these disputes play out. A claims file containing privileged material was shared through an unprotected cloud link, and opposing counsel got in. The magistrate judge found waiver, calling it "the cyber world equivalent of leaving its claims file on a bench in the public square" (No. 1:15cv00057, W.D. Va. Feb. 9, 2017). The district judge reversed the waiver finding, concluding the precautions were reasonable under the circumstances, 2017 WL 4368617 (W.D. Va. Oct. 2, 2017). The insurer kept its privilege, but only after months of litigation over whether one shared link was reasonable. If that question is ever asked about your AI tools, you want to answer it with records, not arguments.
Why "Inadvertent Disclosure" May Not Save You
Federal Rule of Evidence 502(b) protects inadvertent disclosures when reasonable steps were taken to protect the information. Typing client information into an AI tool is deliberate. If a court treats the vendor as an outsider, the inadvertence safety net may not apply.
A Refund Is Not a Remedy
If a vendor leaks your data, you may have a contract claim. But standard terms commonly cap the vendor's liability at roughly what you paid in the prior twelve months and exclude consequential damages (OpenAI, Services Agreement; AWS, Customer Agreement). A year of software fees does nothing for a client whose litigation strategy is now discoverable. Waivers cannot be undone.
A well-documented dedicated environment will not make privilege disputes disappear, but it will give you a strong, documented defense when one arises.
6. Side-by-Side Comparison
| Question | Standard Enterprise Plan | Dedicated Single-Tenant Environment |
|---|---|---|
| Who else uses the hardware? | Many other customers. | Only your organization. |
| Who holds the encryption key? | Usually the vendor. | You, in your own cloud account or an external key manager. |
| How does data travel? | Usually over the public internet, encrypted. | Over private connections from your network. |
| Can the system reach the internet? | Usually, yes. | No. Outbound connections are blocked. |
| How long does the AI provider keep prompts? | Often up to 30 days, unless Zero Data Retention is approved. | Nothing kept by the model provider. |
| Where do chat histories live? | In the vendor's systems. | In your database, under your keys and retention policy. |
| If the vendor is subpoenaed? | It can produce readable data. | It holds only scrambled data. Readable copies come through you. |
| Proof if privilege is challenged? | The vendor's assurances. | Your own access logs and configuration records. |
7. Seven Questions to Ask Before Client Data Goes In
These questions are simple to ask. The answers are where it gets difficult. For each, here is what a weak answer tends to sound like.
- Who holds the encryption key, and can we turn it off? Weak answer: "Everything is encrypted with AES-256." That tells you how the data is locked, not who holds the key.
- Is anyone else's data processed on the same hardware? Weak answer: "Your data is logically separated." That means software separation on shared machines.
- How long do you keep our prompts and outputs, and is that in the signed contract? Weak answer: "We never train on your data." Training and retention are different questions.
- Does your retention commitment cover every company that touches our data? Weak answer: "Our subprocessors meet industry standards." Ask for the list and their terms.
- Where do our chat histories and documents live, and who can read them? Weak answer: "In our secure cloud." Whose account, whose keys, and which employees?
- Can the system make outbound connections to the internet? Weak answer: "Only to trusted services." Which ones, and who decides?
- If you receive a subpoena with a gag order, what could you actually hand over? Weak answer: "We would fight it." Being willing to fight is not the same as having nothing readable to produce.
Why the Answers Are Harder to Read Than the Questions
Vendors use words like "private," "dedicated," and "isolated" to mean very different things. To tell them apart, you need someone who understands both the law and the technology. That combination is rare. Your IT team will know the systems but may not know what Rule 1.6 or FRE 502 requires. Your counsel will know the rules but may not be able to tell whether a vendor's architecture diagram matches its contract.
If no one at the table speaks both languages, you will have to take your vendor at its word, and the safety of your clients' data will depend on it. For most, the practical answer is an outside advisor who can read a vendor's technical blueprints and spot the legal risks buried in them.
9. Conclusion
Enterprise AI plans are a real improvement over consumer chatbots, and for plenty of work they are enough. But "we don't train on your data" is the beginning of the conversation, not the end.
For your most sensitive matters, the questions that count are simpler and harder: Who else is on the hardware? Who holds the key? What does the vendor keep, and for how long? If someone came asking, what could the vendor hand over?
A single-tenant environment lets you answer each of those with "only us." If you are weighing whether your practice needs one, or want a second set of eyes on the answers your current vendor has given you, I am glad to talk it through.