The AI productivity argument is now compelling enough that most professional services firms have stopped arguing against it. The question has shifted from “should we use AI” to “how do we use it without creating a problem.”
The answer a lot of firms have landed on is: carefully. Use ChatGPT, but don’t put anything sensitive in. Use it for drafts, not for real files. Use the enterprise version — that one’s supposed to be fine.
These are rationalisations. Not policies.
Here is what the risk actually looks like, why “being careful” is not a durable answer, and what a real solution involves.
The actual data flow when you use a public AI tool
When a lawyer types a summary of a client matter into ChatGPT to help draft a letter, that text leaves the firm’s network. It travels to OpenAI’s servers, is processed there, and a response comes back. What happens to that text after the fact depends on which version of the tool is being used, whether the account has specific data handling agreements in place, and what the current terms of service say — which have changed multiple times.
Most consumer and standard business accounts have historically allowed inputs to be used to improve underlying models. Enterprise agreements offer stronger commitments: data not used for training, limited retention periods. Microsoft’s Copilot products, through certain M365 licensing tiers, extend similar commitments.
The issue is not that these commitments are necessarily dishonest. The issue is that “data not used for training” and “data not leaving your firm’s control” are different things. The information is still transmitted over the internet, still processed on third-party infrastructure, still subject to whatever legal obligations, security incidents, or policy changes affect that infrastructure in the future.
For most industries, that is an acceptable trade-off. For professional services — where client confidentiality is not a preference but a professional and frequently legal obligation — it deserves more scrutiny than it usually gets.
Why “being careful” is not a policy
The risk is rarely a single catastrophic disclosure. It is the cumulative pattern.
A lawyer uses ChatGPT to help draft a demand letter — no client name, just the situation. Then to summarise a deposition — changed the names, kept the facts. Then to draft a regulatory response — pasted in a few paragraphs from the file because it was faster. Each individual decision seemed defensible. The aggregate is a pattern of client information leaving the firm’s network on a regular basis.
The other problem is consistency. “Be careful with client data” is not a rule a team can apply uniformly, because it requires individual judgement at the moment of highest time pressure — when someone is trying to get something done quickly. The variance in how different people interpret a vague policy is not small. You will find out what that variance looks like at the worst possible time.
This is why regulators in legal and accounting have been increasingly specific in their guidance. The Law Society of Ontario, the Law Society of British Columbia, and CPA Canada have all issued guidance on AI tool use in practice. The consistent thread across all of it: professional obligations around confidentiality apply regardless of the tool, and firms are responsible for understanding where client data goes when they use third-party services.
What the enterprise tier does and doesn’t solve
ChatGPT Enterprise and Microsoft Copilot with appropriate licensing are meaningfully better than consumer tools for data handling. For many firms, they are probably sufficient.
But there is a category of firm for which cloud-based AI — regardless of the vendor’s commitments — is not the right architecture:
Firms handling litigation involving commercially sensitive information where even a theoretical risk of third-party disclosure is unacceptable to the client. Firms advising on M&A transactions under strict confidentiality requirements. Accounting firms whose clients operate in regulated industries with explicit data residency requirements. Any firm that has had a client ask, directly, where their information goes when the firm uses AI tools — and found the honest answer uncomfortable.
For these firms, the question is not which cloud vendor to trust. It is whether cloud-based AI is the right approach at all.
What on-premise AI actually means
An on-premise AI deployment runs entirely on hardware the firm owns or controls — a server on-site or on infrastructure the firm manages directly. Queries are processed locally. Nothing leaves the network. No third party has access to the prompts, the outputs, or the usage patterns.
This is not a new concept repackaged. Firms have hosted their own practice management software, document systems, and email servers for decades, precisely because they could not depend on a third party’s infrastructure for sensitive operations. On-premise AI is the same logic applied to a newer category of tool.
What has changed is feasibility. Two or three years ago, running a capable language model on local hardware required significant compute investment and produced results noticeably worse than the frontier cloud models. That gap has closed substantially. Open-source models — Meta’s Llama series, the Mistral family — are now capable enough for most professional services use cases on hardware that is within reach for mid-sized firms.
The practical use cases on-premise AI handles well for legal and accounting firms include:
- Summarising long documents — case files, contracts, financial statements — without sending them anywhere
- Drafting standard correspondence, memos, and client-facing reports
- Synthesising research across multiple sources
- Generating time entry descriptions from notes
- Answering questions against the firm’s own internal knowledge base
These are high-value, time-consuming tasks that currently absorb significant hours across most practices.
What it actually involves
On-premise AI is not a product you purchase and activate. It involves model selection for your specific use cases, hardware provisioning, deployment and configuration, integration with your existing document and matter management systems, and staff training on effective use.
That is worth being honest about. The total cost — hardware, implementation, integration, and training — is a real investment, and the answer to “is it worth it” will not be the same for every firm.
For smaller practices, a realistic starting point is to implement on-premise AI for the specific workflows with the highest confidentiality exposure, use vetted cloud tools for lower-risk tasks, and expand based on demonstrated return. The two approaches can coexist. What matters is that the decision about which category each workflow falls into is made deliberately, not by individual staff members under time pressure.
The question worth asking now
Most professional services firms are somewhere on a spectrum between two positions they cannot hold indefinitely: avoiding AI entirely while competitors adopt it, or using public AI tools with client data in ways that do not fully withstand scrutiny.
The firms that navigate this well will treat it as an infrastructure decision — the same way they treat decisions about where client files are stored or how email is managed — rather than a cultural policy answered with “use your judgement.”
On-premise AI is not the right answer for every practice. But it is worth understanding what it actually involves before deciding it isn’t.