Using multiple LLMs to do complex work — privately

There's a quiet assumption that you pick "the best model" and use it for everything. In real work that's wrong twice over: no model is best at every task, and routing everything — raw — to one public model is a privacy liability you can't take in a regulated org. The answer is many models behind one governed path.

Route by capability and cost

Complex work decomposes into different kinds of steps — fast classification, deep reasoning, code generation, long-context synthesis. Each has a model that's best, and often cheapest, for it. A coordinator breaks the job into a plan and routes each step to the right model.

Example: processing a stack of contracts. A fast, cheap model classifies and routes each document; a strong reasoning model does the clause-by-clause risk analysis; a long-context model synthesizes the portfolio summary. Forcing one premium model to do all three would cost more and do the easy parts no better. Routing gets you higher quality and lower cost.

The privacy problem with raw multi-model

Multi-model only helps if it doesn't leak your data. Sending raw, PII-laden context to a handful of public APIs multiplies the exposure — now your sensitive data is in several third parties' systems instead of one. For regulated organizations, that's a non-starter (see what goes wrong with raw rollouts).

The answer: a governed gateway in front of the fleet

Put a governed gateway between your data and the models. Every request is authenticated; PII is stripped before anything leaves your boundary; policy and access are enforced; the call is routed to the right model; the output is filtered; and every action is logged. Sessions are ephemeral. The models — whichever you route to — see only scrubbed, scoped data.

Example: the contract stack again, but governed. Names, parties, and figures are tokenized before any model sees them; each model works on the redacted version; results are re-hydrated locally. You used three different providers and not one of them ever held your clients' identities.

Multi-model gives you the quality. The governed gateway gives you the privacy. The point is that you don't have to choose between them.

Why this is the durable pattern

Models will keep changing — a new leader every few months. If your architecture is welded to one provider, every shift is a migration. If it's "many models behind one governed gateway," swapping or adding a model is a config change, and your data governance never moves. You get the best of whatever's current, privately, without re-architecting. That's how you do genuinely complex work without handing your proprietary data to someone else's training set.