An assistant for many companies: where it breaks first

A chatbot for one company is a week’s exercise. The same thing for ten companies on one system is a different problem, and the difference is not the model or the answer quality — it is isolation.

Retrieval breaks first

Retrieval breaks first. The model does not “remember” a tenant’s data, but it reads whatever you hand it, and the retrieval layer decides what gets handed over. A tenant filter applied after the semantic search rather than built into the index is a leak waiting for a cleverly phrased query. The rule: tenant is a partition key in the index itself, not a filter over the results.

Then the instructions

Instructions break second. When the system prompt is assembled by concatenating text from tenant settings, you have given every tenant write access to the most dangerous line you own. Tenants fill in templates; they do not write instructions.

Then the cost

Cost breaks third, and it is discovered in the invoice rather than in testing. One tenant uploads a thousand-page file and asks about it a thousand times, and everyone else pays for it. A per-tenant rate limit, a per-conversation token ceiling, and a counter they can see in their own panel — three things you build on day one or build after a painful bill.

And what never shows in a demo

Fourth is the one that never shows up in a demo: what happens when the assistant does not know. An assistant that guesses looks smarter in testing and is more dangerous in production, especially when wired to a system that executes. An answer that says “I do not know, and here is the page that does” is more useful than a confident wrong one, and that is a design decision rather than a model one.

And when it is wired to something that executes

When the assistant is wired to Odoo or an internal system, every point above gets heavier: a read leaks, a write damages. So what reads is separated from what writes, and every writing action is explicitly named, bounded, and logged under whoever asked for it.