Design an LLM customer-support assistant for a SaaS company

I would start by asking what it should handle: common how-to questions only, or account-specific requests such as billing status? Which languages and channels, what volume, and which actions can it take without a human? Let us say chat on the web app, English first, with read-only account access and refunds handled by people.

The flow: a fast intent classifier routes greetings, how-to questions, account questions and complaints. For how-to questions, hybrid retrieval over help articles and resolved tickets feeds a model that answers with links to sources. For account questions, the assistant calls tools such as get_subscription with the logged-in user's identity, so it can only see that customer's data. If confidence is low, the customer asks for a person, or the topic is sensitive, like cancellations or legal issues, it hands off to an agent with a short summary so the customer does not repeat themselves.

For safety, actions with side effects go through approval, tool outputs and documents are treated as untrusted, and personal data is masked in logs. Before launch I would evaluate on a few hundred historical tickets for correctness, groundedness and correct escalation, then roll out gradually and track resolution without escalation, reopen rate, satisfaction, cost per conversation and latency.