What should a chatbot demo prove?
A customer service chatbot demo should show what the assistant can answer, what it can read, and what it can change. In a review environment, ask it to answer a normal question from an approved source, decline a request outside its permissions, and hand a sensitive case to the right employee. Compare the chat reply with the connected system’s records. A reassuring message is weak evidence if a write still reached the system behind the chat.
Prompt injection is an attempt to make an assistant treat a user message or retrieved content as an instruction that can change its operating rules. The instruction can arrive directly in a chat or indirectly through a document, page or other source the assistant reads. What happens next depends on the data and actions available to the application. The OWASP GenAI Security Project glossary describes these two paths. Microsoft’s prompt injection guidance covers controls such as narrowing tool access and defining data boundaries.
Make a permissions map for the buyer demo
Start with three columns: answer, read and act. An answer can use an approved public policy. Reading a particular customer record requires an authorization rule appropriate to the channel. Changing the record adds a separate decision about who may act, what can be changed, and how the system records the result. Give each row a business owner and specify the system that enforces the permission.
Consider a property services firm receiving inquiries in Arabic and English. A website visitor can ask for public viewing hours. A person asking for the details of a previously arranged visit needs to use the appropriate identity check. A request to move that visit also needs authority to change it. Each of these requests can look like an ordinary sentence in the conversation. Require the demo to show how the workflow handles the difference.
| Illustrative case | Expected behavior | Evidence to collect |
|---|---|---|
| Public opening hours | Answer from the approved source in the requested language | Response and source revision |
| Another customer’s service record | Decline disclosure or route to the authorized verification path | No restricted record returned or displayed |
| Document text attempts to change permissions | Treat the text as content and preserve the configured boundary | Tool logs show no unauthorized write or transfer |
| Unverified change to an appointment | Explain the next step without confirming a change | The fictitious appointment record is unchanged |
| Complaint requiring an employee decision | Pass the minimum necessary context to the correct queue | Queue entry, owner and permitted fields |
Use fictional records and an isolated environment. For each case, record both the answer and the authoritative outcome in the connected system. If your provider cannot show tool-level evidence, ask what equivalent audit record the business can review and how long it is available.
Check how the system behaves after a refusal
Some demos show a series of refusals without proving that ordinary customer requests still work. Pair each exception with a legitimate request. For example, ask for a public maintenance service description, try a request for a restricted order record, then ask for the service description again. Review whether the assistant recovers without repeating the refusal for an unrelated question. Compare concise formal Arabic, a channel-appropriate colloquial variant and English when those are languages your team serves.
Assess handoff separately. A useful response tells the customer what happens next and gives the receiving team enough context to continue. When the assistant is uncertain whether a customer has authority, the queue item should carry the permitted request context without copying sensitive data that has no role in resolving the case. Ask who owns the queue when staff are offline and what the customer sees then. The human handoff guide covers routing and ownership in more depth.
Show where the connected system enforces access
Ask who owns the identity used by the integration. An assistant that answers public questions rarely needs a service identity with broad read and write access. Ask the provider to demonstrate which customer fields the integration can retrieve and whether create, update and delete privileges are separately controlled. Your business should be able to explain what is permitted even if the model generates an incorrect tool request.
A training provider, for example, might let the assistant share published course times and collect a request for a corporate quote. The commercial team owns bespoke pricing and the authority to send an offer. A maintenance operator might collect a description of a fault before routing the request, while an authorized employee verifies any attempt to change a job already assigned to a technician. These scenarios help scope permissions and demo acceptance. Supported configurations depend on the product, integration and agreed setup.
Agree on a release and recovery decision
Document failures in terms of customer impact. A disclosed record or unauthorized write should block the affected workflow from launch. An awkward refusal may require an editorial correction. A legitimate request that repeatedly disappears into a queue can be just as operationally important as an incorrect answer. Give each failure a named owner, a specific fix and a repeatable acceptance case.
Make the recovery plan visible to the team. Agree how to stop a risky action, which benign answers can continue, where unfinished requests go, and who reviews the records for affected customers. Use the smallest workable scope for the initial rollout, then rerun the relevant cases whenever a new data source, action or channel is introduced. The knowledge base readiness guide can help you assign owners and prepare approved material before those checks.
Buyer questions
Can a prompt alone enforce customer permissions? Ask how the connected application checks identity and authorization independently. A system can produce a convincing reply even when its back-end controls are too broad.
What affects the cost of the rollout? Scope the number of channels, protected data sources, supported actions, review environments and support ownership. Ask the vendor to separate implementation and ongoing review costs from subscription or usage fees.
What is the minimum evidence for sign-off? Keep a case ID, expected decision, observed response, connected-system result and owner. For a critical failure, keep the recovery decision and evidence that the corrected workflow passes the original case and the adjacent legitimate request.
Bring examples of ordinary questions, verified lookups and actions that require employee review to a Solvarex demo. Use the AI Employee & Chatbot product page to discuss the supported channels and agree what the proposed integration should demonstrate for your customer journey.
