An AI assistant you cannot inspect is an AI assistant you cannot trust with customers.
When someone reports a strange answer, the only useful question is "what actually happened", and answering it requires the conversation, the sources, and the reasoning in one place.
The list
Conversations with the essentials on each row: when it started, who it was with, how many messages, the current state, what it cost, whether anything was blocked, and whether it turned into a ticket.
Filters that reflect what you actually go looking for:
- A date range, for "last Tuesday afternoon".
- Only conversations with blocks, which is the shortcut to every conversation where the assistant refused something. This is the list to skim weekly.
- Anonymous or logged-in, since they behave differently and it is often one or the other you care about.
- A search across what customers typed, so you can find the conversation about a specific product.
The conversation
Open one and you get the thread as it happened, customer on one side and assistant on the other, plus a summary of the customer, when it started, how long it ran, total cost, and any linked ticket.
Under each assistant reply, the working:
- Which sources it used, as clickable links resolved to real page titles.
- How it got there: whether the first search succeeded, whether it had to try alternative phrasings, whether it asked a clarifying question, or whether the answer came from cache.
- Tokens and cost for that specific message.
- Why it was blocked, where it was, named plainly.
That last field is where the value concentrates. An answer rejected for citing a source it was not given tells you something quite different from one rejected as an out-of-scope question, and the log distinguishes them.
What you do with it
Diagnose a complaint. A customer says they were told the wrong thing. You find the conversation, read what was said, and see which page said it. Usually the fix is a page edit.
Spot content problems in bulk. Filter to conversations with blocks and patterns appear fast. The same subject refused eight times is a page you have not written.
Understand your spend. Cost per message, so an expensive week has an explanation rather than a mystery.
Privacy, deliberately
Anonymous visitors are shown as anonymous, with the last part of their IP address masked. Enough to tell two simultaneous conversations apart, not enough to build a profile.
Conversations can be deleted individually or in bulk, and deleting one takes its messages with it.