Knowledge Hub/Ebook
Ebook

# AI Chat Without the Risk: A Public Sector Leader's Guide to Grounded Conversational AI

Hallucinations. Data leakage. Compliance failures. Here's the practical framework for deploying conversational AI in government and education environments — safely, responsibly, and in a way that builds rather than erodes public trust.

22 min read
Government & SLED March 2025 Download Ebook

There is a version of AI chat deployment in government that ends in a front-page headline. An AI chatbot tells a citizen their benefits have been discontinued when they haven't. A school district chatbot references a policy that was repealed two years ago. A city council AI assistant invents a planning regulation that doesn't exist, and a resident relies on it for a development application. These are not hypothetical scenarios. They have happened, and they will happen again — to organisations that deploy AI chat without understanding the fundamental architecture that separates safe from unsafe.

## The Opportunity — and the Concern

Conversational AI represents a genuine step-change in citizen self-service capability. Not the 'press 1 for billing' decision-tree chatbot that government websites have been deploying since 2015 — but AI that actually understands natural language, handles follow-up questions, asks clarifying questions, and synthesises answers from multiple information sources. The potential to reduce contact centre volume, improve citizen satisfaction, and extend government service availability beyond business hours is real and substantial.

The concern is equally real. Standard consumer-grade AI chat systems — including many products being marketed to governments as 'AI assistants' — use large language models (LLMs) that were pretrained on vast quantities of internet text. These models have extensive general knowledge, which makes them useful for many tasks, but that general knowledge is a liability in government contexts. When a citizen asks about their eligibility for a specific state programme, the correct answer is the answer in your policy documentation — not the closest plausible answer the model can construct from its training data.

> "The risk is not that AI chat will give citizens a wrong answer because it's not smart enough. The risk is that it will give a wrong answer because it's too confident — constructing a plausible, fluent, authoritative-sounding response from general knowledge that contradicts your actual policy."

> — Keyspider Government AI Practice, 2025

## Why Hallucinations Happen (And Why Standard AI Chat Is Unsuitable for Government)

Large language models generate text by predicting what words are most likely to follow the words that came before them, based on patterns learned from their training data. They do not retrieve information from a database and report it. They generate text that is statistically likely to be correct — which is very different from text that is verified to be correct.

This architecture means LLMs have no reliable mechanism for saying 'I don't know'. When asked a question they cannot confidently answer from training data, they do not return an error. They generate the most plausible-sounding response the model can construct — even if that response is factually incorrect, outdated, or entirely fabricated. This phenomenon is called hallucination, and it is not a bug that will be fixed in the next model version. It is an inherent property of the statistical text generation architecture.

For a consumer AI assistant helping someone write an email or brainstorm ideas, occasional hallucinations are a manageable nuisance. For a government AI system providing citizens with information about their legal rights, benefit eligibility, application deadlines, or regulatory requirements, hallucinations are a governance failure. They create false reliance, potential legal liability, and — when they become public — exactly the kind of trust-destroying headline that sets back AI adoption in the sector by years.

## Grounded AI: The Architecture That Makes Government AI Chat Safe

### What grounding means

A grounded AI chat system is one in which the language model is architecturally constrained to generate answers only from documents retrieved from a defined, controlled content index — not from its pretrained world knowledge. The technical pattern that implements this constraint is called Retrieval-Augmented Generation (RAG).

In a RAG architecture, when a citizen asks a question, the system first performs a semantic search against your indexed content to find the most relevant documents. Those documents — and only those documents — are then provided to the language model as context. The language model's task is then to synthesise an answer from those specific documents, not to answer from general knowledge. The model cannot 'reach beyond' the retrieved documents to its pretrained knowledge base.

How RAG works — a plain-language explainer

Think of a grounded AI system as a very sophisticated research assistant who is only allowed to answer questions using the documents you hand them. They cannot consult any outside sources, rely on their own background knowledge, or make inferences beyond what the documents say. If the answer isn't in your documents, they say so. Every answer they give includes a citation to the exact document it came from. This is the model for safe AI chat in government contexts.

### Citations: the accountability mechanism

In a properly implemented grounded AI chat system, every answer includes a citation identifying the source document from which the answer was derived. This serves three critical functions for government deployments. First, it allows citizens to verify the answer against the source — and if the source is wrong, the error is traceable and correctable. Second, it provides an audit trail for the AI system's responses, which is essential for FOIA compliance (AI-generated responses that a citizen relied upon are likely discoverable under Open Records laws). Third, it builds citizen trust — a cited answer signals that the system is drawing from official documentation, not guessing.

## Five Non-Negotiable Requirements for Public Sector AI Chat

1. 1 Grounding in organisation content only — the AI must be architecturally constrained to retrieve answers exclusively from your indexed content. Verify this technically, not through vendor assurance. Ask to see the architecture diagram showing how retrieval is enforced before language model generation. Ask what happens when you query about something not in the index. The answer should be a 'no information found' response, not a general-knowledge answer.
2. 2 Citations on every answer — every AI-generated response must display the source document(s) from which it was derived, with a direct link to the source. This is not optional. It is a governance requirement for any public-facing AI system in a government or education context. Systems that generate answers without citations are not suitable for deployment regardless of how accurate their answers appear to be in demos.
3. 3 Role-based access controls — for deployments that serve multiple audiences (public citizens, authenticated residents, staff, specific programme participants), the AI must respect access boundaries. A staff-facing chat system should not surface confidential HR information to unauthenticated users. A student-facing chatbot should not surface faculty-only content. These controls must be enforced at the index level, not through prompt-based instructions.
4. 4 No training on your data — your organisation's content, citizen queries, and conversation logs must never be used to train or fine-tune the AI model. This requirement must be explicit in your data processing agreement. Many consumer-grade AI services use customer data to improve their models by default — this is disqualifying for government use due to potential exposure of sensitive citizen information and Open Records implications.
5. 5 Comprehensive audit logs — every conversation must be logged with timestamp, session identifier, query, AI response, and source citations. These logs must be retained for a period consistent with your organisation's records retention schedule, be producible in response to FOIA requests, and be accessible to authorised administrators for quality review. The absence of audit logs is disqualifying.

## Real-World Deployment Patterns

### Pattern 1: Public-facing citizen chat on government websites

The most common first deployment. A chat widget is embedded on a citizen-facing website, indexed against public programme information, FAQs, and service guides. The AI answers questions about service eligibility, application procedures, office locations, document requirements, and deadlines. The system is unauthenticated — it cannot see any citizen-specific data — and all answers are drawn exclusively from publicly published content.

Key considerations for this pattern: Ensure the search index is kept current with content updates (policy changes, form updates, office closures). Configure fallback behaviour for out-of-scope questions — the system should gracefully direct users to a contact channel when it cannot find an answer. Include a clear disclosure that the user is interacting with an AI system.

### Pattern 2: Authenticated staff intranet chat

A chat interface deployed on the staff intranet, indexed against internal policy manuals, HR documentation, legislative instruments, and operational procedures. Authenticated users see content appropriate to their role. HR staff can ask questions about enterprise agreement provisions; caseworkers can ask about eligibility rules; all staff can access general procedure guides.

Key considerations: Identity provider integration (Active Directory, Azure AD, Okta) is required for role-based content scoping. Pilot with a single department before broad rollout. Establish a process for content owners to flag AI responses that are based on outdated documentation.

### Pattern 3: Hybrid citizen portal (authenticated self-service)

The most sophisticated deployment pattern: a citizen portal where authenticated users (residents who have created accounts) can ask questions about their specific interactions with the agency — application status, payment history, upcoming appointments — combined with general programme information. This pattern requires integration between the AI chat system and transactional backend systems.

Key considerations: This pattern should only be attempted after successful deployment of the unauthenticated public-facing pattern. The integration between AI and transactional systems requires careful security architecture. Start with read-only integration (the AI can answer questions about a citizen's data, but cannot modify it). FOIA implications require specific legal review in this pattern.

## The FOIA Implications of AI Chat Logs

This is the legal consideration that most government AI chat deployments fail to address in advance. When a citizen interacts with a government AI chat system, those conversation logs are likely records of the public body under Open Records law — and may be subject to FOIA disclosure upon request.

Legal consideration — seek counsel

The following is not legal advice and does not substitute for review by your agency's legal counsel. Conversation logs between citizens and government AI systems may constitute public records subject to disclosure under FOIA or your state's equivalent Open Records statute. This is particularly likely if the AI provided information that a citizen relied upon in their dealings with the agency. Proactively design your AI chat deployment with a records retention policy that addresses: (1) how long conversation logs are retained, (2) what data is logged vs anonymised, (3) how logs will be retrieved and reviewed in response to a disclosure request, (4) whether chat logs fall under any applicable exemptions. Consult your agency's legal counsel and records officer before go-live.

## Vendor Evaluation Checklist for AI Chat

Use this checklist when evaluating AI chat vendors for public sector deployment

Grounding architecture is documented and verifiable

Ask for an architecture diagram showing how RAG is implemented. Verify that the LLM receives only retrieved documents as context, not pretrained world knowledge.

Source citations appear on every response

Verify in a live demo — query something specific and confirm the citation appears with a link to the source document.

Clear 'no information found' behaviour for out-of-scope queries

Test with questions outside your indexed content and verify the system declines rather than fabricates an answer.

No training on customer data — contractually guaranteed

Require explicit contractual prohibition in the data processing agreement, not just a vendor statement.

Role-based access control with index-level enforcement

Ask how audience segmentation works. Verify it is enforced at the index level, not through query-time filtering.

Comprehensive audit logging with export capability

Verify what is logged (query, response, citations, timestamp, session ID), how logs are retained, and how they can be exported.

WCAG 2.1 AA compliance for the chat interface

Require a third-party audit report or VPAT. Test keyboard navigation, screen reader compatibility, and colour contrast.

Data residency within your required jurisdiction

Confirm data storage region in the contract, not just in a verbal assurance.

Transparent AI disclosure to users

Verify the interface clearly discloses AI involvement — typically required under state AI transparency laws and best practice.

Fallback to human agent routing

Confirm the system can seamlessly route to a live agent when the AI cannot answer, with conversation context passed to the agent.

Real-time content index updates

Verify that policy updates on your website are reflected in the AI's answers within minutes, not hours.

Government reference clients

Ask for three references at government or education organisations with comparable scale and compliance requirements.

FOIA records management guidance

Ask how the vendor supports customers in managing chat logs as potential public records.

Hallucination rate measurement

Ask how the vendor measures and monitors for hallucinated responses in production. The answer reveals how seriously they take accuracy.

Deployment timeline with contractual commitments

Require a milestone-based deployment schedule with written commitments, not aspirational estimates.

## What Good Looks Like: A Sample Citizen Conversation

Here is an example of what a properly implemented grounded AI chat interaction looks like for a citizen enquiring about rental assistance eligibility. Note the citation, the precision of the answer, and the graceful handling of a follow-up question that goes beyond the available information.

> "Citizen: 'Am I eligible for emergency rental assistance if I lost my job last month?' AI: 'Based on the Emergency Rental Assistance Programme guidelines, you may be eligible if you meet all of the following: (1) Your household income is at or below 80% of the Area Median Income for your county; (2) You have experienced a financial hardship directly related to COVID-19 or a qualifying economic disruption; and (3) You are at risk of housing instability, including past-due rent. A recent job loss is considered a qualifying hardship. You will need to provide documentation including your most recent pay stubs, a termination letter or employer confirmation, and your current lease agreement. [Source: Emergency Rental Assistance Programme Eligibility Guide, Section 3.2 — last updated February 2025] Citizen: How much can I receive? AI: The maximum assistance amount depends on your household size and county median income level. The programme covers up to 12 months of past-due rent plus up to 3 months of future rent. For specific amounts applicable to your household size and county, please refer to the benefit calculator on the programme page, or contact your local housing office. [Source: Emergency Rental Assistance Programme — Benefit Amounts and Calculation, Section 4.1]'"

> — Example interaction demonstrating grounded AI with citations

This interaction demonstrates the key properties of safe government AI chat: precision (specific eligibility criteria, not vague generalities), citation (source document and section clearly identified), appropriate scope (the AI answers what it knows and directs to other resources for what it doesn't), and plain language (the answer is accessible to a citizen without policy expertise).

Related resources

Keyspider AI Assistant — product overview

How Keyspider AI Assistant works, what it grounds answers on, and how it deploys.

How to Stop Your AI Chatbot from Making Things Up

Technical and practical explainer on AI hallucinations and how to prevent them.

How a State Agency Deployed AI Assistant

Case study: 38% contact centre reduction, Phase 2 AI Assistant deployment.

The SLED Procurement Checklist

52 questions to ask every AI vendor before signing.

### See grounded AI chat in action on your own content

Book a demo with our public sector team. We'll demonstrate Keyspider AI Assistant configured on a sample of your actual website content — with citations on every answer and zero hallucinations.

Book a Demo

In this ebook

Ready to see it in action?

Book a 15-minute demo with our SLED team. No sales pressure, just a live look at what's possible for your organisation.

Book a Demo

Topics

Ebook AI Assistant Public Sector Hallucinations Grounded AI FOIA

## Ready to give your users better answers?

AI Search, AI Assistant, and Workplace Search. Deployed in days, not months. See it live on your own content.

Book a Demo Talk to Sales

No credit card required · Live in 2 weeks · Cancel anytime