SAMI — iSAMS Assistant
A retrieval-augmented AI assistant that answers practical "how do I…?" questions about iSAMS school management software, grounded in a curated knowledge base rather than general model knowledge.
Context
iSAMS is school management software with a large surface area — admissions, attendance, timetabling, finance, pastoral care, and reporting. Answering everyday “how do I…?” questions about it usually means raising a support ticket or relying on a colleague’s memory.
SAMI is a free, community-run assistant that answers those questions in seconds, day or night, publicly available at sami.ethancorp.com.
Challenge
A general-purpose chatbot is the wrong tool here: confident but wrong answers about a system of record are worse than no answer at all. The assistant had to stay grounded in how iSAMS actually works, admit when it does not know, and improve over time without a full retraining cycle — while remaining cheap enough to run as a free community service.
Approach
Built as a retrieval-augmented system rather than a model wrapper. A curated knowledge base of 17 in-depth iSAMS guides is chunked into approximately 6,400 embedded vectors; each question retrieves the top five matches above a 0.5 similarity floor, and the system prompt requires the model to answer strictly from retrieved material and say it is not sure rather than fill gaps from general knowledge. Contextual query rewriting lets follow-up questions retrieve correctly without the user repeating themselves.
Around that core: streaming chat over WebSocket with auto-titled, stateful sessions; thumbs-up/down feedback on every answer feeding an admin review queue, where approved feedback is ingested back into the knowledge base by an automated daily job; and an admin console covering usage and cost trends, user management, and direct knowledge-base editing.
Quality is gated rather than assumed — a 20-question evaluation set including deliberate trick questions is graded by an LLM judge before prompt or model changes ship, alongside 218 automated tests run with lint on every push.
Impact
- Retrieval-augmented generation over a curated knowledge base of 17 iSAMS guides
- Approximately 6,400 embedded vectors, retrieved top-5 with a 0.5 similarity floor
- Contextual query rewriting so follow-up questions retrieve correctly
- Streaming WebSocket chat with auto-titled, stateful conversations
- Feedback loop with admin review queue and an automated daily knowledge-base ingest
- Admin console for usage and cost tracking, user management, and knowledge-base editing
- 20-question evaluation set with LLM-as-judge grading as a quality gate
- 218 automated tests (151 backend, 67 frontend) with lint and test run in CI
Stack
Next.js 16 (App Router), React 19, and TypeScript on the front end; Python 3.11, FastAPI, and async SQLAlchemy on the back end; PostgreSQL for application data and Qdrant for vector search; any OpenAI-compatible API for chat and embeddings; orchestrated with Docker Compose and tested in GitHub Actions.
Capabilities Demonstrated
- Retrieval-augmented generation
- Vector search
- LLM application architecture
- FastAPI and Next.js delivery
- Evaluation and quality gates
- Human-in-the-loop feedback
