RAG in production: a document assistant at INSEI

What it is

A document assistant answering staff questions about internal procedures, contacts and resources, in service across two sites.

What was hard

Nothing leaves the institution and nothing is paid for. Solutions relying on an external service were ruled out from the start.

What it proves

A reference question set and an automated judge, put in place before the optimisations: that is what makes it possible to say a change improved anything.

INSEI is a French public higher-education institution based in Suresnes. It trains professionals working in inclusive education and carries out the associated research: schooling for pupils with disabilities, accessibility, pedagogical adaptation. It teaches students, trains practitioners already in post, and publishes resources for the field.

My apprenticeship there runs from April 2023 to September 2026. The project described here was carried out by two people.

Answers that already exist, impossible to find

Staff look for information that already exists. Internal procedures, contacts, HR resources: it is all on the intranet, and all hard to find again. What was asked for is an assistant that answers in natural language, citing its sources.

A prototype that gives rough answers is quick to build from documented building blocks. What follows is what separates it from a system an institution agrees to operate: hosting constraints, source content that is not clean, and a way of saying whether one version is better than the last.

Nothing leaves, nothing is paid for

Nothing leaves the institution. Language model, embedding model, vector store, observability, logging: everything runs on internal servers. This was not a preference but a condition of acceptance, and it rules out from the start any solution relying on a service hosted elsewhere.

Zero budget. No licence, no paid service, no per-request billing.

The source content is not clean. An intranet is made of heterogeneous pages, directory profiles and attachments, with a self-signed certificate and a page structure that varies from one content type to the next.

How the assistant is built

Document assistant architecture

Adding a site means writing one file

1. At ingestion, once

2. Sites

   pages, profiles, attachments

3. Adapters

   one file per site

   The CMS declares where to read

   each page's useful region, supplied by the source

4. Generic engine

   section chunking, embedding

5. Vector store

   the meaning of passages

   Keyword index

   the exact terms

6. On every question asked

7. Question

   asked in a chat interface

8. Routing

   directory or content, two indexes and two weightings

9. Hybrid retrieval

   keywords and meaning combined

   excerpts capped per page

   Reranking

   implemented, off by default, never measured

10. Language model, hosted on premises

    writes the answer and cites its sources

    Cache and observability

    answers cached, requests traced

11. Answer

    with the pages it came from

Three layers, separated for a specific reason: what changes from one site to the next must not contaminate what does not.

An adapter layer. An abstract contract defines what a site must supply: how to authenticate, how to turn a page into indexable documents, which system prompt, which collection. Each site is a file fulfilling that contract, and the API discovers them at start-up. Adding a site means writing one file.

A generic engine. Ingestion, chunking, embedding, retrieval: none of this code knows any particular site. It knows there are person profiles and content pages, and it falls back cleanly on pages when a site has no directory.

An API compatible with the domain’s standard, so it plugs into an existing chat interface rather than requiring one to be written. A bespoke interface did exist: it is no longer in the repository, and the documentation still describes it.

14blocksproductionThe assistant is not one program but fourteen: the store holding the documents, the AI model doing the writing, the cache speeding things up, the monitoring keeping watch, and everything wiring them together. Each runs in its own box, on the institution's servers. make up the deployed stack: vector store, language model, cache, observability, container metrics, proxy, API.

On the institution’s intranet, the index holds on the order of 300profiles, roughlyproductionContact profiles indexed: 300 profiles, roughly (production). and 700pages, roughlyproductionPages and documents indexed: 700 pages, roughly (production).. These are two different kinds of content, and the pipeline handles them through two distinct routes, which explains what follows.

reproducible artefact

Idempotent, incremental ingestion: reindexing duplicates nothing, and pages deleted at the source disappear from the index.

The decisions that mattered

The CMS tells the search engine where to look

This decision was not made in the code, and that is what makes it interesting.

Extracting the useful content of a web page requires knowing which region to read. The reflex solution is to write selectors in the engine’s code, which is what was done first. It has a defect that only shows up over time: every template change on the CMS side silently breaks ingestion, and the person making the change has no reason to know another system depends on it.

The chosen approach moves the responsibility: the CMS itself publishes, for each page, where its content region sits, and the ingestion engine consumes that. This required a change on the CMS side, and therefore a negotiation with the team that owns it, and a convention to be upheld on both sides. In exchange, changing a template no longer breaks anything downstream, and the engine stays ignorant of the sites it serves: plugging in a source means writing an adapter, not touching the engine.

Measure before optimising

A set of 88questionstest setQuestions in the evaluation set: 88 questions (test set). was built and versioned, each carrying its expected answer and the source where that answer should be found, including 3questionstest setNegative controls in the evaluation set: 3 questions (test set). that check the system says it does not know rather than making an answer up. An automated judge reads the retrieved context and rules in binary terms: either the expected answer is in it, or it is not.

That harness, put in place before the optimisations, is what makes the rest legible. Without it, comparing two pipeline versions means comparing two impressions.

Hybrid retrieval was born of a failure

Purely semantic retrieval confused neighbouring concepts: two documents with similar titles and entirely unrelated content. Combining it with keyword search fixed the problem, with different weightings depending on whether the query seeks a person or a procedure. It is not a theoretical refinement, it is the answer to an observed failure, and the harness said whether it worked.

A memory with three windows

The first behaviour kept the whole history, which made the topic drift as soon as a user moved on to a new question. The current rule distinguishes three cases: a question containing a pronoun is a follow-up and sees a wide history, a self-contained question opens a new topic and sees almost none, and query rewriting works on an intermediate window.

documented trade-off

Hybrid lexical and semantic retrieval adopted after a purely semantic approach failed, with different weightings depending on whether the query seeks a person or a procedure.

constraint met

Processing entirely on the institution’s own servers, no external service in the stack, zero budget.

What the harness measured

What moving to hybrid retrieval changed

Over the reference set, the share of questions whose retrieved context actually held the answer went from 27%test setQuestions whose retrieved context held the answer, at the start: 27 % (test set). to 86%test setThe assistant finds the right document nine times out of ten, compared with one time in four at the start of the project.. In between: the move from purely semantic to hybrid retrieval, and a change of embedding model.

That is the project’s result, and it is also its method: the number only means something because the harness existed before the changes it measures.

One caveat comes with it. Both runs covered the question set versioned in the repository, and the second dates from early June 2026, but neither run’s output was kept: the numbers are recorded in the project’s architecture document, not in replayable results files.

One independent measurement, on what was measurable outside the network

The pipeline routes each question to one of two paths, directory or content, with distinct weightings, filters and indexes. That routing is a pure text function, querying neither the vector store nor a model: it is therefore measurable outside the institution’s network, unlike everything else. It sends 96.47% of questionstest setRouting accuracy: 96.47 % of questions (test set, n = 85). the right way.

The expected route is derived by me from the source recorded in the set. It is an explicit assumption, not a ground truth the project produced.

Two nuances the documentation blurs

The reranker is implemented but not enabled by default: the honest wording is “available”, not “the pipeline reranks”. And user identification in traces is attribution, useful for knowing who asked what; it is not authentication, and this page does not present it as such.

What these figures do not say

There is no floor. The measured progression says hybrid beats semantic alone. It does not say what keyword search alone would give, at a far lower cost. Without that third point, we know hybrid fixed a symptom, not what it actually contributes.

The reranker has never been measured. It is disabled by default, which is a defensible cost decision, but one made without a number to back it.

There is no long-term memory, history is lost on every reload. The interface reaches the first level of accessibility, not the second. And no business-gain indicator was ever established: we do not know how much time the assistant saves, because nobody measured how long it took before.

If I started over

I would distrust heuristics that work. Measuring the routing surfaced a defect in my own code: the rule recognising a directory question fires, among other things, on two consecutive capitalised words, a pattern meant to catch a first name and surname, which in practice catches any pair of capitalised words. The defect stayed invisible as long as nobody counted, because a misrouted question still returns something plausible. The measurement gave me a bug rather than a score.

I would keep documentation in step with the code, or not at all. The architecture document still describes an interface that no longer exists. Documentation that lags is worse than none, because it is believed: whoever picks the project up will look for a file that is gone and lose confidence in the rest. On a project explicitly built to be handed over, that is the costliest inconsistency, and the only one nobody reports, because it breaks no test.

I would measure usage before building. The business-gain indicator should have been established at the start, while it was still possible to time a request handled the old way. Once the assistant was in place, the baseline was gone.