Enterprise RBAC AI Execution Engine
Nexus Enterprise RAG Console
An enterprise AI knowledge base built around a problem that is rarely a modelling problem at all — institutional knowledge trapped in documents, and no way to let people query it safely. The system pairs role-based access control with a grounded retrieval pipeline, so the answer a user receives is drawn only from documents their clearance tier permits, and every answer carries its sources.
Technology
The problem
Knowledge inside an organisation is scattered across documents that ordinary people cannot safely be given. Existing AI assistants compound the problem: they answer confidently from whatever they can reach, they cannot cite their sources, they apply no notion of who is allowed to see what, and they give the business no visibility into what the tool is costing to run. The access question, not the model, is the actual product requirement.
Approach
- 01
Made access control part of retrieval rather than part of the interface. The caller's clearance tier filters candidate chunks server-side, so a restricted document is never even a candidate — it cannot leak through a prompt, a citation or a UI oversight.
- 02
Split ingestion from querying. Document processing is slow, I/O-heavy and failure-prone, so it runs on a Celery worker with Redis as the broker. The request path stays responsive and the UI polls status rather than blocking.
- 03
Grounded every answer. The pipeline embeds the question, retrieves by cosine distance in pgvector, then streams a response restricted to those chunks and emits explicit source references. When retrieval finds nothing, it should say so rather than invent.
- 04
Streamed the response instead of waiting for it. Server-sent events emit discrete stages — embed, retrieve, sources, token deltas, done — so the interface can show real progress and the first content appears before the model finishes.
- 05
Treated operational accountability as a feature. Every query's token estimate and cost is recorded, and an append-only activity log captures who did what, from where, and when.
- 06
Wrote real provisioning. Admins create accounts and the server generates the credential, delivering it out of band by email rather than returning a password in an API response.
Architecture
- 01
Document upload
An administrator uploads a document (50 MB ceiling, enforced on both client and server). The upload creates a record and returns immediately.
- 02
Async ingestion
A Celery worker extracts text with PyMuPDF, chunks it, generates embeddings and stores vectors with page references. Status is polled; failures are visible rather than silent.
- 03
Clearance-filtered retrieval
The question is embedded and matched against pgvector using cosine distance over an HNSW index — with a Python cosine fallback so the same code runs on SQLite for local work.
- 04
Grounded streaming answer
Only retrieved chunks are sent to the model. The response streams back with numbered source citations, and an empty retrieval produces an explicit no-answer rather than a guess.
- 05
Cost & audit accounting
Token estimates and duration are logged per query and rolled up for analytics, while an append-only activity feed records authentication, provisioning, searches and administration.
Access is enforced at retrieval time. Restricted content is never a candidate for the model, so it cannot leak through a prompt or a UI bug.
Key features
Challenges
Enforcing clearance inside a vector search
Similarity search does not understand permissions, and filtering after retrieval is a data leak with extra steps. The filter has to be part of the query, which meant reasoning carefully about how the tier constraint is applied before distances are ranked rather than afterwards.
Making asynchronous ingestion trustworthy
Background work is easy to launch and easy to lose. Each document carries explicit status, the interface polls it, and failures surface as a state an operator can see instead of a document that silently never became searchable.
Provisions credentials without leaking them
User creation normally returns the password in the response body, which means it ends up in logs, browser history and network traces. The server generates the credential and delivers it out of band, so the secret never travels back to the caller.
Protecting the accounts that administer everything
Destructive endpoints are the ones that matter. Administrator accounts are guarded from blocking or deletion in both the API and the interface, and destructive operations require explicit typed confirmation.
What I learned
Retrieval-augmented generation inherits every access-control problem the underlying store already has, and then adds a new one — the model can say anything. Pushing permissions into the retrieval query, rather than filtering results afterwards, is the difference between a knowledge base that is safe to expose to an organisation and one that is not.
Scope & current state
Being precise about scope
- This was built as an independent engineering project against a realistic enterprise brief. It is not a production deployment for a paying client.
- The repository ships both a Django/DRF API and a Vite + React SPA, and currently includes a SQLite fallback path (USE_SQLITE=1) for local work alongside the primary PostgreSQL + pgvector store.
- Some frontend views in the current repository state are populated from local mock data modules rather than live API responses.