Retrieval quality is the whole game
Model quality plateaus quickly; retrieval quality does not. Most disappointing knowledge assistants are retrieving the wrong three passages, and no amount of prompt engineering recovers from that.
Chunking is domain-specific
Fixed-size chunking destroys tables, policy clauses and procedural steps. Chunk along the document's own structure, keep headings with their content, and store enough metadata to filter before you rank.
Stale documents quietly lie
A production system needs an ingestion pipeline with change detection and deletion, not a one-off import. The most dangerous answer is a confident one sourced from a superseded policy.
Permission leakage is the real risk
Access control has to be enforced at retrieval time against the requesting user, not applied to the generated answer afterwards. Design for that on day one; retrofitting it is a rewrite.