Skip to content
iam.alan.abreu
← Lab
BUILDINGDraftStarted 2025-03

Grounded retrieval for runbooks

Can retrieval over internal runbooks answer on-call questions without inventing steps that do not exist?

Domains
  • AI
  • Reliability
Technologies
  • pgvector
  • PostgreSQL
  • TypeScript
  • Embeddings

Hypothesis

Retrieval quality, not model size, is the limiting factor for useful operational answers.

Objective

Answer on-call questions strictly from indexed runbooks, with citations and an explicit no-answer state.

Architecture

Documents are chunked by heading, embedded and stored with their source anchor. Answers must cite at least one retrieved chunk or refuse.

Implementation

Placeholder — implementation notes will be published as the experiment progresses.

Findings

Placeholder — findings pending.

Next steps

Build an evaluation set that specifically tests refusal behaviour.

Linked from

Related suggestions