RAGSpine
Reference

Extension points

Typed seams for providers, stores, retrieval, chunking, routing, workflow matching, connectors, and service edges.

RAGSpine's pluggability is not a plugin registry — it is plain structural typing. Each external dependency is a Python Protocol; the core depends on the abstraction, never on a vendor SDK. Adding a provider, vector store, reranker, or OCR engine touches one new file, and every heavy SDK is lazy-imported so the core imports cleanly and runs fully offline with the deterministic MockProvider.

All seams below are @runtime_checkable Protocols — your implementation does not need to subclass anything, it only needs the right method signatures. mypy --strict covers the core, and the anthropic / openai / sentence-transformers / paddleocr SDKs are only imported inside the concrete implementations, never in the seam.

The seams

LLMProvider

corespine (re-exported via agent/llm_provider.py) — chat(messages, tools) returns a ChatCompletion; OpenAI chat-completions shape.

EmbeddingBackend

retrieval/lexical/retrieval.py — batch texts to vectors for the injectable vector channel.

ListwiseJudge

retrieval/rerank/listwise_rerank.py — listwise rerank: return candidate indices best-first.

OcrBackend

extraction/extractors/pdf_scanned_extractor.py — recognize a single scanned page image.

NarrativeRetriever

agent/agent.py — the narrative retrieval seam injected into the orchestrator.

TaskQueue

service/tasks/task_queue.py — async job queue (FakeQueue in tests, RQQueue in prod).

SourceConnector

ingestion/source/connector.py — iter_documents() yields RawDoc from a knowledge-base source (filesystem, HTTP, Notion, in-memory).

RelationExtractor

graph/extractor.py — extract(chunks) returns graph edges; the opt-in narrative-relation slot beside build_relation_graph.

Chunker

retrieval/chunking/chunker.py — text plus DocumentMeta to lineage-preserving chunks; built-ins and third-party entry points.

LibraryRouter / FilterExtractor

retrieval/routing and filtering — deterministic library selection and optional structured metadata-filter extraction.

TemplateMatcher

workflows/matching.py — rank metadata-only runnable template references for natural-language scaffolding.

Current retrieval and workflow seams

Chunker.chunk(text, meta, *, max_chars, overlap_chars) -> list[Chunk] owns chunk strategy. make_chunker supports deterministic default, layout, parent-child, sentence-window, laws, QA, book, and semantic implementations plus the ragspine.chunkers entry-point group. Implementations must preserve doc_id and source_locator; hierarchy/window data is additive, not a substitute for lineage.

LibraryRouter selects one or more RoutableLibrary IDs. MultiIndexRetriever invokes the selected NarrativeRetrievers independently and RRF-fuses their results; a router failure falls back to all libraries. FilterExtractor can propose metadata predicates, but filters remain validated narrowing operations.

TemplateMatcher receives catalog references containing match metadata, not workflow prompts or credentials. The built-in lexical and ONNX matchers return ranked candidates; scaffold policy applies the score/margin threshold and validates the selected template. A custom matcher does not gain file-write or execution authority.

LLMProvider

Owned by corespine and re-exported from src/ragspine/agent/llm_provider.py — the single method the agent's tool-use loop drives. It speaks the OpenAI chat-completions shape (one chat method). AnthropicProvider lazy-imports the anthropic SDK and maps the shape onto the Anthropic API; MockProvider needs neither key nor network.

Prop

Type

0.3.0 migration. This replaced the old create_message(*, system, messages, tools) -> ProviderResponse. The seam now lives in the shared family core corespine so ragspine and its sibling packages share one provider contract — see ADR 0012.

EmbeddingBackend

src/ragspine/retrieval/lexical/retrieval.py — the dependency-injection point for the vector channel. The default is none (pure BM25); inject this to add a vector channel.

Prop

Type

ListwiseJudge

src/ragspine/retrieval/rerank/listwise_rerank.py — the optional LLM listwise reranker seam. The real implementation is Claude (via build_listwise_prompt / parse_listwise_response); tests use a deterministic fake. Falls back to RRF order if absent.

Prop

Type

OcrBackend

src/ragspine/extraction/extractors/pdf_scanned_extractor.py — the scanned-PDF OCR/VLM seam. The real backend (PaddleOCR) runs on Ubuntu + GPU; logic tests use a fake so the render → map → threshold → review flow is fully testable on a GPU-less machine.

Prop

Type

NarrativeRetriever

src/ragspine/agent/agent.py — the narrative retrieval implementation injected into answer_question. Duck-typed; when omitted, the narrative path degrades honestly.

Prop

Type

TaskQueue

src/ragspine/service/tasks/task_queue.py — the async job queue. RQQueue (RQ + Redis) for production, FakeQueue (synchronous inline) for tests. rq / redis are lazy-imported inside RQQueue only.

Prop

Type

SourceConnector

src/ragspine/ingestion/source/connector.py — the knowledge-base source seam. A connector yields raw documents from wherever your KB lives; the default FilesystemConnector needs no extra, and the HTTP / Notion connectors lazy-import httpx behind the [connectors] extra.

Prop

Type

Built-ins register by name in a module-level table; third-party connectors are discovered through the ragspine.source_connectors entry-point group (built-in names win on a clash). Select one with the make_source_connector(spec=None, **kwargs) factory or the RAGSPINE_SOURCE_CONNECTOR env var (none / NoneNone):

Prop

Type

RelationExtractor

src/ragspine/graph/extractor.py — the opt-in narrative-relation slot consumed by build_relation_graph(..., relation_extractor=None). With the default None, the base graph is byte-identical (no extra edges). See ADR 0015.

Prop

Type

Select via make_relation_extractor(spec=None, *, provider=None, profile=None, **kwargs) or RAGSPINE_RELATION_EXTRACTOR (noneNone; deterministic / rule / cooccurrence → the deterministic default; llm / on → the LLM extractor, honestly degrading to None when no provider is available).

A model-extracted edge is useful but untrusted. LLMRelationExtractor writes the derived=model-derived and verified=unverified markers into every edge's metadata, takes lineage from the chunk (the caller) rather than the model's self-report, and drops any edge whose endpoint the deterministic SecurityGate refuses. A model-asserted relation can never read as a controlled, verified fact.

Implement a Protocol and inject it

Because the seams are structural, you implement the method(s) and pass the instance in — no registration, no base class. Here is an OpenAI-backed LLMProvider, grounded in the real chat signature and the ChatCompletion shape (the dataclasses come from corespine and are re-exported from ragspine.agent.llm_provider):

my_openai_provider.py
from typing import Any

from openai import OpenAI  # lazy: only your file imports the SDK
from corespine import ChatCompletion, Choice, ResponseMessage, Usage


class OpenAIProvider:
    """A custom LLMProvider — no subclassing, just the chat method."""

    def __init__(self, model: str = "gpt-4o") -> None:
        self._client = OpenAI()
        self._model = model

    def chat(
        self,
        messages: list[dict[str, Any]],
        *,
        tools: list[dict[str, Any]] | None = None,
    ) -> ChatCompletion:
        # The messages are already in the OpenAI chat-completions shape, so they pass
        # through; the SDK response is the same shape, so the mapping back is direct.
        resp = self._client.chat.completions.create(
            model=self._model,
            messages=messages,
            tools=tools or [],  # adapt / omit as needed
        )
        choice = resp.choices[0]
        message = ResponseMessage(role="assistant", content=choice.message.content)
        u = resp.usage
        usage = (
            Usage(
                prompt_tokens=u.prompt_tokens,
                completion_tokens=u.completion_tokens,
                total_tokens=u.total_tokens,
            )
            if u is not None
            else None
        )
        return ChatCompletion(
            choices=(Choice(index=0, message=message, finish_reason=choice.finish_reason or "stop"),),
            usage=usage,
            model=resp.model,
            id=resp.id,
        )

Inject it exactly where MockProvider would go:

from ragspine.agent.agent import answer_question
from ragspine.storage.fact_store import SqliteFactStore
from my_openai_provider import OpenAIProvider

store = SqliteFactStore("data/fact_metric.db")
store.init_schema()
result = answer_question("...", store, OpenAIProvider())

The structured channel's anti-fabrication guard does not trust provider prose for the number — a found fact is deterministically rendered from the fact value, and a no-fact result is rewritten to "not found" regardless of model output. Swapping the provider cannot defeat the guard.

On this page