Changelog

This page tracks user-facing changes per release: API shapes, configuration surfaces, packaging, and dependencies. For the reasoning behind a change, follow the linked ADR. For symbol-level detail, run make docs against the tagged source.

0.11.0 (current source snapshot)

Natural-language workflow scaffolding and a safe website preview contract. This work is present on main; publication status is deliberately not inferred from source metadata.

Added

• The installed shorthand ragspine "description" and explicit ragspine workflow create command, with exact-template selection, reuse opt-out, lexical/ONNX/auto matching, JSON/YAML output, and safe cross-platform file writes.
• A 1,000-item bundled catalog: 7 manually curated workflows and 993 generated descriptors across exactly 39 industries, 27 use cases, and five archetypes. Dify/n8n marketplace pages may be factual research references, but upstream workflow bodies, prompts, and credentials are neither copied nor downloaded at runtime.
• JSON, YAML, and TOML workflow ingestion normalized to a JSON-compatible mapping, with duplicate key, size, depth, structural-node, string, YAML-alias, symlink, and special-file guards.
• Workflow preview schema v1: deterministic redacted nodes/edges/geometry for rendering without prompts, variables, provider configuration, credentials, or executable configuration.
• Read-only catalog/detail/scaffold HTTP endpoints, page caching/ETags, and matcher fallback.

Security behavior

• Scaffolding and preview never execute a workflow. Existing Dify/n8n run surfaces remain behind RAGSPINE_DIFY_RUN_ENABLED=false by default.
• The service bounds selected workflow request bodies before buffering and strips echoed validation inputs/context from 422 responses.

0.10.0

Retrieval product presets, history semantics, and persisted small-to-big expansion.

Added

• Metadata filters with eq, ne, in, nin, gt, gte, lt, lte, and between, plus an optional automatic extraction seam. Filters only narrow and preserve order.
• Multi-library routing and MultiIndexRetriever: per-library retrieval, RRF fusion, library_id provenance, and an all-library fallback when routing fails.
• Economy retrieval mode (economy / bm25 / lexical) that constructs no embedding backend or vector store.
• Parent-child/small-to-big chunking plus store-level parent_id, heading, window_text, and parent_locator persistence/migration. The child remains the citation; expanded prompt_text is generation-only, and restricted children cannot leak through their windows.
• answer_question(..., history=...) and HTTP ask history. Prior turns are generation context only, never intent/security/retrieval input or evidence. See ADR 0017 and ADR 0018.

0.9.0

Source connectors, relation-graph provenance, domain chunking & streaming. This release opens three more opt-in seams, a streaming answer endpoint, .txt ingestion, and two ergonomics helpers — all default-off and byte-identical when unselected.

Added

• KB source-connector seam. A SourceConnector Protocol (iter_documents() -> Iterable[RawDoc], make_source_connector / RAGSPINE_SOURCE_CONNECTOR) makes the ingestion source pluggable: filesystem (the zero-dependency default), in_memory, http, and notion. Third-party connectors register through the ragspine.source_connectors entry-point group. See Ingestion.
• Opt-in RelationExtractor slot with provenance markers. A RelationExtractor Protocol beside build_relation_graph (make_relation_extractor / RAGSPINE_RELATION_EXTRACTOR). The default None leaves the base graph byte-identical; the deterministic co-occurrence extractor adds clean-lineage co_occurs_with edges; the opt-in LLM extractor (behind [llm]) stamps every edge model-derived + unverified, screens both endpoints through the SecurityGate, and never lets RESTRICTED text reach the model. See ADR 0015.
• Domain chunk presets — laws / qa / book. Three layout-aware RAGSPINE_CHUNKER presets that only tune heading detection per document family (clause / chapter / question-answer pairing), inheriting the base budget, parent_id, and locators. See Retrieval.
• Streaming ask endpoint — POST /v1/ask/stream (SSE). The same answer as /v1/ask, streamed token-by-token. The full guard chain (FAQ short-circuit, dual-channel retrieval, anti-fabrication rewrite, provenance) runs to completion before the first byte is streamed, so a fabrication can never be half-streamed and then retracted. See HTTP API.
• .txt narrative ingestion. Plain-text files join the narrative path (extract_txt_narrative): UTF-8, blank-line paragraph splitting, one segment per block with a para={N} locator — zero-dependency and deterministic. See Ingestion.
• Ergonomics helpers. AgentResult.answer_plain (the answer with the inline （来源：…） suffix stripped, for UIs that render citations from sources), the keyword-only Fact.metric(...) builder (order-independent construction over the frozen 10-tuple), and SqliteFactStore.has_source_doc(source_doc_id) (an existence probe for incremental refresh). See Python API.

Dependencies

• One new optional, permissive-license, lazy-imported extra: [connectors] (httpx, needed only by the http / notion connectors). No new non-optional runtime dependency; the offline default path is unchanged.

0.8.1

Extension-seam hardening. Three governance seams from the breadth backlog are formalized so third-party backends can plug in through the invariant conformance packs, never around them. Pure structural work — every default path stays byte-identical, and the anti-fabrication ratchet holds at zero.

Added

• TraceSink seam + privacy-trace conformance. The privacy-aware trace sink is a registrable seam (make_trace_sink / RAGSPINE_TRACE_SINK), so observability can fan out to OpenTelemetry / files through a conformance pack that rejects or scrubs any payload carrying an answer / fact value / chunk text. A real OtelTraceSink ships behind the optional [otel] extra; a lineage-leaking sink fails the pack.
• FactStore Protocol. The structured store is now a @runtime_checkable Protocol with the stdlib-sqlite SqliteFactStore as the default (make_fact_store / RAGSPINE_FACT_STORE), opening DuckDB / Postgres adapters as a follow-up. Its conformance pack binds the two storage-side invariants — found-determinism (a miss returns empty, never a fabricated value) and provenance survival (source_doc_id + locator persist through upsert/query) — with reverse-proof stubs that must fail.
• Extractor provenance conformance pack. Every registered extractor (xlsx · pptx · pdf · docx · pptx-rich) is asserted over real fixtures to carry non-null source_doc_id + cell locators; a lineage-dropping stub fails on both dimensions.

Dependencies

• One new optional, permissive-license, lazy-imported extra: [otel] (OpenTelemetry, Apache-2.0). No new non-optional runtime dependency.

0.8.0

The second competitor-parity batch (workstreams W8–W12), benchmarked against the mainstream stacks (LlamaIndex · LangChain / LangGraph · Haystack · RAGFlow · Weaviate · Vespa · Jina · Cohere). Like 0.7.0, every addition is opt-in and default-off: with nothing selected the offline deterministic loop is byte-identical, and each new backend inherits the RESTRICTED two-exit isolation and provenance conformance packs.

Added

• Post-retrieval postprocessor chain. A NodePostprocessor seam (make_postprocessor / RAGSPINE_POSTPROCESSOR, e.g. "mmr,lost_in_middle") that runs after rerank and before prompt assembly — MMR diversity de-dup, lost-in-the-middle reordering, and deterministic extractive context compression. All zero-model; the LLMLingua-2 / LLM compressor is a seam follow-up. Compression writes a separate prompt_text key, leaving text and every reference field byte-identical.
• LLM query transformation. HyDE, RAG-Fusion (reuses RRF), step-back prompting, and Adaptive-RAG routing on the existing QueryRewriter / IntentParser seam (make_query_transform / RAGSPINE_QUERY_TRANSFORM, RAGSPINE_ADAPTIVE). Each generated sub-query re-runs the deterministic security gate; HyDE's hypothetical document is a retrieval probe, never a citable fact.
• RAPTOR + chunking strategies. A RAPTOR multi-granularity tree (deterministic threshold clustering; is_synthesis cluster summaries that carry provenance and are never citable facts) plus sentence-window and semantic chunking on the Chunker seam (RAGSPINE_CHUNKER, make_raptor_* / RAGSPINE_RAPTOR*).
• ColBERT & SPLADE rerankers. Token-level late-interaction (MaxSim) and learned-sparse scoring on the existing ListwiseJudge seam via make_reranker (RAGSPINE_RERANKER=colbert|splade, [colbert] / [splade]). Shipped as rerankers; the heavy multi-vector / sparse retrieval indexes are a follow-up.
• ColPali visual-document retrieval. Page-as-image late interaction on a VisualEmbedder seam (make_visual_embedder / RAGSPINE_VISUAL_EMBEDDER, [colpali]), for chart- and figure-dense reports where OCR→text loses layout. Opt-in and GPU; the deterministic MaxSim orchestration and RESTRICTED-at-the-door isolation are complete, real GPU end-to-end is a follow-up.

Dependencies

• New optional, permissive-license, lazy-imported extras only: [colbert], [splade], [colpali] (all via fastembed, Apache-2.0). No new non-optional runtime dependency; the offline default path is unchanged.

0.7.0

The quality-depth batch (workstreams W1–W7): the default retrieval loop becomes genuinely semantic, gains an offline rerank brain, taps the Spine family's document stack, adds groundedness measurement, and grows a deterministic knowledge graph. Opt-in additions are default-off; the lean pure-BM25 path stays byte-identical.

Changed

• Dense-on-by-default when the extra is present. ServiceConfig.embedding now defaults to "auto": with [embed-onnx] installed the shipped loop is genuinely hybrid (BM25 + ONNX semantic → RRF) with no config; with no extra it resolves to None and the lean pure-BM25 path is byte-identical. RAGSPINE_EMBEDDING gains onnx / auto.

Added

• Real semantic embedding default (W1). OnnxEmbeddingBackend (via fastembed, paraphrase-multilingual-MiniLM-L12-v2, 384-dim multilingual, offline, deterministic) behind [embed-onnx], registered as onnx / auto in make_embedding_backend.
• Local cross-encoder reranker (W2). CrossEncoderReranker (fastembed ms-marco-MiniLM, offline, deterministic) on the ListwiseJudge seam via a new make_reranker factory (RAGSPINE_RERANKER, [rerank]) — the offline rerank brain; the LLM listwise judge stays the higher-cost option.
• Family document stack (W3). The default OcrBackend is now the family's offline OCR (pdfspine → ocrspine) and the scanned-PDF path is actually wired into ingestion (was enqueue-only); a new .docx Extractor (docspine, [doc]) and an opt-in .pptx Extractor (pptspine, [ppt]); docx / pptx cell fills and nested tables now flow into the StyledGrid IR.
• Contextual Retrieval + layout chunking (W4). A deterministic context header (title · entity · period · heading) injected into index text only (RAGSPINE_CONTEXTUAL), plus layout-aware and parent-child / small-to-big chunking on the Chunker seam (RAGSPINE_CHUNKER).
• Faithfulness / groundedness eval (W5). Two new ratcheted gates — claim-level faithfulness and free-text answer-accuracy — folded into the same baseline ratchet, with an offline deterministic entailment default. See Evaluation.
• Agentic depth, opt-in (W6). LLM query decomposition (RAGSPINE_QUERY_DECOMPOSE), corrective retrieval / CRAG grade→act loop (RAGSPINE_CORRECTIVE), and a bounded multi-turn conversation-memory skeleton — each re-asserts the security gate and isolation per sub-query / turn.
• GraphRAG (W7). A new graph/ domain: a deterministic structured relation graph over the controlled dimensions (subsidiary roll-up, peer comparison, derivation tracing — fully cited), a GraphStore Protocol with an in-process default plus a networkx adapter (RAGSPINE_GRAPH_STORE, [graph]), and a narrative-GraphRAG extraction / community skeleton (opt-in, behind [graph] + [llm]). See Channels.

Dependencies

• New optional, permissive-license, lazy-imported extras only: [embed-onnx], [rerank], [doc], [ppt], [graph]. The offline, zero-heavy-SDK default path is preserved.

0.3.0

The headline is an LLM provider seam migration onto the shared family core, plus an optional active rate limiter. See ADR 0012 for the full rationale.

Changed (breaking for custom providers)

• LLMProvider now lives in corespine and is re-exported from ragspine.agent.llm_provider. The interface moved from create_message(*, system, messages, tools) -> ProviderResponse to the OpenAI chat-completions shape:

  def chat(
      self,
      messages: list[dict[str, Any]],
      *,
      tools: list[dict[str, Any]] | None = None,
  ) -> ChatCompletion: ...

  The system prompt is now the first {"role": "system", ...} message rather than a separate argument, and ProviderResponse is replaced by corespine.ChatCompletion (choices → ResponseMessage with content + tool_calls, plus usage / model / id). Custom providers must migrate — see Extension points.

Added

• Active TPM rate limiting. Set RAGSPINE_TOKENS_PER_MINUTE (int, default 0 = off) and the service wraps the provider in corespine's RateLimitedProvider — a sliding-window soft limit that stays under a tokens-per-minute ceiling, complementary to the SDK's passive max_retries 429 backoff. See Configuration.
• [all] extra — pip install "rag-spine[all]" pulls in every runtime capability ([pdf,ocr,llm,embed,vector,service]; excludes [dev]). See Installation.

Dependencies

• Adds one non-optional runtime dependency: corespine>=0.1.0 (the thin shared family core; it has empty dependencies of its own). The offline-first, zero-heavy-SDK default path is preserved.

The Spine family

RAGSpine is one package in a small family that shares the thin corespine core. Current sibling releases: