Semantic Cache Guide

The SWIRL Semantic Cache is a separately licensed SWIRL Enterprise capability, introduced in SWIRL 5. It keeps a searchable, permission-aware copy of documents that users have retrieved through SWIRL — so the best version of a document can be found again instantly, versions of the same document are clustered together, and one version can be designated (or automatically elected) as the canonical version for a given question.

The Semantic Cache does not change SWIRL's core promise: data stays in its original systems. The cache holds only documents that SWIRL users actually retrieved with their own credentials, and every cache hit is badged so users can verify it against the live source.

This guide covers: licensing, how documents enter the cache, searching the cache, version clustering, canonical election, pinning, freshness checks, administration, and troubleshooting.

Licensing

The Semantic Cache is licensed separately from SWIRL Enterprise search and chat. Two fields in the SWIRL license govern it:

The license is supplied as signed JSON in the SWIRL_LICENSE environment variable (typically via .env for Docker Compose deployments):

SWIRL_LICENSE=<license-json>

The license is cryptographically signed; editing any field invalidates it. To add the Semantic Cache to an existing deployment, request a new license from SWIRL that includes "cache": true.

Behavior When Unlicensed or Expired

When the cache is not licensed (or its expiration date has passed):

• Federated search continues to work normally. The seeded cache SearchProvider responds with zero results rather than an error.
• All cache ingestion is skipped — nothing new is written to the cache.
• Version clustering and canonical election are disabled.
• Manually curating a document into the cache returns HTTP 402 with an explanatory message.
• RAG falls back to fetching document bodies live from the source instead of reading them from the cache.

License status is re-checked periodically at runtime, so replacing the license applies without a restart.

Checking Cache License Status

• The server startup banner reports Semantic Cache: Licensed | Expired | Unlicensed.
• The agent status endpoint reports features.cache as a boolean.
• The Admin console license panel shows CACHE LICENSED (with the expiration date), CACHE EXPIRED, or CACHE UNLICENSED.

How Documents Enter the Cache

The cache is populated from documents users retrieve through SWIRL — never by crawling. There are three ingestion paths:

1. RAG auto-cache. When SWIRL fetches a document's full text to generate an AI Insight, the extracted text is cached automatically.
2. User curation. A user can add a specific result to the cache from its result card in the Galaxy UI. If only a snippet is available, SWIRL upgrades it by fetching the full body with the user's credentials.
3. Eager pre-caching (optional, off by default). When SWIRL_EAGER_FEDERATED_PROMOTE_ENABLED=True, the top federated results of each search are cached automatically (up to SWIRL_EAGER_FEDERATED_PROMOTE_MAX, default 5, per search).

For each cached document, SWIRL stores:

• Vectors of the document's text chunks in a Qdrant collection (default swirl_corpus), used for semantic retrieval and version clustering.
• The full document body (compressed) in an object-storage tier, used for RAG without re-fetching.
• A document signal record in the SWIRL database that tracks the document's URL, state (known → indexed → pinned), freshness timestamps, and version lineage.

Cached documents are firm-wide by default: caching and pinning operate at the deployment level, while workspaces scope sources, not documents.

To remove a document from the cache, use the un-index action on its result card, or POST /swirl/cache/unindex/.

Searching the Cache

The cache is searched through a standard SearchProvider seeded with SWIRL Enterprise: "Files - SWIRL Cache" (connector SwirlCorpusConnector). It is active and default out of the box, so every federated search includes cache results alongside live sources.

• Queries are embedded and matched against cached document chunks by vector similarity; results below the SWIRL_CORPUS_MIN_SCORE floor (default 0.50) are not returned.
• Cache results carry a cache payload with the document signal id, freshness information, and clustering data, which the Galaxy UI uses to render badges.
• Version clustering only operates on cache results. If a search does not include the "Files - SWIRL Cache" provider, every result is treated as a singleton and no canonical version is shown.

An optional hybrid mode (SWIRL_CORPUS_HYBRID=True) combines dense vectors with sparse BM25 scoring. Enabling it on an existing deployment requires the one-time upgrade_corpus_to_hybrid migration — see Management Commands.

Version Clustering

Organizations accumulate many copies of the same document — drafts, finals, exports, copies in email and chat. The Semantic Cache groups these into version clusters at search time:

• Each cached result is compared to its nearest neighbors in the vector store; documents whose similarity exceeds SWIRL_VERSION_SIMILARITY_THRESHOLD (default 0.95) are linked into one cluster.
• In the Galaxy UI, a cluster is collapsed to its best version, with the other versions available from the result card.
• Results from live (non-cache) sources are never clustered — they appear as singletons.

The default threshold of 0.95 is deliberately strict: it clusters true versions of the same document while keeping distinct documents on the same topic apart. Lowering it increases the risk of unrelated documents merging into one cluster.

Canonical Election

Within a version cluster, SWIRL can elect one version as canonical — the version users should rely on. Election runs automatically when SWIRL_CANONICAL_AUTO_ELECT_ENABLED=True (the default) and combines:

• Source authority — for example, a document-management system outranks a file share copy (weights are configurable per deployment).
• Naming convention — "final" beats "draft-v2-copy".
• Recency — newer versions score higher.
• Format authority — for example, PDF outranks an editable copy for finished documents.
• User signals — a user pin adds a large bonus; a prior rejection applies a penalty.

Two guards keep elections honest:

• Relevance gate: a cluster is only eligible for canonical treatment when its best member scores at least SWIRL_CANONICAL_QUERY_RELEVANCE_MIN (default 0.10) and at least SWIRL_CANONICAL_QUERY_RELEVANCE_RATIO (default 0.6) of the strongest cluster for the query. Irrelevant clusters never display a canonical badge.
• Explicit scopes win: elections recorded for an explicit matter or topic scope take precedence over firm-wide (unscoped) elections.

The elected version is badged Auto-selected in the Galaxy UI and, when SWIRL_CANONICAL_MIXER_PROMOTION=True (default), is promoted in result ordering. Election runs are recorded and reviewable via the canonical review API (GET /swirl/canonical/).

Pinning

Pinning lets a user (typically a curator or knowledge manager) declare the correct version of a document for a specific query. A pinned document is badged Pinned and outranks automatic election for that query.

Pinning a Document

Pin from the result card in the Galaxy UI, or via the API:

POST /swirl/canonical/pin-for-query/
{
  "signal_id": 123,
  "query": "data retention policy"
}

• Pins are per query: different documents can be pinned for different queries (multi-pin).
• Pins are firm-wide ("All Workspaces"): a pin applies to everyone, regardless of the workspace it was created from.
• If you previously pinned a different document for the same query, the API returns HTTP 409 with the existing pin; re-send with "force": true to replace it.

Approving or Rejecting an Automatic Election

Reviewers can confirm or overturn automatic elections from the canonical review queue:

• POST /swirl/canonical/<run-id>/approve/ — approves the elected winner, which pins it for the run's query; other cluster members are marked as superseded alternates.
• POST /swirl/canonical/<run-id>/reject/ — rejects the election, recording a reason (wrong version, wrong document, not authoritative, duplicate, and so on) that penalizes the rejected version in future elections.

Unpinning

Unpinning (POST /swirl/canonical/<signal-id>/unpin/, or from the result card) is neutral: it removes the pin without penalizing the document. The cluster simply re-elects on merits at the next search — possibly choosing the same document again, now badged Auto-selected rather than Pinned.

Freshness and Liveness

Because cached copies can outlive their sources, every cache result carries freshness metadata:

• Verify on use. Each cache result reports when it was last verified against its source. Results that have not been verified recently (or whose verification age exceeds the staleness window) are badged so users know to verify before relying on them. This check uses stored metadata only — no network calls at query time.
• No longer available from source. When a cache result has no corroborating live result in the same search, the Galaxy UI can trigger an on-demand liveness probe (POST /swirl/cache/check_liveness/). The probe uses the caching user's stored credentials and returns a verdict of alive, gone, or unknown. Only a definitive 404/410 marks a document as no longer available — transient errors never do.

Liveness checks are read-only: SWIRL never deletes a cached document automatically. Removing a document from the cache is always an explicit user or admin action (POST /swirl/cache/unindex/).

Administration

Services

The Semantic Cache uses two additional services, both included in the SWIRL Enterprise Docker Compose stack:

Key Settings

All settings live in the environment (.env for Docker Compose). Defaults are production defaults; most deployments only ever change the ones in the first table.

Canonical election weights (source authority, naming convention, recency, format, pin bonus, rejection penalty) are configurable via SWIRL_CANONICAL_WEIGHTS; contact SWIRL support before tuning them.

Management Commands

Run these inside the SWIRL container (Docker) or from the install directory:

docker compose exec app python manage.py <command>

API Summary

Troubleshooting

Related Guides

• User Guide (Enterprise) — searching, result cards, AI Insights.
• Admin Guide — deployment-wide configuration and operations.
• AI Search Guide — AI Providers and RAG configuration.
• MCP Server (Enterprise) — the MCP server uses the cache opportunistically when licensed.