SWIRL for Backstage

Install SWIRL for Backstage

This page installs SWIRL for Backstage end to end: the SWIRL container, the npm package, the Backstage configuration, the first collator run, and the checks that prove it works.

Prerequisites

A Backstage app on the new backend system. The engine module is a backend-plugin-module and does not work with the legacy backend.
Node 22 or 24, which is what the package declares.
Docker, or a Kubernetes cluster, to run the SWIRL container.
Network access from the SWIRL container to the Backstage backend, for the JWKS endpoint, and from the Backstage backend to SWIRL on port 8000.
About 2.5 GiB of memory and about 7 GB of disk for the image.

Step 1: run SWIRL

Docker Compose

Save this as compose.yaml. It is the file shipped in the SWIRL repository at docker/backstage/compose.yaml. The JWKS URL is filled in for a Backstage backend on the host, on port 7007.

services:
  swirl:
    image: swirlai/swirl-backstage:0.1.1
    container_name: swirl-backstage
    ports:
      - "8000:8000"
    volumes:
      - swirl-data:/data
    environment:
      ALLOWED_HOSTS: "localhost,127.0.0.1,swirl,swirl-backstage"
      CSRF_TRUSTED_ORIGINS: "http://localhost:8000"
      SWIRL_ADMIN_PASSWORD: "<choose-a-password>"
      SQL_ENGINE: django.db.backends.sqlite3
      SQL_DATABASE: /data/db.sqlite3
      CELERY_BROKER_URL: redis://127.0.0.1:6379/0
      CELERY_RESULT_BACKEND: redis://127.0.0.1:6379/0
      SWIRL_TANTIVY_DATA_DIR: /data/tantivy
      SWIRL_TANTIVY_WRITER_HEAP_MB: "64"
      SWIRL_SPACY_MODEL_EN: en_core_web_sm
      SWIRL_BACKSTAGE_JWKS_URL: "http://host.docker.internal:7007/api/search/.backstage/auth/v1/jwks.json"
      SWIRL_BACKSTAGE_AUDIENCE: "search"
      SWIRL_PRINT_ADMIN_TOKEN: "true"
    healthcheck:
      test:
        - CMD
        - python
        - -c
        - "import sys, urllib.request; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/swirl/sapi/health/backstage/', timeout=4).status == 200 else 1)"
      interval: 5s
      timeout: 5s
      start_period: 10s
      retries: 18
    extra_hosts:
      - "host.docker.internal:host-gateway"
    restart: unless-stopped

volumes:
  swirl-data:

docker compose up -d
docker compose logs -f swirl

One container runs Redis, Django, and Celery against the /data volume. The database is SQLite on the same volume. On the 0.1.1 image the container reports healthy in about 15 seconds on a first start, and in 6 to 30 seconds on a restart.

SWIRL_ADMIN_PASSWORD is the password for the SWIRL admin user. It is what you sign in with at the SWIRL admin UI, http://localhost:8000/admin/, which is where the GitHub and Confluence pages configure a source. Set it before the first start. There is no other way in.

SWIRL_PRINT_ADMIN_TOKEN: "true" prints the admin API token to the container log on start. The verify steps below use it. Remove it once you have copied the token.

To build the image yourself instead of pulling it, run this in a SWIRL checkout:

docker build --build-arg SWIRL_PROFILE=backstage -t swirlai/swirl-backstage:dev .

Kubernetes

Save this as swirl-backstage.yaml and apply it. It is the manifest shipped at docker/backstage/k8s.yaml. One replica, strategy Recreate, one PersistentVolumeClaim.

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: swirl-backstage-data
  labels:
    app: swirl-backstage
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
---
apiVersion: v1
kind: Service
metadata:
  name: swirl-backstage
  labels:
    app: swirl-backstage
spec:
  selector:
    app: swirl-backstage
  ports:
    - name: http
      port: 8000
      targetPort: 8000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: swirl-backstage
  labels:
    app: swirl-backstage
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: swirl-backstage
  template:
    metadata:
      labels:
        app: swirl-backstage
    spec:
      containers:
        - name: swirl
          image: swirlai/swirl-backstage:0.1.1
          ports:
            - name: http
              containerPort: 8000
          env:
            - name: ALLOWED_HOSTS
              value: "localhost,127.0.0.1,swirl-backstage"
            - name: SQL_ENGINE
              value: django.db.backends.sqlite3
            - name: SQL_DATABASE
              value: /data/db.sqlite3
            - name: CELERY_BROKER_URL
              value: redis://127.0.0.1:6379/0
            - name: CELERY_RESULT_BACKEND
              value: redis://127.0.0.1:6379/0
            - name: SWIRL_TANTIVY_DATA_DIR
              value: /data/tantivy
            - name: SWIRL_TANTIVY_WRITER_HEAP_MB
              value: "64"
            - name: SWIRL_SPACY_MODEL_EN
              value: en_core_web_sm
            - name: SWIRL_BACKSTAGE_JWKS_URL
              value: "http://backstage:7007/api/search/.backstage/auth/v1/jwks.json"
            - name: SWIRL_BACKSTAGE_AUDIENCE
              value: search
            - name: SECRET_KEY
              valueFrom:
                secretKeyRef:
                  name: swirl-backstage
                  key: SECRET_KEY
                  optional: true
          volumeMounts:
            - name: data
              mountPath: /data
          startupProbe:
            httpGet:
              path: /swirl/sapi/health/backstage/
              port: http
            periodSeconds: 5
            failureThreshold: 24
          readinessProbe:
            httpGet:
              path: /swirl/sapi/health/backstage/
              port: http
            periodSeconds: 10
            failureThreshold: 3
          livenessProbe:
            httpGet:
              path: /swirl/sapi/health/backstage/
              port: http
            periodSeconds: 30
            failureThreshold: 5
          resources:
            requests:
              cpu: "500m"
              memory: "768Mi"
            limits:
              memory: "2Gi"
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: swirl-backstage-data

kubectl apply -f swirl-backstage.yaml
kubectl rollout status deployment/swirl-backstage

Create the secret out of band rather than putting a key in the manifest:

kubectl create secret generic swirl-backstage \
    --from-literal=SECRET_KEY="$(python -c 'import secrets; print(secrets.token_urlsafe(50))')"

Step 2: install the engine module

First edit .yarnrc.yml in the app root and add the @swirl-search scope to npmPreapprovedPackages:

nodeLinker: node-modules
npmMinimalAgeGate: 3d
npmPreapprovedPackages:
  - '@backstage/*'
  - '@swirl-search/*'

@backstage/create-app ships npmMinimalAgeGate: 3d, which quarantines any package published in the last 72 hours unless its scope is preapproved. Without this line the install below fails with YN0016 for the first three days after every SWIRL release. Preapproving the one scope is the Backstage-native escape hatch. Do not set npmMinimalAgeGate: 0, which turns the check off for every package in the repository.

Then add the package:

yarn --cwd packages/backend add @swirl-search/backstage-plugin-search-backend-module-swirl@0.1.1

Add one line to packages/backend/src/index.ts, after the search plugin:

backend.add(import('@backstage/plugin-search-backend'));
backend.add(
  import('@swirl-search/backstage-plugin-search-backend-module-swirl'),
);

Remove any other search engine module from the backend, such as @backstage/plugin-search-backend-module-pg or the Elasticsearch module. Backstage accepts one search engine. Leave the collator modules in place.

Step 3: configure app-config.yaml

Add the SWIRL block. baseUrl is the only required key. Every other key has a default, and all of them are listed in the configuration reference.

search:
  swirl:
    # The only required key. The URL your Backstage backend uses to reach SWIRL.
    baseUrl: http://localhost:8000

Use the address the Backstage backend can reach. On Kubernetes that is the service name, for example http://swirl-backstage:8000. The whole block is backend visible only and never reaches the browser.

The module skips registration with a logged warning when search.swirl is absent, so a backend without the block still boots.

Step 4: point SWIRL at Backstage's JWKS

The engine module authenticates to SWIRL with a Backstage plugin token. SWIRL verifies that token against the Backstage backend's JWKS endpoint. Two environment variables configure it, and both must be set or the path is disabled.

Variable	Meaning	Example
`SWIRL_BACKSTAGE_JWKS_URL`	The full JWKS URL of the Backstage backend. Empty disables Backstage token verification entirely.	`http://backstage:7007/api/search/.backstage/auth/v1/jwks.json`
`SWIRL_BACKSTAGE_AUDIENCE`	The plugin id the token is minted for. Must equal `search.swirl.audience` in app-config.	`search` (the default)

The path segment in the JWKS URL is the plugin id, so it matches the audience. Keep both at search unless you have a reason to change them together.

The URL must resolve from inside the SWIRL container, not from your laptop. On Docker Desktop use http://host.docker.internal:7007/... for a backend running on the host. On Linux that name only resolves when the compose file carries the extra_hosts entry shown above.

Step 5: run the collators and confirm the index

Start or restart the Backstage backend. The catalog and TechDocs collators run on their normal schedule, and the first run happens a few seconds after start. The backend log records one indexing run per document type. On the smoke-tested setup the first index was live 3.2 seconds after the backend started, with no manual trigger.

Confirm that SWIRL has a live index. The token below is the admin API token from the container log.

curl -s -H "Authorization: Token <admin-api-token>" \
    http://localhost:8000/swirl/index/

{
  "types": [
    {"type": "software-catalog", "live": "20260904T002915-625358", "doc_count": 9, "bytes": 28136, "updated": 1788481755.794762, "open": null},
    {"type": "techdocs", "live": null, "doc_count": 0, "bytes": 0, "updated": null, "open": null}
  ]
}

Read the fields as they are, not as they look. doc_count is the document count. live is the live generation id, <compact-timestamp>-<microseconds>. updated is a Unix epoch float, not an ISO string. open names a generation that is mid-run, and is null the rest of the time. A type with "live": null has no index yet, which is what a stock create-app shows for techdocs, because it ships no TechDocs content.

An empty types list means no collator run has finished. See Troubleshooting.

Step 6: verify

First the health endpoint. It needs no token and returns HTTP 200 only when Redis, the Celery search worker, and the Tantivy reader are all up.

curl -s http://localhost:8000/swirl/sapi/health/backstage/

{
  "ok": true,
  "redis": {"ok": true, "url": "redis://127.0.0.1:6379/0"},
  "celery_search_worker": {"ok": true, "workers": ["celery@f0eac55fdd85"]},
  "tantivy": {"ok": true, "types": ["software-catalog"]},
  "license": {"edition": "community", "backstage": true}
}

The worker name is the container hostname, so it differs on every run. types lists the document types that have a live index. A stock create-app shows software-catalog only.

Then one search straight at SWIRL, against the indexed lane only. Search for something the catalog actually holds. A stock create-app ships the entities example-website, example-grpc-api and examples, so example-website is a safe term:

curl -s -H "Authorization: Token <admin-api-token>" \
    "http://localhost:8000/swirl/search/?qs=example-website&providers=backstage-index&backstage_types=software-catalog"

Then the same search through Backstage, which is the path that matters:

curl -s -H "Authorization: Bearer <backstage-user-token>" \
    "http://localhost:7007/api/search/query?term=example-website"

Expect a result set shaped like this, trimmed to one result and to the fields that matter here:

{
  "results": [
    {
      "type": "software-catalog",
      "document": {
        "title": "example-website",
        "location": "/catalog/default/component/example-website",
        "namespace": "default",
        "kind": "Component",
        "lifecycle": "experimental"
      },
      "highlight": {
        "preTag": "<cdfdc5fb-1511-46f0-8cbf-5a1d698ff560>",
        "postTag": "</cdfdc5fb-1511-46f0-8cbf-5a1d698ff560>",
        "fields": {"title": "<...>example-website</...>"}
      }
    }
  ],
  "numberOfResults": 1
}

The highlight markers are not literal <em> tags. SWIRL wraps each hit term in a marker pair drawn at random when the instance starts, so that text arriving from a source cannot forge a highlight. Backstage rewrites the pair when it renders. A raw response therefore carries a random tag pair, not <em>.

Results with a type of software-catalog or techdocs came from the index. Results with a type of swirl-federated came from a connected source.

Running with more than one Backstage pod

Multiple Backstage pods are the normal case on Kubernetes. All of them point at one SWIRL service. The rules are short.

One SWIRL replica. The container runs its own Redis and holds its own copy of the index. A second replica would hold a second, separate index.
Deployment strategy Recreate. The default RollingUpdate would start a second pod that cannot attach the same volume.
The index lives on a PersistentVolumeClaim. A restart does not reindex. SWIRL reopens the index from disk and serves again in seconds.
Collators may run on any Backstage pod. Each indexing run opens its own generation in SWIRL and swaps it in at the end. A run from one pod never half-writes the index another pod is reading.

When the SWIRL pod is down, Backstage search returns an error rather than stale results. Catalog browsing, TechDocs, and every other plugin are unaffected. When SWIRL comes back, search works again with the index it already had. Active-passive high availability is on the Enterprise roadmap and is not in this release.