Development and architecture

Setup

Create a virtual environment and install the package in editable mode:

python3 -m venv .venv
source .venv/bin/activate
pip install -e .

Include optional semantic-search dependencies when needed:

pip install -e ".[semantic]"

Tests

Run the Python test suite:

python3 -m unittest -v

Retrieval evaluation

Retrieval quality is a versioned CI contract under benchmarks/retrieval/. The pull-request profile uses a deterministic 1,000-session corpus:

python3 -m pip install -r benchmarks/retrieval/requirements-ci.txt
python3 -m benchmarks.retrieval.evaluate \
  --profile pr --sessions 1000 --semantic --output-dir /tmp/recall-retrieval

Scheduled evaluation reuses the same specification and generator at larger scales:

python3 -m benchmarks.retrieval.evaluate --profile nightly --sessions 10000 \
  --semantic --output-dir /tmp/recall-retrieval-10k
python3 -m benchmarks.retrieval.evaluate --profile nightly --sessions 100000 \
  --semantic --output-dir /tmp/recall-retrieval-100k

Each run writes results.json and summary.md. Golden texts use the configured real local embedding model. Scale-only filler sessions use deterministic zero vectors so CI measures SQLite storage and the production matrix/ranking path without claiming to benchmark 100,000 unique model inferences.

PR change detection skips retrieval only when every changed path is a known-safe documentation or static asset. Unknown paths run fail-safe. The GitHub branch rule should require only Retrieval CI / Required retrieval gate, not the conditional worker job.

Type-check the Pi extension:

npx tsc --project tsconfig.recall.json

Architecture

The recall.py module and CLI remain the compatibility facade. The data lifecycle is separated into focused modules:

• recall_core.ingestion discovers Claude Code, Pi, and Codex transcripts and normalizes their records into a shared message shape.
• recall_core.indexing owns the SQLite schema, FTS5 indexes, connections, and incremental persistence.
• recall_core.retrieval owns query filters, fuzzy and regular-expression search, ranking, and match-preview formatting.

Semantic embedding and CLI/TUI orchestration currently remain in the facade. Keeping them there avoids changing optional-dependency loading and the public module surface while the core is separated into subsystems.

Demo assets

Regenerate the README walkthrough with VHS:

vhs docs/recall-demo.tape

The tape runs real Recall commands from the checkout. For its semantic-search segment, install the optional dependencies and build embeddings first:

python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[semantic]"
recall index --semantic