Skip to content

Running the tests

The unit and integration suite

Test-only dependencies are in backend/requirements-dev.txt and are deliberately not in requirements.txt.

docker compose exec backend pip install -r requirements-dev.txt
docker compose exec -w /app/tests -e PYTHONPATH=/app backend python -m pytest .

Two things about that command are not optional:

  • Run from /app/tests. The test modules import each other by bare name, so the tests directory has to be the working directory.
  • PYTHONPATH=/app is what lets the subprocess-spawning migration tests import app.

Why the dev requirements file exists

pytest had been pip-installed by hand into the running container. A rebuild silently removed the ability to run the suite at all — over a thousand tests, unrunnable, with nothing to say why.

The suite covers access and grants (test_access.py, test_category_grants.py, test_article_grants.py, test_deck_grants.py, test_question_detail_access.py, test_answer_safety.py), retrieval (test_hybrid_search.py, test_global_search.py, test_ai_mode_matching.py), the article library, the question pipeline, and the API contract.

The API contract test

backend/tests/api-contract.json is the API surface as it stood when it was last accepted: every method and path, the parameters each takes, and the status codes it answers with. test_api_contract.py compares the live app against it and fails on any difference, naming the routes that moved.

After deliberately adding or changing a route:

docker compose run --rm --no-deps -e DATABASE_URL=sqlite:// -e PYTHONPATH=/app \
  -w /app backend python -m tests.test_api_contract --write

Regenerating it produces a diff to review. That is the point: removing a route should be visible in review as "this client is about to break", not discovered by the client.

Regenerate docs/openapi.json at the same time — see What is here.

End to end

docker-compose.test.yml brings up a complete separate stack: its own Postgres, its own Redis, its own volumes, its own network, its own ports.

docker compose -f docker-compose.test.yml up -d --build
docker compose -f docker-compose.test.yml run --rm seed
cd e2e && npx playwright test
docker compose -f docker-compose.test.yml down -v     # -v: take the data with it

Nothing there touches the running site. The point of an end-to-end test is to do the destructive things a real user can do — sit a session, delete an article, sign out everywhere — and none of that may happen to somebody's actual work. Neither Postgres nor Redis publishes a host port, so they cannot collide with the real ones.

The database is thrown away with the stack. A test suite that depends on data surviving between runs is a test suite that passes on your machine.

There are also OIDC end-to-end notes in e2e/oidc-e2e.md.

The front end

Vitest, alongside the components:

cd frontend && npm test