Skip to content

Progress and analysis

Every figure on these screens is counting something specific. This page says what.

Throughout: only completed, non-expired, non-repetition attempts are counted.

Per session

/sessions/:attemptId breaks a sitting down question by question: what was chosen, whether it was right, how long it took, and how everyone else did on the same question. A question's status is one of correct, incorrect, skipped, withdrawn, unanswered.

The post-session "what to study next" panel uses the same ranking as the standalone recommendations below, grouped by articles, disciplines or systems.

GET /study-tools/attempts/{attempt_id}/questions/{question_id}/responses gives the class-wide distribution across the options for one question, scoped to attempts the asker is allowed to see.

Across everything

Endpoint Counts
GET /study-tools/performance-by-category Answered, correct and accuracy per category. A question counts in its primary category and in every additional category linked to it.
GET /study-tools/completion?days= Bank-wide answered count, percent correct, seconds per question, over an optional trailing window.
GET /study-tools/answer-split The same answers counted two ways — see below.
GET /study-tools/performance-over-time A running cumulative accuracy line by date, plus a point per session.
GET /study-tools/readiness Your accuracy on your latest answer per question, plus a peer comparison.
GET /study-tools/recommendations?group= Ranked focus areas. group is disciplines, articles or systems.

Answer split — effort versus knowledge

The same rows, counted twice:

  • All attempts — every answer ever given. This is effort.
  • Latest attempt — the most recent verdict per question. This is what you currently know.

Answers that were right with a hint are broken out separately in both.

What is locked, and until when

Figure Unlocks at
Performance over time 40 answers and 3 completed sessions
Readiness — your own score 40 answers
Readiness — the peer comparison additionally 10 questions shared with other learners, and at least 3 other learners

A number computed from four answers is noise presented as a measurement, so it is not shown at all.

Readiness, precisely

"Your score" is accuracy over your latest answer to each question you have answered. The peer figure is the expected accuracy of other learners on the same questions you answered — not on the bank as a whole — so it is a like comparison rather than a comparison against a different paper.

How recommendations are ranked

priority = max(gap, 0) × relevance + 0.25 × unseen_share × relevance
  • gap is shrunk readiness against your own overall accuracy as the baseline, using empirical-Bayes shrinkage with a prior weight of 8 answers. A topic you have answered twice is pulled hard toward your own average rather than being declared a weakness.
  • relevance is the active exam's published blueprint weight if you have one, and otherwise the topic's share of the bank.
  • The second term is why something you have barely touched can be recommended even when you have not got it wrong.

This is a study aid, not a psychometric exam score, and the code says so.

Behind the tutor and the card queue

services/performance.py computes a different thing again, and it is not a learner-facing screen. Per top-level topic, over the last 30 days — falling back to all-time below 10 recent answers — it produces a smoothed accuracy, pulled toward a neutral 0.55 prior in proportion to how little evidence there is. That feeds two places: the paragraph handed to the tutor's system prompt, and the "weakest categories first" ordering of new flashcards.

A topic is only ever suggested if the library can actually support studying it: its best article must be at least 8,000 characters and it must have at least 5 linked questions. The paragraph given to the tutor carries percentages and topic names only — never question ids or stems.

Your own exam results

/me/exam-results is a private log a learner keeps by hand of real exams they have sat: the date, the kind (score, percentile or pass_fail), the figures, and a note. It is validated for internal coherence and cross-checked against nothing. There is no educator view and no aggregate.