How papers are ranked

Ranking has two layers: a relevance tier from the screening rules, then recency and citations within each tier. No paper is ever dropped.

The full rules are in DSciarrino/SKILLS.md; this page summarizes them.

Layer 1: relevance tier

Every paper gets one rating. Tiers are listed in this order, and every paper in a higher tier ranks above every paper in a lower one.

Tier	Meaning
Preferred	Directly helps build the initial annotation algorithm. Read these first.
Use with caution	Broad or general; useful for orientation and for finding other papers.
Later (refinement)	Relevant but too narrow for the first build; revisit during refinement.
Out of scope	Kept at the bottom with the reason recorded, so the decision can be reversed.

The screening rules

Applied in order; the first rule that decides the rating wins. The rule number appears as a tag on each paper.

Out of scope → Out of scope: disease outside cancer, the microbiome. Exceptions: algorithm, benchmark and dataset papers whose non-cancer samples are only test data go on to rule 3; reusable cell reference maps outside cancer rate Later.
Too specific → Later: one subtype in one disease state, or one particular process.
Algorithm papers → Preferred: methods for cell type identification or annotation, including LLM and agent-based ones. Benchmarks and labeled datasets count here too.
Experimental papers → Preferred only if they define markers: a clear marker set for a cell type makes an experimental paper a direct source of rules.
Very general papers → Use with caution.

"Cell type" versus "cell state" wording is never used as a filter, because the terminology isn't standardized.

Dr. Talkington's own examples: CellTypeAgent rates Preferred (an agent-based algorithm close to this project); the CD4+ T cell reference map in viral infection rates Later (not cancer, and a single subtype).

Layer 2: within a tier

within-tier score = 0.5 × recency + 0.5 × citation score

Score	How it's calculated
Recency	1.0 for the current year, falling by 0.1 per year, reaching 0 at 10 or more years old. Papers from the last 5 years get the recent tag.
Citation score	OpenAlex citations per year since publication, turned into a percentile across all papers (0 = least cited, 1 = most cited). Papers with no count get a neutral 0.5.

Per year so newer papers can compete with older ones.
Percentiles so a few papers with thousands of citations don't flatten everyone else.
Ties go to papers that define markers, then to the more recent paper.

The three scores appear as bars in each paper's Ranking box.

Who decides the tiers

Ratings were assigned by applying the rules in SKILLS.md and are recorded in literature_sweep/screening_decisions.csv, one row per paper with the rule and a reason. Some are flagged needs review, and a few cases are listed in SKILLS.md as open questions for Dr. Talkington. The 50/50 recency/citation split is a starting point, not one of her rules.

Changing a rating

Edit the paper's row in literature_sweep/screening_decisions.csv (relevance, rule, reason).
Rename or move literature_sweep/screened_papers.csv (the script won't overwrite it), then run python apply_screening.py in literature_sweep/.
Run python pipeline/build_index.py in annotation_explorer/ and reload the app.