From gene counts to cell groups · Workflow

Before any method runs, one nuisance has to go. Sequencing does not capture the same number of molecules from every cell: in this dataset the deepest cell yields — as many counts as the shallowest. Compare two cells on raw counts and much of what you see is how well each was sampled, not what it was doing.

The fix is deliberately blunt. Divide every cell by its own total, multiply by a common target, then take log1p. The first part removes depth; the log stops a handful of enormously expressed genes from dominating every distance, and makes a doubling count the same wherever it happens.

Nothing about the biology is added here. This step only makes the later comparisons about the right thing.

Of — measured genes, most are either off in nearly every cell or on at much the same level in all of them. Neither kind can tell two cells apart, and carrying them costs both compute and contrast.

So we keep the genes that vary more than their own expression level predicts. Plot every gene by mean expression against a variability score, and the informative ones stand out above the trend. Taking the top — is the standard working choice.

This is the one step in the workflow that throws information away on purpose: a gene that matters in only a handful of rare cells can fall below the line.

Principal component analysis does not select genes and does not bend anything. It finds the directions along which the cloud of cells varies most, and rotates the coordinate system onto them. Each new axis is a weighted blend of all — input genes, ordered by how much variance it captures.

This is the workhorse step nobody talks about: every step after this one — t-SNE, UMAP and both clustering algorithms — runs on the first 50 principal components, not the raw genes. It is a denoiser as much as a reducer. The components you discard are mostly sampling noise from the — of the matrix that is zero.

And unlike everything that follows, PCA's axes mean something you can read. Click a component in the scree plot to see which genes load onto it.

t-SNE gives up on preserving distances. It converts them into probabilities — how likely is cell j to be picked as a neighbour of cell i — and then arranges points in two dimensions so that the same probabilities come out. Neighbours stay neighbours; everything else is negotiable.

Perplexity sets how many neighbours each point is expected to have. It is the one knob that matters, and moving it does not just polish the picture — it changes what the picture claims. Drag it and watch the rare types detach and re-merge.

The three panels underneath are the part that gets forgotten. They are the same cells, the same perplexity, three random seeds. The islands are stable; their arrangement is not. So the distance between two islands does not show how closely their cell types are related.

UMAP builds a weighted neighbour graph and then finds a two-dimensional layout whose graph looks the same. It is much faster than t-SNE and tends to keep more of the large-scale arrangement, which is why it took over single-cell biology almost completely.

n_neighbors is the real parameter: small values ask the algorithm to respect fine local structure and it will happily shatter a population into fragments; large values ask it to respect the global shape and it will merge distinct types. min_dist is cosmetic — it controls how tightly points may be packed, and nothing else. Moving one changes the analysis; moving the other changes the poster.

Then switch the colouring to donor. The pancreas cells came from four different people, and that fact is written all over the map.

DBSCAN needs no cluster count. It needs a radius ε and a population minPts: a point with at least minPts neighbours inside ε is a core point, touching core points merge into a cluster, and anything left over is labelled noise rather than forced into a group. Being allowed to say “this point belongs to nothing” is genuinely useful.

The catch is that ε is global. This dataset spans a — range in population size, from — down to types with a handful of cells. Turn ε down far enough to keep those rare types separate and the common ones shatter; turn it up enough to hold the common ones together and the rare ones dissolve into noise. No setting does both; try the whole range of the slider.

HDBSCAN runs DBSCAN at every ε at once. Shrinking the radius builds a hierarchy: clusters appear, survive for a while, then split into children. Rather than choosing a radius, it measures how long each candidate cluster persists across that hierarchy and keeps the ones that last. A dense cluster and a sparse cluster can each be selected at the scale where each is stable.

The condensed cluster tree shows that hierarchy. Width is population, downward is decreasing radius, and the highlighted bands are what the excess-of-mass rule selected. min_cluster_size is the only parameter, and it says what you are willing to call a cluster at all.

The section “Evaluate the clusters” compares the two methods at their best, and the scores disagree: DBSCAN wins on ARI while HDBSCAN finds twice as many cell types (10 against 5, counting a type as found when at least half of its cells fall in one cluster; by the F1 ≥ 0.5 rule used in the results table the counts are 12 against 4). ARI rewards getting the bulk of the cells right, and the bulk of these cells are beta, alpha and delta. Lumping everything into three big blobs scores well. The metric you pick decides who wins.

Now the published cell types, the answer key, come out. Two algorithms, four scores, and they do not agree with each other about who won — which is the most useful thing this workflow has to teach.

DBSCAN takes the higher ARI with — clusters and a class-F1 of —. HDBSCAN scores lower on ARI and reaches — — nearly three times better at actually resolving the individual cell types. ARI rewards getting the bulk of the cells right, and the bulk of these cells are beta, alpha and delta, so lumping the rest together is a winning strategy under that metric and a useless one in a lab.

Then the harder question: why is every score mediocre when the UMAP picture looks so clean?