Skip to content

How scMultiBench works

scMultiBench is a benchmark of single-cell multimodal integration methods. The multibench package is its Python API.

What scMultiBench covers

The benchmark groups datasets into four scenarios by how the data were measured. Each scenario is a category value in the API:

  • Vertical: several modalities measured in the same cells, for example CITE-seq or 10x Multiome.
  • Diagonal: modalities measured in different cells, for example scRNA-seq and scATAC-seq from separate experiments.
  • Mosaic: several batches, only some sharing a modality, joined by a paired batch.
  • Cross: several batches, each with every modality. In this package that means RNA and ADT, for example one CITE-seq assay from several donors.
Details

The benchmark evaluates dimension reduction, batch correction, clustering and feature selection.

Its datasets are real and simulated, and cover RNA, ADT and ATAC.

How it evaluates

Each task has its own metric panel. The scores are combined into per-method rank scores. For the full ranking per scenario, see the paper's decision tree (Fig. 6) or the interactive explorer.

Details

The metric panels are:

  • Dimension reduction, batch correction and clustering: scIB metrics such as kBET, iLISI, ASW, graph connectivity, PCR, ARI and NMI.
  • Feature selection: marker overlap and correlation.
  • Every task: runtime and peak memory.

In the package, mtb.evaluate computes the scIB metrics for an embedding.

Next steps