Quickstart¶
Five steps take you from finding methods to plotting your run next to the stored results. Only step 3 runs a method. It needs Linux, Windows or macOS with Apple silicon, and a method environment. The other steps also work on an Intel Mac.
For your own data:
- CITE-seq: the RNA + ADT tutorial, which runs in Colab.
- 10x Multiome, unpaired RNA + ATAC or one file per batch: see Your own data as a dataset folder.
Words used on this site
- embedding
- A table with one row per cell and tens to hundreds of numbers per cell, like
adata.obsm["X_pca"]. - category
- How your cells were measured, for example
verticalfor CITE-seq from one experiment. - variant
- One version of a method for one data type, such as RNA + ADT.
- role
- The name of one input file, such as
rnaoradt. - dataset folder
- One folder that holds the input files of a dataset, such as
rna.h5andcty.csv. - raw counts
- Integer counts per cell and feature, before any normalization.
- demo dataset
- A dataset that
mtb.data.fetchdownloads: a benchmark dataset such asD11, or a smallerminicopy of one such asD22mini. - stored results
- The scores that ship with the package: its own runs of the methods, one dataset per category.
- long table
- A table with one row per method, dataset and metric.
- cell order
- The order of the rows in a method's output. For diagonal data most methods put the RNA cells first, then the ATAC cells.
runwrites each cell's barcode intoembedding.h5, andevaluate(dataset=)matches the label files by them.
import multibench as mtb, pandas as pd
# 1. methods for RNA + ADT measured in several batches (cross)
mtb.find_methods(category="cross", modalities=["rna", "adt"])
# 2. plot the stored results of dataset D52
df = mtb.load_results("cross", dataset="D52")
mtb.plot.bubble(df, metrics=["ARI", "NMI", "cLISI"], save="fig.pdf")
# 3. run and score a method on D52 (Linux, Windows or macOS)
mtb.data.fetch("D52")
mtb.env.install(["StabMap"])
res = mtb.run("D52", "cross", methods=["StabMap"], out_dir="out/")
# 4. score an embedding of your own (adata: your AnnData)
scores = mtb.evaluate(adata, labels="celltype", obsm="X_pca", method="PCA")
# 5. plot your run of step 3 next to the stored results
mine = res.long.assign(method=res.long.method + " (this run)")
mtb.plot.bubble(pd.concat([df, mine]), metrics=["ARI", "NMI", "cLISI"])
- Discover methods covers filtering the methods by category, modality and labels, and ranking them on the stored results.
- Load and plot results covers loading the stored results and drawing the paper's bubble table.
- Run a method covers running a method on a demo dataset or on your own data.
- Score an embedding covers what each metric
measures and what
evaluatetakes. - Add your own method covers putting your scores next to the stored ones, and Are my numbers comparable? what to expect.
Next steps¶
- Vertical / diagonal / mosaic / cross: one notebook per integration task, on a small dataset and on your own data.
- API reference: every function and parameter.