Cross integration: Multiple RNA + ATAC¶
Cross integration combines batches that all measure the same modalities. The task is to remove batch effects and keep the biological structure. This tutorial is for batches that each hold RNA and ATAC peaks. It runs StabMap and scMM on D56mini, 4,500 cells in three batches, cut from the first three batches of the benchmark dataset D56, and then on data in your own format.
Methods run on Linux, and scMM has no Windows or macOS environment. On Windows or macOS, open this notebook in Colab: sections 1 to 4 take about 8 minutes there.
1. Install¶
pip may then print a note about restarting the kernel: no restart is needed, because the pins keep the numpy and pandas already installed.
import numpy, pandas
pins = f"numpy=={numpy.__version__} pandas=={pandas.__version__}"
%pip install -q "multibench-sc>=0.7.4" {pins}
Note: you may need to restart the kernel to use updated packages.
2. Download the data and the environments¶
mtb.data.fetch downloads D56mini (10 MB) once. mtb.env.install downloads the environments for StabMap and scMM. The download is 1.4 GB.
import multibench as mtb
import pandas as pd
pd.set_option("display.max_columns", None) # show every column of the tables
mtb.data.fetch("D56mini")
METHODS = ["StabMap", "scMM"]
envs = mtb.env.install(METHODS)
downloading D56mini (10 MB) ... [env] downloading scmb_scmm2 481 MB ... [env] 481 MB of 481 MB (27 MB/s) [env] unpacking scmb_scmm2 ...
3. Run the methods¶
run runs each method on D56mini and scores its output. The clustering metrics score how well the embedding's clusters and neighbourhoods match the cell types, and how separate the clusters are. The batch metrics measure how well the batches mix. The classification scores measure how well the cell types learned on one batch are predicted in the other batches. Higher is better for every metric.
Details
params={"Method": {"key": value}} sets a method's parameters, and mtb.params_for lists them.
mtb.load_run("out/D56mini") reloads these results later.
res = mtb.run("D56mini", "cross", methods=METHODS, out_dir="out/D56mini")
res.summary
[run] StabMap (cross/D56mini) ... [run] -> scored (14.9s) ARI 0.291 [run] scMM (cross/D56mini) ... [run] -> scored (26.5s) ARI 0.218
| method | status | ARI | NMI | ASW | iASW | iF1 | cLISI | ASW_batch | ARI_batch | NMI_batch | GC | iLISI | OCA | ACA | Spec | Sens | F1 | run_sec | caveat | reason | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | StabMap | scored | 0.2912 | 0.4920 | 0.5016 | 0.5097 | 0.3543 | 0.9370 | 0.8220 | 0.9853 | 0.9672 | 0.8139 | 0.4810 | 0.5124 | 0.3967 | 0.9726 | 0.3967 | 0.3656 | 14.9 | ||
| 1 | scMM | scored | 0.2178 | 0.3618 | 0.5216 | 0.5267 | 0.2795 | 0.8972 | 0.9118 | 0.9851 | 0.9666 | 0.6966 | 0.7437 | 0.4416 | 0.3430 | 0.9678 | 0.3430 | 0.3231 | 26.5 |
4. Plot¶
res.plot() draws the scores as a bubble table. Circle size ranks the rows within each column and colour shows the value scaled per column, so a small gap can look large. res.summary has the values.
Details
Metrics are grouped by family: blue for clustering, green for batch correction, pink for classification. Each family starts with an Overall bar: each row's mean rank over the family's metrics, scaled so that the lowest gets the shortest bar.
A column with fewer than two different values is drawn grey. A figure of one method is grey everywhere.
res.plot()
5. Your own data¶
Your data needs raw RNA counts in one .h5ad file and an ATAC peak matrix of the same cells in another, with a cell type and a batch for each cell. Give the same peaks in every batch.
The methods of this task read three batches.
Recipes cover a 10x .h5 file, labels in a CSV file, no labels and another number of donors.
Details: export
The batches are numbered in the sorted order of their names, read as text: donors A, B and C become batches 1, 2 and 3, and 10 comes before 2.
For one AnnData per batch, call export_dataset once per batch with batch_index=N instead of batch=.
The next cell writes two small example files, my_rna.h5ad and my_atac.h5ad, from 60% of the cells of D56mini, with a batch column. Skip it for your own data.
mtb.data.write_example("D56mini")
wrote my_rna.h5ad: 2,700 cells. RNA counts in X, cell types in obs["celltype"], batches in obs["batch"]. wrote my_atac.h5ad: 2,700 cells. ATAC peak counts in X, cell types in obs["celltype"], batches in obs["batch"].
Read your files. For your own data, change every file name marked # <-. On Colab, upload the files first: click the folder icon on the left, then the upload icon.
import anndata as ad
rna = ad.read_h5ad("my_rna.h5ad") # <- your RNA file, every batch in it
atac = ad.read_h5ad("my_atac.h5ad") # <- your peak file, the same cells
print(rna)
print(atac)
AnnData object with n_obs × n_vars = 2700 × 1000
obs: 'celltype', 'batch'
layers: None (.X)
AnnData object with n_obs × n_vars = 2700 × 5000
obs: 'celltype', 'batch'
layers: None (.X)
Write the dataset folder. Change each name in quotes marked # <- to one the printout above lists for your files, with its prefix, as "obs:celltype".
NAME = "MYCROSS_RNA_ATAC" # <- a new name for your own data
mtb.io.export_dataset(rna, "mydata/" + NAME,
rna="X", # <- where the RNA counts are:
# "X" or "layer:counts"
atac=atac, # the ATAC file, peak counts in X
atac_kind="peak", # the methods here read peaks
labels="obs:celltype", # <- the cell-type column
batch="obs:batch", # <- the batch or donor column
category="cross", overwrite=True)
mtb.scan("mydata/" + NAME, "cross", methods=METHODS)
[scan] 2 of 2 methods have their input files in MYCROSS_RNA_ATAC. 2 of 2 can run on this computer. MYCROSS_RNA_ATAC holds RNA and ATAC, in batches 1-3. [scan] Cells per file: rna1.h5 936, atac_peak1.h5 936, rna2.h5 887, atac_peak2.h5 887, rna3.h5 877, atac_peak3.h5 877.
| method | modalities | runnable | reason | env | caveat | |
|---|---|---|---|---|---|---|
| 0 | StabMap | rna1+rna2+rna3+atac1+atac2+atac3 | True | scmb_r | ||
| 1 | scMM | rna1+rna2+rna3+atac1+atac2+atac3 | True | scmb_scmm2 | scMM reads inputs/rna1.h5, inputs/rna2.h5, inp... |
mine = mtb.run("mydata/" + NAME, "cross", methods=METHODS,
out_dir="out/" + NAME)
mine.summary
[run] StabMap (cross/MYCROSS_RNA_ATAC) ... [run] -> scored (13.0s) ARI 0.296 [run] scMM (cross/MYCROSS_RNA_ATAC) ... [run] -> scored (18.3s) ARI 0.132
| method | status | ARI | NMI | ASW | iASW | iF1 | cLISI | ASW_batch | ARI_batch | NMI_batch | GC | iLISI | OCA | ACA | Spec | Sens | F1 | run_sec | caveat | reason | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | StabMap | scored | 0.2956 | 0.4807 | 0.5213 | 0.5222 | 0.3357 | 0.9293 | 0.8398 | 0.9889 | 0.9675 | 0.8084 | 0.5309 | 0.5188 | 0.3870 | 0.9729 | 0.3870 | 0.3582 | 13.0 | ||
| 1 | scMM | scored | 0.1322 | 0.2629 | 0.5040 | 0.5055 | 0.2044 | 0.8668 | 0.8949 | 0.9909 | 0.9743 | 0.6586 | 0.7680 | 0.4048 | 0.2585 | 0.9659 | 0.2585 | 0.2287 | 18.3 |
mine.plot()
6. Every method of this task¶
8 methods run this task. This notebook runs them all. Its 6 environments download 19.6 GB on a computer without a GPU and 23.0 GB on one with a GPU.
Troubleshooting¶
res.summary[res.summary.status != "scored"] lists each method that failed or was skipped. Its reason column ends with the method's error output.
Details
| symptom | fix |
|---|---|
| a method fails on a small dataset | mtb.scan names what the method needs from the data's size in its caveat column |
| batch metrics use the wrong batches | res.rescore(batch=...), with one batch per cell, scores again without running the methods |
Next steps¶
- Run a method, Score an embedding, Load and plot results and Discover methods.
- Look at an embedding draws a method's embedding as a UMAP.
- The interactive explorer has the full benchmark's rankings, with no install needed.