Diagonal integration: [multiple RNA, multiple ATAC]¶
Diagonal integration combines RNA and ATAC measured in different cells, with no pairing between them. This tutorial is for data in several batches: three RNA batches and three ATAC batches. It runs iNMF and online_iNMF on D37mini, the RNA and ATAC of 3,000 cells of the benchmark dataset D37 with the pairing left out, then on data in your own format.
iNMF and online_iNMF read ATAC as gene-activity scores, made beforehand with a tool such as Signac or ArchR.
iNMF and online_iNMF run on Linux, Windows and macOS (Apple silicon). Or open in Colab: sections 1 to 4 take about 6 minutes there.
1. Install¶
pip may then print a note about restarting the kernel: no restart is needed, because the pins keep the numpy and pandas already installed.
import numpy, pandas
pins = f"numpy=={numpy.__version__} pandas=={pandas.__version__}"
%pip install -q "multibench-sc>=0.7.4" {pins}
Note: you may need to restart the kernel to use updated packages.
2. Download the data and the environments¶
mtb.data.fetch downloads D37mini (12 MB) once. mtb.env.install downloads the environments for iNMF and online_iNMF. On Linux the download is 0.9 GB. On Windows it is 0.3 GB. On macOS it is 0.7 GB.
import multibench as mtb
import pandas as pd
pd.set_option("display.max_columns", None) # show every column of the tables
mtb.data.fetch("D37mini")
METHODS = ["iNMF", "online_iNMF"]
envs = mtb.env.install(METHODS)
downloading D37mini (12 MB) ...
3. Run the methods¶
run runs each method on D37mini and scores its output. The clustering metrics score how well the embedding's clusters and neighbourhoods match the cell types, and how separate the clusters are. The batch metrics measure how well the batches mix. The classification scores measure how well the cell types learned on one batch are predicted in the other batches. Higher is better for every metric.
Details
params={"Method": {"key": value}} sets a method's parameters, and mtb.params_for lists them.
mtb.load_run("out/D37mini") reloads these results later.
res = mtb.run("D37mini", "diagonal", methods=METHODS, out_dir="out/D37mini")
res.summary
[run] iNMF (diagonal/D37mini) ... [run] -> scored (13.2s) ARI 0.492 [run] online_iNMF (diagonal/D37mini) ... [run] -> scored (8.7s) ARI 0.422
| method | status | ARI | NMI | ASW | iASW | iF1 | cLISI | ASW_batch | ARI_batch | NMI_batch | GC | iLISI | OCA | ACA | Spec | Sens | F1 | run_sec | caveat | reason | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | iNMF | scored | 0.4916 | 0.6451 | 0.6689 | 0.6808 | 0.4667 | 0.9696 | 0.9020 | 0.9793 | 0.8819 | 0.7801 | 0.6074 | 0.7858 | 0.5021 | 0.9728 | 0.5021 | 0.4896 | 13.2 | ||
| 1 | online_iNMF | scored | 0.4217 | 0.5941 | 0.6228 | 0.6334 | 0.4382 | 0.9408 | 0.8754 | 0.9808 | 0.9219 | 0.8435 | 0.6265 | 0.7296 | 0.5582 | 0.9671 | 0.5582 | 0.5613 | 8.7 |
4. Plot¶
res.plot() draws the scores as a bubble table. Circle size ranks the rows within each column and colour shows the value scaled per column, so a small gap can look large. res.summary has the values.
Details
Metrics are grouped by family: blue for clustering, green for batch correction, pink for classification. Each family starts with an Overall bar: each row's mean rank over the family's metrics, scaled so that the lowest gets the shortest bar.
A column with fewer than two different values is drawn grey. A figure of one method is grey everywhere.
res.plot()
5. Your own data¶
Your data needs raw RNA counts and ATAC as one .h5ad file per batch, with a cell type for each cell. online_iNMF needs about 5,000 cells in total.
Recipes cover labels in a CSV file and no labels.
Details: export
The RNA batches and the ATAC batches are numbered separately. RNA batch 1 and ATAC batch 1 need not come from the same sample. scJoint, GLUE, iNMF and online_iNMF read exactly three RNA and three ATAC batches. For another number of batches, see another number of donors.
For ATAC as a peak matrix, pass atac_kind="peak". GLUE reads the peak matrix.
The next cell writes six small example files, my_rna1.h5ad to my_rna3.h5ad and my_atac1.h5ad to my_atac3.h5ad, from the first 900 RNA profiles and the first 900 ATAC profiles of each batch of D37mini. Skip it for your own data.
mtb.data.write_example("D37mini")
wrote my_rna1.h5ad: 900 cells. RNA counts in X, cell types in obs["celltype"]. wrote my_rna2.h5ad: 900 cells. RNA counts in X, cell types in obs["celltype"]. wrote my_rna3.h5ad: 900 cells. RNA counts in X, cell types in obs["celltype"]. wrote my_atac1.h5ad: 900 cells. ATAC gene-activity scores in X, cell types in obs["celltype"]. wrote my_atac2.h5ad: 900 cells. ATAC gene-activity scores in X, cell types in obs["celltype"]. wrote my_atac3.h5ad: 900 cells. ATAC gene-activity scores in X, cell types in obs["celltype"].
Read your files. For your own data, change every file name marked # <-. On Colab, upload the files first: click the folder icon on the left, then the upload icon.
import anndata as ad
rna1 = ad.read_h5ad("my_rna1.h5ad") # <- RNA batch 1: your file
rna2 = ad.read_h5ad("my_rna2.h5ad") # <- RNA batch 2: your file
rna3 = ad.read_h5ad("my_rna3.h5ad") # <- RNA batch 3: your file
atac1 = ad.read_h5ad("my_atac1.h5ad") # <- ATAC batch 1: your file
atac2 = ad.read_h5ad("my_atac2.h5ad") # <- ATAC batch 2: your file
atac3 = ad.read_h5ad("my_atac3.h5ad") # <- ATAC batch 3: your file
print(rna1)
print(atac1)
AnnData object with n_obs × n_vars = 900 × 2500
obs: 'celltype'
layers: None (.X)
AnnData object with n_obs × n_vars = 900 × 2500
obs: 'celltype'
layers: None (.X)
Write the dataset folder. Change each name in quotes marked # <- to one the printout above lists for two of your files, with its prefix, as "obs:celltype".
NAME = "MYDIAG_BATCHES" # <- a new name for your own data
rna = [rna1, rna2, rna3] # <- your RNA batches
atac = [atac1, atac2, atac3] # <- your ATAC batches
for b in range(len(rna)): # one call per batch
mtb.io.export_dataset(rna[b], "mydata/" + NAME,
rna="X", # <- the RNA counts:
# "X" or "layer:counts"
atac=atac[b], # the ATAC batch
atac_kind="gene_activity", # <- or "peak"
labels="obs:celltype", # <- the cell-type column
category="diagonal", batch_index=b + 1,
overwrite=True)
mtb.scan("mydata/" + NAME, "diagonal", methods=METHODS)
[scan] 2 of 2 methods have their input files in MYDIAG_BATCHES. 2 of 2 can run on this computer. MYDIAG_BATCHES holds RNA and ATAC, in batches 1-3. [scan] Cells per file: rna1.h5 900, atac_gas1.h5 900, rna2.h5 900, atac_gas2.h5 900, rna3.h5 900, atac_gas3.h5 900.
| method | modalities | runnable | reason | env | caveat | |
|---|---|---|---|---|---|---|
| 0 | iNMF | rna1+rna2+rna3+atac_gas1+atac_gas2+atac_gas3 | True | scmb_r | ||
| 1 | online_iNMF | rna1+rna2+rna3+atac_gas1+atac_gas2+atac_gas3 | True | scmb_r |
mine = mtb.run("mydata/" + NAME, "diagonal", methods=METHODS,
out_dir="out/" + NAME)
mine.summary
[run] iNMF (diagonal/MYDIAG_BATCHES) ... [run] -> scored (13.3s) ARI 0.506 [run] online_iNMF (diagonal/MYDIAG_BATCHES) ... [run] -> scored (8.5s) ARI 0.462
| method | status | ARI | NMI | ASW | iASW | iF1 | cLISI | ASW_batch | ARI_batch | NMI_batch | GC | iLISI | OCA | ACA | Spec | Sens | F1 | run_sec | caveat | reason | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | iNMF | scored | 0.5063 | 0.6608 | 0.6541 | 0.6707 | 0.5097 | 0.9623 | 0.8964 | 0.9852 | 0.9405 | 0.7940 | 0.6184 | 0.7607 | 0.4793 | 0.9696 | 0.4793 | 0.4638 | 13.3 | ||
| 1 | online_iNMF | scored | 0.4615 | 0.6153 | 0.5987 | 0.6172 | 0.4717 | 0.9437 | 0.8912 | 0.9922 | 0.9495 | 0.7948 | 0.6244 | 0.7277 | 0.5477 | 0.9667 | 0.5477 | 0.5467 | 8.5 |
mine.plot()
6. Every method of this task¶
4 methods run this task. This notebook runs them all. Its 3 environments download 4.8 GB on a computer without a GPU and 8.3 GB on one with a GPU.
Troubleshooting¶
res.summary[res.summary.status != "scored"] lists each method that failed or was skipped. Its reason column ends with the method's error output.
Details
| symptom | fix |
|---|---|
| a method fails on a small dataset | mtb.scan names what the method needs from the data's size in its caveat column |
| batch metrics use the wrong batches | res.rescore(batch=...), with one batch per cell, scores again without running the methods |
Next steps¶
- Run a method, Score an embedding, Load and plot results and Discover methods.
- Look at an embedding draws a method's embedding as a UMAP.
- The interactive explorer has the full benchmark's rankings, with no install needed.