Run a method¶
mtb.run runs the
methods you name on a dataset folder, each in its own environment, scores
each output and saves the results. methods=["SCALEX"] runs one method.
Methods run on Linux, and on Windows and macOS (Apple silicon) for the
methods with a Windows or
macOS environment. SCALEX, the example of this
page, runs on Linux and macOS. On an Intel Mac,
mtb.scan(dataset, category, command=True) returns the plan with each
command. Run the call itself on a Linux machine.
Summary
Details
run finds the version of the method that fits the category and
the files of the folder. It converts the files if needed, runs the
method script in <out_dir>/<method>_<dataset>/ and scores its output.
mtb.scan creates no files: with command=True its command column
holds each command, with <out_dir> for the output folder.
Some commands read a file that run writes first under
<out_dir>/<method>_<dataset>/inputs/, such as a converted AnnData,
the renamed peak file of GLUE and Seurat_v3, or the links to the data
files of scMM, scMVP and scJoint. The caveat column of
scan says so. Call mtb.run in the job for those methods, not the
command alone.
Install the method's environment¶
Install the method's environment before its first run.
mtb.env.install(["SCALEX"], dry_run=True) lists what is installed and what
is missing, with the download sizes.
Details
A missing environment marks the method not runnable in mtb.scan, and
run skips it. On an Intel Mac the reason says that the environment
runs only on Linux.
GLUE reads the GENCODE v43 human annotation, which its first run
downloads, and MIRA a logger.py, which the package puts next to its
script. GLUE runs its human script only, so mouse data does not run
through the package. mtb.method_info(m)["setup_hint"] holds these
notes, and the caveat column of scan lists them.
What run returns¶
res is a RunResult. res.summary has one row per method: its status,
run time and scores, and its reason column says why a method failed or
was skipped.
res.embedding(method) returns the method's embedding as an AnnData, in the
method's cell order: obsm["X_emb"]. Its obs holds each cell's label,
and its batch when the run had more than one batch: the label file of each
cell, or the batch= of run. The cell names are the barcodes of the
dataset files. Where two files share a barcode, each name starts with its
label file, such as cty2:AAACCTGAG....
res.summary[res.summary.status != "scored"] # empty when every method scored
emb = res.embedding("SCALEX")
emb.obs.head() # label, and batch, per cell
res.long # the scores, as the long table
res = mtb.load_run("runs/") # the same result, later
run also writes the cell names into each embedding.h5, as the dataset
barcodes, so mtb.evaluate and mtb.io.read find each cell of
the file by its name.
Details
res.records holds one dict per method, with the keys
mtb.RunResult lists. Five methods put the cells in another order than RNA before ATAC and
cty1, cty2, .... uniPort and Seurat_v5 put the ATAC cells first.
StabMap puts its reference batch first, in cross and in the mosaic
patterns that start with an RNA-only batch. SMILE and MultiVI in
mosaic use their own batch order. embedding(method) and
mtb.evaluate(..., dataset=, method=) follow that order.
method_info(m)["supports"] gives each variant's output_kind,
embedding or graph. run scores scMoMaT on the neighbour graph it
writes, as the paper does: ARI, NMI, iF1, cLISI, GC and iLISI. ASW, iASW,
ASW_batch and the classification metrics need an embedding, so scMoMaT
has none. MIRA and Seurat_WNN get the status ran, and the reason says
that they write a graph.
res.rescore(batch=..., labels=..., metrics=...).save() scores the saved
outputs again and writes the new scores. metrics= replaces those scores
and keeps the others. No method runs again.
Set a method's parameters¶
Every method takes parameters.
mtb.params_for(method, category, modalities) lists them with their
defaults, which are the paper's values. A default of None leaves the
setting to the method's library. run(params={method: {name: value}}) sets
them. For several values of one parameter, loop over run, one
out_dir per value, and name each run's rows:
import pandas as pd
p = mtb.params_for("scJoint", "diagonal")
p["tunable"]["epochs_stage1"] # {'default': 20, 'type': 'int'}
runs = []
for v in (10, 30, 50):
r = mtb.run("D28", "diagonal", methods=["scJoint"], out_dir=f"runs/e{v}",
params={"scJoint": {"epochs_stage1": v}})
runs.append(r.long.assign(method=f"scJoint epochs={v}"))
mtb.plot.bubble(pd.concat(runs))
Inputs¶
A dataset folder holds one file per role: a role is the name of one input,
such as rna, atac_gas or cty, the label file.
Dataset folder layout lists the file names, and
mtb.scan(dataset, category) says which methods can run on a folder, and
why not the rest.
mtb.method_info("SCALEX")["supports"][0]["modalities"] # ['rna', 'atac_gas']
mtb.scan("D28", "diagonal")
Details
Each entry of supports is one variant: the version of a method for
one category and one set of roles. For a method with several variants,
run runs the one with the most of the folder's modalities.
modalities=["rna", "adt"] picks another.
scan checks that each file is there and that no matrix is transposed.
It also checks that each label file has as many rows as its matrix, and
that files that must hold the same cells do. Its reason column names
what is wrong.
Dataset folder layout¶
Put the files of one dataset in one folder, <data_path>/<DATASET>/, for
example data/MYDATA/. The folder name is the dataset name.
mtb.io.export_dataset writes the folder from an AnnData or MuData.
| Category | Files |
|---|---|
vertical |
rna.h5 with adt.h5, atac.h5 or both, from the same cells, and cty.csv |
diagonal |
rna.h5 and rna_cty.csv for the RNA cells, atac_peak.h5, atac_gas.h5 or both, and atac_cty.csv for the ATAC cells |
mosaic |
one file per batch and modality, numbered by batch (rna1.h5, rna2.h5, atac2.h5 ...), and one label file per batch (cty1.csv, cty2.csv ...) |
cross |
as mosaic, with atac_peak1.h5, atac_peak2.h5 ... for ATAC |
Each .h5 file holds matrix/data (features x cells, the transpose of
AnnData.X), matrix/features and matrix/barcodes. R's rhdf5 shows the
dimensions reversed, so write a cells x features matrix from R. A label file
is a one-column CSV: a header line, then one label per cell, in the order of
the cells.
Vertical, mosaic and cross methods read ATAC as peaks. In diagonal,
atac_peak.h5 holds peaks and atac_gas.h5 gene-activity scores of the
same cells, and each method reads one of them or both:
| ATAC files | Diagonal methods |
|---|---|
| both | MultiMAP, Seurat_v3 |
atac_peak.h5 |
Seurat_v5, GLUE |
atac_gas.h5 |
SCALEX, scBridge, VIPCCA, Portal, uniPort, sciCAN, scJoint, iNMF, online_iNMF, Conos |
For several RNA and several ATAC batches in diagonal, scJoint, GLUE, iNMF
and online_iNMF read exactly 3 of each. They are numbered rna1.h5,
atac_gas1.h5 or atac_peak1.h5, rna_cty1.csv and atac_cty1.csv, then 2
and 3. For another number of batches, see Another number of donors.
Mosaic and cross methods read exactly the batches of one of these patterns. Number yours to match one:
| Category | Batch 1 | Batch 2 | Batch 3 | Methods |
|---|---|---|---|---|
| mosaic | rna | rna+adt | adt | Multigrate, scMoMaT, StabMap |
| mosaic | rna | rna+atac | atac | Multigrate, MultiVI, Cobolt, scMoMaT, SMILE, StabMap |
| mosaic | rna+adt | rna+atac | rna | Multigrate, scMoMaT, StabMap, UINMF |
| mosaic | rna+adt | rna+atac | adt | Multigrate, scMoMaT, StabMap |
| cross | rna+adt | rna+adt | rna+adt | Multigrate, totalVI, scMM, scMDC, scMoMaT, Concerto, sciPENN, StabMap, UINMF, MOFA2 |
| cross | rna+atac | rna+atac | rna+atac | Multigrate, scMM, scMDC, scMoMaT, UnitedNet, StabMap, UINMF, MOFA2 |
| cross | adt+atac | adt+atac | Multigrate, scMoMaT, StabMap, UINMF, MOFA2 | |
| cross | rna+adt+atac | rna+adt+atac | Multigrate, scMoMaT, StabMap, UINMF, MOFA2 | |
| cross | rna+adt | rna+adt | UINMF | |
| cross | rna+atac | rna+atac | UINMF |
Your own data as a dataset folder¶
mtb.io.export_dataset writes an AnnData or MuData as a dataset folder,
which scan and run read.
For a Cell Ranger .h5 file, labels in a CSV file, data without labels or
another number of donors, see
Recipes for your own data.
Give raw counts for RNA, ADT and peaks. The methods normalise the data themselves. The folder stores each matrix dense, so drop genes or peaks you do not need first.
With ATAC, set atac_kind= to "peak" or "gene_activity". Most methods
read one of the two, and method_info(m)["atac"] names it. MultiMAP and
Seurat_v3 read both. scan and run skip a method whose file holds
the other one, also when methods= names it.
For CITE-seq, section 5 of the RNA + ADT tutorial writes the folder. Other layouts:
import mudata as md # pip install mudata
mdata = md.read_h5mu("my_multiome.h5mu")
# labels in mdata.obs; use rna:celltype for mdata["rna"].obs
mtb.io.export_dataset(mdata, "data/MYMULTIOME", rna="rna", atac="atac",
atac_kind="peak", labels="obs:celltype",
category="vertical")
mtb.scan("data/MYMULTIOME", "vertical", modalities=["rna", "atac"])
# on Linux, Windows or macOS: the RNA + ATAC methods; batch = each cell's sample
res = mtb.run("data/MYMULTIOME", "vertical", out_dir="out/",
modalities=["rna", "atac_peak"], batch=mdata.obs["sample"])
Keep all samples in one folder and pass each cell's sample as batch=.
atac_peak keeps the methods that read RNA and ATAC peaks.
import anndata as ad
rna = ad.read_h5ad("lung_rna.h5ad")
atac = ad.read_h5ad("lung_atac.h5ad") # other cells than rna
mtb.io.export_dataset(rna, "data/LUNG", atac=atac, atac_kind="peak",
labels="obs:cell_type", category="diagonal")
peak_methods = mtb.find_methods("diagonal", modalities=["atac_peak"])
mtb.scan("data/LUNG", "diagonal", methods=peak_methods)
scan checks the four methods that read peaks. With peaks alone, only
GLUE passes the file check. To add gene activity, see
My ATAC is peaks only.
A mosaic or cross dataset can arrive as one file per batch. Here
cite, multiome and rna_only hold one batch each.
kw = dict(labels="obs:cell_type", category="mosaic")
mtb.io.export_dataset(cite, "data/LAB", adt="obsm:protein",
batch_index=1, **kw)
mtb.io.export_dataset(multiome, "data/LAB", rna="rna", atac="atac",
atac_kind="peak", batch_index=2, **kw)
mtb.io.export_dataset(rna_only, "data/LAB", batch_index=3, **kw)
mtb.scan("data/LAB", "mosaic")
batch_index=N writes the whole object as batch N. Number the batches
to match a pattern the methods read, as
Dataset folder layout lists them. For cross, write each donor file the same way with
category="cross". One file with a donor column is split by batch=:
mtb.io.export_dataset(adata, "data/MYCITE3", adt="obsm:protein",
labels="obs:celltype", batch="obs:donor",
category="cross")
The cross methods for RNA + ADT read batches 1-3. Dataset folder layout lists every batch pattern.
Details: files and labels
mtb.io.read reads a file of the folder back as an AnnData.
A label file is a one-column CSV: a header line x, then one label
per cell, for example
pd.Series(labels, name="x").to_csv(path, index=False).
A 10x Multiome AnnData read with gex_only=False holds genes and
peaks in one X. Select each part with a feature filter:
rna="X[feature_types=Gene Expression]" and
atac="X[feature_types=Peaks]". Without a filter on such data,
export_dataset raises ValueError. The recipe
A 10x .h5 file reads the file and splits it.
Running an export again raises FileExistsError for the files already
in the folder. overwrite=True replaces them. A call that fails
writes nothing. An export also raises FileExistsError when the
folder holds files of other data that it does not write: files with
other cells, and after batch= files of other batch numbers. With
batch_index=, a call checks its own batch number. The message names
the first file. overwrite=True does not remove them. Write data with
fewer batches into a new folder.
mtb.scan checks the folder for each method. Its caveat column
flags values that are not whole numbers. When an expected file is
missing, scan names the file it found instead.
Details: ATAC
The ATAC file name depends on the category. Vertical reads atac.h5.
method_info(m)["atac"] names the form a method reads. Diagonal reads
atac_peak.h5 for peaks and atac_gas.h5 for gene activity. Mosaic
reads atac<i>.h5. The older name atac.h5 for gene activity still
works.
GLUE and Seurat_v3 read peak names such as chr1:100-200. scan and
run also skip these two methods when the peak names cannot be
rewritten in that form, such as peak_0. With such names, scan
cannot tell peaks from gene activity. The other methods run, with a
caveat that names the file. scBridge reads the whole folder, and scan
does not check its ATAC file. For a method skipped for its ATAC file,
export the ATAC form it reads.
Recipes for your own data¶
Each task tutorial has a section "Your own data". It reads .h5ad files
that hold the cell types in obs, writes a dataset folder and runs the
methods on it. These recipes cover four other starting points. Run them in
the tutorial notebook of your task. There mtb is imported, and METHODS
is the list of methods from its section 2. Its check cell and run cell
use NAME, the folder name that a recipe sets. The recipes write to
mydata/ and call the cell-type column celltype, as the tutorials do.
A 10x .h5 file¶
Cell Ranger writes filtered_feature_bc_matrix.h5. One file holds one
sample of cells measured in two modalities, which is the data of a
vertical task. scan and run do not read this
file. Read it with scanpy first.
import scanpy as sc
adata = sc.read_10x_h5("filtered_feature_bc_matrix.h5", # <- your file
gex_only=False)
print(adata.var["feature_types"].value_counts())
gex_only=False keeps the peaks or the antibody counts. Without it, scanpy
returns the genes alone. The printout lists the feature types of the file.
A 10x Multiome file holds Gene Expression and Peaks, and a CITE-seq
file holds Gene Expression and Antibody Capture.
scanpy warns that variable names are not unique, here and in the next
steps. Do not call var_names_make_unique on adata: on the whole object
it also renames an antibody that has the name of a gene. The split below
calls it on the genes only.
The file holds no cell types. Add them to adata now, before the split,
with the blocks of Labels in a CSV file, on adata
whatever the tutorial, then come back here for the split. Without
cell types, leave the labels= line out of the export below, as in
No labels.
Split adata into one AnnData per modality and write the folder. For
10x Multiome, in the vertical RNA + ATAC tutorial:
kind = adata.var["feature_types"]
rna = adata[:, kind == "Gene Expression"].copy()
rna.var_names_make_unique() # the gene names only
atac = adata[:, kind == "Peaks"].copy()
NAME = "MY10X" # <- a name for your data
mtb.io.export_dataset(rna, "mydata/" + NAME,
rna="X",
atac=atac,
atac_kind="peak",
labels="obs:celltype",
overwrite=True)
For CITE-seq, in the vertical RNA + ADT tutorial:
kind = adata.var["feature_types"]
rna = adata[:, kind == "Gene Expression"].copy()
rna.var_names_make_unique() # the gene names only
adt = adata[:, kind == "Antibody Capture"].copy()
NAME = "MY10X" # <- a name for your data
mtb.io.export_dataset(rna, "mydata/" + NAME,
rna="X",
adt=adt,
labels="obs:celltype",
overwrite=True)
When export_dataset says that a matrix holds 0 features, compare the type
name in the split with the feature types printed above. A type that is not
printed is not in adata: read the file again with gex_only=False.
Then run the check cell and the run cell of that tutorial. They use
NAME, so they work on this folder:
For 10x Multiome the check lists atac.h5 and atac_peak.h5: one matrix
under two names, one copy on disk.
Each matrix is stored dense and read dense: features × cells × 8 bytes of
memory. 150,000 peaks and 10,000 cells take 12 GB. Keep the genes and
peaks you need before the export. export_dataset raises ValueError
when a modality would hold two feature types, for example rna="X" on the
object before the split.
Labels in a CSV file¶
export_dataset reads the cell types from obs, not from a file. Read the
CSV with pandas and add the column. The barcodes in the CSV must be unique:
a duplicated barcode stops the block with the pandas error
cannot reindex on an axis with duplicate labels. For diagonal data, do
the recipe once for rna and once for atac.
In the code, adata is the object that holds your cells. In a tutorial
that reads one object, it is adata. In the RNA + ATAC tutorials it is
rna, and the name adata does not exist there: write rna for adata
in every line of this recipe. With another name, such as adt or rna1,
write the object whose cells the CSV lists.
import pandas as pd
ann = pd.read_csv("annotation.csv", index_col=0) # <- your file, barcodes first
lab = ann["cluster_annotation"].astype(str).str.strip() # <- your column
adata.obs["celltype"] = lab.where(~lab.isin(["", "NA", "nan"]))
print(adata.obs["celltype"].notna().sum(), "of", adata.n_obs, "cells have a label")
pandas matches the rows by barcode, so the CSV may be in any order. The
third line turns a blank label and the text NA into a missing value,
because a label file reads them as missing. When export_dataset names
another such text, add it to that list.
The last line counts the cells that got a label. When it prints 0, or
fewer cells than you expect, stop here. Some barcodes of the CSV are then
spelled differently from adata.obs_names, often by an ending such as
-1. Compare ann.index[:3] with adata.obs_names[:3], and do not go on
until the number is right.
When the barcodes of the CSV lack the -1 ending that the cell names have,
put the next line into the block above, after the line that reads the CSV,
and run the block again. The line that reads the CSV comes first, so that a
second run does not add the ending twice.
A cell without a label cannot be scored, and export_dataset refuses it.
When some cells have no label, keep the labelled cells.
A second object of the same cells, such as atac, needs the same cells:
atac = atac[adata.obs_names].copy(). Then go on with the export cell and
labels="obs:celltype".
No labels¶
Cell types are used for the scores. A few methods also read them as an
input. The check names those methods: their reason says
cty.csv is missing. The label file that it names depends on the layout.
For diagonal data it is rna_cty.csv, for some methods together with
atac_cty.csv: the reason then holds rna_cty.csv is missing. or
rna_cty.csv, atac_cty.csv are missing. Without cell types, leave labels=
out of the export and pass evaluate=False to run.
The two blocks below are those of the vertical RNA + ADT tutorial. In
another tutorial, do the same by removing the labels= line of that
tutorial's export cell, and pass evaluate=False in its run cell.
NAME = "MYDATA" # <- a name for your data
mtb.io.export_dataset(adata, "mydata/" + NAME,
rna="X", # <- where the RNA counts are
adt="obsm:protein", # <- where the proteins (ADT) are
overwrite=True)
mtb.scan("mydata/" + NAME, "vertical", methods=METHODS)
mine = mtb.run("mydata/" + NAME, "vertical", methods=METHODS,
out_dir="out/" + NAME, evaluate=False)
mine.summary[["method", "status"]]
The status ran means that the method ran and nothing was scored.
skipped marks a method that needs cell types as an input. Each method
that ran has a folder in out/MYDATA/. For most methods the embedding is
the file embedding.h5 in it. Without evaluate=False the methods run as
well, and the status ran says that nothing was scored.
Diagonal data with cell types for the RNA cells only is the same case.
run scores a diagonal folder only when both sides have labels. Leave
labels= out and pass evaluate=False. No score compares methods on such
data: scores of the RNA cells alone do not change with where a method puts
the ATAC cells.
When you have cell types later, add them to the folder and score the saved
embeddings. No method runs again. Here adata is the object whose cells
the folder holds. In the vertical RNA + ATAC tutorial it is rna.
mtb.io.export_dataset(adata, "mydata/" + NAME, rna=None, labels="obs:celltype",
overwrite=True)
mine = mtb.load_run("out/" + NAME).rescore()
mine.save() # the new scores into out/MYDATA
mine.summary
export_dataset writes the labels in the cell order of the files already
in the folder, whatever the row order of adata is by then. It refuses
labels for other cells.
This line is for a folder without batch numbers that holds one set of
cells, as the vertical tutorials write it. For cross, mosaic and diagonal
data it does not apply: run the export cell of your tutorial again with its
labels= line, and then the rescore line above. The files keep their
cell order. Diagonal data needs the column in both files.
With cell types for some of the cells only, keep the labelled cells before
the export, as in the recipe above. evaluate raises when a label is
missing: score an embedding on its labelled cells, as
mtb.evaluate(emb[emb.obs["label"].notna()], labels="label").
Another number of donors¶
A batch is a donor, a sample or an experiment. A cross method reads a fixed
number of batches: three for RNA + ADT and for RNA + ATAC, and two for
ADT + ATAC and for RNA + ADT + ATAC.
Dataset folder layout lists them. With another number of donors,
treat the data as a vertical task: keep all cells in one folder and give
run the donor of each cell.
Do this in the vertical tutorial of your modalities. Its METHODS are
vertical methods, and its section 2 installs their environments.
ADT + ATAC has no vertical task: for that data, keep two donors, as
shown further down, with two names in the list.
NAME = "MYDONORS" # <- a name for your data
mtb.io.export_dataset(adata, "mydata/" + NAME,
rna="X", # <- where the RNA counts are
adt="obsm:protein", # <- where the proteins (ADT) are
labels="obs:celltype", # <- the cell-type column
overwrite=True) # no batch= here
mine = mtb.run("mydata/" + NAME, "vertical", methods=METHODS,
out_dir="out/" + NAME,
batch=adata.obs["donor"]) # <- your donor column
mine.summary
The methods do not read batch=. The summary then has the batch
metrics ASW_batch, ARI_batch, NMI_batch, GC and iLISI, which say
how well the donors mix. batch= takes the column itself, not its name. run reads
the folder, and the folder holds no obs columns. The values are matched to
the cells by barcode. Every cell needs a donor: run stops when one is missing.
For diagonal data with another number of donors than 3, do the same in the
diagonal RNA + ATAC tutorial. Export rna and atac with all their cells,
without batch_index=, and give run the donor of every RNA and ATAC cell:
import pandas as pd
mine = mtb.run("mydata/" + NAME, "diagonal", methods=METHODS,
out_dir="out/" + NAME,
batch=pd.concat([rna.obs["donor"], atac.obs["donor"]])) # <- your column
To use the cross tutorial instead, keep as many donors as its methods read. First print the donor names of your file. This block changes nothing.
Write as many of the printed names as the methods read into the list of the
next block, in place of A, B and C. Then run it.
When none of the names is in the file, no cell is kept, and the export stops:
The data holds 0 cells, so there is nothing to write. When one name is
wrong, the cells of the other donors are kept, and the check names the
files of the batch that is not there.
When the list holds a name that is not in the file, adata ends with fewer
cells or with none, and a corrected list does not bring them back. Run the
read cell of the tutorial again to get the data back, and then this block.
In a cross folder with more batches than a method reads, the method reads
the first ones, and the check says so:
StabMap reads batches 1-3 of 5. Batches 4-5 are not used. With fewer, it
names the files that are not there: rna3.h5, adt3.h5 are missing. The
batches are numbered in the sorted order of the donor names, read as text,
so 10 comes before 2.
GPU or CPU?¶
Most methods also run without a GPU, only slower. Six need an NVIDIA
GPU. On a host without one, scan and run mark them not runnable.
[m for m in mtb.find_methods() if mtb.method_info(m)["gpu"] == "required"]
# ['scBridge', 'moETM', 'SMILE', 'sciCAN', 'UnitedNet', 'iPOLNG']
mtb.method_info("scJoint")["cpu_params"]
# {'use_cuda': ''}
Details
scJoint and scMDC use CUDA unless told otherwise. On a host without a
GPU, run adds the flags in method_info(m)["cpu_params"] and
prints one line to stderr. A key you pass in params wins.
Six torch environments, env_sciPENN, matilda, scmb_scjoint,
scmb_torch, scmb_scmm2 and scmb_multigrate2, also come as smaller
CPU-only builds.
mtb.env.install picks them on a host without a GPU, and
flavor="cpu" or "gpu" overrides that. scMM never uses a GPU, so
scmb_scmm2 comes as its CPU build on every host. Both builds
install under the environment's own name, and the flavor column of
mtb.env.install(..., dry_run=True) reports the installed one.
Next: score an embedding of your own, then draw the bubble table.