Mosaic integration¶
Mosaic integration has several batches where only some share a modality - an RNA-only batch, an ATAC-only batch, and a paired batch that bridges them. Mosaic methods are the most layout-sensitive of the four categories: each supports specific per-batch modality patterns, so scan() per dataset is the source of truth for what applies.
This tutorial covers, end to end:
- installing the package and the per-method environments
- the on-disk data layout this category expects
- seeing what runs on a dataset (
scan) and what each method exposes for tuning - running several methods for real and reading their metrics
- the standard figures: a per-dataset panel and an across-datasets summary
- running the same pipeline on your own dataset, demonstrated for real
Reference dataset: D45 (32,151 cells). The stored results shipped
with these notebooks were produced on it, so every table below reproduces.
Contents - 1 Install · 2 Data layout · 3 What can I run · 4 Paper coverage · 5 What can I tune · 6 Run the analysis · 7 Reading the metrics · 8 Figures · 9 Your own dataset · Troubleshooting
1. Install¶
Two layers: the multibench package (~2 MB) and the conda environments of the
methods you run. On Colab, run the first cell and let the kernel restart
once - then just keep running from the next cell.
# Colab ships without conda; this provisions it (the kernel restarts ONCE).
# On a machine that already has conda, this cell does nothing.
import importlib.util, shutil
def _has(mod):
try:
return importlib.util.find_spec(mod) is not None
except ModuleNotFoundError:
return False
if shutil.which("conda") or shutil.which("mamba"):
print("conda available - nothing to do")
elif _has("google.colab"):
!pip -q install condacolab
import condacolab
condacolab.install() # restarts the kernel; afterwards, continue below
else:
print("no conda found - install it first (mamba recommended); see the installation guide")
conda available - nothing to do
import importlib.util, os
if importlib.util.find_spec("multibench") is None:
!git clone --depth 1 https://github.com/DSichang/scMultiBench.git
%cd scMultiBench
!pip -q install -e ".[eval]"
elif os.path.isdir("/content/scMultiBench"):
# reused Colab runtime: refresh the editable install to the latest code,
# then drop the already-imported modules so the NEXT import sees it -
# a live kernel never re-reads changed files on its own
%cd /content/scMultiBench
!git pull -q
!pip -q install -e ".[eval]"
import importlib, sys
for _m in [m for m in list(sys.modules) if m == "multibench" or m.startswith("multibench.")]:
del sys.modules[_m]
importlib.invalidate_caches()
print("multibench refreshed to the latest repository state")
else:
print("multibench already installed")
multibench already installed
Now the environments for the methods this tutorial runs
(StabMap, scMoMaT). env install skips anything already present;
other tiers are one flag away (--category mosaic: ~45 GB (7 envs); no flag:
the whole benchmark, 29 envs / 175 GB).
import sys
# --packed uses prebuilt archives where published (minutes instead of a
# 10-30 min build); envs without one fall back to the lockfile build.
!{sys.executable} -m multibench env install --methods StabMap,scMoMaT --packed --run
scmb_r [have ] <- StabMap scmb_torch [have ] <- scMoMaT
import warnings; warnings.filterwarnings("ignore")
%matplotlib inline
from pathlib import Path
import pandas as pd
pd.set_option("display.max_colwidth", None) # never truncate a `reason`
pd.set_option("display.max_columns", None) # never hide a metric column
pd.set_option("display.width", 200)
import multibench as mtb
DATASET = "D45"
CATEGORY = "mosaic"
RESULTS = Path("results") if Path("results").exists() else Path("notebooks/results")
mtb.data.fetch('D45', 'D46') # reference data (290 MB, 97 MB); no-op when present
print("multibench", mtb.__version__)
multibench 0.2.1
2. The data layout¶
A dataset is a folder of flat files; the folder name is the dataset name.
describe_layout prints the exact filenames for each category:
print(mtb.describe_layout(CATEGORY))
Put your files in <data_path>/<DATASET_NAME>/ , e.g. ./data/MYDATA/
(dataset = the folder NAME; data_path = the folder that CONTAINS it)
LAYOUT FOR MOSAIC:
rna1.h5 rna2.h5 atac2.h5 atac3.h5 <- numbered, one per batch
cty1.csv cty2.csv cty3.csv <- one per batch
(numbered files live in the SAME flat dir; there is no batch column)
Modality roles and the filenames they resolve to:
rna rna.h5 - gene expression
adt adt.h5 - surface protein (CITE-seq antibody-derived tags)
atac atac.h5 - chromatin accessibility
atac_gas atac.h5 - ATAC as GENE-ACTIVITY scores <-- note: plain atac.h5
atac_peak peak.h5 - ATAC as PEAKS <-- note: peak.h5, NOT atac.h5
rna1/rna2/... rna1.h5, rna2.h5, ... - one file per BATCH (mosaic/cross)
adt1/adt2/... adt1.h5, adt2.h5, ... - one file per BATCH (mosaic/cross)
cty cty.csv - cell-type labels, ONE label set (vertical)
rna_cty / atac_cty rna_cty.csv, atac_cty.csv - one label file PER MODALITY, used when RNA and ATAC come from different cells (diagonal)
cty1/cty2/... cty1.csv, cty2.csv, ... - one label file per BATCH (mosaic/cross)
!! ATAC: the role name does NOT guarantee the representation.
atac_gas resolves to atac_gas.h5 if present, otherwise FALLS BACK
to atac.h5 - and in the shipped multiome datasets (D12-D17) atac.h5
contains PEAKS, not gene activity. Only D27/D28 ship a real
atac_gas.h5. Verified by feature names: 12 atac.h5 files are peaks
(chr1:3094772-3095489), 2 atac_gas.h5 files are gene activity.
atac_peak resolves to atac_peak.h5, else peak.h5.
This matters because methods disagree: Portal/SCALEX/iNMF/sciCAN/
Conos/VIPCCA/scJoint need GENE ACTIVITY, while MultiVI/moETM/scMM/
MIRA/scMVP/Seurat_WNN/GLUE need PEAKS. Feeding the wrong one runs
to completion and returns a plausible but WRONG embedding - no error.
Check what you actually have before trusting a cross-dataset result.
MODALITY FILE FORMAT (.h5) - easiest route first:
mtb.io.to_canonical(src, dst) converts an .h5ad and writes
everything below correctly. Prefer it over building the file by hand.
If you do build it yourself, ALL THREE datasets are required:
matrix/data the matrix, stored FEATURES x CELLS
matrix/features one entry per feature (row of matrix/data)
matrix/barcodes one entry per cell (column of matrix/data)
e.g. 2,000 genes x 5,000 cells -> matrix/data has shape (2000, 5000),
matrix/features has 2000 entries and matrix/barcodes has 5000.
NOTE this is the TRANSPOSE of the scanpy/AnnData convention
(AnnData.X is cells x genes). scan() rejects a transposed file, and
a file with only matrix/data fails with a KeyError about 'features'.
Labels are a single-column CSV: one header line (typically 'x'),
then one cell-type label per cell; the evaluator reads the first
column and skips the header.
mosaic: Several batches where only SOME share a modality; a paired batch bridges the others.
ENVIRONMENTS
Every method runs in its OWN conda env (they need mutually
incompatible framework versions). scan() marks a method NOT runnable
if its env is missing, so a sweep never starts one that cannot finish.
multibench env doctor # what is needed / what is missing
multibench env install --run # build them all from lockfiles
Then: mtb.scan('MYDATA') -> mtb.run_all('MYDATA', '<category>', out_dir=...)
The modality files are HDF5 with three required datasets:
| dataset | contents | shape |
|---|---|---|
matrix/data |
the matrix, features x cells | (n_features, n_cells) |
matrix/features |
one name per feature | (n_features,) |
matrix/barcodes |
one id per cell | (n_cells,) |
Note this is the transpose of the scanpy/AnnData convention (AnnData.X is
cells x genes). Two safety nets exist: mtb.io.to_canonical(src, dst) converts
an .h5ad correctly, and scan() rejects a transposed file at preflight instead
of letting a method fail half an hour in.
The label CSV is the one file you author by hand, so its schema in full:
a single column with one header line (typically x) and one label per
cell, in the same order as matrix/barcodes of the matching modality
file(s) - the evaluator reads the first column and skips the header line.
Where a category uses several label files (cty1.csv, rna_cty.csv, ...),
each aligns with its own batch or modality. The next cell prints the head of a
shipped one - this is the whole format:
cty = sorted((mtb.config.DEFAULT.data_path / DATASET).glob("*cty*.csv"))[0]
print(cty.name)
print(*open(cty).read().splitlines()[:4], sep="\n")
cty1.csv "x" "CD4 Naive" "CD4 Memory" "CD4 Naive"
ATAC caution. Methods disagree about the ATAC representation - some need gene-activity scores, others need peaks - and feeding the wrong one runs to completion and returns a plausible but wrong embedding, with no error.
describe_layoutabove states which file resolves where; check what your files actually contain before trusting a result.
From AnnData to canonical, executed. Most real data starts as .h5ad;
mtb.io.to_canonical writes the layout above correctly (including the
transpose). Converting a small demo object end to end:
import importlib.util
if importlib.util.find_spec("anndata"):
import anndata as ad, numpy as np, h5py, tempfile, os
tmp = tempfile.mkdtemp()
demo = ad.AnnData(X=np.random.poisson(2.0, size=(120, 40)).astype(float))
demo.obs_names = [f"cell{i}" for i in range(120)]
demo.var_names = [f"gene{i}" for i in range(40)]
src = os.path.join(tmp, "demo.h5ad"); demo.write_h5ad(src)
dst = os.path.join(tmp, "rna.h5")
mtb.io.to_canonical(src, dst)
with h5py.File(dst) as f:
print("keys :", sorted(f["matrix"].keys()))
print("shape:", f["matrix/data"].shape, "(features x cells - transposed for you)")
else:
print("skipped - anndata not installed here (an evaluation-only dependency;",
"a full `pip install -e .` on your own machine brings it in)")
keys : ['barcodes', 'data', 'features'] shape: (40, 120) (features x cells - transposed for you)
3. What can I run on this dataset?¶
scan inspects the folder and reports every method that can run - and, for the
rest, exactly why not (missing file, missing environment, wrong layout). Nothing
executes, so this is instant and safe.
avail = mtb.scan(DATASET, category=CATEGORY)
avail[avail.runnable][["method", "modalities", "env", "output_kind",
"n_tunable", "runtime_tier"]]
| method | modalities | env | output_kind | n_tunable | runtime_tier | |
|---|---|---|---|---|---|---|
| 0 | Cobolt | rna1+rna2+atac2+atac3 | scmb_cobolt3 | embedding | 2 | slow |
| 1 | MultiVI | rna1+atac3+rna2+atac2 | scmb_multivi | embedding | 0 | very_slow |
| 2 | Multigrate | rna1+rna2+atac2+atac3 | scmb_multigrate2 | embedding | 2 | very_slow |
| 3 | SMILE | rna2+atac2+rna1+atac3 | env_sciPENN | embedding | 0 | slow |
Methods that are not runnable come with a reason rather than a silent absence:
not_ok = avail[~avail.runnable][["method", "modalities", "reason"]]
not_ok.head(5) if len(not_ok) else "(everything in this category runs here)"
| method | modalities | reason | |
|---|---|---|---|
| 4 | Multigrate | rna1+rna2+adt2+adt3 | FileNotFoundError: Multigrate/D45/mosaic: input files not found on disk: {'adt2': '/media/disk2/Sichang/scmbench_pkg/data/D45/adt2.h5', 'adt3': '/media/disk2/Sichang/scmbench_pkg/data/D45/adt3.h5'}. Available files in /media/disk2/Sichang/scmbench_pkg/data/D45: ['D45.zip', 'atac2.h5', 'atac3.h5', 'cty1.csv', 'cty2.csv', 'cty3.csv', 'rna1.h5', 'rna2.h5'] |
| 5 | StabMap | rna1+rna2+rna3+adt1+atac2 | FileNotFoundError: StabMap/D45/mosaic: input files not found on disk: {'rna3': '/media/disk2/Sichang/scmbench_pkg/data/D45/rna3.h5', 'adt1': '/media/disk2/Sichang/scmbench_pkg/data/D45/adt1.h5'}. Available files in /media/disk2/Sichang/scmbench_pkg/data/D45: ['D45.zip', 'atac2.h5', 'atac3.h5', 'cty1.csv', 'cty2.csv', 'cty3.csv', 'rna1.h5', 'rna2.h5'] |
| 6 | scMoMaT | rna1+rna2+rna3+adt1+atac2 | FileNotFoundError: scMoMaT/D45/mosaic: input files not found on disk: {'rna3': '/media/disk2/Sichang/scmbench_pkg/data/D45/rna3.h5', 'adt1': '/media/disk2/Sichang/scmbench_pkg/data/D45/adt1.h5'}. Available files in /media/disk2/Sichang/scmbench_pkg/data/D45: ['D45.zip', 'atac2.h5', 'atac3.h5', 'cty1.csv', 'cty2.csv', 'cty3.csv', 'rna1.h5', 'rna2.h5'] |
4. How much of the paper does this cover?¶
scan() answers "what runs on THIS dataset". A different question: how many of
the methods the paper benchmarks for mosaic does this package wire at all?
Stated explicitly so you never mistake a dataset limitation for full coverage.
from multibench.engine import registry
PAPER = {'vertical': ['totalVI', 'sciPENN', 'Concerto', 'scMSI', 'Matilda', 'MOFA2', 'Multigrate', 'UINMF', 'scMoMaT', 'Seurat_WNN', 'scMM', 'scMDC', 'moETM', 'VIMCCA', 'iPOLNG', 'MIRA', 'UnitedNet', 'scMVP'], 'diagonal': ['scBridge', 'Portal', 'SCALEX', 'VIPCCA', 'Seurat_v3', 'MultiMAP', 'Seurat_v5', 'sciCAN', 'Conos', 'iNMF', 'online_iNMF', 'scJoint', 'GLUE', 'uniPort'], 'mosaic': ['MultiVI', 'scMoMaT', 'StabMap', 'Cobolt', 'UINMF', 'Multigrate', 'SMILE', 'scMM', 'moETM', 'UnitedNet', 'totalVI', 'sciPENN'], 'cross': ['totalVI', 'scMoMaT', 'UnitedNet', 'sciPENN', 'Concerto', 'scMDC', 'StabMap', 'UINMF', 'scMM', 'MOFA2', 'Multigrate', 'PASTE', 'PASTE2', 'SPIRAL', 'GPSA']}
IMPUTATION_ONLY = ['scMM', 'moETM', 'UnitedNet', 'totalVI', 'sciPENN']
paper = PAPER[CATEGORY]
wired = sorted({m for m in mtb.list_methods()
if any(v.when.get("category") == CATEGORY
for v in registry.get(m).variants)})
missing = [m for m in paper if m not in wired]
print(f"paper benchmarks {len(paper)} methods for {CATEGORY}; this package wires {len(wired)}")
if missing:
print("not wired here:", ", ".join(missing))
imp = [m for m in missing if m in IMPUTATION_ONLY]
if imp:
print(" the paper evaluates these only via IMPUTATION, which is not wired:",
", ".join(imp))
else:
print("full parity with the paper for this category")
paper benchmarks 12 methods for mosaic; this package wires 6 not wired here: UINMF, scMM, moETM, UnitedNet, totalVI, sciPENN the paper evaluates these only via IMPUTATION, which is not wired: scMM, moETM, UnitedNet, totalVI, sciPENN
5. What can I tune?¶
params_for reports each method's defaults and, where the upstream script
exposes any, the tunable hyperparameters. An empty tunable is honest: many
upstream scripts hardcode their hyperparameters, and this package never edits
upstream code, so it reports rather than pretends.
# a registry question, so it needs no data on disk
from multibench.engine import registry
rows = []
for m in sorted(mtb.list_methods(category=CATEGORY)):
p = None
try:
p = mtb.params_for(m, CATEGORY)
except Exception: # multi-variant: try each variant's modalities
for v in registry.get(m).variants:
if v.when.get("category") != CATEGORY:
continue
try:
p = mtb.params_for(m, CATEGORY, v.when.get("modalities"))
break
except Exception:
continue
if p is None: # variant selection needs a concrete dataset
rows.append({"method": m, "n_tunable": 0, "tunable": "(see scan() on your dataset)"})
continue
rows.append({"method": m, "n_tunable": len(p.get("tunable") or {}),
"tunable": ", ".join(sorted((p.get("tunable") or {}))[:6])})
pd.DataFrame(rows).sort_values("n_tunable", ascending=False).reset_index(drop=True)
| method | n_tunable | tunable | |
|---|---|---|---|
| 0 | Cobolt | 2 | batch_size, lr |
| 1 | Multigrate | 2 | epochs, lr |
| 2 | MultiVI | 0 | |
| 3 | SMILE | 0 | |
| 4 | StabMap | 0 | |
| 5 | scMoMaT | 0 | |
| 6 | sciPENN | 0 | (see scan() on your dataset) |
| 7 | totalVI | 0 | (see scan() on your dataset) |
6. Run the analysis¶
run_all is the whole pipeline in one call: resolve inputs -> run each method
in its own conda env -> load the outputs -> compute scIB metrics. Here:
StabMap, scMoMaT on D46. Both methods run in minutes to tens of minutes on D46 - run_sec below is the measured time on our host.
(This demo runs on D46, whose layout fits these methods; mosaic layouts vary per dataset - see the note above.)
res = mtb.run_all("D46", CATEGORY,
methods=['StabMap', 'scMoMaT'],
out_dir="/tmp/tutorial_mosaic")
res.summary
[run_all] StabMap (mosaic/D46) ...
/media/disk2/Sichang/.conda/envs/scmbench_eval/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm
[run_all] -> CHAIN_OK (65.2s) 0.574
[run_all] scMoMaT (mosaic/D46) ...
[run_all] -> CHAIN_OK_GRAPH_METHOD (188.4s) 0.2817
| method | status | run_sec | output_kind | emb_shape | n_tunable | label_order | label_order_confidence | ARI | NMI | ASW | iASW | iF1 | cLISI | ASW_batch | GC | iLISI | label_order_note | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | StabMap | CHAIN_OK | 65.2 | embedding | [21416, 50] | 0 | cty1.csv+cty2.csv+cty3.csv | 0.5517 | 0.5740 | 0.7291 | 0.5426 | 0.5313 | 0.4882 | 0.9944 | 0.8100 | 0.9545 | 0.0273 | None |
| 1 | scMoMaT | CHAIN_OK_GRAPH_METHOD | 188.4 | graph | [21416, 2] | 0 | cty1.csv+cty2.csv+cty3.csv | 0.8356 | 0.2817 | 0.5446 | 0.4152 | 0.3471 | 0.3415 | 0.9882 | 0.4944 | 0.4239 | 0.1250 | None |
run_all picked the right label files, resolved the label order (see
label_order in the summary), and scored every embedding. Your first figure -
the methods you just ran, in the paper's panel layout:
res.plot()
7. Reading the metrics¶
Two families, matching the paper's grouping. All are higher = better, on [0, 1] except ARI (can be slightly negative at chance level).
| family | metrics | what they measure |
|---|---|---|
| clustering / bio-conservation | ARI, NMI, ASW, iASW, iF1, cLISI |
does the embedding separate the annotated cell types? |
| batch correction | ASW_batch, GC, iLISI (+ opt-in kBET) |
are the batches mixed within each cell type? |
Notes that save confusion later:
iASW/iF1are isolated-label scores; this benchmark scores every label, so they exist even on a single-batch dataset.- batch metrics appear only when the dataset has real batches - their absence on a single-batch dataset is correct, not missing data.
kBETis opt-in (mtb.evaluate(..., slow_metrics=True)): it is much slower than the other metrics.
8. The figures¶
Section 6 plotted the methods you ran; the same call scales to the whole
field. The stored full-benchmark sweep for D45 (every wired method,
default settings - hours of compute, shipped with the repository so this
reproduces instantly) gives the paper-style panel:
Per-dataset panel, in the paper's layout: methods as rows (best first),
metrics as columns grouped by task family - blues for DR & clustering, greens
for batch correction - each family led by an Overall rank column. Read each
circle as two encodings: its size is the method's rank within that column
(largest = rank 1) and its colour is the metric's value (min-max scaled
within the column, darker = higher); the Overall columns use bar length and
colour the same way. This is the full stored benchmark for D45; to produce the same files for your own
data, mtb.run_all(DATASET, CATEGORY, out_dir=...) without methods= runs
everything runnable and writes summary.csv and long.csv into out_dir.
long = pd.read_csv(RESULTS / "long_all_D45.csv")
fig = mtb.plot.bubble(long)
fig.set_dpi(110)
fig
Across datasets. The summary figure needs every method to have results on
every dataset it is averaged over - otherwise absence and performance get mixed
into the same bar. The results folder therefore ships TWO mosaic datasets swept
with the same benchmark: D45 and D45s (a 60% cell subsample of D45
under a new name). The code keeps the intersection of methods present in
both, so the matrix behind the figure is complete by construction - copy this
pattern with your own same-category datasets. The encoding is the paper's:
each metric's bar is its grand rank - the min-max scaled mean rank across
the datasets - with length and colour both carrying it, and Overall is the
min-max scaled mean rank of the grand ranks, exactly as in the paper's summary
panels. Both metric families appear because both datasets are multi-batch.
a = pd.read_csv(RESULTS / "long_all_D45.csv").assign(dataset="D45")
b = pd.read_csv(RESULTS / "long_all_D45s.csv").assign(dataset="D45s")
both = sorted(set(a.method) & set(b.method)) # complete matrix, by construction
pair = pd.concat([a, b], ignore_index=True)
pair = pair[pair.method.isin(both)]
print(f"{len(both)} methods with results on both datasets")
mtb.plot.bubble(pair, aggregate="summary",
title=f"Summary of 2 mosaic datasets, {len(both)} methods")
4 methods with results on both datasets
9. Your own dataset - for real¶
Everything above used shipped data. This section does what you will actually do:
put files in a folder, point the package at it, and get scored results - executed
here on a dataset the package has never seen (a 60% cell subsample of
D46 under a new name, built with ordinary h5py/pandas code you can
adapt to your own export pipeline).
import os, shutil
import h5py
import numpy as np
import pandas as pd
def subsample_dataset(src_dir, dst_dir, frac=0.6, seed=0):
"""Copy a dataset to a new name, keeping a random fraction of the cells.
Files sharing a cell count get the SAME kept-cell index, so modality files
and their label CSVs stay aligned - which is exactly the property your own
export pipeline must preserve. The output is the canonical layout:
matrix/data as features x cells, plus matrix/features and matrix/barcodes.
"""
rng = np.random.default_rng(seed)
os.makedirs(dst_dir, exist_ok=True)
counts, keep = {}, {}
for fn in sorted(os.listdir(src_dir)):
p = os.path.join(src_dir, fn)
if fn.endswith(".h5"):
with h5py.File(p) as f:
if "matrix/data" in f:
counts[fn] = f["matrix/data"].shape[1] # features x cells
elif fn.endswith(".csv"):
counts[fn] = len(pd.read_csv(p))
for n in set(counts.values()):
k = max(50, int(n * frac))
keep[n] = np.sort(rng.choice(n, size=k, replace=False))
for fn, n in counts.items():
sp, dp = os.path.join(src_dir, fn), os.path.join(dst_dir, fn)
idx = keep[n]
if fn.endswith(".csv"):
pd.read_csv(sp).iloc[idx].to_csv(dp, index=False)
else:
with h5py.File(sp) as f, h5py.File(dp, "w") as g:
grp = g.create_group("matrix")
grp.create_dataset("data", data=np.asarray(f["matrix/data"])[:, idx])
if "matrix/features" in f:
grp.create_dataset("features", data=np.asarray(f["matrix/features"]))
if "matrix/barcodes" in f:
grp.create_dataset("barcodes", data=np.asarray(f["matrix/barcodes"])[idx])
return dst_dir
DATA_ROOT = "/tmp/mydata"
src = mtb.config.DEFAULT.data_path / "D46"
subsample_dataset(src, f"{DATA_ROOT}/MYDATA_mosaic", frac=0.6)
sc = mtb.scan(f"MYDATA_mosaic", category=CATEGORY, data_path=DATA_ROOT)
print(f"{int(sc.runnable.sum())} of {len(sc)} methods can run on MYDATA_mosaic")
2 of 7 methods can run on MYDATA_mosaic
mine = mtb.run_all(f"MYDATA_mosaic", CATEGORY,
methods=['StabMap', 'scMoMaT'],
out_dir=f"{DATA_ROOT}/out_mosaic",
data_path=DATA_ROOT)
mine.summary
[run_all] StabMap (mosaic/MYDATA_mosaic) ...
[run_all] -> CHAIN_OK (36.6s) 0.5597
[run_all] scMoMaT (mosaic/MYDATA_mosaic) ...
[run_all] -> CHAIN_OK_GRAPH_METHOD (137.6s) 0.1995
| method | status | run_sec | output_kind | emb_shape | n_tunable | label_order | label_order_confidence | ARI | NMI | ASW | iASW | iF1 | cLISI | ASW_batch | GC | iLISI | label_order_note | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | StabMap | CHAIN_OK | 36.6 | embedding | [12849, 50] | 0 | cty1.csv+cty2.csv+cty3.csv | 0.5314 | 0.5597 | 0.7285 | 0.5388 | 0.5288 | 0.4869 | 0.9920 | 0.8071 | 0.9588 | 0.0500 | None |
| 1 | scMoMaT | CHAIN_OK_GRAPH_METHOD | 137.6 | graph | [12849, 2] | 0 | cty1.csv+cty2.csv+cty3.csv | 0.7053 | 0.1995 | 0.5267 | 0.3802 | 0.3400 | 0.3259 | 0.9841 | 0.4627 | 0.4544 | 0.1357 | None |
mine.plot()
Both reached CHAIN_OK on this subsample when we ran them; run_sec above is the measured time. Two methods rather than three because that is every wired mosaic method D46's layout admits - the four-method D45 figures in section 8 are where this category's methods are compared side by side.
For your real data the only work is producing the canonical files: export each
modality with mtb.io.to_canonical (from .h5ad) or the h5py pattern above,
write one label CSV per the layout in section 2, and the same three calls -
scan, run_all, plot - do the rest.
Troubleshooting¶
| symptom | meaning | fix |
|---|---|---|
scan says not runnable: input files not found |
a required file is absent | the reason names the exact file and lists what IS in the folder |
scan says env missing |
that method's conda env is not built | multibench env install --run |
... looks like cells x features |
matrix stored transposed | re-export with mtb.io.to_canonical |
| a method FAILs in seconds | wrong input representation or layout | read res.failures.iloc[0]["error"] - the full command line and stderr tail are there |
| a method TIMEOUTs | slow, not broken | raise timeout=; runtime tiers in scan are measured, not guessed |
label_order_confidence low |
several label files fit the cell count | check label_order_candidates in the record |
Note - why is UINMF not in mosaic?¶
Two verified blockers, recorded in the registry: its upstream script derives the
second batch's unshared features from the FIRST batch's object (harmless when
both unshared blocks are the same modality, fatal otherwise), and its two-batch
shape fits no mosaic dataset shipped here. A method scan() reports as runnable
must actually run, so it is deliberately not offered.
Next steps¶
- the other three tutorials: vertical, diagonal, cross
- the hosted interactive explorer: https://shiny.maths.usyd.edu.au/scMultiBench/ - the full benchmark's rankings, browsable without installing anything
SETUP.md- measured install cost and the smallest end-to-end checkmtb.method_info(name)- everything the registry knows about one methodmtb.sweep(...)- one method over a range of one hyperparameter