Run a method¶

scMultiBench runs the original, unmodified method scripts from the reference repository, it never edits them. mtb.run reads a method's declaration from the registry, picks the right variant from the inputs you pass, builds the exact command line, runs it inside that method's own conda environment, and loads the typed output back into Python.

TL;DR

import multibench as mtb

res = mtb.run(
    "SCALEX", "diagonal",
    inputs={
        "rna": "data/D35/rna1.h5",
        "atac_gas": "data/D35/atac_gas1.h5",
    },
    out_dir="runs/scalex_d35",
)
res.output.shape   # -> (13914, 10)  joint embedding (n_cells x latent dims)

The run model¶

Each method is declared once in the registry (multibench/engine/methods.yaml) as a MethodSpec. A spec lists the categories and tasks the method supports, the conda env it needs, and one variant per (category, modality-set) it can handle. A single mtb.run call walks this path:

Look up the spec for method and collect the modality roles from your inputs (auxiliary roles like data_dir are ignored here).
Select the variant whose when: matches (category, modalities). The match is exact on the set of modality roles, {"rna", "atac_gas"} selects the diagonal RNA + gene-activity variant, not the vertical one.
Convert inputs to the canonical scMultiBench .h5 (unless convert=False) into a per-run inputs/ folder.
Build the command — [interpreter, entrypoint, *args, *params] — with the entrypoint resolved against the reference repo_path.
Wrap and execute via cmd_template (default conda run -n <env> {cmd}) in the per-run out_dir working directory.
Load the output by its declared kind (e.g. an embedding ndarray) into a RunResult.

Each method needs its own conda environment

scMultiBench orchestrates per-tool environments, it does not install the methods for you. The env each method needs is declared in the registry, for SCALEX it is scmb_torch_v2 (see mtb.method_info("SCALEX")["env"]), and it must exist on your machine (conda env list to check). mtb.run shells out with conda run -n <env> ...; if the env is missing, the subprocess fails and mtb.run raises a RuntimeError carrying the captured stderr tail.

Inputs: roles and format flexibility¶

inputs is a dict mapping each modality role to a file path. Roles are the method's own argument names from the registry, for SCALEX's diagonal variant they are rna and atac_gas (gene-activity scores derived from ATAC).

You don't have to pre-convert your data. By default mtb.run routes every modality input through mtb.io.to_canonical, which adapts common formats to the canonical scMultiBench .h5 (HDF5 with matrix/data as features × cells, matrix/features, matrix/barcodes):

Input you pass	Handled by `to_canonical`
`.h5ad`	`anndata.read_h5ad`
`.csv` / `.tsv`	parsed to a matrix
`.loom`	`anndata.read_loom`
in-memory `AnnData`	used directly
already-canonical `.h5`	passed through unchanged

# both inputs become canonical .h5 before the method sees them; the demo D35
# files are already canonical, so to_canonical passes them through unchanged
mtb.run(
    "SCALEX", "diagonal",
    inputs={
        "rna": "data/D35/rna1.h5",
        "atac_gas": "data/D35/atac_gas1.h5",
    },
    out_dir="runs/scalex_d35",
)

Pass convert=False when your inputs are already in the exact layout the method expects and you want scMultiBench to hand the paths through verbatim:

mtb.run(..., convert=False)   # no format adaptation; paths used as-is

You can also convert by hand. to_canonical writes a fresh canonical .h5 for formats that need adapting (e.g. .h5ad), and returns the input path unchanged when it is already canonical; read_canonical is the inverse, canonical .h5 -> AnnData:

from pathlib import Path

# a real .h5ad would be written to out; a canonical .h5 is returned as-is
path = mtb.io.to_canonical(
    "data/D35/rna1.h5",
    out=Path("inputs/rna.h5"),
)
adata = mtb.io.read_canonical(path)   # canonical .h5 -> AnnData (cells x genes)

Inspecting variants before you run¶

Use mtb.method_info to see which categories, tasks, environment, and variant entrypoints a method declares, this tells you the modality roles each variant expects.

mtb.method_info("SCALEX")
# {'id': 'SCALEX', 'language': 'python', 'categories': ['diagonal'],
#  'tasks': ['clustering', 'batch'], 'env': 'scmb_torch_v2', 'atac': 'gene_activity',
#  'needs_labels': False, 'status': 'verified', 'setup_hint': '',
#  'variants': ['tools_scripts/SCALEX/main_SCALEX.py']}

If the (category, modalities) you pass has no matching variant, mtb.run raises a KeyError that lists every (category, modality-set) the method does support, so the fix is usually to adjust your inputs keys.

status tells you what is wired

In v1, methods with status: "verified" have had their commands

Every registry method is wired and run-verified: its command template has been cross-checked against the real entrypoint and executed end-to-end on a reference dataset.

Validated example: SCALEX diagonal integration on D35¶

This is a real, GPU-validated run. SCALEX performs diagonal integration (unpaired modalities bridged by shared features): an RNA batch (rna1) and an ATAC batch summarised as gene-activity scores (atac_gas1) from dataset D35.

run_scalex_d35.py

import multibench as mtb

res = mtb.run(
    "SCALEX",
    category="diagonal",
    task="clustering",
    inputs={
        "rna": "data/D35/rna1.h5",
        "atac_gas": "data/D35/atac_gas1.h5",
    },
    out_dir="runs/scalex_d35",
)

print(res.output.shape)   # -> (13914, 10)   joint embedding (cells x latent dims)
print(res.cmd)            # the exact argv that ran, including the conda wrapper

Use absolute paths for inputs and out_dir

The method script runs with out_dir as its working directory (step 5 above), and out_dir is also passed to the script as its save path. So a relative out_dir (or relative inputs) is resolved against the subprocess cwd, not your shell's cwd, which makes the outputs land in the wrong place. Pass absolute paths for both the modality inputs and out_dir (as above) so everything resolves regardless of where the subprocess runs. No cmd_template is needed: the default conda run -n {env} {cmd} fills {env} from the spec (scmb_torch_v2 for SCALEX), so the run lands in the right conda env automatically.

mtb.run returns a RunResult dataclass:

Field	Type	What it holds
`method`	`str`	the method id you ran
`out_dir`	`Path`	the per-run working directory
`cmd`	`list[str]`	the exact wrapped command line that executed
`output`	`object`	the primary loaded output, here an embedding `ndarray`
`extra`	`dict`	any additional declared outputs, keyed by filename
`stdout` / `stderr`	`str`	captured subprocess streams

res.output            # numpy ndarray, shape (13914, 10), the joint embedding
res.extra             # {} for SCALEX (no extra outputs)
res.cmd[:3]           # [<conda>, 'run', '-n']  -> the cmd_template wrapper

The embedding flows straight into evaluation and plotting (labels and batch are the per-cell ground-truth and batch vectors for the dataset, see Evaluate a run):

metrics = mtb.evaluate(res.output, category="diagonal", task="clustering",
                       labels=labels, batch=batch)

Choosing a `cmd_template`¶

cmd_template decides how the built command is executed; {cmd} is the placeholder for the argv. The default targets the method's declared env:

cmd_template = "conda run -n {env} {cmd}"   # default; {env} filled from the spec

Override it to run elsewhere, e.g. an explicit env name, a Singularity container, or an HPC scheduler wrapper:

Explicit conda envSingularity / ApptainerRun in the current env

mtb.run(..., cmd_template="conda run -n scmb_torch_v2 {cmd}")

mtb.run(..., cmd_template="singularity exec multibench.sif {cmd}")

mtb.run(..., cmd_template="{cmd}")   # no wrapper, use the active interpreter

repo_path points at the checkout of the reference scMultiBench method scripts; it defaults to config.DEFAULT.repo_path. The variant entrypoint (e.g. tools_scripts/SCALEX/main_SCALEX.py) is resolved relative to it.

Reading other output kinds¶

The primary output is loaded by its declared kind. embedding / imputed / markers / graph outputs come back as numpy arrays; labels as a list of strings; coords as a path you handle yourself. Any additional files a variant declares land in res.extra, keyed by filename.

Graph-output methods do not return an embedding

A few methods emit a cell-graph rather than a coordinate embedding, notably scMoMaT, MIRA, and Seurat_WNN. For these, res.output is the graph representation, not a (cells × dims) embedding, and downstream steps that assume an embedding (e.g. ASW-style metrics) will not apply directly. Check mtb.method_info(method)["status"] and the variant's output kind before wiring such a run into an embedding-based comparison.

Next: turn a run's output into scIB metrics in Evaluate a run, then render the bubble table.