Python API for developers

This page is for developers who import bootstrapper or change its code. To run Bootstrapper, see the commands and the config files.

This reference covers the bootstrapper package, except the model scripts in bootstrapper/models.

• bootstrapper: bootstrapper, blockwise, catalog, cli, config, configs, derive, discover, evaluate, inputs, params, predict, prepare, refine, segment, styles, train, training, utils, view, write
• bootstrapper.data: data, bbox, convert, mask, merge, paths, scale_pyramid, tissue_mask
• bootstrapper.eval: eval, compute_errors, compute_metrics
• bootstrapper.gp: gp, add_2d_lsds, add_aff_errors, add_lsd_errors, add_lsds, clahe_augment, create_labels, create_mask, custom_grow_boundary, defect_augment, deform_augment, expand_labels, gamma_augment, impulse_noise_augment, obfuscate_labels, reject_sparse_labels, renumber, smooth_augment
• bootstrapper.post: post, blockwise, blockwise.extract_frags, blockwise.watershed_frags, blockwise.waterz_agglom, cc, connected_components, merge_tree, naming, seeds, sqlite, watershed, watershed_mutex, ws

bootstrapper

Bootstrapper makes dense 3D instance segmentations of microscopy volumes from sparse 2D labels.

The bs command is bootstrapper.cli.cli. The package exports prepare, train, predict, segment, evaluate, refine, view and utils as click commands.

bootstrapper.blockwise

Run daisy and volara tasks blockwise, and raise when a block fails.

daisy 1.x counts a failed block as done. run_blockwise and run_volara_task check the task states and raise instead, and Monitor prints one short line per task.

• volara_log_dir(seg_dataset_prefix): Return <container>_volara_logs/<prefix in the container> beside the container. Use a hashed name in the working directory for paths with ':' or '='. Daisy cannot pass these characters to workers.

• check_task_states(task_states): Raise RuntimeError when a task has failed or orphaned blocks.

• class Monitor(server, names), a subclass of CLMonitor: A daisy monitor that shows the progress bars under short task names and prints one line per task in place of daisy's Execution Summary. When a block fails or is orphaned, Monitor prints daisy's full summary.

  • on_task_start(task_id, task_state)
  • on_acquire_block(task_id, task_state)
  • on_release_block(task_id, task_state)
  • on_task_done(task_id, task_state)
  • on_block_failure(block, exception, context)
  • on_server_exit()

• run_blockwise(tasks, multiprocessing=True, name=None): Run daisy.run_blockwise, but raise if any block failed or was orphaned. name, if given, replaces the printed task id only when the expanded task list contains one task.

• run_volara_task(task, multiprocessing=True, name=None): Run volara's BlockwiseTask.run_blockwise, but raise if any block failed.

  Drop cached blocks before each run. Otherwise, volara skips completed blocks and can leave stale output after the database resets.

bootstrapper.catalog

The models, what each reads and gives, and the chains that make one round.

• class Model(name: str, short: str, inputs: tuple, outputs: tuple, shipped: bool): One model, with its name and short form, what it reads and gives, and whether a checkpoint ships.
  • reads_raw
• download_fix(name, round_path=None): Return the commands that download the checkpoint of the shipped model name.
• parse_chain(text): Parse models by name or short form, split by spaces, commas or arrows. Case does not matter.
• check_round(chain): Refuse a chain unless it starts from raw, gives each model what it reads, and ends in 3D affinities.
• valid_chains(): Return every chain that check_round accepts, in name order.
• default_chain(labels_2d): Return the default chain. 2D labels train 2d_mtlsd for the shipped 3d_affs_from_2d_mtlsd, and 3D labels train 3d_mtlsd.
• describe(chain): Return the chain joined by arrows, with a note for each model that reads only part of the model before it. Example: 2d_mtlsd → 3d_affs_from_2d_lsd; 3d_affs_from_2d_lsd reads only the 2d_lsds.

bootstrapper.cli

This module defines the bs command group and bs run.

bs run runs the steps of a round in file order. Without --only or --from, bs run skips the leading steps of a directory round that are done and unchanged, but repeats every selected step of a single-file round.

Each run that starts a step writes a receipt into runs/. The entry point is cli.

• class CommandGroup, a subclass of click.Group: The bs group lists commands in round order and accepts short aliases (prep, pred, infer, seg, eval). It logs unexpected errors to a file.
  • list_commands(ctx)
  • get_command(ctx, cmd_name)
  • invoke(ctx)
• cli: the click command of bs.
• run: the click command of bs run.
• done(path, todo): Count the leading steps of todo that are done. A step is done when its newest receipt in path/runs reports it done and no earlier step has run since. Its keys must also match the receipt, and every output path must exist.
• refuse_stale(path): Refuse a round whose run tails differ from the ones bs prepare derives now.
• plan(path, todo, volumes): Return the dry-run lines, which say what each step reads and writes. Refuse a read that is not there and that no earlier step writes.
• write_receipt(path, notes, volumes, steps): Write a round file of the steps that ran, each with the keys it read, below one note per step.
• volume_ops(step, config): Return the operations for a volume. Include the mask operation only when the volume has a target mask.
• run_refine(ctx, step): Run a refine step's ops in order on each volume. Each op reads the output of the op before it. A volume's op table, such as [run.refine.<volume>.mask], merges over the step's.

bootstrapper.config

A config file contains one step or a whole round. [volumes.<name>] describes the data; [train.<name>], [predict.<name>], [segment], [segment.<name>], [refine] and [evaluate] are steps. A step needs a name only when its kind has several steps. Below your keys, a run tail of [run.<step>] tables contains the keys bs prepare derives.

load and load_round read the steps of a file or a round, and load_volumes reads its volumes.

• class Volume(name: str, keys: dict): One [volumes.<name>] table, with the volume's name and its keys.
  • array(ref): Return the path under ref when ref is one of this volume's array keys. Otherwise ref is a path, and array returns it as it is.
• class Step(kind: str, name: str, keys: dict, volumes: dict = field(default_factory=dict)): One step table, with its kind, its name, your keys over its run tail, and its per-volume tables.
  • label
• load(path, tail=True): Return the steps of a file, each over its run tail. With tail=False, return your keys alone.
• load_round(path, tail=True): Return (file, steps) pairs in run order, for the numbered files of a folder or for one file.
• numbered(path): Return (number, file, step) for each step of a round in run order, numbered from 01.
• matches(sel, n, f, s): Return True when sel is the step's number, kind or kind.name, or a prefix of its file name.
• prepared_round(path): Return the round containing path when bs prepare wrote its run tail. Return None for files without a run tail. For a step file without volumes and with a volumes.toml beside it, the round is its folder.
• load_volumes(path): A file's volumes come from its [volumes.<name>] tables, or from the volumes.toml beside it when the file has none. A glob makes one volume per match, named after the match's container. The matches of a volume's globs pair up in sorted order.
• volume_name(path, taken): Make the table name from the container's file name, with dashes for dots. training.zarr gives training-zarr, and a second volume gets training-zarr-2.
• steps_of(path, kind, name=None): Return the steps of one kind in a file, or the one step called name.
• runs(step): Return (volume name, config) pairs, or (None, config) for a step without per-volume tables. Merge each volume's tables, such as params and db, key by key over the step's.
• paths(config, key): Yield the paths under key in config, in its tables and in its arrays of tables.
• ops_beside(keys): Return a refine table's ops beside its keys, as a file stores them.
• with_tail(step, tail): Return the step over its run tail. Per-volume settings override step settings, which override values in the run tail.
• unset(table, *above): Return the keys of table that no table in above sets. A sub-table in both keeps only the keys that it alone has.
• refuse_unknown(where, keys, valid, hint=''): Refuse the first key not in valid. The error suggests the nearest valid key, or lists the valid keys.

bootstrapper.configs

Model setups and their checkpoints.

MODEL_DIR contains the model scripts, and MODEL_URLS gives the download URL of each shipped checkpoint. The functions copy a model's scripts into a setup and find or download its checkpoints.

• checkpoint_iteration(checkpoint): Return the training iteration in a checkpoint name, or None if the name has none.

  model_checkpoint_5000 and model_checkpoint_5000.ckpt both give 5000.

• output_level(checkpoint, chain_str=''): Return the group that predict writes its outputs in: <iteration>, or <iteration>--from--<chain_str>.

• find_checkpoints(setup_dir): Return the checkpoints of a setup by iteration, including those one level down.

• copy_model_scripts(model_name, setup_dir, style='train', cli_edit=True): Copy a model's scripts, without checkpoints, into setup_dir. With cli_edit, offer to edit net_config.json.

• download_checkpoints(model_name, setup_dir, style='prepare'): Download and unzip the shipped checkpoint of model_name into setup_dir.

• checkpoint_path(setup_dir, checkpoint): Find a checkpoint by path, with an optional .ckpt suffix. Also check the setup and one level below it for the same iteration. Return None when there is none.

bootstrapper.data

Array utilities: the commands of bs utils, and the masks bs prepare writes.

bootstrapper.data.bbox

bs utils bbox: crop an array to the bounding box of its positive voxels.

• bbox: the click command of bs utils bbox.

bootstrapper.data.convert

bs utils convert: convert a 3D image or a directory of 2D images to a Zarr array.

• read_from(in_path)
• convert: the click command of bs utils convert.

bootstrapper.data.mask

bs utils mask, and the training masks that bs prepare writes.

mask writes a mask of an image (nonzero voxels, closed in xy) or of labels (labels > 0). training_mask returns the path of a volume's training mask, and write_training_mask writes that mask.

• make_raw_mask(in_ds, out_ds, block): Write one block of an image mask from nonzero voxels. Close twice in xy with a disk of radius 10.
• make_obj_mask(in_ds, out_ds, block): Write one block of a labels mask, which is 1 where labels > 0.
• training_mask(volume, out_array): Return the path of the training mask that the volume's training_mask selects. Return out_array for "everywhere", or for "labels" grown by mask_dilation voxels. Return an explicit mask path unchanged, or None for the trainer's labels > 0 mask.
• write_training_mask(volume, out_array): Write a volume's training mask at out_array. With "labels", the mask is labels > 0 grown by mask_dilation voxels, in xy only with 2d_labels. With "everywhere", the mask is ones over the whole labels array, or ones on the labelled sections with 2d_labels.
• mask: the click command of bs utils mask.

bootstrapper.data.merge

bs utils merge: apply the merges of a LUTs file to a segmentation, blockwise.

• quick_merge_block(in_ds, out_ds, luts, block): Write one block of the merged segmentation, in which each id in a merge group becomes the group's id.
• merge: the click command of bs utils merge.

bootstrapper.data.paths

The output path of a mask of an input array.

• mask_dataset_path(in_array, out_array=None): Resolve the output path of the mask of an input array.

  For a scale-level input with out_array=None, use the same level name in a separate mask array. For example, raw/s0 gives raw_mask/s0.

  Args:
      in_array (str): Path to the input zarr array.
      out_array (str, optional): The path the user gives. None derives it from in_array.
  
  Returns:
      str: Path to the output mask array.
  
  Raises:
      click.ClickException: If the output path is the input array.

bootstrapper.data.scale_pyramid

bs utils scale-pyramid: write coarser or finer scale levels of an array, blockwise.

The command samples label arrays and never averages them. It averages image arrays to downscale and interpolates them to upscale.

• is_label_array(name, dtype): Return True for uint32, uint64, or a basename containing label, lbl, ids, mask or seg (case-insensitive).

  Averaging id 5 and id 9 gives id 7, which may belong to an unrelated object. The name check also detects masks with narrower dtypes, such as uint8.

• scale_block(in_array, out_array, factor, mode, labels, block): Write one block of a rescaled level. Sample ids for labels, and average or interpolate an image.

• scale_array(in_array, out_array, factor, write_size, mode, labels): Rescale in_array by factor into out_array with daisy, in blocks of write_size.

• parse_factor(value): Parse comma- or space-separated integers into a tuple.

• scale_pyramid: the click command of bs utils scale-pyramid.

bootstrapper.data.tissue_mask

bs utils tissue-mask: a tissue mask with no holes, for bs refine mask.

• tissue_counts(raw, cz, z0): The nonzero voxel count per section in the chunk that starts at z0.
• neighbours(src, z, bad): The union of raw > 0 on the nearest sections below and above z that are not in bad.
• hull(m): The convex hull of a 2D mask, joined with the mask. An empty mask stays empty.
• envelope(m, r): A 2D mask closed with a disk of radius r, with its holes filled.
• mask_block(raw, out, bad, radius, z0): Write the tissue mask of the sections from z0 to the end of its chunk, and return its voxel count.
• tissue_mask: the click command of bs utils tissue-mask.

bootstrapper.derive

The run tail has the keys that bs prepare derives for each step from the volumes and the keys above. derive reads array metadata and net_config.json files, never voxels. A key that you set above stays out of the tail, and the keys that bs prepare derives from it use your value.

• derive(volumes, steps, root): Return the [run.<step>] table of each step, in step order. root contains the round's model setups and training masks.

bootstrapper.discover

Find the arrays of a zarr container from their metadata alone.

• class Array(path: str, name: str, shape: tuple, dtype: str, voxel_size: tuple, offset: tuple): An array found in a container, with its metadata.
  • role: A guess that the user's answer confirms: "mask" when the name contains mask (any case), "labels" for int32, int64, uint32 or uint64, and "image" otherwise.
  • roi
• container(path): Return the zarr container that a path is in, up to its .zarr. For a path without .zarr, return its folder.
• is_array(path): Return True when path is a Zarr format 2 array.
• read(path, root=''): Read metadata for the array at path. For OME-Zarr, remove singleton t and c axes from the reported shape. Raise click.ClickException for unsupported OME metadata.
• arrays(group): Return every array under a zarr group, in path order. Skip an array that bootstrapper cannot read, and print the reason to stderr.
• scales(array): For an array named like s0, return each array in its folder with a scale-level name. For OME-Zarr, return the levels of its image.
• candidates(raw, arrays): Return the labels and masks that fit the raw. An array fits when it has the raw's voxel size and a region that overlaps the raw's.

bootstrapper.eval

The metrics and error maps of bs evaluate.

bootstrapper.eval.compute_errors

Error maps of a segmentation against the predictions that made it.

compute_errors computes affinities or LSDs from the segmentation. It writes their differences from the predictions as an error map and a thresholded error mask.

• compute_errors(seg_dataset, pred_dataset, mask_dataset, out_map_dataset, out_mask_dataset, thresholds=(0.1, 1.0), roi_offset=None, roi_shape=None, return_arrays=False, **kwargs): Write the error map and error mask of a segmentation against a prediction.

  A prediction name with 3d_lsds needs lsd_sigma, and one with 3d_affs needs aff_neighborhood (one offset per channel). The maps cover the intersection of the prediction, the segmentation, the mask and the ROI, with the ROI snapped outward to the prediction voxel grid. The function always writes both output datasets, and with return_arrays it also returns the batch.

• compute_stats(array): Mean and std of an array, with the count and fraction of voxels greater than zero.

bootstrapper.eval.compute_metrics

Scores of a segmentation against ground truth, and the report of one segmentation.

compute_metrics gives VOI and NVI against ground-truth labels, and ERL and nERL against ground-truth skeletons. seg_report counts objects and foreground.

• read_skeletons(gt_skeletons_file, roi): Read skeletons from a graphml file.

• seg_report(seg_dataset, mask_dataset=None, roi_offset=None, roi_shape=None): Report the object count, the foreground fractions, the foreground voxels outside the mask, and the ids and ROI fractions of the 3 largest objects. Count exactly, inside the ROI, in z slabs one chunk deep.

• compute_metrics(seg_dataset, gt_labels_dataset, gt_skeletons_file, mask_dataset=None, roi_offset=None, roi_shape=None): Score a segmentation against ground truth inside the ROI.

  Label comparisons apply the mask and return VOI and NVI. Skeleton comparisons use the mask's ROI but ignore its voxel values, and return ERL, nERL, and merge and split counts.

bootstrapper.evaluate

bs evaluate: score each segmentation of a round.

evaluate reports the object count, the foreground and the ids of the largest objects of each segmentation. It also compares each segmentation with ground-truth labels or skeletons when the step has [evaluate.gt], and with the predictions that made it when the step has [evaluate.pred]. The entry point is evaluate.

• get_seg_datasets(seg_datasets_prefix): Return the segmentation arrays under a prefix, without the error maps that evaluate pred writes.
• get_eval_config(config, config_file, mode, suffix=True, volume=None, **kwargs): Build the evaluate config for one volume and mode, with the given keywords. Add the volume to out_result when there is one, and the mode when suffix is True.
• run_gt_evaluation(config, seg_ds): Return the scores of seg_ds against the ground-truth labels or skeletons of [evaluate.gt].
• run_pred_evaluation(config, seg_ds): Write the error map and mask of seg_ds against the prediction of [evaluate.pred], and return their statistics.
• print_report(report): Print the object count, the foreground fractions and the ids of the largest objects in a report from seg_report.
• short_names(seg_datasets): Return short names, such as 'mws b-0.3/fill_holes', that include the method, the name parts that differ, and the refine suffix. Return the full paths when two names clash.
• summary(stats): Format the metrics, the error-mask fraction or the object count of one segmentation for the closing summary.
• run_evaluation(config, config_file, mode='pred', suffix=True, volume=None, **kwargs): Report every segmentation, then evaluate it in mode gt or pred. In mode report, stop after the report. Return one row per segmentation for the closing summary.
• evaluate: the click command of bs evaluate.

bootstrapper.gp

Gunpowder nodes for training and evaluation.

The nodes compute LSD targets and error maps, augment intensities and labels, and make synthetic labels. The model train.py scripts build their pipelines from them.

bootstrapper.gp.add_2d_lsds

A gunpowder node that computes 2D local shape descriptors, one section at a time.

• class Add2DLSDs, a subclass of AddLSDs: Create a 2D local shape descriptor for each voxel, one section at a time.

  Args:

      segmentation (:class:`ArrayKey`): The array storing the segmentation
          to use.
  
      descriptor (:class:`ArrayKey`): The array of the shape descriptor to
          generate.
  
      lsds_mask (:class:`ArrayKey`, optional): A binary output mask with the same
          shape as the descriptors. The mask is 0 at background voxels.
          Use it to exclude background from the training loss.
  
      unlabelled (:class:`ArrayKey`, optional): A binary array with 0 for
          unlabelled areas. The node sets lsds_mask to 0 at these voxels
          to exclude them from training.
  
      sigma (float or tuple of float): The standard deviation of the Gaussian
          kernel in world units. A float applies to all axes; a tuple uses (z, y, x).
          The descriptors use y and x, and z sets the requested context.
          Use z=0 to avoid extra context in z.
  
      downsample (int, optional): Downsample the segmentation mask to extract
          the statistics with the given factor. The default is 1 (no
          downsampling).

bootstrapper.gp.add_aff_errors

A gunpowder node that computes an affinity error map and mask from a segmentation and predicted affinities.

• class AddAffErrors(segmentation, seg_affs, pred_affs, error_map, error_mask, neighborhood, labels_mask=None, thresholds=(0.1, 1.0), array_specs=None), a subclass of BatchFilter: Compute an affinity error map and error mask from a segmentation and predicted affinities.
  • setup()
  • prepare(request)
  • process(batch, request)

bootstrapper.gp.add_lsd_errors

A gunpowder node that computes an LSD error map and mask from a segmentation and predicted LSDs.

• class AddLSDErrors(segmentation, seg_descriptor, pred_descriptor, error_map, error_mask, thresholds=(0.1, 1.0), labels_mask=None, sigma=5.0, downsample=1, array_specs=None), a subclass of BatchFilter: Compute a local shape descriptor error map and error mask from a segmentation and predicted descriptors.

  Args:

      segmentation (:class:`ArrayKey`): The array storing the segmentation
          to use.
  
      seg_descriptor (:class:`ArrayKey`): The array of the shape descriptor to
          generate.
  
      pred_descriptor (:class:`ArrayKey`): The array of the shape descriptor to
          compute errors against.
  
      error_map (:class:`ArrayKey`): The mean squared difference between
          seg_descriptor and pred_descriptor, averaged across channels at each voxel.
  
      error_mask (:class:`ArrayKey`): A binary mask of the error_map values strictly
          between the two thresholds.
  
      thresholds (tuple of float): The exclusive lower and upper bounds of the
          error mask.
  
      labels_mask (:class:`ArrayKey`, optional): The array to use as a mask
          for the errors. The errors are zero where the mask is zero.
  
      sigma (float or tuple of float): The standard deviation of the Gaussian
          kernel in world units. A float applies to every axis of the
          segmentation.
  
      downsample (int, optional): Downsample the segmentation mask to extract
          the statistics with the given factor. The default is 1 (no
          downsampling).

  • setup()
  • prepare(request)
  • process(batch, request)

bootstrapper.gp.add_lsds

A gunpowder node that computes the 3D local shape descriptors of a segmentation.

• compute_lsds(segmentation, sigma, voxel_size, downsample, crop): Compute LSDs of segmentation inside the voxel slices crop, with the whole array as context. Anchor the downsampling grid at the first voxel.

• class AddLSDs(segmentation, descriptor, lsds_mask=None, unlabelled=None, sigma=5.0, downsample=1), a subclass of gp.BatchFilter: Create a local shape descriptor for each voxel, as lsd.train.gp.AddLocalShapeDescriptor does with Gaussian accumulation and all components.

  Args:

      segmentation (:class:`ArrayKey`): The array storing the segmentation
          to use.
  
      descriptor (:class:`ArrayKey`): The array of the shape descriptor to
          generate.
  
      lsds_mask (:class:`ArrayKey`, optional): A binary output mask with the same
          shape as the descriptors. The mask is 0 at background voxels.
          Use it to exclude background from the training loss.
  
      unlabelled (:class:`ArrayKey`, optional): A binary array with 0 for
          unlabelled areas. The node sets lsds_mask to 0 at these voxels
          to exclude them from training.
  
      sigma (float or tuple of float): The standard deviation of the Gaussian
          kernel in world units. A float applies to every axis of the
          segmentation.
  
      downsample (int, optional): Downsample the segmentation mask to extract
          the statistics with the given factor. The default is 1 (no
          downsampling).

  • setup()
  • prepare(request)
  • process(batch, request)

bootstrapper.gp.clahe_augment

A gunpowder node that applies CLAHE contrast augmentation at random, section by section.

• class ClaheAugment(array, clip_limit_range=(0.01, 0.1), signal_threshold=0.01, p=1.0), a subclass of gp.BatchFilter: Apply CLAHE (Contrast Limited Adaptive Histogram Equalization) with a random clip limit.

  Args:
      array (:class`ArrayKey`):
          The intensity array to modify.
      clip_limit_range (tuple of float):
          The minimum and maximum clip limits for uniform random sampling.
      signal_threshold (float):
          Apply CLAHE only when a slice's range (max - min) exceeds this value.
          Skip other slices to avoid amplifying noise.
      p (float):
          The probability of applying this augmentation.

  • setup()
  • skip_node(request)
  • process(batch, request)

bootstrapper.gp.create_labels

A gunpowder node that provides synthetic 3D labels.

• class CreateLabels(array_key, anisotropy_range=None, shape=(20, 20, 20), dtype=np.uint32, voxel_size=None, background_prob=0.1), a subclass of gp.BatchProvider: Generate synthetic 3D label arrays.

  Args:
      array_key (gp.ArrayKey): The key of the array to provide labels for.
      anisotropy_range (tuple): The range of anisotropy values to use for label generation.
      shape (tuple): The shape of the labels array.
      dtype (numpy.dtype): The data type of the labels.
      voxel_size (tuple): The voxel size of the labels.
      background_prob (float): The probability that the node sets a label id to
          background.

  • setup()
  • provide(request)

bootstrapper.gp.create_mask

A gunpowder node that makes a uint8 mask where the input array is greater than zero.

• class CreateMask(in_array, out_array), a subclass of gp.BatchFilter: Create a uint8 mask that is 1 where the input array is greater than zero.

  Args:
      in_array (gp.ArrayKey): The input array.
      out_array (gp.ArrayKey): The output array.

  • setup()
  • prepare(request)
  • process(batch, request)

bootstrapper.gp.custom_grow_boundary

A gunpowder node that grows the boundaries between labels.

• class CustomGrowBoundary(labels, mask=None, max_steps=1, background=0, only_xy=False), a subclass of gp.BatchFilter: Grow a boundary between regions in a label array. The node does not grow a boundary at the border of the batch or of the optional mask. For each label, erode a random number of voxels from 0 to max_steps.

  Args:

      labels (:class:`ArrayKey`):
  
          The array containing labels.
  
      mask (:class:`ArrayKey`, optional):
  
          A mask with 0 in unknown regions, which prevents erosion at boundaries between labelled and unknown regions.
  
      max_steps (``int``, optional):
  
          The maximum erosion distance in voxels.
  
      background (``int``, optional):
  
          The label to assign to the boundary voxels.
  
      only_xy (``bool``, optional):
  
          Do not grow a boundary in the z direction.

  • process(batch, request)

bootstrapper.gp.defect_augment

A gunpowder node that adds section defects such as missing sections, low contrast and deformation.

• class DefectAugment(intensities, prob_missing=0.05, prob_low_contrast=0.05, prob_artifact=0.0, prob_deform=0.0, contrast_scale=0.1, artifact_source=None, artifacts=None, artifacts_mask=None, deformation_strength=20, axis=0, p=1.0), a subclass of BatchFilter: Augment intensities with missing or low-contrast sections, artifacts from a separate source, or deformed sections.

  Args:

      intensities (:class:`ArrayKey`):
  
          The key of the array of intensities to modify.
  
      prob_missing(``float``):
      prob_low_contrast(``float``):
      prob_artifact(``float``):
      prob_deform(``float``):
  
          The probabilities per section for missing data, low contrast, an artifact
          (see param ``artifact_source``), or deformation. Their sum must not exceed 1.
          The node sets each missing section to 0 or 1, with equal probability.
  
      contrast_scale (``float``, optional):
  
          Scale intensity differences from the section mean by this factor when ``prob_low_contrast`` > 0.
  
      artifact_source (class:`BatchProvider`, optional):
  
          A gunpowder batch provider for intensities (:class:`ArrayKey` ``artifacts``) and an
          alpha mask (:class:`ArrayKey` ``artifacts_mask``). The node uses it when
          ``prob_artifact`` > 0.
  
      artifacts(:class:`ArrayKey`, optional):
  
          The key of the artifact intensities in ``artifact_source``.
  
      artifacts_mask(:class:`ArrayKey`, optional):
  
          The key of the artifact alpha mask in ``artifact_source``. The node
          blends the artifacts with ``intensities`` through this mask.
  
      deformation_strength (``int``, optional):
  
          The strength of the section deformation in voxels, for
          ``prob_deform`` > 0. The deformation models a fold by shifting the
          section contents toward a randomly oriented line in the section.
          The node sets the line, dilated by 10 voxels, to 0.
  
      axis (``int``, optional):
  
          The axis along which the sections are cut.
  
      p (``float``, optional):
  
          The probability of applying the augmentation, from 0 to 1. The default is 1.0 (always apply).
          A lower value skips some batches and reduces augmentation time.

  • setup()
  • teardown()
  • skip_node(request)
  • prepare(request)
  • process(batch, request)

bootstrapper.gp.deform_augment

A gunpowder node for elastic deformation that keeps raw and labels aligned.

• class DeformAugment, a subclass of gp.DeformAugment: gp.DeformAugment with the transform sampled at the true control-point positions.

  gunpowder 1.4.4 and 1.5.0 resample the control grid with an endpoint-aligned zoom for each requested ROI. Raw and labels get different zooms and do not align.

bootstrapper.gp.expand_labels

A gunpowder node that grows labels into the background, one section at a time.

• class ExpandLabels(labels, background=0, expansion_factor=1), a subclass of gp.BatchFilter: Grow each label into the background by up to expansion_factor voxels, section by section.
  • process(batch, request)

bootstrapper.gp.gamma_augment

A gunpowder node that changes the gamma of intensity arrays at random.

• class GammaAugment(arrays, gamma_min=0.8, gamma_max=1.2, slab=None, p=1.0), a subclass of BatchFilter: Apply random gamma transformations to intensity arrays.

  Parameters:
      arrays (:class:`ArrayKey` or list of :class:`ArrayKey`s):
          The intensity arrays to modify, with gamma sampled independently for each array and slab.
      gamma_min (``float``):
          The minimum gamma to sample.
      gamma_max (``float``):
          The maximum gamma to sample. It must be >= gamma_min.
      slab (``tuple`` of ``int``, optional):
          The shape of the slabs in which the node changes gamma. -1 stands
          for the full size of the array along that axis.
          For example, a slab of::
  
              (2, -1, -1, -1)
  
          changes gamma separately for each slice ``[0:2,:]``,
          ``[2:4,:]``, ... of 4D data.
      p (``float``, optional):
          The probability of applying this augmentation.

  • setup()
  • skip_node(request)
  • process(batch, request)

bootstrapper.gp.impulse_noise_augment

A gunpowder node that adds random impulse noise to intensity arrays.

• class ImpulseNoiseAugment(arrays, pixel_p=0.1, range=(0, 1), p=1.0), a subclass of gp.BatchFilter: Add random-valued impulse noise to an intensity array or a list of intensity arrays.

  Args:

      arrays (:class:`ArrayKey` or list of :class:`ArrayKey`s):
  
          The intensity arrays to modify. The node applies the same noise pattern to each.
  
      pixel_p (``float``):
  
          The per-pixel probability of noise.
  
      range (``tuple``, default: (0,1) ):
  
          The range of the noise values, drawn from a uniform distribution.
          For a fixed noise value, set both ends of the range to that value.
  
      p (``float``, optional):
  
          The probability of applying this augmentation.

  • setup()
  • skip_node(request)
  • process(batch, request)

bootstrapper.gp.obfuscate_labels

A gunpowder node that adds random splits, merges and artifacts to labels.

• class ObfuscateLabels(in_labels, out_labels, num_tries=5, p_split=0.1, p_merge=0.1, p_artifact=0.1), a subclass of gp.BatchFilter: Apply random splits, merges, and artifacts to a copy of a 3D label array.

  Parameters:
      in_labels (gp.ArrayKey): The input label array.
      out_labels (gp.ArrayKey): The array in the batch for the modified labels.
      num_tries (int): The number of attempts for each operation type.
      p_split (float): The per-attempt probability of a split operation.
      p_merge (float): The per-attempt probability of a merge operation.
      p_artifact (float): The per-attempt probability of adding artifacts.

  • setup()
  • prepare(request)
  • process(batch, request)

bootstrapper.gp.reject_sparse_labels

A gunpowder node that rejects batches with too few labelled voxels, and stops when no batch passes.

• class TooFewLabels, a subclass of BaseException: No window passed the labelled-fraction rule. It is a BaseException, because gunpowder wraps an Exception in a long request dump that hides its message.

• class RejectSparseLabels(mask, min_masked), a subclass of gp.BatchFilter: Request batches until the mean of the mask is more than min_masked, like gp.Reject.

  Raise TooFewLabels when max_rejected batches in a row fail.

  Args:
      mask (gp.ArrayKey): The labelled-voxel mask.
      min_masked (float): The labelled fraction a batch must exceed.

  • setup()
  • provide(request)

bootstrapper.gp.renumber

A gunpowder node that gives each connected component of a label array its own id.

• class Renumber(labels), a subclass of gp.BatchFilter: Find connected components of the same value, and replace each component with a new label.

  Args:

      labels (:class:`ArrayKey`):
  
          The label array to modify.

  • process(batch, request)

bootstrapper.gp.smooth_augment

A gunpowder node that applies a random Gaussian blur to an intensity array.

• class SmoothAugment(array, blur_min=0.5, blur_max=1.5, slab=None, p=1.0), a subclass of gp.BatchFilter: Blur an intensity array with a Gaussian filter, with sigma drawn uniformly between blur_min and blur_max.

  Args:

      array (:class:`ArrayKey`):
  
          The intensity array to modify.
  
      blur_min (``float``):
      blur_max (``float``):
  
          The minimum and maximum sigma in voxels. The node samples sigma uniformly for each slab.
      slab (``tuple`` of ``int``, optional):
          The shape of the slabs in which the node blurs. -1 stands for the
          full size of the array along that axis.
          For example, a slab of::
  
              (2, -1, -1, -1)
  
          blurs each slice ``[0:2,:]``,
          ``[2:4,:]``, ... of 4D data separately.

  • setup()
  • skip_node(request)
  • process(batch, request)

bootstrapper.inputs

Open native Zarr arrays with funlib, or read OME-Zarr image levels (OME-NGFF 0.4 in Zarr format 2). For OME-Zarr, remove singleton time and channel axes and use its scale and translation in nm. docs/inputs.md gives the rules.

• class OmeLayout(index: tuple, axis_names: tuple, voxel_size: tuple, offset: tuple, image_path: str, level_paths: tuple): The layout of an OME-Zarr image level. index selects t and singleton c at 0; the paths are absolute.
• open_ds(path, mode='r', *, kind=None, **kwargs): Open native arrays with funlib, or OME-Zarr levels without copying data. For OME-Zarr, use its geometry in nm and axes z y x or c z y x. kind "raw" takes z y x or c z y x; "labels" and "mask" take z y x. kwargs go to funlib for a native array; for OME-Zarr, the keywords that funlib does not take go to zarr.
• refuse_zarr3(path, **kwargs): Raise ClickException for a Zarr format 3 node, which OME-NGFF 0.5 writes. The zarr version that bootstrapper uses reads format 2.
• refuse_group(path, **kwargs): Raise ClickException for an OME-Zarr image group, which contains one array per level.
• levels(path, **kwargs): Return the full level paths of the OME-Zarr image group at path, in declared order. Return [] when path is not a group or has no multiscales records. Raise ClickException for malformed or ambiguous records.
• ome(path, **kwargs): Read the OmeLayout from the nearest image group whose multiscales lists this array. Return None when no multiscales lists it. Raise ClickException for malformed or unsupported metadata.

bootstrapper.params

The keys that bs prepare writes into the config files, with each key's value and what it does. A table name has * in place of a model or volume name. bs prepare writes an off key as a commented example.

• class Key(value: object, help: str, off: bool = False): A key that bs prepare writes, with its value, what it does, and whether bs prepare comments it out.
• defaults(table): Return the values of a table's keys that are on.

bootstrapper.post

The segmentation methods of bs segment, and their helpers.

watershed, watershed_mutex and connected_components each give one method's entry point. The other modules supply the blockwise tasks, the graph database, the output names and the seeds.

bootstrapper.post.blockwise

Volara blockwise tasks for watershed segmentation.

bootstrapper.post.blockwise.extract_frags

Volara fragment extraction for mutex watershed, plain and seeded.

• class ExtractFrags, a subclass of VolaraExtractFrags: ExtractFrags with the fragment filter written back into the array.

  replace_values returns a new array, which the parent discards, so its filter has no effect. The parent's remove_small_objects(min_size=...) also drops fragments of exactly remove_debris voxels under skimage 0.26.

  • filter_avg_fragments(affs, fragments_data, filter_value)
  • get_fragments(affs_data)

• class SeededExtractFrags, a subclass of ExtractFrags: ExtractFrags seeded from an integer labels array on the affs grid.

  mwatershed keeps every nonzero seed voxel in its seed's fragment and never joins two seed ids. Seeded fragments skip the fragment filters. The database node attribute seed_id is the id of the seed voxels in a fragment, or 0 for an unseeded fragment, stored as text.

  • compute_fragments(affs_data, seeds)

  • get_fragments(affs_data, seeds): Return the fragments, with each seeded fragment's id set to its seed id + seeds.size.

    Unseeded mwatershed ids are at most the voxel count, so the shift keeps the two apart.

  • watershed_in_block(block, affs, frags, rag_provider, mask=None, seeds=None)

  • process_block_func()

bootstrapper.post.blockwise.watershed_frags

A volara task that makes watershed fragments from affinities and writes their nodes to the graph database.

• class WatershedFrags, a subclass of BlockwiseTask: Extract fragments from affinities with a seeded watershed, block by block. Write fragment positions and sizes to the volara graph database (SQLite or PostgreSQL) for waterz agglomeration.
  • task_name
  • write_roi
  • voxel_size
  • write_size
  • context_size
  • num_voxels_in_block
  • output_datasets
  • drop_artifacts()
  • init()
  • init_out_array()
  • compute_fragments(affs_data)
  • filter_avg_fragments(affs, fragments_data, filter_value)
  • epsilon_agglomerate_fragments(affs_data, fragments_data)
  • get_fragments(affs_data)
  • watershed_in_block(block, affs, frags, rag_provider, mask=None)
  • process_block_func()

bootstrapper.post.blockwise.waterz_agglom

A volara task that writes waterz merge scores to the fragment graph.

• class WaterzAgglom, a subclass of BlockwiseTask: A blockwise task that scores supervoxel RAG edges with waterz.

  For each block, agglomerate fragments with waterz and build a merge tree from the merge history. Write each RAG edge's merge score to the volara graph database (SQLite or PostgreSQL). Thresholded connected components on the stored graph produce the global segmentation.

  • task_name
  • write_roi
  • write_size
  • context_size
  • output_datasets
  • drop_artifacts()
  • init()
  • agglomerate_in_block(block, affs, frags, rag_provider)
  • process_block_func()

bootstrapper.post.cc

Connected components of thresholded affinities, compiled with numba.

• compute_connected_component_segmentation(hard_aff: np.ndarray): Compute connected components from affinities.

  Args:
      hard_aff: Thresholded boolean short-range affinities, with shape (3, z, y, x).
          Channels follow z, y, x order. Each affinity connects a voxel to its neighbor at offset -1 along that axis.
  
  Returns:
      The segmentation, with shape (z, y, x).

bootstrapper.post.connected_components

The cc method of bs segment thresholds the affinities, then labels connected components.

The entry point is cc_segmentation. It runs as one block over the ROI.

• cc_segmentation(config): Segment the affinities in the ROI with connected components, as one block, and write the fragments and the segmentation.

bootstrapper.post.merge_tree

A merge tree over a waterz merge history, for the merge score of two fragments.

• class MergeTree(leaf_nodes=None): A merge tree over a waterz merge history, accelerated with numba.

  Adapted from lsd (lsd.post.merge_tree). Build the tree by replaying a merge history with merge. find_merge(u, v) and find_merges(us, vs) return the score where fragments first join one segment, or None and NaN, respectively, for fragments that never merge.

  The tree stores nodes in flat arrays indexed by dense ids, and maps sparse fragment ids through \_idx. The compiled query loop matches lsd's Cython implementation.

  • merge(u, v, target, score)
  • find_merges(us, vs): Run find_merge over sequences of ids, and return a float64 array. A NaN entry means that the pair never merges, where find_merge returns None.
  • find_merge(u, v)

bootstrapper.post.naming

Output names from segmentation params, and the params recorded on each output.

• fmt(value, sep='_'): Format a value for a name. An all-equal list collapses to one value, and a nested list joins with '.'.
• build_name(params): Build a name from the non-None parameters listed in CODES, in that order. A boolean renders as its bare code when True, or as the code followed by 0 when False.
• mws_segmentation_name(params): The name watershed_mutex gives its segmentation for these params.
• inputs_differ(store, run_params): Return the warning text if the output at store came from other inputs, else None.
• dump_params(store, params): Record the resolved params on the output dataset's zarr attrs.
• dump_lut_params(lut_path, params): Write the resolved params beside a LUT, as <lut_path>.json.

bootstrapper.post.seeds

Graph constraints and final ids for user-seeded mutex watershed. The graph database stores seed_id as text, so ids above 2^31 fit in a PostgreSQL column. Readers convert seed_id to int.

• link_seeds(db, roi, weights): Chain fragments of each seed id with must-link edges (+SEED_LINK). For different seed ids, add a mutex edge (-SEED_LINK) wherever mutex watershed could merge them. Store these constraints in the seed_link edge attribute. weights maps each affinity edge attribute to its GraphMWS (weight, bias), such as {"zyx_aff": global_bias}.
• seed_range(store, num_workers): The smallest and largest id in a seeds array, read chunk by chunk.
• seed_lut(db, lut, roi, max_seed): Rewrite the LUT for every fragment. Set each seeded segment to its seed id, and number other segments from max_seed + 1.

bootstrapper.post.sqlite

The SQLite graph database that the blockwise segmentation tasks share.

• class SQLite, a subclass of VolaraSQLite: volara's SQLite graph database, with a 10-minute lock timeout in place of the 5-second default of sqlite3. Block workers share the file. On a busy disk, one commit can lock it for more than 5 seconds.
  • open(mode='r')

bootstrapper.post.watershed

The ws method of bs segment makes watershed fragments, agglomerates them with waterz, then writes one segmentation per threshold.

The entry point is watershed_segmentation. Fragment arrays use Zarr, LUT files go in lut_dir, and the graph database stores fragment nodes and edges for blockwise runs.

• watershed_segmentation(config): Run the ws method of a flat segment config: fragments, waterz merge scores, then one segmentation for each threshold.

bootstrapper.post.watershed_mutex

The mws method of bs segment runs mutex watershed over the fragments and the affinity graph.

The entry point is mutex_watershed_segmentation. A seeds array, if given, keeps each seed id in one object. Fragment arrays use Zarr, LUT files go in lut_dir, and the graph database stores fragment nodes and edges for blockwise runs.

• mutex_watershed_segmentation(config): Run the mws method of a flat segment config: fragments, edges, mutex watershed, then relabel.

bootstrapper.post.ws

Watershed fragments from affinities or boundary distances, for one block.

• watershed_from_boundary_distance(boundary_distances, boundary_mask, return_seeds=False, id_offset=0, min_seed_distance=10): Watershed fragments from a boundary distance map, seeded at its local maxima.

  Returns (fragments, max_id), and the seeds as well with return_seeds. Ids start after id_offset.

• watershed_from_affinities(affs, max_affinity_value=1.0, fragments_in_xy=False, return_seeds=False, min_seed_distance=10): Extract initial fragments from affinities with a watershed transform. Return the fragments and the largest seed ID, which can exceed the largest fragment ID.

  Returns:
      (fragments, max_id)
      or
      (fragments, max_id, seeds) if return_seeds == True

bootstrapper.predict

bs predict: run a model's predict.py script over a volume.

With one GPU or none, and one worker, the script tiles the ROI with a gunpowder Scan. With more of either, daisy runs the script over ROI blocks on the selected GPUs or CPUs. The entry point is predict.

• predict_blockwise(config): Run the model script as daisy workers over the blocks of the ROI.

• get_devices(num_gpus): Return the devices this process may use, as CUDA_VISIBLE_DEVICES tokens.

  Keep UUID and MIG device tokens as they are, without converting them to numeric IDs.

• call_predict(config): Run the model script as one daisy worker, on device number worker_id modulo the number of devices. An empty device token runs the script on the CPU. When daisy stops the worker, the worker kills the script.

• refuse_incompatible_output(path, **wanted): Raise ClickException when an existing output does not match this run, because prepare_ds would rewrite its metadata.

• resolve_checkpoint(setup_dir, checkpoint): Return the checkpoint file for a config's checkpoint path, also when the file is one level down in the setup.

• get_pred_config(config, **kwargs): Return the plan of one predict run: the model script, its arguments and GPUs, and for daisy the block ROIs. Prepare the output arrays, and refuse an existing one that does not match.

• run_prediction(config_file, step=None, **kwargs): Run the [predict.*] steps of config_file in file order, or the one named step.

• predict: the click command of bs predict.

bootstrapper.prepare

bs prepare: collect data settings through a dialogue or flags, then write the round. For an existing round, add the volume that the flags give and rewrite its run tails.

• class Draft(keys: dict = field(default_factory=dict), name: str = '', raw: discover.Array = None, arrays: tuple = (), images: tuple = (), source: str = '', labels_read: str = '', labels: discover.Array = None, taken: frozenset = frozenset(), round: str = ''): One volume's answers so far, and what bs prepare read to ask the next question.
• class Answers(volumes: dict, chain: tuple = ()): The facts of a round: each volume's keys by name, and the models in run order.
• class Question(flag: str, prompt: str, help: str, example: str, parse: Callable, default: Callable = None, when: Callable = None, show: Callable = None, need: Callable = None, note: Callable = None)
• parse_raw(text, d)
• pick_raw(text, d)
• parse_voxel_size(text, d)
• parse_labels(text, d)
• labels_2d(labels): Return (is_2d, text). is_2d is True when no two adjacent sections both have labels. Read the coarsest scale with the same dtype and z voxel size as labels. First read about 16 sampled sections; if no labelled sample has labels on the next section, read every section.
• parse_labelling(text, d)
• parse_mask(text, d)
• parse_untraced(text, d)
• parse_target_mask(text, d)
• parse_gt(text, d)
• parse_name(text, d)
• parse_models(text, answers)
• ask(questions, state, flags, terminal): Answer each question that applies, from flags, terminal input, or defaults, and return the final state. At the terminal, '?' explains a question and '<' goes back one answer.
• ask_volumes(round_dir, flags, terminal): Return the volumes' keys by name. Flags give one volume, and the dialogue gives as many as the user enters.
• summary(name, d, taken=()): dw09-zarr: raw/s1, sparsely labelled gt_boundaries/s1 (3D; other objects around them), ground truth gt_boundaries/s1
• write_round(path, answers): Write volumes.toml if absent, one numbered file of user keys per step, and then their run tails. For a path ending in .toml, store all steps in one file. Return the files written.
• add_volume(path, flags, terminal): Add the volume that flags give to the round folder at path, in its volumes.toml.
• add_steps(path): Write an evaluate step when a volume has ground truth and the round folder lacks that step. Return the files written.
• add_mask_op(path): Add a [refine.mask] table to the first file whose refine step has no mask operation, and remove that file's run tail. bs prepare calls this when a new volume has a target mask. The operation sets labels outside the mask to 0. Return [file], or [] when no file changes.
• stale(path): Return a list of files whose run tails differ from values derived from the volumes and user keys. Each result is (file, text, new run tail, {table: {key: (old, new)}}).
• fetch_checkpoints(path): Download each shipped checkpoint that a predict step of the round at path reads and MODEL_DIR does not have.
• refresh(path): Download the shipped checkpoints the round lacks, then write the run tails that stale finds. Copy each training setup's model scripts if train.py is missing, and write the training masks. Return (file, {table: {key: (old, new)}}) for each file whose tail changed.
• set_in_channels(setup_dir, volumes): Set in_channels in net_config.json for a setup that reads raw. Use the channel count for c z y x arrays, or 1 otherwise.
• prepare: the click command of bs prepare.

bootstrapper.refine

bs refine: run blockwise operations on a segmentation.

Filter objects by size, size outliers or z extent; remove or merge ids; or zero labels outside a mask. For morphology (dilate, erode, opening, closing and fill-holes), use bs refine morph --op; the other operations have their own commands. bs run chains the operations of a [refine] step.

• refine: the click command of bs refine.
• outlier_filter: the click command of bs refine outlier-filter.
• size_filter: the click command of bs refine size-filter.
• z_filter: the click command of bs refine z-filter.
• remap: the click command of bs refine remap.
• mask: the click command of bs refine mask.
• morph: the click command of bs refine morph.

bootstrapper.segment

bs segment: make segmentations from affinities.

The method is ws (watershed and waterz agglomeration), mws (mutex watershed) or cc (connected components). DEFAULTS gives the default parameters of each method. The entry point is segment; the methods are in bootstrapper.post.

• parse_params(param_str): Return a -p value as a Python literal, or as the text when it does not parse as one.
• parse_shape(value): Parse coordinates from an integer list or a space- or comma-separated string. Return "roi" or None unchanged.
• get_method_params(method, params): Return the -p key=value pairs as a dict. Refuse a key that the method does not take.
• get_seg_config(config, method, **kwargs): Build a flat segment config from the step keys, the given options, and the method parameters over their defaults. For ws and mws, also set a LUT folder and a graph database.
• run_segmentation(config, mode='ws', **kwargs): Build a flat configuration, then segment with mode (ws, mws or cc).
• segment: the click command of bs segment.

bootstrapper.styles

The colors of the bs messages, with one palette for each step.

• create_style_palette(base_color)
• get_style(style, stype='info')
• cli_echo(text, style='default', stype='info')
• cli_confirm(text, style='default', stype='prompt', **kwargs)

bootstrapper.train

bs train: train a model on the labelled volumes of a [train.*] step.

setup_train checks the samples, assigns their batch shares, and writes train.toml for the setup's train.py. run_training then runs that script. The entry point is train.

• setup_train(config_file, step=None, **kwargs): Check the samples and assign their batch shares, then write train.toml in the setup. Return the paths of train.py and train.toml.
• probabilities(samples, sampling): Return each sample's batch share, weighted by labelled voxels ("labels"), equally ("equal"), or inversely ("inv"). Accept custom weights as a list in sample order or a table by volume.
• labelled_voxels(sample): Return the number of voxels a sample trains on, which is the number of nonzero voxels in its training mask, or in its labels when it has no mask.
• labelled_sections(path, kind='labels'): Return [first z, end z, labelled voxels], with z in nm, for each run of adjacent sections that have a nonzero voxel. The trainers draw only windows whose output touches a run, so they do not keep rejecting samples from unlabelled sections.
• check_inside(name, path, outer_name, outer_path): Raise ClickException when the array at path reaches past the array at outer_path. Without this check, the gunpowder Pad node stops training later. name and outer_name are the roles: raw, labels or mask.
• run_training(config_file, step=None, **kwargs): Run the setup's train.py on the train.toml that setup_train writes.
• train: the click command of bs train.

bootstrapper.training

The PyTorch Lightning training loop that the model train.py scripts share.

fit trains a model on gunpowder batches and saves checkpoints and snapshots in the setup. It resumes from the newest checkpoint.

• device_warning(): Return one warning line when training would run on the CPU because the GPU grant is empty or no GPU is visible. Return None when a GPU is available.

• class LabelsTooSmall, a subclass of BaseException: A window did not fit inside the labels (or mask) during training. It is a BaseException, because gunpowder wraps an Exception in a long request dump that hides its message.

• class LabelledLocation(message), a subclass of gp.RandomLocation: A gp.RandomLocation that raises LabelsTooSmall(message) when no window position fits the labels.

  • prepare(request)

• window_extent(output_size, voxel_size, deform, shift=None): Return the y x extent, in nm, of the labels that one output window reads through deform and shift.

  A rotation widens the window up to its diagonal, and the scale farthest from 1 enlarges it by |1 / scale - 1| times its span. The control-point jitter and the section shifts add 3 standard deviations at each side. Two more voxels, and the rounding up to whole voxels, cover the rounding that gunpowder does.

• labelled_crops(samples, output_size, deform, shift=None): Return (sample, roi, probability, location) for each run of labelled sections that bs train found in a sample. The roi contains every window inside the labels (or mask) whose output touches the run. The probability is the sample's batch share times the run's fraction of the sample's labelled voxels. location is the LabelledLocation node that draws the windows of the run. Raise ValueError when the labels (or mask) are smaller in y or x than the window_extent of deform and shift.

• class GunpowderDataset(create_pipeline, *args, **kwargs), a subclass of IterableDataset: An infinite dataset over a gunpowder pipeline. Each dataloader worker builds its own copy of the pipeline and requests batches from it. The constructor creates the pipeline once, so an input error stops training before the workers start.

• class StepProgressBar, a subclass of RichProgressBar: Show progress against max_steps instead of the unknown per-epoch batch count of the infinite dataset (which renders as '/--').

  • total_train_batches

• class SnapshotCallback(setup_dir, voxel_size, save_every), a subclass of Callback: Save a zarr snapshot of the batch and predictions at step 1 and every save_every steps. Convert float arrays to [0, 255] uint8 only when their values fit in [-1, 1] and include a negative value. Save other arrays unchanged.

  • on_train_batch_end(trainer, pl_module, outputs, batch, batch_idx)

• fit(model, dataset, setup_dir, max_iterations, save_checkpoints_every, snapshot_callback, num_workers=None, devices=None): Train model on dataset, starting from the newest checkpoint in setup_dir if there is one.

  fit saves a checkpoint to setup_dir every save_checkpoints_every iterations. When num_workers is None, fit uses half the available CPUs, rounded down, with a minimum of 1 and a maximum of 32. When devices is None, fit uses one device on Lightning's automatic accelerator, which can be the CPU.

bootstrapper.utils

bs utils: crop, mask, merge and rescale arrays, convert images to Zarr, and download checkpoints.

Each command except download-ckpts comes from a module of bootstrapper.data.

• utils: the click command of bs utils.
• download_ckpts: the click command of bs utils download-ckpts.

bootstrapper.view

bs view: show what the steps of a round wrote, a training snapshot, or array paths in neuroglancer. layer_type picks the layer type of each array from its name, dtype and channels.

• view: the click command of bs view.
• create_coordinate_space(voxel_size, is_2d): Return the neuroglancer dimensions of a training snapshot, 2D or 3D, in nm.
• process_dataset(f, ds, is_2d): Return the data, voxel size and voxel offset of one snapshot array, without its singleton z in 2D.
• snapshot_layers(zarr_path): Return one layer for each array of a training snapshot.
• layer_type(name, dtype, channels): Use rgb channels 0, 1, 2 for affinities, LSDs and raw with three or more channels, such as the sections of a 2D batch. Use segmentation for an integer array whose name contains label or mask, or whose values have 4 or more bytes. Use a gray image otherwise.
• round_layers(path, chosen): Return, for each volume, the raw and then what the chosen steps wrote. When no step is chosen, also return the volume's own arrays. Assign each array to the last step that writes it. A refine output beside a segmentation belongs to refine. Show the raw and the last array of the first volume with gt, or of the first volume when no volume has gt.

bootstrapper.write

Build the TOML text of a config file. Each key in params.KEYS gets a comment that says what it does, and off keys and tables appear as comments. The run tail goes below a banner, and bs prepare replaces only the tail.

• file_text(title, doc): Return the TOML text of doc below the comment title.
• tail_text(round_path, run): Return the banner, then the [run.*] tables of run.
• splice(text, tail): Return text with tail in place of its run tail. Every line outside the banner and the [run.*] tables stays as it is, except blank lines at the end.