Administrative Command-Line Tooling

These commands can be used to inspect, reformat, convert, and prepare a dataset for testing. We are explicitly noting them as administrative commands to allow you to see how the system works but are not promising that the interfaces for these commands will not change or the output. For example, the inspect subcommand is currently the bare minimum and the intent is for it to provide more information later on. With that in mind, using these commands in operational scripts should be considered a risk. Please contact us if there is functionality you rely on that you would like to have a stronger backwards compatibility contract.

lsst-images-admin

Administrative tools for lsst.images files.

Usage

lsst-images-admin [OPTIONS] COMMAND [ARGS]...

convert

Convert a legacy FITS file to a new lsst.images format.

The output format is chosen from OUTPUT’s extension.

Usage

lsst-images-admin convert [OPTIONS] INPUT OUTPUT

Options

--type <type_>

Legacy input type; overrides auto-detection.

Options:

visit_image | cell_coadd

--skymap <skymap>

Pickled skymap (required for cell coadds unless –butler is given).

--butler <butler>

Butler repository to resolve the skymap (cell coadds only).

--collection <collection>

Butler collection holding the skymap (required with –butler).

--overwrite

Overwrite OUTPUT if it exists.

--preserve-quantization, --no-preserve-quantization

Preserve quantization-compressed pixel values so they can be written back out losslessly (visit images only). On by default.

Arguments

INPUT

Required argument

OUTPUT

Required argument

diagram

Generate a composition diagram of an lsst.images model.

Pass a schema name (e.g. visit-image or cell-coadd) to diagram the abstract model, or --from-file to diagram the concrete structure of a serialized file, which collapses unions such as the PSF to the type actually stored. Use --list to see the available schema names.

Usage

lsst-images-admin diagram [OPTIONS] [MODEL]

Options

--from-file <from_file>

Diagram the concrete structure of a serialized file instead of a model.

--format <fmt>

Output format.

Default:

'mermaid'

Options:

mermaid | dot | tree

--expand <TYPE>

Expand a type that is collapsed by default. Repeatable.The name can match the public name (‘Image’) or the serialization name (‘ImageSerializationModel’).

--collapse <TYPE>

Render TYPE as a leaf without expanding its fields. Repeatable; useful for noisy sub-trees such as –collapse ButlerInfo. The name can match the public name (‘Image’) or the serialization name (‘ImageSerializationModel’).

--expand-leaves

Expand the serialization-helper leaves that are collapsed by default.

--attributes

Show scalar fields too; by default only model composition is shown.

--hide-field <NAME>

Drop every field with this name, along with any sub-tree reachable only through it. Repeatable; useful for clipping pervasive payload fields such as ‘–hide-field data’ and ‘–hide-field table’.

--hide-type <TYPE>

Drop a type entirely, removing every edge that points at it (unlike –collapse, which keeps the node as a leaf). Repeatable; matches the public or serialization name, e.g. ‘–hide-type TableModel’.

--serialization-names

Label nodes with serialization-model class names instead of public class names.

-o, --output <output>

Write to a file instead of stdout.

--list

List available schema names and exit.

Arguments

MODEL

Optional argument

extract_test_data

Extract test fixtures from a Rubin data repository.

Usage

lsst-images-admin extract_test_data [OPTIONS] COMMAND [ARGS]...

dp2

Extract test data from a butler repository.

Usage

lsst-images-admin extract_test_data dp2 [OPTIONS]

Options

-b, --butler-repo <butler_repo>

Path to the butler repository.

-d, --testdata-dir <testdata_dir>

Path to the testdata_images directory.

-c, --collection <collection>

Collection to use for most data products.

--visit-images, --no-visit-images

Whether to extract preliminary_visit_image or visit_image datasets.

--difference-images, --no-difference-images

Whether to extract difference_image datasets.

--coadds, --no-coadds

Whether to extract coadd datasets.

--camera, --no-camera

Whether to extract the camera.

--skymap, --no-skymap

Whether to extract the skymap.

fuzz-masked-image

Shuffle the proprietary pixels of MaskedImage files for public release.

Each FILE is read in whatever format it is given, its image, mask, and variance planes are shuffled within tiles (one shared permutation per tile keeps the three planes mutually consistent), and the result is written beside the input with SUFFIX inserted before the extension. The output format follows that extension. Every other plane and all metadata are written back unchanged, and each output is re-read to confirm the proprietary planes really changed.

Usage

lsst-images-admin fuzz-masked-image [OPTIONS] [FILES]...

Options

--seed <seed>

Seed for pixel shuffling.

Default:

1

--suffix <suffix>

Text inserted before the extension of each output file.

Default:

'.fuzzed'

--tile-shape <Y X>

Override the shuffle block and FITS compression tile; default is the cell size if the object has a cell grid, else the full image.

--quantize-level <quantize_level>

RICE quantization level for the lossy image/variance planes.

Default:

16.0

--compression-seed <compression_seed>

FITS tile-compression dither seed (FITS output only).

Default:

1

--overwrite

Overwrite existing output files.

Arguments

FILES

Optional argument(s)

inspect

Print basic information about an lsst.images file.

Reports the schema URL, container format version, and the public Python class registered for the file’s schema (when known) without deserializing pixel data.

Usage

lsst-images-admin inspect [OPTIONS] FILE

Arguments

FILE

Required argument

minify

Subset a real data file into a small test fixture.

Usage

lsst-images-admin minify [OPTIONS] INPUT OUTPUT

Options

--overwrite

Overwrite OUTPUT if it exists.

Arguments

INPUT

Required argument

OUTPUT

Required argument

reformat

Rewrite an lsst.images file in a different container format.

Reads INPUT and writes it back out to OUTPUT, choosing the format from OUTPUT’s extension. This is the easy way to, for example, turn a FITS file into an NDF for testing.

Usage

lsst-images-admin reformat [OPTIONS] INPUT OUTPUT

Options

--overwrite

Overwrite OUTPUT if it exists.

Arguments

INPUT

Required argument

OUTPUT

Required argument

verify-rewrite

Compare rewritten datasets against their originals in a butler repo.

Usage

lsst-images-admin verify-rewrite [OPTIONS] COMMAND [ARGS]...

stage4

Compare rewritten visit/difference images and downstream tables in NEW_COLLECTION against the originals in OLD_COLLECTION of REPO.

Usage

lsst-images-admin verify-rewrite stage4 [OPTIONS] REPO OLD_COLLECTION
                                        NEW_COLLECTION

Options

--where <where>

Query string to constraint the comparison.

--old-prefix <old_prefix>

Prefix for the old dataset type names.

--new-prefix <new_prefix>

Prefix for the new dataset type names.

Arguments

REPO

Required argument

OLD_COLLECTION

Required argument

NEW_COLLECTION

Required argument