IngestTestBase

class lsst.obs.base.ingest_tests.IngestTestBase

Bases: object

Base class for tests of gen3 ingest. Subclass from this, then unittest.TestCase to get a working test suite.

Attributes Summary

curatedCalibrationDatasetTypes	List or tuple of Datasets types that should be present after calling writeCuratedCalibrations.
dataIds	list of butler data IDs of files that should have been ingested.
file	Full path to a file to ingest in tests.
filterLabel	The lsst.afw.image.FilterLabel that should be returned by the above file.
ingestDatasetTypeName	The DatasetType to use for the ingest.
ingestDir	Root path to ingest files into.
instrumentClass	The instrument class.
instrumentClassName	The fully qualified instrument class name.
instrumentName	The name of the instrument.
rawIngestTask	The task to use in the Ingest test.
seed_config	Location of a seed configuration file to pass to butler create.
visits	A dictionary mapping visit data IDs the lists of exposure data IDs that are associated with them.

Methods Summary

checkRepo([files])	Check the state of the repository after ingest.
setUp()
setUpClass()
tearDownClass()
testCopy()
testDefineVisits()
testDirect()
testFailOnConflict()	Re-ingesting the same data into the repository should fail.
testHardLink()
testInPlace()	Test that files already in the directory can be added to the registry in-place.
testLink()
testSymLink()
testWriteCuratedCalibrations()	Test that we can ingest the curated calibrations, and read them with loadCamera both before and after.
verifyIngest([files, cli, fullCheck])	Test that RawIngestTask ingested the expected files.

Attributes Documentation

curatedCalibrationDatasetTypes: list[str] | None = None

List or tuple of Datasets types that should be present after calling writeCuratedCalibrations. If None writeCuratedCalibrations will not be called and the test will be skipped.

dataIds: list[DataCoordinate] = []

list of butler data IDs of files that should have been ingested.

file: str = ''

Full path to a file to ingest in tests.

filterLabel: lsst.afw.image.FilterLabel = None

The lsst.afw.image.FilterLabel that should be returned by the above file.

ingestDatasetTypeName: str = 'raw'

The DatasetType to use for the ingest.

If this is not an Exposure dataset type the tests will be more limited.

ingestDir: str = ''

Root path to ingest files into. Typically obs_package/tests/; the actual directory will be a tempdir under this one.

instrumentClass

The instrument class.

instrumentClassName

The fully qualified instrument class name.

Returns:

str

The fully qualified instrument class name.

instrumentName

The name of the instrument.

Returns:

str

The name of the instrument.

rawIngestTask: str = 'lsst.obs.base.RawIngestTask'

The task to use in the Ingest test.

seed_config: str | None = None

Location of a seed configuration file to pass to butler create.

Useful if additional formatters or storage classes need to be defined.

visits: dict[DataCoordinate, DataCoordinate] = {}

A dictionary mapping visit data IDs the lists of exposure data IDs that are associated with them. If this is empty (but not None), visit definition will be run but no visits will be expected (e.g. because no exposures are on-sky observations).

Methods Documentation

checkRepo(files: list[str] | None = None) → None

Check the state of the repository after ingest.

This is an optional hook provided for subclasses; by default it does nothing.

Parameters:

fileslist [str], or None

List of files to be ingested, or None to use self.file

setUp() → None

classmethod setUpClass() → None

classmethod tearDownClass() → None

testCopy() → None

testDefineVisits() → None

testDirect() → None

testFailOnConflict() → None

Re-ingesting the same data into the repository should fail.

testHardLink() → None

testInPlace() → None

Test that files already in the directory can be added to the registry in-place.

testLink() → None

testSymLink() → None

testWriteCuratedCalibrations() → None

Test that we can ingest the curated calibrations, and read them with loadCamera both before and after.

verifyIngest(files: list[str] | None = None, cli: bool = False, fullCheck: bool = False) → None

Test that RawIngestTask ingested the expected files.

Parameters:

fileslist [str], or None

List of files to be ingested, or None to use self.file

clibool, optional

Unused.

fullCheckbool, optional

If True, read the full raw dataset and check component consistency. If False check that a component can be read but do not read the entire raw exposure.

Notes

Reading all the ingested test data can be expensive. The code paths for reading the second raw are the same as reading the first so we do not gain anything by doing full checks of everything. Only read full pixel data for first dataset from file. Don’t even do that if we are requested not to by the caller. This only really affects files that contain multiple datasets.