Testing the Package

Unit Tests

The unit tests for this package consist of butler (Gen 2) tests for each camera, and metadata translation tests.

Butler Tests

The butler tests use the testing infrastructure from lsst.obs.base and there is a test file for each camera named tests/test_fooCam.py. There are additional tests in those files that are specific to the camera mappers in this package and that improve test coverage. For these to run some test data must be ingested into the Butler directory. New data can be added as follow:

• Generate a raw test data file (or files) that is as small as possible. The butler tests will work even if the data array is empty so long as those dimensions are reflected in the butler tests. To retain the correct dimensional information one technique is to zero out the pixel array and compress with gzip. The following code shows one method of zeroing out all the amplifiers:

  from astropy.io import fits
  import numpy
  
  aim = fits.open("rawfile.fits")
  aim.info()  # inspect to ensure the extension numbers are plausible
  for i in range(16):
      amp = aim[i+1]
      amp.data = numpy.zeros(amp.data.shape, dtype=amp.data.dtype)
  aim.writeto('zeroed.fits')
  

  Do not use a .gz extension as we still need the template strings to work to be able to locate the file.

• This file or files should be added to the test butler repository in data/input. Create directory data/input/fooCam, if required, and into that directory add a file _mapper with contents lsst.obs.lsst.fooCam.FooCamMapper and then run:

  ingestImages.py ./data/input/fooCam --mode=copy fooCamtest.fits
  

  specifying the path to the test files. This will store them in the butler repository.

• To test calibrated data butler access, if data are available, the individual image, mask, and variance planes, can be zeroed and put into the repository using the appropriate butler.put call to the test butler repository. This writes the file as tile compressed by default.

  Alternatively you can zero the file out and compress it using fpack and then use ingestCalibs.py to add it to the test repository:

  ingestCalibs.py ./data/input/fooCam --mode=copy --validity=99999 --output ./data/input/fooCam/CALIB calibfile.fits
  

• Copy one of the tests/test_cam.py files most closely related to your camera. You can include tests for processed data here but if you only have raw files with no calibrations it may be easiest to use test_ucd.py. These tests should be modified to reflect your test data dimensions and metadata.

Metadata Translation

The astro_metadata_translator header translators (currently defined in lsst.obs.lsst.translators) are tested in tests/test_translator.py. There is a test method per camera and the reference headers are stored as YAML files in tests/headers. YAML is used to simplify the movement of these tests into astro_metadata_translator at some point in the future. A header file can be generated in this format by running:

translate_header -d testfile.fits > tests/headers/testfile.yaml

The gen 2 translators are tested in tests/test_parsetask.py. Again, there is a test method per camera and the test files are read directly from the data/input test data directory (not using a butler).

CI Tests

Integration tests are run from within the ci_lsst package.