File fitsCompression.h¶
-
namespace
lsst
Class for a simple mapping implementing a generic AstrometryTransform.
Remove all non-astronomical counts from the Chunk Exposure’s pixels.
Forward declarations for lsst::utils::Cache
For details on the Cache class, see the Cache.h file.
It uses a template rather than a pointer so that the derived classes can use the specifics of the transform. The class simplePolyMapping overloads a few routines.
A base class for image defects
Numeric constants used by the Integrate.h integrator routines.
Compute Image Statistics
- Note
Gauss-Kronrod-Patterson quadrature coefficients for use in quadpack routine qng. These coefficients were calculated with 101 decimal digit arithmetic by L. W. Fullerton, Bell Labs, Nov 1981.
- Note
The Statistics class itself can only handle lsst::afw::image::MaskedImage() types. The philosophy has been to handle other types by making them look like lsst::afw::image::MaskedImage() and reusing that code. Users should have no need to instantiate a Statistics object directly, but should use the overloaded makeStatistics() factory functions.
-
namespace
afw
-
namespace
fits
Functions
-
ImageCompressionOptions::CompressionAlgorithm
compressionAlgorithmFromString
(std::string const &name)¶ Interpret compression algorithm expressed in string.
-
std::string
compressionAlgorithmToString
(ImageCompressionOptions::CompressionAlgorithm algorithm)¶ Provide string version of compression algorithm.
-
ImageCompressionOptions::CompressionAlgorithm
compressionAlgorithmFromCfitsio
(int cfitsio)¶ Convert compression algorithm from cfitsio to ImageCompressionOptions::CompressionAlgorithm.
-
int
compressionAlgorithmToCfitsio
(ImageCompressionOptions::CompressionAlgorithm algorithm)¶ Convert ImageCompressionOptions::CompressionAlgorithm to cfitsio.
-
ImageScalingOptions::ScalingAlgorithm
scalingAlgorithmFromString
(std::string const &name)¶ Interpret scaling algorithm expressed in string.
-
std::string
scalingAlgorithmToString
(ImageScalingOptions::ScalingAlgorithm algorithm)¶ Provide string version of compression algorithm.
-
struct
ImageCompressionOptions
¶ - #include <fitsCompression.h>
Options for tile compression of image pixels
Tile compression is a feature provided by cfitsio, where contiguous parts of the image (“tiles”, e.g., rows, multiple rows, blocks or the entire image) are compressed separately. The aim of this struct is to hold the parameters used to configure the compression.
Floating-point images can be losslessly compressed (quantizeLevel=0.0) using (only) the GZIP or GZIP_SHUFFLE compression algorithms, but the compression factor so achieved is modest (e.g., ~ 10% compression). Better compression factors can be achieved if floating-point images are first quantised into integer images. This can be done by cfitsio (through the quantizeLevel parameter) or through use of the ImageScalingOptions.
The Compression is specified by:
the compression algorithm
the tile size
the quantization level (for quantization applied by cfitsio; for floating-point images)
Due to bugs, cfitsio may require setting the quantizeLevel to a value other than zero when compressing integer data, but in this case it should have no effect.
Public Types
-
enum
CompressionAlgorithm
¶ Compression algorithms
cfitsio’s compression algorithms are #defines; these have a namespace.
We deliberately don’t support HCOMPRESS: it doesn’t appear to be useful to us (e.g., lossy) and it requires extra configuration.
Values:
-
NONE
¶ No compression.
-
GZIP
¶ Standard GZIP compression.
-
GZIP_SHUFFLE
¶ GZIP compression with shuffle (most-significant byte first)
-
RICE
¶ RICE compression.
-
PLIO
¶ PLIO compression.
-
-
typedef ndarray::Array<long, 1, 1>
Tiles
¶
Public Functions
-
ImageCompressionOptions
(CompressionAlgorithm algorithm_, Tiles tiles_, float quantizeLevel_ = 0.0)¶ Custom compression.
-
ImageCompressionOptions
(CompressionAlgorithm algorithm_, std::vector<long> tiles_, float quantizeLevel_ = 0.0)¶
-
ImageCompressionOptions
(CompressionAlgorithm algorithm_, int rows = 1, float quantizeLevel_ = 0.0)¶ Compression by rows or entire image
- Parameters
[in] algorithm_
: Compression algorithm to use[in] rows
: Number of rows per tile (0 = entire image)[in] quantizeLevel_
: cfitsio quantization level
Public Members
-
CompressionAlgorithm
algorithm
¶ Compresion algorithm to use.
-
float
quantizeLevel
¶ quantization level: 0.0 = none requires use of GZIP or GZIP_SHUFFLE
-
struct
ImageScale
¶ - #include <fitsCompression.h>
Scale to apply to image
Images are scaled to the type implied by the provided BITPIX using the provided scale and zero-point:
value in memory = BZERO + BSCALE * value in FITS
In addition to scaling, a random field of values distributed [0,1) may be added before quantisation (“fuzz”); this preserves the expectation value of the floating-point image, while increasing the variance by 1/12.
Public Functions
-
ImageScale
(int bitpix_, double bscale_, double bzero_)¶ Constructor
We make BZERO an integer multiple of BSCALE, because cfitsio notes: “This helps to ensure the same scaling will be performed if the
file undergoes multiple fpack/funpack cycles”.
The BLANK is 255 for BITPIX=8 since FITS specifies that uses unsigned char; otherwise it is set to the maximum int for the appropriate signed integer.
Convert to an array of pixel values to write to FITS
- Return
Array of pixel values, appropriately scaled.
- Parameters
[in] image
: Image to scale[in] forceNonfiniteRemoval
: Force removal of non-finite values? This is useful for lossless compression, because cfitsio doesn’t mask out non-finite values, and they end up contaminating the entire tile.[in] fuzz
: Add random values before quantising?[in] tiles
: Tile dimensions[in] seed
: Seed for random number generator
-
-
class
ImageScalingOptions
- #include <fitsCompression.h>
Options for scaling image pixels
Scaling (quantisation) of floating-point images is important in order to achieve respectable compression factors. Unfortunately, scaling a floating-point image means losing some information, in two ways: values are quantised so values in between the quanta will be rounded up or down; and values outside the range of supported values in the integer image will be clipped. This implementation is based on the successful implementation used by Pan-STARRS.
cfitsio provides a facility for scaling image pixels on write (see ImageCompressionOptions.quantizeLevel), but that facility is limited (it provides a single scaling option, and doesn’t use our masks when collecting statistics). This scaling facility provides multiple scaling options, and could be extended to support more in the future (e.g., logarithmic or asinh scaling).
Scaling is specified by:
the scaling algorithm: the algorithm used to determining the scales (and therefore the dynamic range).
the bits per pixel: 8,16,32,64,-32,-64 (negative means floating-point but you probably don’t want to use those here); larger values give more dynamic range, produce larger images (which effect may be negated by compression).
fuzz: whether to add a random field of values distributed [0,1) before quantisation; this preserves the expectation value of the floating-point image, while increasing the variance by 1/12.
seed: seed for the random number generator used by the fuzz.
maskPlanes: a list of mask planes to be ignored when doing statistics.
quantizeLevel: for the STDEV_* algorithms, specifies the ratio of the quantum size to the image standard deviation.
quantizePad: for the STDEV_POSITIVE and STDEV_NEGATIVE algorithms, specifies how many standard deviations to allow on the short side.
bscale, bzero: for the MANUAL algorithm, specifies the BSCALE and BZERO to use.
Scaling algorithms are:
NONE: no scaling or quantisation at all. The image goes out the way it came in.
RANGE: scale based on the dynamic range in the image. This preserves dynamic range at the cost of sensitivity to low-level fluctuations.
STDEV_POSITIVE: the scale is set to sample the standard deviation of the image (bscale = stdev/quantizeLevel) and the dynamic range extends principally positive (allowing only quantizePad standard deviations negative).
STDEV_NEGATIVE: similar to STDEV_POSITIVE, but the dynamic range extends principally negative.
STDEV_BOTH: the scale is set similar to STDEV_POSITIVE, but the dynamic range is shared between the positive and negative sides.
MANUAL: the scale is set manually. We do what we’re told, no more, no less.
Perhaps this one class could/should have been polymorphic, with different subclasses for different algorithms? But I went with a C-like approach keying off the enum, probably because I was influenced by Pan-STARRS’ C code.
Unnamed Group
Determine the scaling for a particular image
- Parameters
[in] image
: Image for which to determine scaling[in] mask
: Mask for image (used to measuring statistics)
Public Types
-
enum
ScalingAlgorithm
¶ Values:
-
NONE
No scaling.
-
RANGE
¶ Scale to preserve dynamic range.
-
STDEV_POSITIVE
¶ Scale based on the standard deviation. dynamic range positive.
-
STDEV_NEGATIVE
¶ Scale based on the standard deviation, dynamic range negative.
-
STDEV_BOTH
¶ Scale based on the standard deviation, dynamic range positive+negative.
-
MANUAL
¶ Scale set manually.
-
Public Functions
-
ImageScalingOptions
() Default Ctor
Scaling is disabled by default.
-
ImageScalingOptions
(ScalingAlgorithm algorithm_, int bitpix_, std::vector<std::string> const &maskPlanes_ = {}, int seed_ = 1, float quantizeLevel_ = 4.0, float quantizePad_ = 5.0, bool fuzz_ = true, double bscale_ = 1.0, double bzero_ = 0.0) General purpose Ctor
- Parameters
[in] algorithm_
: Scaling algorithm to use[in] bitpix_
: Bits per pixel (8,16,32,64,-32,-64), or 0 to match pixel type exactly.[in] maskPlanes_
: Mask planes to ignore when doing statistics[in] seed_
: Seed for random number generator when fuzzing[in] quantizeLevel_
: Divisor of the standard deviation for STDEV_* scaling[in] quantizePad_
: Number of stdev to allow on the low side (for STDEV_POSITIVE/NEGATIVE)[in] fuzz_
: Fuzz the values when quantising floating-point values?[in] bscale_
: Manually specified BSCALE (for MANUAL scaling)[in] bzero_
: Manually specified BZERO (for MANUAL scaling)
-
ImageScalingOptions
(int bitpix_, double bscale_ = 1.0, double bzero_ = 0.0) Manual scaling Ctor
- Parameters
[in] bitpix_
: Bits per pixel (8,16,32,64,-32,-64), or 0 to match pixel type exactly.[in] bscale_
: Manually specified BSCALE[in] bzero_
: Manually specified BZERO
Public Members
-
ScalingAlgorithm
algorithm
Scaling algorithm to use.
-
int
bitpix
Bits per pixel (0, 8,16,32,64,-32,-64)
-
bool
fuzz
Fuzz the values when quantising floating-point values?
-
int
seed
Seed for random number generator when fuzzing.
-
float
quantizeLevel
Divisor of the standard deviation for STDEV_* scaling.
-
float
quantizePad
Number of stdev to allow on the low/high side (for STDEV_POSITIVE/NEGATIVE)
-
double
bscale
Manually specified BSCALE (for MANUAL scaling)
-
double
bzero
Manually specified BZERO (for MANUAL scaling)
Private Functions
Convert image,mask to arrays.
-
template<typename
T
, intN
>
ImageScaledetermineFromRange
(ndarray::Array<T const, N, N> const &image, ndarray::Array<bool, N, N> const &mask, bool isUnsigned = false, bool cfitsioPadding = true) const¶ Determine scale using the RANGE algorithm
- Parameters
[in] image
: Image for which to determine scaling[in] mask
: Mask for image (used to measuring statistics)[in] isUnsigned
: Is the output using an unsigned integer?[in] cfitsioPadding
: Pad the low end like cfitsio does (for BITPIX=32)?
-
template<typename
T
, intN
>
ImageScaledetermineFromStdev
(ndarray::Array<T const, N, N> const &image, ndarray::Array<bool, N, N> const &mask, bool isUnsigned = false, bool cfitsioPadding = true) const¶ Determine scale using the STDEV algorithm
- Parameters
[in] image
: Image for which to determine scaling[in] mask
: Mask for image (used to measuring statistics)[in] isUnsigned
: Is the output using an unsigned integer?[in] cfitsioPadding
: Pad the low end like cfitsio does (for BITPIX=32)?
-
namespace
detail
¶ Functions
Create a PixelArray suitable for an image with the nominated BITPIX
- Return
A new PixelArray.
- Parameters
[in] bitpix
: Bits per pixel (0,8,16,32,64,-32,-64) for output.[in] array
: Array with pixel values to convert.
-
template<typename
T
>
structBitpix
¶ - #include <fitsCompression.h>
FITS BITPIX header value by C++ type.
-
template<>
structBitpix
<double>¶ Public Static Attributes
-
int const
value
= -64
-
int const
-
template<>
structBitpix
<float>¶ Public Static Attributes
-
int const
value
= -32
-
int const
-
template<>
structBitpix
<std::int16_t> Public Static Attributes
-
int const
value
= 16
-
int const
-
template<>
structBitpix
<std::int32_t> Public Static Attributes
-
int const
value
= 32
-
int const
-
template<>
structBitpix
<std::int64_t> Public Static Attributes
-
int const
value
= 64
-
int const
-
template<>
structBitpix
<std::uint16_t> Public Static Attributes
-
int const
value
= 16
-
int const
-
template<>
structBitpix
<std::uint32_t> Public Static Attributes
-
int const
value
= 32
-
int const
-
template<>
structBitpix
<std::uint64_t> Public Static Attributes
-
int const
value
= 64
-
int const
-
template<>
structBitpix
<std::uint8_t> Public Static Attributes
-
int const
value
= 8
-
int const
-
template<typename
T
>
classPixelArray
: public lsst::afw::fits::detail::PixelArrayBase - #include <fitsCompression.h>
Typed array of pixel values.
Public Functions
-
PixelArray
()
-
PixelArray
(PixelArray const&)
-
PixelArray
(ndarray::Array<T, 1, 1> const &array) Construct from an ndarray::Array of the same type
Copies pointers only, no deep copy.
-
template<typename
U
>PixelArray
(ndarray::Array<U, 1, 1> const &array) Construct from an ndarray::Array of different type
Allocates memory and sets values: necessary to support the type change.
-
~PixelArray
()
-
void const *
getData
() const Return a void* array of the pixels.
-
-
class
PixelArrayBase
- #include <fitsCompression.h>
Abstract base class for an array of pixel values
For writing pixels out with the C-based cfitsio API, it’s helpful to have a type-agnostic holder of the pixel values (because although we can template on the array type, the output type can be configured at run-time, which means we can’t use ndarray::Array as the carrier). This is essentially a C++-ish void* array.
Subclassed by lsst::afw::fits::detail::PixelArray< T >
Public Functions
-
virtual
~PixelArrayBase
()
-
virtual void const *
getData
() const = 0 Return a void* array of the pixels.
-
std::size_t
getNumElements
() const Return the number of pixels.
Protected Functions
-
PixelArrayBase
(std::size_t num)
-
virtual
-
ImageCompressionOptions::CompressionAlgorithm
-
namespace