ToArrow

class lsst.daf.butler.arrow_utils.ToArrow

Bases: ABC

An abstract interface for converting objects to an Arrow field of the appropriate type.

Attributes Summary

data_type	Arrow data type for the field this converter produces.
field	Arrow field this converter produces.
name	Name of the field.
nullable	Whether the field permits null or None values.

Methods Summary

append(value, column)	Append an object's arrow representation to a list.
dictionary_encoded()	Return a new converter with the same name and type, but using dictionary encoding (to 32-bit integers) to compress duplicate values.
finish(column)	Convert a list of values constructed via append into an Arrow array.
for_datetime(name[, nullable])	Return a converter for astropy.time.Time, stored as TAI nanoseconds since 1970-01-01.
for_primitive(name, data_type, nullable)	Return a converter for a primitive type already supported by Arrow.
for_region(name[, nullable])	Return a converter for lsst.sphgeom.Region.
for_timespan(name[, nullable])	Return a converter for lsst.daf.butler.Timespan.
for_uuid(name[, nullable])	Return a converter for uuid.UUID.

Attributes Documentation

data_type

Arrow data type for the field this converter produces.

field

Arrow field this converter produces.

name

Name of the field.

nullable

Whether the field permits null or None values.

Methods Documentation

abstract append(value: Any, column: list[Any]) → None

Append an object’s arrow representation to a list.

Parameters:

valueobject

Original value to be converted to a row in an Arrow column.

columnlist

List of values to append to. The type of value to append is implementation-defined; the only requirement is that finish be able to handle this list later.

dictionary_encoded() → ToArrow

Return a new converter with the same name and type, but using dictionary encoding (to 32-bit integers) to compress duplicate values.

abstract finish(column: list[Any]) → Array

Convert a list of values constructed via append into an Arrow array.

Parameters:

columnlist

List of column values populated by append.

static for_datetime(name: str, nullable: bool = True) → ToArrow

Return a converter for astropy.time.Time, stored as TAI nanoseconds since 1970-01-01.

Parameters:

namestr

Name of the schema field.

nullablebool

Whether the field should permit null or None values.

Returns:

to_arrowToArrow

Converter instance.

static for_primitive(name: str, data_type: DataType, nullable: bool) → ToArrow

Return a converter for a primitive type already supported by Arrow.

Parameters:

namestr

Name of the schema field.

data_typepyarrow.DataType

Arrow data type object.

nullablebool

Whether the field should permit null or None values.

Returns:

to_arrowToArrow

Converter instance.

static for_region(name: str, nullable: bool = True) → ToArrow

Return a converter for lsst.sphgeom.Region.

Parameters:

namestr

Name of the schema field.

nullablebool

Whether the field should permit null or None values.

Returns:

to_arrowToArrow

Converter instance.

static for_timespan(name: str, nullable: bool = True) → ToArrow

Return a converter for lsst.daf.butler.Timespan.

Parameters:

namestr

Name of the schema field.

nullablebool

Whether the field should permit null or None values.

Returns:

to_arrowToArrow

Converter instance.

static for_uuid(name: str, nullable: bool = True) → ToArrow

Return a converter for uuid.UUID.

Parameters:

namestr

Name of the schema field.

nullablebool

Whether the field should permit null or None values.

Returns:

to_arrowToArrow

Converter instance.