ApdbCassandra¶
-
class
lsst.dax.apdb.
ApdbCassandra
(config: lsst.dax.apdb.apdbCassandra.ApdbCassandraConfig)¶ Bases:
lsst.dax.apdb.Apdb
Implementation of APDB database on to of Apache Cassandra.
The implementation is configured via standard
pex_config
mechanism usingApdbCassandraConfig
configuration class. For an example of different configurations check config/ folder.Parameters: - config :
ApdbCassandraConfig
Configuration object.
Attributes Summary
partition_zero_epoch
Start time for partition 0, this should never be changed. Methods Summary
countUnassociatedObjects
()Return the number of DiaObjects that have only one DiaSource associated with them. dailyJob
()Implement daily activities like cleanup/vacuum. getDiaForcedSources
(region, object_ids, …)Return catalog of DiaForcedSource instances from a given region. getDiaObjects
(region)Returns catalog of DiaObject instances from a given region. getDiaSources
(region, object_ids, visit_time)Return catalog of DiaSource instances from a given region. makeField
(doc)Make a ConfigurableField
for Apdb.makeSchema
(drop)Create or re-create whole database schema. store
(visit_time, objects, sources, …)Store all three types of catalogs in the database. tableDef
(table)Return table schema definition for a given table. Attributes Documentation
-
partition_zero_epoch
= DateTime("1970-01-01T00:00:00.000000000", TAI)¶ Start time for partition 0, this should never be changed.
Methods Documentation
-
countUnassociatedObjects
() → int¶ Return the number of DiaObjects that have only one DiaSource associated with them.
Used as part of ap_verify metrics.
Returns: - count :
int
Number of DiaObjects with exactly one associated DiaSource.
Notes
This method can be very inefficient or slow in some implementations.
- count :
-
dailyJob
() → None¶ Implement daily activities like cleanup/vacuum.
What should be done during daily activities is determined by specific implementation.
-
getDiaForcedSources
(region: lsst.sphgeom._sphgeom.Region, object_ids: Optional[Iterable[int]], visit_time: lsst.daf.base.dateTime.dateTime.DateTime) → Optional[pandas.core.frame.DataFrame]¶ Return catalog of DiaForcedSource instances from a given region.
Parameters: - region :
lsst.sphgeom.Region
Region to search for DIASources.
- object_ids : iterable [
int
], optional List of DiaObject IDs to further constrain the set of returned sources. If list is empty then empty catalog is returned with a correct schema. If
None
then returned sources are not constrained. Some implementations may not support latter case.- visit_time :
lsst.daf.base.DateTime
Time of the current visit.
Returns: - catalog :
pandas.DataFrame
, orNone
Catalog containing DiaSource records.
None
is returned ifread_forced_sources_months
configuration parameter is set to 0.
Raises: - NotImplementedError
May be raised by some implementations if
object_ids
isNone
.
Notes
This method returns DiaForcedSource catalog for a region with additional filtering based on DiaObject IDs. Only a subset of DiaSource history is returned limited by
read_forced_sources_months
config parameter, w.r.t.visit_time
. Ifobject_ids
is empty then an empty catalog is always returned with the correct schema (columns/types). Ifobject_ids
isNone
then no filtering is performed and some of the returned records may be outside the specified region.- region :
-
getDiaObjects
(region: lsst.sphgeom._sphgeom.Region) → pandas.core.frame.DataFrame¶ Returns catalog of DiaObject instances from a given region.
This method returns only the last version of each DiaObject. Some records in a returned catalog may be outside the specified region, it is up to a client to ignore those records or cleanup the catalog before futher use.
Parameters: - region :
lsst.sphgeom.Region
Region to search for DIAObjects.
Returns: - catalog :
pandas.DataFrame
Catalog containing DiaObject records for a region that may be a superset of the specified region.
- region :
-
getDiaSources
(region: lsst.sphgeom._sphgeom.Region, object_ids: Optional[Iterable[int]], visit_time: lsst.daf.base.dateTime.dateTime.DateTime) → Optional[pandas.core.frame.DataFrame]¶ Return catalog of DiaSource instances from a given region.
Parameters: - region :
lsst.sphgeom.Region
Region to search for DIASources.
- object_ids : iterable [
int
], optional List of DiaObject IDs to further constrain the set of returned sources. If
None
then returned sources are not constrained. If list is empty then empty catalog is returned with a correct schema.- visit_time :
lsst.daf.base.DateTime
Time of the current visit.
Returns: - catalog :
pandas.DataFrame
, orNone
Catalog containing DiaSource records.
None
is returned ifread_sources_months
configuration parameter is set to 0.
Notes
This method returns DiaSource catalog for a region with additional filtering based on DiaObject IDs. Only a subset of DiaSource history is returned limited by
read_sources_months
config parameter, w.r.t.visit_time
. Ifobject_ids
is empty then an empty catalog is always returned with the correct schema (columns/types). Ifobject_ids
isNone
then no filtering is performed and some of the returned records may be outside the specified region.- region :
-
classmethod
makeField
(doc: str) → lsst.pex.config.configurableField.ConfigurableField¶ Make a
ConfigurableField
for Apdb.Parameters: - doc :
str
Help text for the field.
Returns: - configurableField :
lsst.pex.config.ConfigurableField
A
ConfigurableField
for Apdb.
- doc :
-
makeSchema
(drop: bool = False) → None¶ Create or re-create whole database schema.
Parameters: - drop :
bool
If True then drop all tables before creating new ones.
- drop :
-
store
(visit_time: lsst.daf.base.dateTime.dateTime.DateTime, objects: pandas.core.frame.DataFrame, sources: Optional[pandas.core.frame.DataFrame] = None, forced_sources: Optional[pandas.core.frame.DataFrame] = None) → None¶ Store all three types of catalogs in the database.
Parameters: - visit_time :
lsst.daf.base.DateTime
Time of the visit.
- objects :
pandas.DataFrame
Catalog with DiaObject records.
- sources :
pandas.DataFrame
, optional Catalog with DiaSource records.
- forced_sources :
pandas.DataFrame
, optional Catalog with DiaForcedSource records.
Notes
This methods takes DataFrame catalogs, their schema must be compatible with the schema of APDB table:
- column names must correspond to database table columns
- types and units of the columns must match database definitions, no unit conversion is performed presently
- columns that have default values in database schema can be omitted from catalog
- this method knows how to fill interval-related columns of DiaObject (validityStart, validityEnd) they do not need to appear in a catalog
- source catalogs have
diaObjectId
column associating sources with objects
- visit_time :
-
tableDef
(table: lsst.dax.apdb.apdbSchema.ApdbTables) → Optional[lsst.dax.apdb.apdbSchema.TableDef]¶ Return table schema definition for a given table.
Parameters: - table :
ApdbTables
One of the known APDB tables.
Returns: - table :
- config :