ApdbSql¶
- class lsst.dax.apdb.ApdbSql(config: ApdbSqlConfig)¶
Bases:
Apdb
Implementation of APDB interface based on SQL database.
The implementation is configured via standard
pex_config
mechanism usingApdbSqlConfig
configuration class. For an example of different configurations checkconfig/
folder.- Parameters:
- config
ApdbSqlConfig
Configuration object.
- config
Methods Summary
Return the number of DiaObjects that have only one DiaSource associated with them.
dailyJob
()Implement daily activities like cleanup/vacuum.
getDiaForcedSources
(region, object_ids, ...)Return catalog of DiaForcedSource instances from a given region.
getDiaForcedSourcesHistory
(start_time, end_time)Returns catalog of DiaForcedSource instances from a given time period.
getDiaObjects
(region)Returns catalog of DiaObject instances from a given region.
getDiaObjectsHistory
(start_time, end_time[, ...])Returns catalog of DiaObject instances from a given time period including the history of each DiaObject.
getDiaSources
(region, object_ids, visit_time)Return catalog of DiaSource instances from a given region.
getDiaSourcesHistory
(start_time, end_time[, ...])Returns catalog of DiaSource instances from a given time period.
Returns catalog of SSObject instances.
makeField
(doc)Make a
ConfigurableField
for Apdb.makeSchema
([drop])Create or re-create whole database schema.
reassignDiaSources
(idMap)Associate DiaSources with SSObjects, dis-associating them from DiaObjects.
store
(visit_time, objects[, sources, ...])Store all three types of catalogs in the database.
storeSSObjects
(objects)Store or update SSObject catalog.
tableDef
(table)Return table schema definition for a given table.
Returns dictionary with the table names and row counts.
Methods Documentation
- countUnassociatedObjects() int ¶
Return the number of DiaObjects that have only one DiaSource associated with them.
Used as part of ap_verify metrics.
- Returns:
- count
int
Number of DiaObjects with exactly one associated DiaSource.
- count
Notes
This method can be very inefficient or slow in some implementations.
- dailyJob() None ¶
Implement daily activities like cleanup/vacuum.
What should be done during daily activities is determined by specific implementation.
- getDiaForcedSources(region: Region, object_ids: Iterable[int] | None, visit_time: DateTime) DataFrame | None ¶
Return catalog of DiaForcedSource instances from a given region.
- Parameters:
- region
lsst.sphgeom.Region
Region to search for DIASources.
- object_idsiterable [
int
], optional List of DiaObject IDs to further constrain the set of returned sources. If list is empty then empty catalog is returned with a correct schema.
- visit_time
lsst.daf.base.DateTime
Time of the current visit.
- region
- Returns:
- catalog
pandas.DataFrame
, orNone
Catalog containing DiaSource records.
None
is returned ifread_sources_months
configuration parameter is set to 0.
- catalog
- Raises:
- NotImplementedError
Raised if
object_ids
isNone
.
Notes
Even though base class allows
None
to be passed forobject_ids
, this class requiresobject_ids
to be not-None
.NotImplementedError
is raised ifNone
is passed.This method returns DiaForcedSource catalog for a region with additional filtering based on DiaObject IDs. Only a subset of DiaSource history is returned limited by
read_forced_sources_months
config parameter, w.r.t.visit_time
. Ifobject_ids
is empty then an empty catalog is always returned with a correct schema (columns/types).
- getDiaForcedSourcesHistory(start_time: DateTime, end_time: DateTime, region: Region | None = None) DataFrame ¶
Returns catalog of DiaForcedSource instances from a given time period.
- Parameters:
- start_time
dafBase.DateTime
Starting time for DiaForcedSource history search. DiaForcedSource record is selected when its
midPointTai
falls into an interval betweenstart_time
(inclusive) andend_time
(exclusive).- end_time
dafBase.DateTime
Upper limit on time for DiaForcedSource history search.
- region
lsst.sphgeom.Region
, optional Region to search for DiaForcedSources, if not specified then whole sky is searched. If region is specified then some returned records may fall outside of this region.
- start_time
- Returns:
- catalog
pandas.DataFrame
Catalog containing DiaObject records.
- catalog
Notes
This part of API may not be very stable and can change before the implementation finalizes. Some implementations may not support region filtering, they will return records from the whole sky.
- getDiaObjects(region: Region) DataFrame ¶
Returns catalog of DiaObject instances from a given region.
This method returns only the last version of each DiaObject. Some records in a returned catalog may be outside the specified region, it is up to a client to ignore those records or cleanup the catalog before futher use.
- Parameters:
- region
lsst.sphgeom.Region
Region to search for DIAObjects.
- region
- Returns:
- catalog
pandas.DataFrame
Catalog containing DiaObject records for a region that may be a superset of the specified region.
- catalog
- getDiaObjectsHistory(start_time: DateTime, end_time: DateTime, region: Region | None = None) DataFrame ¶
Returns catalog of DiaObject instances from a given time period including the history of each DiaObject.
- Parameters:
- start_time
dafBase.DateTime
Starting time for DiaObject history search. DiaObject record is selected when its
validityStart
falls into an interval betweenstart_time
(inclusive) andend_time
(exclusive).- end_time
dafBase.DateTime
Upper limit on time for DiaObject history search.
- region
lsst.sphgeom.Region
, optional Region to search for DiaObjects, if not specified then whole sky is searched. If region is specified then some returned records may fall outside of this region.
- start_time
- Returns:
- catalog
pandas.DataFrame
Catalog containing DiaObject records.
- catalog
Notes
This part of API may not be very stable and can change before the implementation finalizes.
- getDiaSources(region: Region, object_ids: Iterable[int] | None, visit_time: DateTime) DataFrame | None ¶
Return catalog of DiaSource instances from a given region.
- Parameters:
- region
lsst.sphgeom.Region
Region to search for DIASources.
- object_idsiterable [
int
], optional List of DiaObject IDs to further constrain the set of returned sources. If
None
then returned sources are not constrained. If list is empty then empty catalog is returned with a correct schema.- visit_time
lsst.daf.base.DateTime
Time of the current visit.
- region
- Returns:
- catalog
pandas.DataFrame
, orNone
Catalog containing DiaSource records.
None
is returned ifread_sources_months
configuration parameter is set to 0.
- catalog
Notes
This method returns DiaSource catalog for a region with additional filtering based on DiaObject IDs. Only a subset of DiaSource history is returned limited by
read_sources_months
config parameter, w.r.t.visit_time
. Ifobject_ids
is empty then an empty catalog is always returned with the correct schema (columns/types). Ifobject_ids
isNone
then no filtering is performed and some of the returned records may be outside the specified region.
- getDiaSourcesHistory(start_time: DateTime, end_time: DateTime, region: Region | None = None) DataFrame ¶
Returns catalog of DiaSource instances from a given time period.
- Parameters:
- start_time
dafBase.DateTime
Starting time for DiaSource history search. DiaSource record is selected when its
midPointTai
falls into an interval betweenstart_time
(inclusive) andend_time
(exclusive).- end_time
dafBase.DateTime
Upper limit on time for DiaSource history search.
- region
lsst.sphgeom.Region
, optional Region to search for DiaSources, if not specified then whole sky is searched. If region is specified then some returned records may fall outside of this region.
- start_time
- Returns:
- catalog
pandas.DataFrame
Catalog containing DiaObject records.
- catalog
Notes
This part of API may not be very stable and can change before the implementation finalizes.
- getSSObjects() DataFrame ¶
Returns catalog of SSObject instances.
- Returns:
- catalog
pandas.DataFrame
Catalog containing SSObject records, all existing records are returned.
- catalog
- classmethod makeField(doc: str) ConfigurableField ¶
Make a
ConfigurableField
for Apdb.- Parameters:
- doc
str
Help text for the field.
- doc
- Returns:
- configurableField
lsst.pex.config.ConfigurableField
A
ConfigurableField
for Apdb.
- configurableField
- makeSchema(drop: bool = False) None ¶
Create or re-create whole database schema.
- Parameters:
- drop
bool
If True then drop all tables before creating new ones.
- drop
- reassignDiaSources(idMap: Mapping[int, int]) None ¶
Associate DiaSources with SSObjects, dis-associating them from DiaObjects.
- Parameters:
- idMap
Mapping
Maps DiaSource IDs to their new SSObject IDs.
- idMap
- Raises:
- ValueError
Raised if DiaSource ID does not exist in the database.
- store(visit_time: DateTime, objects: DataFrame, sources: DataFrame | None = None, forced_sources: DataFrame | None = None) None ¶
Store all three types of catalogs in the database.
- Parameters:
- visit_time
lsst.daf.base.DateTime
Time of the visit.
- objects
pandas.DataFrame
Catalog with DiaObject records.
- sources
pandas.DataFrame
, optional Catalog with DiaSource records.
- forced_sources
pandas.DataFrame
, optional Catalog with DiaForcedSource records.
- visit_time
Notes
This methods takes DataFrame catalogs, their schema must be compatible with the schema of APDB table:
column names must correspond to database table columns
types and units of the columns must match database definitions, no unit conversion is performed presently
columns that have default values in database schema can be omitted from catalog
this method knows how to fill interval-related columns of DiaObject (validityStart, validityEnd) they do not need to appear in a catalog
source catalogs have
diaObjectId
column associating sources with objects
- storeSSObjects(objects: DataFrame) None ¶
Store or update SSObject catalog.
- Parameters:
- objects
pandas.DataFrame
Catalog with SSObject records.
- objects
Notes
If SSObjects with matching IDs already exist in the database, their records will be updated with the information from provided records.
- tableDef(table: ApdbTables) TableDef | None ¶
Return table schema definition for a given table.
- Parameters:
- table
ApdbTables
One of the known APDB tables.
- table
- Returns:
- tableRowCount() Dict[str, int] ¶
Returns dictionary with the table names and row counts.
Used by
ap_proto
to keep track of the size of the database tables. Depending on database technology this could be expensive operation.- Returns:
- row_counts
dict
Dict where key is a table name and value is a row count.
- row_counts