Apdb¶
-
class
lsst.dax.apdb.
Apdb
(config, afw_schemas=None)¶ Bases:
object
Interface to L1 database, hides all database access details.
The implementation is configured via standard
pex_config
mechanism usingApdbConfig
configuration class. For an example of different configurations check config/ folder.Parameters: - config :
ApdbConfig
- afw_schemas :
dict
, optional Dictionary with table name for a key and
afw.table.Schema
for a value. Columns in schema will be added to standard APDB schema.
Methods Summary
countUnassociatedObjects
()Return the number of DiaObjects that have only one DiaSource associated with them. dailyJob
()Implement daily activities like cleanup/vacuum. getDiaForcedSources
(object_ids, dt[, …])Returns catalog of DiaForcedSource instances matching given DiaObjects. getDiaObjects
(pixel_ranges[, return_pandas])Returns catalog of DiaObject instances from given region. getDiaSources
(object_ids, dt[, return_pandas])Returns catalog of DiaSource instances given set of DiaObject IDs. getDiaSourcesInRegion
(pixel_ranges, dt[, …])Returns catalog of DiaSource instances from given region. isVisitProcessed
(visitInfo)Test whether data from an image has been loaded into the database. lastVisit
()Returns last visit information or None
if visits table is empty.makeSchema
([drop, mysql_engine, …])Create or re-create all tables. saveVisit
(visitId, visitTime)Store visit information. storeDiaForcedSources
(sources)Store a set of DIAForcedSources from current visit. storeDiaObjects
(objs, dt)Store catalog of DiaObjects from current visit. storeDiaSources
(sources)Store catalog of DIASources from current visit. tableRowCount
()Returns dictionary with the table names and row counts. Methods Documentation
-
countUnassociatedObjects
()¶ Return the number of DiaObjects that have only one DiaSource associated with them.
Used as part of ap_verify metrics.
Returns: - count :
int
Number of DiaObjects with exactly one associated DiaSource.
- count :
-
dailyJob
()¶ Implement daily activities like cleanup/vacuum.
What should be done during daily cleanup is determined by configuration/schema.
-
getDiaForcedSources
(object_ids, dt, return_pandas=False)¶ Returns catalog of DiaForcedSource instances matching given DiaObjects.
This method returns lsst.afw.table catalog with schema determined by the schema of L1 database table. Re-mapping of the column names may be done for some columns (based on column map passed to constructor) but types or units are not changed.
Parameters: - object_ids :
Collection of DiaObject IDs
- dt :
datetime.datetime
Time of the current visit
- return_pandas :
bool
Return a
pandas.DataFrame
instead oflsst.afw.table.SourceCatalog
.
Returns: - catalog :
lsst.afw.table.SourceCatalog
orNone
Catalog contaning DiaForcedSource records.
None
is returned ifread_sources_months
configuration parameter is set to 0 or whenobject_ids
is empty.
-
getDiaObjects
(pixel_ranges, return_pandas=False)¶ Returns catalog of DiaObject instances from given region.
Objects are searched based on pixelization index and region is determined by the set of indices. There is no assumption on a particular type of index, client is responsible for consistency when calculating pixelization indices.
This method returns lsst.afw.table catalog with schema determined by the schema of APDB table. Re-mapping of the column names is done for some columns (based on column map passed to constructor) but types or units are not changed.
Returns only the last version of each DiaObject.
Parameters: - pixel_ranges :
list
oftuple
Sequence of ranges, range is a tuple (minPixelID, maxPixelID). This defines set of pixel indices to be included in result.
- return_pandas :
bool
Return a
pandas.DataFrame
instead oflsst.afw.table.SourceCatalog
.
Returns: - catalog :
lsst.afw.table.SourceCatalog
orpandas.DataFrame
Catalog containing DiaObject records.
- pixel_ranges :
-
getDiaSources
(object_ids, dt, return_pandas=False)¶ Returns catalog of DiaSource instances given set of DiaObject IDs.
This method returns lsst.afw.table catalog with schema determined by the schema of APDB table. Re-mapping of the column names is done for some columns (based on column map passed to constructor) but types or units are not changed.
Parameters: - object_ids :
Collection of DiaObject IDs
- dt :
datetime.datetime
Time of the current visit
- return_pandas :
bool
Return a
pandas.DataFrame
instead oflsst.afw.table.SourceCatalog
.
Returns: - catalog :
lsst.afw.table.SourceCatalog
,pandas.DataFrame
, orNone
Catalog contaning DiaSource records.
None
is returned ifread_sources_months
configuration parameter is set to 0 or whenobject_ids
is empty.
-
getDiaSourcesInRegion
(pixel_ranges, dt, return_pandas=False)¶ Returns catalog of DiaSource instances from given region.
Sources are searched based on pixelization index and region is determined by the set of indices. There is no assumption on a particular type of index, client is responsible for consistency when calculating pixelization indices.
This method returns lsst.afw.table catalog with schema determined by the schema of APDB table. Re-mapping of the column names is done for some columns (based on column map passed to constructor) but types or units are not changed.
Parameters: - pixel_ranges :
list
oftuple
Sequence of ranges, range is a tuple (minPixelID, maxPixelID). This defines set of pixel indices to be included in result.
- dt :
datetime.datetime
Time of the current visit
- return_pandas :
bool
Return a
pandas.DataFrame
instead oflsst.afw.table.SourceCatalog
.
Returns: - catalog :
lsst.afw.table.SourceCatalog
,pandas.DataFrame
, orNone
Catalog containing DiaSource records.
None
is returned ifread_sources_months
configuration parameter is set to 0.
- pixel_ranges :
-
isVisitProcessed
(visitInfo)¶ Test whether data from an image has been loaded into the database.
Used as part of ap_verify metrics.
Parameters: - visitInfo :
lsst.afw.image.VisitInfo
The metadata for the image of interest.
Returns: - visitInfo :
-
lastVisit
()¶ Returns last visit information or
None
if visits table is empty.Visits table is used by ap_proto to track visit information, it is not a part of the regular APDB schema.
Returns:
-
makeSchema
(drop=False, mysql_engine='InnoDB', oracle_tablespace=None, oracle_iot=False)¶ Create or re-create all tables.
Parameters:
-
saveVisit
(visitId, visitTime)¶ Store visit information.
This method is only used by
ap_proto
script froml1dbproto
and is not intended for production pipelines.Parameters: - visitId :
int
Visit identifier
- visitTime :
datetime.datetime
Visit timestamp.
- visitId :
-
storeDiaForcedSources
(sources)¶ Store a set of DIAForcedSources from current visit.
This methods takes lsst.afw.table catalog, its schema must be compatible with the schema of L1 database table:
- column names must correspond to database table columns
- some columns names may be re-mapped based on column map passed to constructor
- types and units of the columns must match database definitions, no unit conversion is performed presently
- columns that have default values in database schema can be omitted from afw schema
Parameters: - sources :
lsst.afw.table.BaseCatalog
orpandas.DataFrame
Catalog containing DiaForcedSource records
-
storeDiaObjects
(objs, dt)¶ Store catalog of DiaObjects from current visit.
This methods takes lsst.afw.table catalog, its schema must be compatible with the schema of APDB table:
- column names must correspond to database table columns
- some columns names are re-mapped based on column map passed to constructor
- types and units of the columns must match database definitions, no unit conversion is performed presently
- columns that have default values in database schema can be omitted from afw schema
- this method knows how to fill interval-related columns (validityStart, validityEnd) they do not need to appear in afw schema
Parameters: - objs :
lsst.afw.table.BaseCatalog
orpandas.DataFrame
Catalog with DiaObject records
- dt :
datetime.datetime
Time of the visit
-
storeDiaSources
(sources)¶ Store catalog of DIASources from current visit.
This methods takes lsst.afw.table catalog, its schema must be compatible with the schema of L1 database table:
- column names must correspond to database table columns
- some columns names may be re-mapped based on column map passed to constructor
- types and units of the columns must match database definitions, no unit conversion is performed presently
- columns that have default values in database schema can be omitted from afw schema
Parameters: - sources :
lsst.afw.table.BaseCatalog
orpandas.DataFrame
Catalog containing DiaSource records
- config :