DiaPipelineTask¶
- class lsst.ap.association.DiaPipelineTask(initInputs=None, **kwargs)¶
Bases:
PipelineTask
Task for loading, associating and storing Difference Image Analysis (DIA) Objects and Sources.
Attributes Summary
Methods Summary
associateDiaSources
(diaSourceTable, ...)Associate DiaSources with DiaObjects.
createNewDiaObjects
(unAssocDiaSources)Loop through the set of DiaSources and create new DiaObjects for unassociated DiaSources.
Empty (clear) the metadata for this Task and all sub-Tasks.
Get metadata for all tasks.
Get the task name as a hierarchical name including parent task names.
getName
()Get the name of the task.
Get a dictionary of all tasks as a shallow copy.
makeField
(doc)Make a
lsst.pex.config.ConfigurableField
for this task.makeSubtask
(name, **keyArgs)Create a subtask as a new instance as the
name
attribute of this task.mergeAssociatedCatalogs
(preloadedDiaSources, ...)Merge the associated diaSource and diaObjects to their previous history.
mergeCatalogs
(originalCatalog, newCatalog, ...)Combine two catalogs, ensuring that the columns of the new catalog have the same dtype as the original.
purgeDiaObjects
(bbox, wcs, diaObjCat[, ...])Drop diaObjects that are outside the exposure bounding box.
run
(diaSourceTable, legacySolarSystemTable, ...)Process DiaSources and DiaObjects.
runForcedMeasurement
(diaObjects, ...)Forced Source Measurement
runQuantum
(butlerQC, inputRefs, outputRefs)Do butler IO and transform to provide in memory objects for tasks
run
method.Test the sorted DataFrame index for duplicates.
timer
(name[, logLevel])Context manager to log performance data for an arbitrary block of code.
updateObjectTable
(diaObjects, diaSources)Update the diaObject table with the new diaSource records.
writeToApdb
(updatedDiaObjects, ...)Write to the Alert Production Database (Apdb).
Attributes Documentation
Methods Documentation
- associateDiaSources(diaSourceTable, solarSystemObjectTable, diffIm, diaObjects)¶
Associate DiaSources with DiaObjects.
Associate new DiaSources with existing DiaObjects. Create new DiaObjects fron unassociated DiaSources. Index DiaSource catalogue after associations. Append new DiaObjects and DiaSources to their previous history. Test for DiaSource and DiaObject duplications. Compute DiaObject Summary statistics from their full DiaSource history. Test for duplication in the updated DiaObjects.
- Parameters:
- diaSourceTable
pandas.DataFrame
Newly detected DiaSources.
- solarSystemObjectTable
pandas.DataFrame
Preloaded Solar System objects expected to be visible in the image.
- diffIm
lsst.afw.image.ExposureF
Difference image exposure in which the sources in
diaSourceCat
were detected.- diaObjects
pandas.DataFrame
Table of DiaObjects from preloaded DiaObjects.
- diaSourceTable
- Returns:
- associatedDiaSources
pandas.DataFrame
Associated DiaSources with DiaObjects.
- newDiaObjects
pandas.DataFrame
Table of new DiaObjects after association.
- associatedSsSources
pandas.DataFrame
Table of new ssSources after association.
- associatedDiaSources
- createNewDiaObjects(unAssocDiaSources)¶
Loop through the set of DiaSources and create new DiaObjects for unassociated DiaSources.
- Parameters:
- unAssocDiaSources
pandas.DataFrame
Set of DiaSources to create new DiaObjects from.
- unAssocDiaSources
- Returns:
- results
lsst.pipe.base.Struct
Results struct containing:
- diaSources
pandas.DataFrame
DiaSource catalog with updated DiaObject ids.
- diaSources
- newDiaObjects
pandas.DataFrame
Newly created DiaObjects from the unassociated DiaSources.
- newDiaObjects
- nNewDiaObjects
int
Number of newly created diaObjects.
- nNewDiaObjects
- results
- getFullMetadata() TaskMetadata ¶
Get metadata for all tasks.
- Returns:
- metadata
TaskMetadata
The keys are the full task name. Values are metadata for the top-level task and all subtasks, sub-subtasks, etc.
- metadata
Notes
The returned metadata includes timing information (if
@timer.timeMethod
is used) and any metadata set by the task. The name of each item consists of the full task name with.
replaced by:
, followed by.
and the name of the item, e.g.:topLevelTaskName:subtaskName:subsubtaskName.itemName
using
:
in the full task name disambiguates the rare situation that a task has a subtask and a metadata item with the same name.
- getFullName() str ¶
Get the task name as a hierarchical name including parent task names.
- Returns:
- fullName
str
The full name consists of the name of the parent task and each subtask separated by periods. For example:
The full name of top-level task “top” is simply “top”.
The full name of subtask “sub” of top-level task “top” is “top.sub”.
The full name of subtask “sub2” of subtask “sub” of top-level task “top” is “top.sub.sub2”.
- fullName
- getName() str ¶
Get the name of the task.
- Returns:
- taskName
str
Name of the task.
- taskName
See also
getFullName
Get the full name of the task.
- getTaskDict() dict[str, weakref.ReferenceType[lsst.pipe.base.task.Task]] ¶
Get a dictionary of all tasks as a shallow copy.
- Returns:
- taskDict
dict
Dictionary containing full task name: task object for the top-level task and all subtasks, sub-subtasks, etc.
- taskDict
- classmethod makeField(doc: str) ConfigurableField ¶
Make a
lsst.pex.config.ConfigurableField
for this task.- Parameters:
- doc
str
Help text for the field.
- doc
- Returns:
- configurableField
lsst.pex.config.ConfigurableField
A
ConfigurableField
for this task.
- configurableField
Examples
Provides a convenient way to specify this task is a subtask of another task.
Here is an example of use:
class OtherTaskConfig(lsst.pex.config.Config): aSubtask = ATaskClass.makeField("brief description of task")
- makeSubtask(name: str, **keyArgs: Any) None ¶
Create a subtask as a new instance as the
name
attribute of this task.- Parameters:
- name
str
Brief name of the subtask.
- **keyArgs
Extra keyword arguments used to construct the task. The following arguments are automatically provided and cannot be overridden:
config
.parentTask
.
- name
Notes
The subtask must be defined by
Task.config.name
, an instance ofConfigurableField
orRegistryField
.
- mergeAssociatedCatalogs(preloadedDiaSources, associatedDiaSources, diaObjects, newDiaObjects, diffIm)¶
Merge the associated diaSource and diaObjects to their previous history.
- Parameters:
- preloadedDiaSources
pandas.DataFrame
Previously detected DiaSources, loaded from the APDB.
- associatedDiaSources
pandas.DataFrame
Associated DiaSources with DiaObjects.
- diaObjects
pandas.DataFrame
Table of DiaObjects from preloaded DiaObjects.
- newDiaObjects
pandas.DataFrame
Table of new DiaObjects after association.
- preloadedDiaSources
- Returns:
- mergedDiaSourceHistory
pandas.DataFrame
The combined catalog, with all of the rows from preloadedDiaSources catalog ordered before the rows of associatedDiaSources catalog.
- mergedDiaObjects
pandas.DataFrame
Table of new DiaObjects merged with their history.
- updatedDiaObjectIds
numpy.Array
Object Id’s from associated diaSources.
- mergedDiaSourceHistory
- Raises:
- RuntimeError
Raised if duplicate DiaObjects or duplicate DiaSources are found.
- mergeCatalogs(originalCatalog, newCatalog, catalogName)¶
Combine two catalogs, ensuring that the columns of the new catalog have the same dtype as the original.
- Parameters:
- originalCatalog
pandas.DataFrame
The original catalog to be added to.
- newCatalog
pandas.DataFrame
The new catalog to append to
originalCatalog
- catalogName
str
, optional The name of the catalog to use for logging messages.
- originalCatalog
- Returns:
- mergedCatalog
pandas.DataFrame
The combined catalog, with all of the rows from
originalCatalog
ordered before the rows ofnewCatalog
- mergedCatalog
- purgeDiaObjects(bbox, wcs, diaObjCat, diaObjectIds=None, buffer=0)¶
Drop diaObjects that are outside the exposure bounding box.
- Parameters:
- bbox
lsst.geom.Box2I
Bounding box of the exposure.
- wcs
lsst.afw.geom.SkyWcs
Coordinate system definition (wcs) for the exposure.
- diaObjCat
pandas.DataFrame
DiaObjects loaded from the Apdb.
- buffer
int
, optional Width, in pixels, to pad the exposure bounding box.
- bbox
- Returns:
- diaObjCat
pandas.DataFrame
DiaObjects loaded from the Apdb, restricted to the exposure bounding box.
- diaObjCat
- run(diaSourceTable, legacySolarSystemTable, diffIm, exposure, template, preloadedDiaObjects, preloadedDiaSources, preloadedDiaForcedSources, band, idGenerator, solarSystemObjectTable=None)¶
Process DiaSources and DiaObjects.
Load previous DiaObjects and their DiaSource history. Calibrate the values in the diaSourceCat. Associate new DiaSources with previous DiaObjects. Run forced photometry at the updated DiaObject locations. Store the results in the Alert Production Database (Apdb).
- Parameters:
- diaSourceTable
pandas.DataFrame
Newly detected DiaSources.
- legacySolarSystemTable
pandas.DataFrame
Not used
- diffIm
lsst.afw.image.ExposureF
Difference image exposure in which the sources in
diaSourceCat
were detected.- exposure
lsst.afw.image.ExposureF
Calibrated exposure differenced with a template to create
diffIm
.- template
lsst.afw.image.ExposureF
Template exposure used to create diffIm.
- preloadedDiaObjects
pandas.DataFrame
Previously detected DiaObjects, loaded from the APDB.
- preloadedDiaSources
pandas.DataFrame
Previously detected DiaSources, loaded from the APDB.
- preloadedDiaForcedSources
pandas.DataFrame
Catalog of previously detected forced DiaSources, from the APDB
- band
str
The band in which the new DiaSources were detected.
- idGenerator
lsst.meas.base.IdGenerator
Object that generates source IDs and random number generator seeds.
- solarSystemObjectTable
pandas.DataFrame
Preloaded Solar System objects expected to be visible in the image.
- diaSourceTable
- Returns:
- results
lsst.pipe.base.Struct
Results struct with components.
apdbMarker
: Marker dataset to store in the Butler indicating that this ccdVisit has completed successfully. (lsst.dax.apdb.ApdbConfig
)associatedDiaSources
: Catalog of newly associated DiaSources. (pandas.DataFrame
)diaForcedSources
: Catalog of new and previously detected forced DiaSources. (pandas.DataFrame
)diaObjects
: Updated table of DiaObjects. (pandas.DataFrame
)associatedSsSources
: Catalog of ssSource records. (pandas.DataFrame
)
- results
- Raises:
- RuntimeError
Raised if duplicate DiaObjects or duplicate DiaSources are found.
- runForcedMeasurement(diaObjects, updatedDiaObjects, exposure, diffIm, idGenerator)¶
Forced Source Measurement
Forced photometry on the difference and calibrated exposures using the new and updated DiaObject locations.
- Parameters:
- diaObjects
pandas.DataFrame
Catalog of DiaObjects.
- updatedDiaObjects
pandas.DataFrame
Catalog of updated DiaObjects.
- exposure
lsst.afw.image.ExposureF
Calibrated exposure differenced with a template to create
diffIm
.- diffIm
lsst.afw.image.ExposureF
Difference image exposure in which the sources in
diaSourceCat
were detected.- idGenerator
lsst.meas.base.IdGenerator
Object that generates source IDs and random number generator seeds.
- diaObjects
- Returns:
- diaForcedSources
pandas.DataFrame
Catalog of calibrated forced photometered fluxes on both the difference and direct images at DiaObject locations.
- diaForcedSources
- runQuantum(butlerQC, inputRefs, outputRefs)¶
Do butler IO and transform to provide in memory objects for tasks
run
method.- Parameters:
- butlerQC
QuantumContext
A butler which is specialized to operate in the context of a
lsst.daf.butler.Quantum
.- inputRefs
InputQuantizedConnection
Datastructure whose attribute names are the names that identify connections defined in corresponding
PipelineTaskConnections
class. The values of these attributes are thelsst.daf.butler.DatasetRef
objects associated with the defined input/prerequisite connections.- outputRefs
OutputQuantizedConnection
Datastructure whose attribute names are the names that identify connections defined in corresponding
PipelineTaskConnections
class. The values of these attributes are thelsst.daf.butler.DatasetRef
objects associated with the defined output connections.
- butlerQC
- testDataFrameIndex(df)¶
Test the sorted DataFrame index for duplicates.
Wrapped as a separate function to allow for mocking of the this task in unittesting. Default of a mock return for this test is True.
- Parameters:
- df
pandas.DataFrame
DataFrame to text.
- df
- Returns:
bool
True if DataFrame contains duplicate rows.
- timer(name: str, logLevel: int = 10) Iterator[None] ¶
Context manager to log performance data for an arbitrary block of code.
- Parameters:
See also
lsst.utils.timer.logInfo
Implementation function.
Examples
Creating a timer context:
with self.timer("someCodeToTime"): pass # code to time
- updateObjectTable(diaObjects, diaSources)¶
Update the diaObject table with the new diaSource records.
- Parameters:
- diaObjects
pandas.DataFrame
Table of new DiaObjects merged with their history.
- diaSources
pandas.DataFrame
The combined preloaded and associated diaSource catalog.
- diaObjects
- Returns:
- updatedDiaObjects
pandas.DataFrame
Table of DiaObjects updated with the number of associated DiaSources
- updatedDiaObjects
- writeToApdb(updatedDiaObjects, associatedDiaSources, diaForcedSources)¶
Write to the Alert Production Database (Apdb).
Store DiaSources, updated DiaObjects, and DiaForcedSources in the Alert Production Database (Apdb).
- Parameters:
- updatedDiaObjects
pandas.DataFrame
Catalog of updated DiaObjects.
- associatedDiaSources
pandas.DataFrame
Associated DiaSources with DiaObjects.
- diaForcedSources
pandas.DataFrame
Catalog of calibrated forced photometered fluxes on both the difference and direct images at DiaObject locations.
- updatedDiaObjects