Data access and pipeline middleware

• Frequently asked questions
  • When should I use each of the query methods/commands?
    • Collections
    • Dataset Types
    • Dimension Records
    • Datasets
    • Data Ids
  • Where can I find documentation for command-line butler queries?
  • Why do queries using Registry methods return duplicate results?
  • Why are some keys (usually filters) sometimes missing from data IDs?
  • How do I make a dict from a DataCoordinate?
  • How do I avoid errors involving queries for calibration datasets?
  • How do I fix an empty QuantumGraph?
  • How can I make QuantumGraph generation faster?
  • How do I recover from a small number of failed quanta in a big graph?
    • Accepting failures
    • Fixing failures
    • Fixing non-failures
    • Data ID constraints
  • What do I do if a query method/command is slow?
  • How can I get a report on all failures and missing datasets in a run?
    • The Quanta Table
      • Status Definitions for Task Quanta
        • Unknown
        • Successful
        • Blocked
        • Failed
        • Wonky
        • Total
        • Expected
    • The Dataset Table
      • Status Definitions for Datasets
        • Visible
        • Shadowed
        • Predicted Only
        • Unsuccessful
        • Cursed
        • Total
        • Expected
    • Combining group-level pipetask report summaries into a step-level rollup
  • How do I clean up processing runs I don’t need anymore?
    • The Easy Way: deleting everything
    • The Easy Way: delete a bad output runs
    • Deleting only some datasets