1. LargeSynopticSurveyTelescope(LSST)
  2. DataManagementOrganizationand
  3. Management
  4. WilliamO’Mullane,JohnSwinbank,MarioJuricandDMLT
  5. LDM-294
  6. LatestRevision: 2017-07-19
  7. Abstract
  8. ChangeRecord
  9. Contents
      1. 1 Introduction 1
      2. 2 DataManagementConceptualArchitecture 3
      3. 3 DataManagementOrganizationStructure 7
      4. 4 ProjectControls 17
      5. 5 Products 19
      6. 6 RolesinDataManagement 20
      7. 7 DataManagementGroups/Bodies 29
      8. 8 LeadinstitutionsinDM 35
      9. 9 DevelopmentProcess 42
      10. 10 DataManagementProblem/ConflictResolution 43
      11. A DMProductList 45
      12. B ProposedFY2018WBS:1.02C:DataManagementConstruction 49
      13. C DMDiscussionandDecisionMakingProcess 85
      14. D TraceabilitymatrixofDMSR equirementstoOSSRequirements 88
      15. E TraceabilitymatrixofOSS equirementstoDMSR equirements 99
      16. F References 107
      17. G Acronyms 109
  10. DataManagementOrganizationandManagement
  11. 1 Introduction
      1. 1.1 Purpose
      2. 1.2 MissionStatement
      3. 1.3 GoalsandObjectives
  12. 2 Data Management Conceptual Architecture
      1. 2.1 External Interfaces&AuxiliaryData
  13. 3 Data Management Organization Structure
      1. 3.1 Meetings
      2. 3.2 WorkingGroups
      3. 3.3 ExternalStudies
      4. 3.4 DocumentManagement
      5. 3.4.1 Documentation of Cross-Cutting Aspects for services
      6. 3.4.2 DocumentationofServiceLayers
      7. 3.4.3 DraftDocuments
      8. 3.5 ConfigurationControl
      9. 3.5.1 Software Configuration Control
      10. 3.5.2 Hardware Configuration Control
      11. 3.6 RiskManagement
      12. 3.7 QualityAssurance
      13. 3.8 Actionitemcontrol
      14. 3.9 VerificationandValidation
  14. 4 ProjectControls
      1. 4.1 Schedule
      2. 4.2 WorkBreakdownStructure
      3. 4.3 PlanningProcess
  15. 5 Products
  16. 6 RolesinDataManagement
      1. 6.1 DMProjectManager(DMPM)
      2. 6.2 DMDeputyProjectManager(DDMPM)
      3. 6.3 DMSubsystemScientist(DMSS)
      4. 6.4 Project Controller/Scheduler
      5. 6.5 ProductOwner
      6. 6.6 PipelinesScientist
      7. 6.7 SystemsEngineer
      8. 6.8 DM InterfaceScientist(DMIS)
      9. 6.9 SoftwareArchitect
      10. 6.10 OperationsArchitect
      11. 6.11 ReleaseManager(RM)
      12. 6.12 Lead InstitutionSeniorPositions
      13. 6.12.1 Technical/Control Account Manager (T/CAM)
      14. 6.12.2 Institutional Science/Engineering Lead
      15. 6.13 DMScienceValidationScientist
  17. 7 DataManagementGroups/Bodies
      1. 7.1 SystemScienceTeam
      2. 7.1.1 OrganizationandGoals
      3. 7.1.2 Communications
      4. 7.2 DMSystemsEngineeringTeam
      5. 7.2.1 Communications
      6. 7.3 DMLeadershipTeam
      7. 7.3.1 Communications
      8. 7.4 DMChangeControlBoard
      9. 7.5 DMScienceValidationTeam
  18. 8 LeadinstitutionsinDM
      1. 8.1 LSSTTucson
      2. 8.1.1 Science Quality and Reliability Engineering
      3. 8.1.2 LSST International Communications and Base Site
      4. 8.2 PrincetonUniversity
      5. 8.2.1 DataReleaseProduction
      6. 8.3 TheUniversityofWashington
      7. 8.3.1 AlertProduction
      8. 8.4 California Institute of Technology/IPAC
      9. 8.4.1 ScienceUser InterfaceandTools
      10. 8.5 SLAC
      11. 8.5.1 ScienceDataArchiveandDataAccessServices
      12. 8.6 NCSA
  19. 9 DevelopmentProcess
      1. 9.1 Communications
      2. 9.2 Conventions
      3. 9.3 Reviews
  20. 10 Data Management Problem/Conflict Resolution
  21. A DMProductList
  22. B Proposed FY2018 WBS: 1.02C: Data Management Construction
      1. 1.02C.01: SystemManagement
      2. 1.02C.01.01: Project Management
      3. 1.02C.01.02:[Legacy]ScienceDataQuality IntegrationandTest
      4. 1.02C.02: SystemsEngineering
      5. 1.02C.02.01: DataManagementScience
      6. 1.02C.02.02: DMSystemArchitecture
      7. 1.02C.03: AlertProduction
      8. 1.02C.03.00:Management,Leadership&OtherCosts
      9. 1.02C.03.01: SingleFrameProcessing
      10. 1.02C.03.02:CatalogAssociationforAlertProduction
      11. 1.02C.03.03: AlertDistributionSystem
      12. 1.02C.03.04: AlertGenerationPipeline
      13. 1.02C.03.05:ToolsforSciencePipelines
      14. 1.02C.03.06:MovingObjectProcessingSystem(MOPS)
      15. 1.02C.03.07: T ansform fitting on stacks of images
      16. 1.02C.03.08: Integration
      17. 1.02C.04:DataReleaseProduction
      18. 1.02C.04.00:Management,Leadership&OtherCosts
      19. 1.02C.04.01: Software Primitives
      20. 1.02C.04.02: Calibration Products
      21. 1.02C.04.03: ImageCharacterization
      22. 1.02C.04.04: Coaddition
      23. 1.02C.04.05:Detection&Deblending
      24. 1.02C.04.06: Characterization & Measurement
      25. 1.02C.04.07: Maintenance,Quality&Documentation
      26. 1.02C.05:ScienceUser InterfaceandTools
      27. 1.02C.05.00:Management,Leadership,&OtherCosts
      28. 1.02C.05.06: Client-serverQuery&VisualizationFramework
      29. 1.02C.05.07:LSSTSciencePlatform Interfaces
      30. 1.02C.05.08: Applications
      31. 1.02C.05.09: Alert Interfaces
      32. 1.02C.05.10: Integration&Test
      33. 1.02C.06:ScienceDataArchiveandApplicationServices
      34. 1.02C.06.00:Management,Leadership,&OtherCosts
      35. 1.02C.06.01:ScienceDataArchive
      36. 1.02C.06.02:DataAccessServices
      37. 1.02C.06.03: TaskFramework
      38. 1.02C.06.04: Middleware, Infrastructure,andToolkits
      39. 1.02C.07:LSSTDataFacility
      40. 1.02C.07.05:LSSTDataFacilityManagement,ServiceArchitecture,andProjectControls
      41. 1.02C.07.06.01: LDF-offered Services
      42. 1.02C.07.06.02: Reusable Production Services
      43. 1.02C.07.07:Data,Computeand ITSecurityServices
      44. 1.02C.07.08:LDFServiceSoftware
      45. 1.02C.07.09: ITCandFacilities
      46. 1.02C.08: InternationalCommunicationsandBaseSite
      47. 1.02C.08.01:BaseCenter
      48. 1.02C.08.02:ChileanDataAccessCenter
      49. 1.02C.08.03:LongHaulNetworks
      50. 1.02C.09:SystemLevelTesting&ScienceValidation
      51. 1.02C.10:ScienceQualityandReliabilityEngineering
      52. 1.02C.10.01:Management,Leadership,&OtherCosts
      53. 1.02C.10.02: QualityControl,DataspaceServices,andDeveloper Infrastructure
      54. 1.02C.10.02.02:SciencePlatformNotebookEnvironmentforQA,Commissioning&User
      55. Science
  23. C DMDiscussionandDecisionMakingProcess
      1. C.1 Empowerment
      2. C.2 RFCProcess
      3. C.3 ExceptionsandAppeals
      4. C.4 Formalities
  24. D TraceabilitymatrixofDMSR equirementstoOSSRequirements
  25. E TraceabilitymatrixofOSS equirementstoDMSR equirements
  26. F References
  27. G Acronyms
      1. Acronym Description

LargeSynopticSurveyTelescope(LSST)

Back to top


DataManagementOrganizationand

Back to top


Management

Back to top


WilliamO’Mullane,JohnSwinbank,MarioJuricandDMLT

Back to top


LDM-294

Back to top


LatestRevision: 2017-07-19
ThisLSSTdocumenthasbeenapprovedasaContent-ControlledDocumentbytheLSSTDMTech-
nical Control Team. If this document is changed or superseded, the new document will retain
the Handle designation shown above. The control is on the most recent digital document with
thisHandleintheLSSTdigitalarchiveandnotprintedversions.Additionalinformationmaybe
found in the corresponding DM RFC.
LARGESYNOPTICSURVEYTELESCOPE

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19

Back to top


Abstract
This management plan covers the organization and management of the Data Man-
agement(DM)subsystemduringthedevelopment,construction,andcommission-
ing of LSST. It sets out DM goals and lays out the management organization roles
and responsibilities to achieve them. It provides a high level overview of DM archi-
tecture, products and processes. It provides a structured starting point for under-
standingDMandpointerstofurtherdocumentation.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
ii

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19

Back to top


ChangeRecord
Version Date
Description
Ownername
1.1
2004-06-23 Initialversion(Document-139).
J.Kantor
1.2 2011-07-12 UpdatedforPDR.
J.Kantor
1.3 2014-03-07 Updatedforconstructionphase.
J.Kantor
1.4 2014-10-21 SQuaREsectionadded.
J.Kantor
1.5 2014-10-30 AddedLDM-294handle.
J.Kantor
2.0 2015-03-11 Updated with new RFC process, realignment
ofTCT,SAT,DMLT.
J.Kantor
3.0 2017-06-30 Complete overhaul of content, with all new
authors.RewritteninLaTeX.Approvedforre-
leasebyW.O’Mullane.
W.O’Mullane
3.1 2017-07-04 Minor cleanups for review. Approved in
RFC-
358.
W.O’Mullane
3.2 2017-07-19 Editorial fixes and efresh schedule and com-
ponent diagrams
W.O’Mullane
Document source location:
https://github.com/lsst/LDM-294
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
iii

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19

Back to top


Contents
1 Introduction
1
1.1 Purpose .......................................... 1
1.2 MissionStatement .................................... 1
1.3 GoalsandObjectives ................................... 1
2 DataManagementConceptualArchitecture
3
2.1 External Interfaces&AuxiliaryData .......................... 3
3 DataManagementOrganizationStructure
7
3.1 Meetings .......................................... 8
3.2 WorkingGroups ...................................... 9
3.3 ExternalStudies ...................................... 9
3.4 DocumentManagement ................................. 9
3.4.1 Documentation of Cross-Cutting Aspects for services............ 11
3.4.2 DocumentationofServiceLayers ....................... 12
3.4.3 DraftDocuments ................................. 13
3.5 ConfigurationControl .................................. 14
3.5.1 SoftwareConfigurationControl ........................ 14
3.5.2 HardwareConfigurationControl ........................ 15
3.6 RiskManagement .................................... 15
3.7 QualityAssurance .................................... 15
3.8 Actionitemcontrol .................................... 15
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
iv

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
3.9 VerificationandValidation ............................... 16
4 ProjectControls
17
4.1 Schedule .......................................... 17
4.2 WorkBreakdownStructure ............................... 18
4.3 PlanningProcess ..................................... 18
5 Products
19
6 RolesinDataManagement
20
6.1 DMProjectManager(DMPM) .............................. 20
6.2 DMDeputyProjectManager(DDMPM) ........................ 22
6.3 DMSubsystemScientist(DMSS) ............................ 22
6.4 ProjectController/Scheduler .............................. 24
6.5 ProductOwner ...................................... 24
6.6 PipelinesScientist .................................... 24
6.7 SystemsEngineer .................................... 25
6.8 DM InterfaceScientist(DMIS) .............................. 26
6.9 SoftwareArchitect .................................... 26
6.10OperationsArchitect ................................... 27
6.11ReleaseManager(RM) .................................. 27
6.12Lead InstitutionSeniorPositions ............................ 28
6.12.1 Technical/ControlAccountManager(T/CAM) ................ 28
6.12.2 InstitutionalScience/EngineeringLead .................... 28
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
v

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
6.13 DM Science Validation Scientist ............................. 29
7 DataManagementGroups/Bodies
29
7.1 SystemScienceTeam .................................. 29
7.1.1 OrganizationandGoals ............................. 30
7.1.2 Communications ................................. 31
7.2 DMSystemsEngineeringTeam ............................. 31
7.2.1 Communications ................................. 32
7.3 DMLeadershipTeam .................................. 32
7.3.1 Communications ................................. 33
7.4 DMChangeControlBoard ............................... 34
7.5 DMScienceValidationTeam............................... 35
8 LeadinstitutionsinDM
35
8.1 LSSTTucson ........................................ 35
8.1.1 ScienceQualityandReliabilityEngineering ................. 35
8.1.2 LSST International Communications and Base Site ............. 36
8.2 PrincetonUniversity ................................... 37
8.2.1 DataReleaseProduction ............................ 37
8.3 TheUniversityofWashington .............................. 38
8.3.1 AlertProduction ................................. 38
8.4 California Institute of Technology/IPAC ........................ 39
8.4.1 ScienceUser InterfaceandTools ....................... 39
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
vi

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
8.5 SLAC............................................. 40
8.5.1 ScienceDataArchiveandDataAccessServices ............... 40
8.6 NCSA ............................................ 41
9 DevelopmentProcess
42
9.1 Communications ..................................... 43
9.2 Conventions ........................................ 43
9.3 Reviews ........................................... 43
10 DataManagementProblem/ConflictResolution
43
A DMProductList
45
B ProposedFY2018WBS:1.02C:DataManagementConstruction
49
C DMDiscussionandDecisionMakingProcess
85
C.1 Empowerment....................................... 85
C.2 RFCProcess ........................................ 85
C.3 ExceptionsandAppeals ................................. 86
C.4 Formalities ......................................... 87
D TraceabilitymatrixofDMSR equirementstoOSSRequirements
88
E TraceabilitymatrixofOSS equirementstoDMSR equirements
99
F References
107
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
vii

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
G Acronyms
109
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
viii

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19

Back to top


DataManagementOrganizationandManagement

Back to top


1 Introduction
1.1 Purpose
Thisdocumentdefinesthemission,goalsandobjectives,organizationandresponsibilitiesof
the LSST Data Management subsystem (“DM”). The document is currently scoped to define
these elements for the LSST Design, Construction, and Commissioning phases. It does not
addressanyongoingmissionforDMduringLSSTOperations.
1.2 MissionStatement
Standupoperable,maintainable,qualityservicestodeliverhigh-qualityLSSTdataproducts
for science and education, all on time and within reasonable cost.
1.3 GoalsandObjectives
LSSTDataManagementwill:
• Definethedataproducts,dataaccessmechanisms,anddatamanagementandcuration
equirementsforLSST(withapprovalbyothers).
• Assess current and operations-era technologies for use in providing engineered solu-
tionstothe equirements.
• Define a secure computing, communications, and storage infrastructure and services
architectureunderlying DM.
•Select,implement,construct,test,document,anddeploythedatamanagementinfras-
tructure,middleware,applications,andexternalinterfaces.
• Adopt appropriate cybersecurity measures throughout the DM subsystem and espe-
ciallyonexternalfacingservices.
• DocumenttheoperationalproceduresassociatedwithusingandmaintainingDMcapa-
bilities.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
1

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Evaluate, select, ecruit, hire/contract and direct permanent staff, contract, and in-kind
esourcesinLSSTandfrompartnerorganizationsparticipatinginLSSTDMinitiatives.
TheDMgoalsinselectingand,wherenecessary,developingLSSTsoftwaresolutionsare:
• Acquire and/or develop solutions: To achieve its mission, LSST DM prefers to acquire
andconfigureexisting,off-the-shelf,solutions.Wherenosatisfactoryoff-the-shelfsolu-
tionsareavailable,DMdevelopsthesoftwareandhardwaresystemsnecessaryto:
Enable the generation of LSST data products at the LSST Archive and Satellite pro-
cessingcenter,and
Enable the serving of LSST data products from the two LSST DACs (one in the U.S.,
and one in Chile).
• Maintain coherent architecture: DM software architecture is actively managed at the
subsystem level. A well engineered and cleanly designed codebase is less buggy, more
maintainable, and makes developers who work on it more productive. Where there is
no significant impact on capabilities, budget, or schedule, LSST DM prefers to acquire
and/ordevelopreusable,opensource,solutions.
• Support eproducibility and insight into algorithms: Other than when prohibited by li-
censing,security,orothersimilarconsiderations,DMmakesallnewlydevelopedsource
code,andinparticularthatpertainingtoscientificalgorithms,public.Ourprimarygoals
inpublicizingthecodearetosimplify eproducibilityofLSSTdataproductsandtopro-
vide insight into algorithms used. Achieving these goals equires that the software must
beproperlydocumented.
• Opportunities beyond LSST: LSST DM codes may be of interest and (re)used beyond the
LSST project (e.g., by other survey projects, or by individual LSST end-users). While en-
abling or supporting such applications goes beyond LSST’s construction equirements,
cost and schedule-neutral technical and programmatic options that do not preclude
themandallowforfuturegeneralizationshouldbestronglypreferred.
Background decision material on choices made in DM will be documented in technical notes
which will be lodged in DocuShare (see Section 3.4) with “DMTN” series handles..
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
2

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19

Back to top


2 Data Management Conceptual Architecture
The DM Subsystem Architecture is detailed in LDM-148. A few of the higher level diagrams
are eproducedheretoorientthe eaderwithinDM.
During Operations, components of the DM Subsystem will be installed and run in multiple
locations. Theseinclude:
• The Commissioning Cluster in the Base Facility in La Serena, Chile
• The main compute facility at NCSA in Urbana-Champaign
• The US Data Access Center (DAC), also at NCSA in Urbana-Champaign
• TheChileanDACintheBaseFacility
• The Satellite Processing Center at CC-IN2P3 in Lyon, France
Figure 1 shows the various DM components which will be used in operations and the physical
compute environments in which they will be deployed. Bulk data storage and transport be-
tweencomponentsisprovidedbytheDataBackbone.Thiscomplexpieceofinfrastructureis
displayedinFigure3.
Science users will access the data products produced by LSST through the Science Platform,
as showninFigure2.
Figure 4 shows the common infrastructure and services layer which underlies the compute
environments. This does not list specific technologies for management/monitoring, provi-
sioning/deployment, or workload/workflow — these are still being selected — but under
considerationareindustry-standardtoolssuchasNagios,Puppet/vSphere/OpenStack/Kuber-
netes,andPegasus.
2.1 External Interfaces&AuxiliaryData
The DM external interfaces are controlled by the ICDs listed in Table 1.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
3

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
Data Backbone
NCSA
Analysis and
Developer Support
Level 1
Level 2
Base Center
Prompt
Processing
Ingest
Offline
Processing
OCS
Driven
Batch Ctrl
Image and
EFD
Archiving
Level 1
Quality
Control
Alert
Distribution
Telemetry
Gateway
Alert
Filtering
Template &
Calib. Prod.
Production
Data
Release
Production
Level 2
Quality
Control
US Data Access Center
Bulk Data
Distribution
Science
Platform
(DAC)
Observatory
Developer
Services
Integration
& Test
Science
Platform
(Sci. Valid.)
Data
Backbone
Endpoint
Prompt
Processing
Data
Backbone
Endpoint
OCS Batch
Processing
Data
Backbone
Endpoint
Commissioning Cluster
Science
Platform
(Commiss.)
Data
Backbone
Endpoint
RabbitMQ
BBFTP
HTCondor
Satellite Processing CC-IN2P3
DRP
Satellite
Processing
Pegasus /
HTCondor
Tape
Periodic
Calibration
Payload
Template
Generation
Payload
Raw Calib
Validation
Payload
Alert
Production
Payload
Annual
Calibration
Payload
DRP
Payload
MOPS
Payload
Daily Cal.
Update
Payload
Chilean Data Access Center
Science
Platform
(DAC)
Data
Backbone
Endpoint
Science
Users
Staff
Staff
Alert
Users
Community
Alert
Brokers
EPO
Other Data
Partners
Data
Backbone
Endpoint
FIGURE 1: DM components as deployed during Operations. Where components are de-
ployed in multiple locations, the connections between them are labeled with the relevant
communicationprotocols.Sciencepayloadsareshowninblue.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
4

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
Platform Backend Services
LSST Science Platform
LSP Web
APIs
Aspect
LSP Portal
Aspect
LSP
JupyterLab
Aspect
User
Compute
User File
Storage
User
Databases
FIGURE 2: Thesub-componentsoftheSciencePlatform.
Data
Backbone
Endpoint
Tape
Data Butler
Client
Science Image Archive
Science Catalog Archive
Data Backbone Transport and Replication
Metadata &
Provenance
Files
Qserv
Other DBs
Caches
Science
Image
Archive
Science
Catalog
Archive
Science
Image
Archive
Science
Catalog
Archive
Data
Backbone
Endpoint
Caches
Other
Backbone
Clients
Data Butler
Client
Other
Backbone
Clients
FIGURE 3: The Data Backbone links all the physical components of DM.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
5

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
NCSA
Base Center
US Data Access Center
Commissioning Cluster
Satellite Processing CC-IN2P3
Chilean Data Access Center
Workload /
Workflow
Provisioning /
Deployment
HTCondor
vSphere +
Puppet
Management /
Monitoring
Workload /
Workflow
Provisioning /
Deployment
HTCondor
Management /
Monitoring
Workload /
Workflow
Provisioning /
Deployment
HTCondor
Management /
Monitoring
Management /
Monitoring
Workload /
Workflow
Provisioning /
Deployment
HTCondor
Management /
Monitoring
Workload /
Workflow
Provisioning /
Deployment
HTCondor
Management /
Monitoring
Identity
Management
Identity
Management
Replica
FIGURE 4:CommoninfrastructureservicesavailableateachDMlocation.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
6

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
TABLE 1: DM Interface Control Documents
LSE-68 DataAcquisition InterfacebetweenDataManagementandCam-
era
LSE-69 InterfacebetweentheCameraandDataManagement
LSE-72 OCSCommandDictionaryforDataManagement
LSE-75 ControlSystem InterfacesbetweentheTelescopeandDataMan-
agement
LSE-76 Infrastructure InterfacesbetweenSummitFacilityandDataMan-
agement
LSE-77 Infrastructure InterfacesbetweenBaseFacilityandDataManage-
ment
LSE-130 ListofData ItemstobeExchangedBetweentheCameraandData
Management
LSE-131 Data Management Interface Requirements to Support Education
andPublicOutreach
LSE-140 Auxiliary Instrumentation Interface between Data Management
andTelescope
In addition, certain tasks in DM rely on external catalogs and other information. The current
design equires:
• Gaia catalog (Release 2) as a photometry baseline.

Back to top


3 Data Management Organization Structure
This section defines the organizational structure during the period in which the DM Subsys-
temisdevelopedandcommissioned,uptothestartofLSSTObservatoryoperations.
The DM Project Manager (William O’Mullane), Deputy Project Manager (John Swinbank) and
DM Subsystem Scientist (Mario Jurić), who are known collectively as DM Management, lead
the DM Subsystem. The Project Manager has direct responsibility for coordination with the
overall LSST Project Office, the LSST Change Control Board, the LSST Corporation, and LSST
partner organizations on all budgetary, schedule, and esource matters. The Project Scientist
hasprimaryscientificandtechnicalresponsibilitywithinthesubsystemandresponsibilityfor
ensuringthatthescientific equirementsoftheLSSTaresupportedandisamemberofthe
LSSTProjectScienceTeam(PST).
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
7

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
DM views its deliverables as hierarchical tree of
products
, as described in Section 5. The sub-
system organization is based around groups which are responsible for the highest levels of
that product tree (corresponding to Work Breakdown Structure elements at the third level,
i.e.
1.02C.n
). This is illustrated in Figure 5.
Data Management Leadership Team (DMLT)
DM Project Manager
William O’Mullane
DM Deputy Manager
John Swinbank
Project Manager
Victor Krabbendam
DM Subsys-
tem Scientist
Mario Juric
´
Project Scientist
Zeljk
˘
o Ivezic
´
Systems Engineering
Manager
William O’Mullane
Pipeline Scientist
Robert Lupton
Systems Engineer
Tim Jenness
Software Architect
Kian Tat Lim
Operations Architect
Don Petravick
Interface Scientist
Gregory Dubois-Felsmann
Configuration/Release
Manager
???
DM Documentalist
Jonathan Sick
Project Control/Scheduler
Kevin Long
DM Admin
Libby Petrick
LSST Documentalist
Robert McKercher
SUIT
T/CAM
Xiuqin Wu
Product Owner
Gregory Dubois-
Felsmann
Data Access Services
T/CAM
Fritz Mueller
Product Owner
Mario Juric
Science Pipelines
T/CAM
John Swinbank
DRP Owner
Jim Bosch
Alerts Owner
Eric Bellm
SQuaRE
T/CAM
Frossie Economou
Product Owner
Simon Krughoff
Data Facility
T/CAM
Margaret Gelman
Infrastructure Owner
Don Petravick
Proc Systems Owner
Robert Gruendl
LHN & Base Site
T/CAM
Jeff Kantor
Product Owner
Don Petravick
DM Subsystem
Science Team
DMLT
DMCCB
(was TCT)
1
FIGURE 5: DM organization.
3.1 Meetings
As a diverse and distributed organization DM staff will participate in a considerable number
of meetings. NSF and Aura have many rules on meeting attendance and LSST keep policies
updated accordingly in LPM-191 and Document-13760. This includes the travel summary
eporttemplate[Document-13762]everytravelermustfillafterattendingameeting.
A detailed debrief note or presentation may be asked of travelers to specific meetings of
interestbytheDMLT(7.3).
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
8

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
3.2 WorkingGroups
The egular decision making process within DM is based on individual empowerment and a
mechanismtodevelopconsensus.This“RFC”processisdescribedinAppendixC.
However, someissuesindevelopmentofasystemlikeDataManagement equiremoreeffort
toresolvethancanbereasonablyaddressedthroughanRFC.When equired,theDMPM
willaddresstheseissuesthroughthecreationofashort-lived“workinggroup”.Theworking
group will be given a specific narrow charge, it will be a small group (perhaps seven people),
its activities will be bounded in time, and it will have a clear deliverable. Members of the group
will be agreed by the DMLT (7.3) to provide the best technical input from the perspective of all
stakeholders.Membersoftheworkinggroupshoulddiscussintheirlocalorganizationsand
socializerecommendationsaheadofadoption.
3.3 ExternalStudies
TheDMPMmayinitiateorrequeststudiesbyexternalpartiestoinvestigateor eporton
technologicalorotherchoicesfacingtheDMsystem.
3.4 DocumentManagement
DM documents will follow the Systems Engineering Guidelines of LSST. PDF versions of re-
leased documents shall be deposited in DocuShare in accordance with the Project’s Docu-
mentManagementPlan[LPM-51].
LDM-leveldocumentsarechange-controlled,andmaybereleasedonlyonagreementofthe
DMCCB (Section 7.4). Uncontrolled documents, such as technical notes, may be released
when the author decides it is appropriate or is asked to release it by the Project Manager.
ThedocumenttreeforDMisshowninFigure6. Thisisnotexhaustive, butservestogivea
high level overview of the main documents in DM and how they relate to each other. Some
documents shown in red are not yet written.
Figure 6 has a single box for “end user documentation”. However, appropriate web-based,
user-focused documentation is egarded as a major DM deliverable. Figure 7 shows the in-
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
9

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
Observatory System
Spec LSE-30 (OSS)
DM Data Acq
ICD LSE-68
DM Camera
ICD LSE-69
DM Telescope
Control Sys
ICD LSE-75
DM Summit Infra
ICD LSE-76
DM Base Infra
ICD LSE-77
DM EPO ICD
LSE-131
DM Telescope
Aux ICD LSE-140
LSST System
Requirements
LSE-29 (LSR)
LSST Science
Requirements
LPM-17 (SRD)
Specification and Design
Planning
Test and Validation
Coming in 2018
Interface Control
Document (ICD)
Needs Update
DM System
Requirements
LSE-61 (DMSR)
LSST Data Quality
Assurance Plan
LSE-63 (DQAP)
LSST Data Products
LSE-163 (DPDD)
DM Validation
& Test Plan
LDM-503 (SVTP)
DM PMP LDM-294
Config/Release/Deploy
Management
DM Verification
Control (VCD)
Component Archi-
tecture LDM-148
Science Platform
Requirements
LDM-554
Middleware
Requirements
LDM-556
Database Require-
ments LDM-555
Science Platform
Design LDM-542
Middleware
Design LDM-152
Database De-
sign LDM-135
Services & Infras-
tructure LDM-129
Pipeline De-
sign LDM-151
Network De-
sign LSE-78
User Documentation
NCSA Enclave Test
Spec LDM-532
Base Enclave Test
Spec LDM-538
Comm Cluster Test
Spec LDM-541
Data BackBone
Test Spec LDM-535
Data Services Test
Spec LDM-536
Science Platform
Test Spec LDM-540
L1 Test Spec
LDM-533
L2 Test Spec
LDM-534
DBB Infrastructure
Test Spec LDM-537
L2 KPMs LDM-502
Qserv test spec
LDM-552
L2 Test Reports
NCSA Enclave
Test Reports
Base Enclave
Test Reports
Comm Cluster
Test Reports
DBB Test Reports
Data Services
Test Reports
Science Platform
Test Reports
L1 Test Reports
Infrastructure
Test Reports
1
FIGURE 6: Outline of the documentation tree for DM software relating the high level docu-
ments to each other.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
10

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
tendedhierarchyforthisdocumentation.
www.lsst.org
community.lsst.org
www.lsst.io
dr1.lsst.io
Science Platform (JupyterLab) User Guide
pipelines.lsst.io
Firefly Portal Docs (in app)
LSST Science Platform Portal
Account setup
Data Product Reference
Guides
Tutorial projects
(notebooks)
JupyterLab
environment
guides
alerts.lsst.io
L1 Data Reference
Installation Guide
Release Notes
Frameworks
Contribution Guide
Getting Started
Processing Data
Modules
Data ingest
Single Frame Processing
Difference imaging
Coaddition
Multi-epoch Processing
Postprocessing
Observatory interface framework
Measurement framework
Modelling framework
Task framework
Butler (data access) framework
Data structures framework
Geometry framework
Display framework
Logging framework
Debug framework
Verification framework
Build system
firefly-client.lsst.io
lsstsw.lsst.io
lsst-texmf.lsst.io
firefly.lsst.io
developer.lsst.io
qserv.lsst.io
ltd-keeper.lsst.io
ltd-mason.lsst.io
ltd-conveyor.lsst.io
L1 Data Access APIs
Brokers
Data Access Guides
Our goal is to have a
documentation site
published for every software
project, even small projects
and microservices. Sphinx
templates and READMEs will
jumpstart this initiative.
NOTE:
One self-contained
documentation site per data
release.
NOTE:
www.lsst.io will offer a metadata-
driven index and search service
for LSST documentation and
projects, including:
1. Software.
2. Project documents (LDMs, etc.).
3. Technical notes.
4. Papers
NOTE:
Website or application
Documentation site (Sphinx)
Interior documentation page or section
Legend
FIGURE 7: Outline of the web hierarchy for the DM end user documentation.
Service-leveldocumentationfollowsthelayeredservicearchitectureoftheLSSTDataFacility
(seeFigure8).
3.4.1 Documentation of Cross-Cutting Aspects for services
The cross-cutting aspects of the LSST Data Facility, Security and Operational Manageability,
are epresented by the vertical boxes. Documentation of these aspects describes policies,
procedures,andsupportingmanagementframeworks,including:
1.LDFservicemanagementframework:servicecatalog,service-levelagreements(SLAs),
configurationmanagementdatabase(CMDB),servicemonitoring.
2. LDFservicemanagementprocessesandcontextintheoverallproject:incidentresponse,
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
11

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
FIGURE 8: Outline of layered service architecture of the Data Facility.
requestresponse,issuetracking,problemmanagementandtheproblemmanagement
database,changemanagementandchangecontrolauthority,releasemanagement.
3. Overview of the security enclave structure
4. Securitycontrolsandincidentresponseprocedures
5. Disasterrecoveryandcontinuitypolicies
3.4.2 DocumentationofServiceLayers
Theboxatthetopofthefigure, UseCases, epresentssubsystem-levelandproject-levelop-
erational use cases. The next layer, LDF-offered Services, epresents specific services offered
by the Data Facility which satisfy those use cases. Documentation of this layer includes:
1. For each service, a Concept of Operations (ConOps) which summarizes how a service
operates to satisfy a use case. The ConOps describes the operational characteristics
of the production system, context within overall LSST operations, and epresentative
scenarios.
2. For each service, a Theory of Operations, which provides a mental model of a con-
structed system. The Theory of Operations explains how the constructed service both
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
12

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
fulfillstheConOpsandintegrateswiththecross-cuttingaspectsofthefacility.Thedoc-
umentdescribestheoverallarchitectureoftheserviceanddependencyonsupporting
servicelayers;integrationintoaspectsofcomputersecurity,informationsecurityand
business continuity; and integration into incident eporting and response, availability
andcapacitymanagement,andchangemanagement.
The next two layers, Reusable Production Services and Data, Compute, and IT Security Ser-
vices, epresent tiers of supporting service. Documentation of these layers includes a Theory
of Operations, as described above, explaining the dependencies on supporting service and
ITClayers,andintegrationwithcross-cuttingaspectsofthefacility.
The ITC box epresents hardware components supporting all LDF services. Documentation
of ITC describes the system elements at all facility sites, administration within each security
enclave and integration with security operations, the overall provisioning plan, ITC system
monitoringandintegrationintotheservicemonitoringframework,andintegrationintoser-
vicemanagementprocessesincludingconfigurationmanagementandchangemanagement.
The Software box epresents service software components being developed by the LSST Data
Facility. Documentation of software elements follows the standards of the LSST software
stack.
DocumentsaremanagedasconfigurationitemsintheLSSTDataFacilityCMDB.
3.4.3 DraftDocuments
DraftDMdocumentswillbekeptinGitHub. Asingle epositoryperdocumentwillbemain-
tainedwiththeheadrevisioncontainingthe
released
versionwhichshouldmatchtheversion
onDocuShare. Each epositorywillbeincludedasa
submodule
ofasinglegit epositorylo-
cated at
https://github.com/lsst-dm/dm-docs.
Use of Google Docs or Confluence is tolerated but final delivered documents must conform
to the standard LSST format, and hence either produced with LaTeX, using the lsst-texmf
package
1
,orWord,usingtheappropriateLSSTtemplate[Document-9224,Document-11920].
The precursor document should then be erased with a pointer to the baseline document,
1
https://lsst-texmf.lsst.io
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
13

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
storedinGitHub.
End user documentation is planned to be appropriately web based, and the scheme for that
isdescribedinLDM-493.
3.5 ConfigurationControl
Configuration control of documents is dealt with in Section 3.4. Here we consider more the
operationalsystemsandsoftwareconfigurationcontrol.
3.5.1 Software Configuration Control
DMfollowsagitbasedversioningsystembasedonpublicgit epositories. Theapproachis
coveredinthedeveloperguide
https://developer.lsst.io/processes/workflow.html
andis
consistentwiththeSystemsEngineeringManagementPlanLSE-17.Themasterbranchisthe
stable code with development done in
ticket
branches (named with the id of the correspond-
ingJIRATicketdescribingthework). Oncereviewedabranchismergedtomaster, which
should always be functional and releasable. Releases are ecorded by tagging the master
branch; release branches can be created if patches are equired.
AsweapproachcommissioningandoperationsDMwillhavemuchstricterconfigurationcon-
trol. At this point there will be a version of the software which may need urgent patching, a
next candidate release version of the software, and the master. A patch to the operational
version will equire the same fix to be made in the two other versions. The role of the DM
Change Control Board (DMCCB; Section 7.4) becomes very important at this point to ensure
onlyessentialfixesmakeittothelivesystemaspatchesandthat equiredfeaturesarein-
cludedinplannedreleases.
Wecannotescapethefactthatwewillhavemultiplecodebranchestomaintaininoperations
which will lead to an increase in work load. Hence one should consider that perhaps more
manpower may be needed in commissioning to cope with urgent software fixes while con-
tinuing development. The other consideration would be that features to be developed post
commissioning will probably be delayed more than one may think, as maintenance will take
priority.
2
2
WOMidentifiesthisasthemaintenancesurge.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
14

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
3.5.2 Hardware Configuration Control
Onthehardwaresidewehavemultipleconfigurableitems;weneedtocontrolwhichversions
ofsoftwareareonwhichmachines.ThesedaystoolinglikePuppetmakethisreasonablypain-
less. Still the configuration must be carefully controlled to ensure eproducible deployments
providing correct and eproducible results. The exact set of released software and other
tools on each system should be held in a configuration management database. Changes to
theconfigurationshouldbeendorsedbytheDMCCB.
The sizing model for compute hardware purchasing is detailed in LDM-144, LDM-141, and
LDM-138.
3.6 RiskManagement
Risks will be dealt with within the LSST Project framework as defined in LPM-20. Risks in DM
may be sent to the DM Project Manager or Deputy Project Manager at any time for consider-
ation to be included in the formal risk egister (appropriately costed and weighted). All risks
are reviewed egularly by the DM Project Manager and Systems Engineer (minimum each 3
months).
3.7 QualityAssurance
InaccordancewiththeprojectQAplan[LPM-55]wewillperformQAonthesoftwareproducts.
ThisworkwillmainlybecarriedoutbySQuaRE(Section8.1.1).Qualityassuranceheremeans
compliance with project guidelines for production, in our case for software production. A
part of this is to have a verification/validation plan(s) which in and of itself is a major task (see
Section3.9).
3.8 Actionitemcontrol
Actions in DM are tracked as JIRA issues and periodically reviewed at DMLT meetings.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
15

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
3.9 VerificationandValidation
We intend to verify and validate as much of DM as we can before commissioning and oper-
ations. This will be achieved through testing and operations ehearsals/data challenges. The
verification and validation approach is detailed in LDM-503 including a high level test sched-
ule; the top level schedule is given in Figure 9.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
16

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19

Back to top


4 ProjectControls
DMfollowstheLSSTprojectcontrolssystem,asdescribedinLPM-98.Considerationsspecific
to DM are outlined in Section 4.3.
The DM Project Controller is responsible for the PMCS and, in particular, for ensuring that
DM properly complies with our earned value management equirements. The Controller is
the first point of contact for all questions about the PMCS.
4.1 Schedule
The entire LSST project schedule is held in Primavera. Tied to major project milestones we
have a series of DM tests which need to be performed to show readiness for the different
project phases. This is depicted in Figure 9.
FIGURE 9: DM major milestones—designated as LDM-503-
x
—in the LSST schedule. These
milestones are defined at level 2 according to the scheme described in Section 4.3.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
17

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
TABLE 2: DM top level Work Breakdown Structure
WBS Description
Lead Institution
1.02C.01 SystemManagement
LSSTTucson
1.02C.02 SystemsEngineering
LSSTTucson
1.02C.03 AlertProduction
UniversityofWashington
1.02C.04 DataReleaseProduction
PrincetonUniversity
1.02C.05 ScienceUser InterfaceandTools
Caltech IPAC
1.02C.06 ScienceDataArchive
SLAC
1.02C.07 ProcessingControl&Site Infrastructure NCSA
1.02C.08 InternationalCommunications.&BaseSite LSSTTucson
1.02C.09 Systems Integration&Test
NCSA&LSSTTucson
1.02C.10 ScienceQuality&ReliabilityEngineering LSSTTucson
4.2 WorkBreakdownStructure
While the original DM WBS is laid out in LPM-43 with definitions provided in LPM-44, the new
WBSiscurrentlydescribedinAppendixB,whichisexpectedtoreplacethecontentsofLPM-43
upon approval by the LSST CCB.
The WBS provides a hierarchical index of all hardware, software, services, and other deliver-
ables which are equired to complete the LSST Project. It consists of alphanumeric strings
separated by periods. The first component is always “1”, eferring to the LSST Construction
Project. “02C” in the second component corresponds to Data Management Construction.
Subdivisionsthereofareindicatedbyfurtherdigits.Thesesubdivisionscorrespondtoteams
within the DM project. The top level WBS elements are mapped to the lead institutes in Ta-
ble 2; the lead institutions roles are outlined in Section 8. The various groups involved in the
WBS are briefly described in Section 7.
4.3 PlanningProcess
Milestones have been defined to describe the major goals of the DM subsystem throughout
theconstructionproject.Eachmilestonehasadescription,aduedate,andalevel.Fourlevels
aredefined:
Level 1
The most important milestones exposed at the NSF level.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
18

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
Level 2
Cross-subsystem milestones (for example, DM milestones that affect the Camera
Subsystem).
Level 3
Cross-team milestones within DM (for example, Middleware milestones that affect
theDRPTeam).
Level 4
Internal milestones within a team.
The major DM subsystem tests described in Section 4.1 are defined as level 2 milestones.
Teamsplantheirworktowardseachtestbydefiningaseriesoflevel3milestones. Teams
may define level 4 milestones for their own use.
Resources to achieve the milestones throughout the duration of construction have been al-
located by means of
planning packages
loaded into the PMCS. Each top level WBS within DM
(per Table 2) is divided into some tens of planning packages, each of which addresses some
partofthe DMbaselinedesignwith aclearlydefinedscope, deliverable, esourcecost, and
enddate.
Astheduedateforworkapproaches,theactions equiredtocompleteeachplanningpackage—
and hence meet the associated milestones—must be defined in detail. The DM team divides
the year into two six month long
cycles
, running from November through May (the “spring
cycle”)andfromJunethroughOctober(the“fallcycle”). Atthestartofeachcycle,theDM
Leadership Team (Section 7.3) agrees on the detailed plan of work for the cycle, and this is
loaded in to JIRA as a series of “epics”, corresponding to projects of a few person-months
duration,eachwithdefinedstartandenddatesand esourceloading. TheDMteam ecords
workandtracksprogressagainstepicsusingJIRA;theProjectController(Section6.4)arranges
for this information to be ingested to and made available within the PMCS.
ThisprocessisdescribedindetailinDMTN-020.

Back to top


5 Products
TheproductsofDMarenotthedataproductsdefinedinLSE-163, athertheyaretheartifacts,
systemsandservicesweneedtoproducethoseproducts.Section2outlinesthehighestlevel
of this for DM while Appendix A defines the complete product tree for DM and it is picto-
rially epresentedatatrimmedlevelinFigure10. LDM-148providesatraceofproductsto
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
19

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
equirements, while Appendix A proves a full list with technical manager, WBS element and
product owner for each. Our primary guiding equirements come from LSE-30, the Observa-
tory System Specification (OSS), with Appendix D tracing DM equirements [LSE-61] to OSS,
andAppendixEtracingtherelevantOSS equirementstoDM.
Figure 10 contains the WBS element associated with the component as well as any git repos-
itoriesbelongingtothem. Sincethefigurestopsatlevel3mostgit epositorieswillonlybe
found in the full list in Appendix A.
Everygit epositoryshouldappearinAppendixAandhencehaveatechnicalmanagerand
product owner identified. The table is hierarchical hence if the manager/owner is not filled
in (or the individual is no longer with the project) we may go to the parent element man-
ager/owner. Some work remains to finish rationalizing the components and epositories.
Every JIRA component should map to one row in Appendix A thus providing a contact for that
component.SomeJIRAcomponentsarenotphysicalproducts-theyshouldstillappearinthe
Appendix A which is the single source for DM of component ownership. Some work is also
neededtorationalizetheJIRAcomponents.

Back to top


6 RolesinDataManagement
ThissectiondescribestheresponsibilitiesassociatedwiththerolesshowninFigure5.
6.1 DMProjectManager(DMPM)
TheDMProjectManagerisresponsiblefortheefficientcoordinationofallLSSTactivitiesand
responsibilitiesassignedtotheDataManagementSubsystem.TheDMProjectManagerhas
the responsibility of establishing the organization, esources, and work assignments to pro-
videDMsolutions. TheDMProjectManagerservesastheDM epresentativeintheLSST
Project Office and in that role is responsible for presenting DM initiative status and submit-
ting new DM initiatives to be considered for approval. Ultimately, the DM Project Manager,
in conjunction with his/her peer Project Managers (Telescope, Camera), is responsible for
deliveringanintegratedLSSTsystem. TheDMProjectManager eportstotheLSSTProject
Manager. Specific responsibilities include:
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
20

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
Data Management
.
Data Access Center
Bulk Distrib
Proposal Manager
Resource Manager
.
Data Backbone
1.02C.06.02.01
Data Butler Client
daf_persistence/db/daf_fmt_*
Transport/Repl
.
Science Cat Archive
1.02C.06.02.03
Qserv DBMS
qserv/partition/scisql
.
Science Cat DBs
.
Science Img Archive
1.02C.06.02.05
Global Metadata
1.02C.06.01.01
Provenance DB
.
Developer services
1.02C.10.02.03.01
Build/Unit Test
sconsUtils/base/lsstsw/lsst_build
1.02C.10.02.03.04
Devel Comm Tools
1.02C.10.02.03.03
Doc Infrastructure
lsst-texmf/templates/lsstDoxygen
1.02C.10.02.03.01
SW Version Control
1.02C.10.02.03.05
Issue Tracking
1.02C.10.02.03.02
Packaging/Distrib
lsst/shebangtron/lsst_dm_stack_demo
Identity Manager
.
Infrastructure
Batch Proc
Infra Systems
.
Task Execution
1.02C.06.03
Activator Bases
1.02C.06.03
Pipeline Config
pex_config
1.02C.06.04.01
Logging
log
1.02C.06.03
Multi-Core Task
1.02C.06.03
Multi-Node Task
pipe_base/ctrl_pool
1.02C.06.03
SuperTask
pipe_supertask/pipe_base/pex_exceptions
.
Integration Test
1.02C.10.02.03.01
Contin Integration
Jenkins
1.02C.01.01
Precursor Data
obs_*/validation_data_*/testdata_*/afwdata
.
IT Environments
.
Archive IT
Archive Center Env
Archive DAC Env
DAC Integ Env
Archive DBB Env
DBB Integ Env
Dev Env
L1 Integration Env
L2 Integration Env
Satellite Env
Science Valid Env
.
Base IT
Base Center Env
Base DAC Env
Base DBB Env
Comm Cluster Env
1.02C.03
Level 1 System
1.02C.03.03
Alert Distribution
1.02C.03.03
Alert Filtering
L1 Offline Proc
.
L1 OCS Components
Archiver
ctrl_iip
Catchup Archiver
ctrl_iip
EFD Tranform
Header Generator
OCS Batch Proc
ctrl_iip
Pointing Publisher
Prompt Proc
ctrl_iip
Telem Gateway
ctrl_iip
.
L1 Science Payloads
.
Offline Alert Prod
.
Prompt Alert Prod
1.02C.04.02
Aux Tel Spec Pipe
1.02C.04.02
Daily Calibration
1.02C.04.02
Offline Calibration
pipe_drivers
1.02C.04.02
Prompt Calibration
pipe_drivers
1.02C.04.02
CBP Control
1.02C.03
L1 Quality Control
.
Level 1 Services
.
Archiving Services
Pointing Service
.
Prompt Services
.
Level 2 System
1.02C.04.07
L2 Quality Control
validate_drp/verify_metrics/ci_hsc
.
L2 Science Payloads
1.02C.04.02
CPP Quality Control
1.02C.04.02
Periodic Cal Prod
1.02C.04.02
Annual Cal Prod
.
Data Release Prod
1.02C.06.01.01
DRP-Internal DB
daf_ingest
1.02C.03.04
Template Gen Prod
.
Production Exec
Campaign Manager
Job Activator
Pre-Flight Activator
Workflow Manager
ctrl_orca/ctrl_platform_*/ctrl_execute/ctrl_stats/ctrl_provenance
Workload Manager
.
Science Platform
.
LSP JupyterLab
1.02C.10.02.02.05
JupyterLab Activator
1.02C.10.02.02.06
JupyterLab Deploy
1.02C.10.02.02.01
JupyterLab Env
1.02C.05.07
JupyterLab SUIT Intf
1.02C.10.02.02.04
JupyterLab SW Env
.
LSP Portal
1.02C.05.07
Firefly Python APIs
firefly_client
1.02C.05.06
Firefly
firefly
1.02C.05.09
SUI Alert Interfaces
1.02C.05.08
Portal Applications
user workspace
Portal Interfaces
Xiuqin Wu
1.02C.05.07
Visualizers
.
LSP Web APIs
1.02C.06.02.05
Catalog Access
dax_dbserv
1.02C.06.02.05
Cat Metadata Acc
dax_metaserv
1.02C.06.02.05
Img Metadata Acc
dax_metaserv
1.02C.06.02.04
Image Access
dax_imgserv
1.02C.06.02.02
Web Framework
dax_webserv/dax_webservcommon
.
DM Networks
1.02C.07.08.06
Arc Extl Net
1.02C.07.08.03 (moving to 1.02C.08)
Base Network
.
Chilean Nat
1.02C.08.03.01.03
Summit - AURA
1.02C.08.03.01.04
DWDM Equipment
1.02C.08.03.01.01A
La Serena - AURA
1.02C.08.03.01.01
La Ser - Santi
.
Int/US WANs
1.02C.08.03.02.01
SCL - MIA 100 Gbps
1.02C.08.03.02.02
Network Mgmt
1.02C.08.03.02.03
SCL - BR Spectrum
1.02C.08.03.02.01
US National WAN
1.02C.08.03
Long-Haul Nets
.
Science Algorithms
1.02C.04.06
Aperture Corr
1.02C.03.01
Artifact Detection
meas_algorithms
1.02C.03.01
Artifact Interp
1.02C.04.06
Association/Match
1.02C.03.07
Astrometric Fit
jointcal/meas_astrom/meas_mosaic
1.02C.04.03
Background Estim
meas_algorithms
1.02C.04.03
Background Ref
1.02C.03.02
DIAObj Association
1.02C.03.04
DCR Template Gen
1.02C.04.05
Deblending
meas_deblender
1.02C.03.04
Img Decorrelation
ip_diffim
1.02C.04.04
Image Coaddition
coadd_utils/coadd_chisquared
1.02C.03.01
ISR
pipe_tasks/ip_isr
1.02C.04.06
Measurement
meas_base/meas_algorithms/meas_extensions_*/meas_modelfit
.
Orbit/Ephemeris
1.02C.03.06
Attribution/Precov
mops_daymops
1.02C.03.06
Ephemeris Calc
mops_night
1.02C.03.06
Orbit Fitting
1.02C.03.06
Orbit Merging
1.02C.03.06
Tracklet Gen
mops_daymops
1.02C.03.07
Photometric Fit
jointcal/meas_mosaic
1.02C.03.05
Proper Motion
1.02C.04.03
PSF Estim Large
1.02C.03.01
PSF Estim Small
meas_algorithms
1.02C.04.04
PSF Matching
Raw Meas Cal
1.02C.03.01
Reference Catalogs
meas_algorithms
1.02C.03.02
Reference Match
1.02C.03.01
Spatial Models
afw
1.02C.04.05
Source Detection
1.02C.04.06
Star/Galaxy Sep
1.02C.03.04
Template Storage
1.02C.03.04
Variability Char
.
Science Primitives
1.02C.03.05
Camera Descr
afw
1.02C.03.05
Chromaticity Utils
afw
1.02C.04.01
Convolution
afw
1.02C.03.05
Approx 2-D Fields
afw
1.02C.04.01
Footprints
afw
1.02C.03.05
Fourier Transforms
afw
1.02C.03.05
Common Functions
afw
.
Geometry
1.02C.03.05
Cartesian Geom
1.02C.03.05
Coord Transforms
afw/astshim
1.02C.06.04.03
Spherical Geom
sphgeom/skypix/skymap/geom/afw
1.02C.04.01
Images
afw
1.02C.04.01
MC Sampling
afw
1.02C.04.01
Num Integration
afw
1.02C.04.01
Num Optimization
afw
1.02C.04.01
PhotoCal Repr
afw
1.02C.06.02.01
Property/Metadata
daf_base
1.02C.03.05
Point-Spread Funcs
meas_algorithms/shapelet
1.02C.04.01
Random Numbers
afw
1.02C.04.01
Science Tools
afw/utils
1.02C.04.01
Basic Statistics
afw
1.02C.04.01
Tables
afw
1.02C.03.05
Tree Structures
afw
1.02C.04.01
Warping
afw
.
QC Dashboard
1.02C.10.02.01.04
Alert QC
1.02C.10.02.01.01
QC Harness
validate_base
1.02C.10.02.01.02
QC Notifications
1.02C.10.02.01.03
QC Reports
1
FIGURE 10: DMproducttree. -thereareover200products,thistreeistoconveyandideaof
the products and is truncated to make it somewhat legible. The full list is given in Appendix A
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
21

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Manage the overall DM System
• Define scope and request funding for DM System
• Develop and implement the DM project management and control process, including
earnedvaluemanagement
• ApprovetheDMWorkBreakdownStructure(WBS),budgetsand esourceestimates
• Approve or execute as appropriate all DM outsourcing contracts
• Convene and/or participate in all DM reviews
• Co-chair the DM Leadership Team (Section 7.3)
6.2 DMDeputyProjectManager(DDMPM)
The PM and deputy will work together on the general management of DM and any specific PM
tasksmaybedelegatedtothedeputyasneededandagreed. IntheabsenceofthePMthe
deputycarriesfullauthorityanddecisionmakingpowersofthePM.TheDMProjectManager
will keep the Deputy Project Manager informed of all DM situations such that the deputy may
effectively act in place of the Project Manager when absent.
6.3 DMSubsystemScientist(DMSS)
TheDMSubsystemScientist(DMSS)hastheultimateresponsibilityforensuringDMinitiatives
providesolutionsthatmeettheoverallLSSTsciencegoals.Assuch,thispersonleadsthedefi-
nitionandunderstandingofthesciencegoalsanddeliverablesoftheLSSTDataManagement
SystemandisaccountableforcommunicatingthesetotheDMengineeringteam.
TheDMSubsystemScientist eportstotheLSSTProjectScientist. TheDMSSisamemberof
the LSST Change Control Board and the Project Science Team. He/she chairs and directs the
work of the DM System Science Team (Section 7.1).
Specificresponsibilitiesandauthoritiesinclude:
• CommunicateswithDMsciencestakeholders(LSSTProjectScientistandTeam,advisory
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
22

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
bodies, the science community) to understand their needs and identifies aspects to be
satisfiedbytheDMSubsystem.
• Develops, maintains, and articulates the vision of DM products and services responsive
tostakeholderneeds.
•WorkswiththeLSSTProjectScientisttocommunicatetheDMSystemvisiontoDMstake-
holders. Works with the DM Project Manager to communicate and articulate the DM
System vision and equirements to the DM construction team.
• Regularly monitors DM construction team progress and provides feedback to the DM
Project Manager to ensure the continual understanding of and adherence to the DM
vision, equirements, and priorities.
• Develops and/or evaluates proposed changes to DM deliverables driven by schedule,
budget,orotherconstraints.
•ProvidesadvicetotheDMProjectManageronscience-drivenprioritizationofconstruc-
tion activities.
• Validates the science quality of DM deliverables and the capability of all elements of the
DM System to achieve LSST science goals.
• Serves as Data Management Liaison as requested by LSST Science Collaborations
• Providessafe,effective,efficientoperationsinarespectfulworkenvironment.
Specific authorities include:
• Definesthevisionandhigh-level equirementsoftheDMproductsandservices equired
to deliver on LSST science goals.
• DefinesthescienceacceptancecriteriaforDMdeliverables(bothfinalandintermediate)
andvalidatesthattheyhavebeenmet(ScienceValidation).
• HiresorappointsDMSystemScienceTeamstaff andotherdirect eportsanddefines
theirresponsibilities.
• Advises and consents to the appointments of institutional DM Science Leads.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
23

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• DelegatesauthorityandresponsibilityasappropriatetoinstitutionalScienceLeadsand
other members of the DM System Science Team.
• Represents and speaks for the LSST Data Management.
• Convenes and/or participates in all DM reviews.
• Co-Chairs the DM Leadership Team
6.4 Project Controller/Scheduler
TheDMProjectControllerisresponsibleforintegratingDM’sagileplanningprocesswiththe
LSSTProjectManagementandControlSystem(PMCS).Specificresponsibilitiesinclude:
• Assist T/CAMs in developing the DM plan
• Synchronize the DM plan, managed as per Section 4.3, with the LSST PMCS
• Ensure that the plan is kept up-to-date and milestones are properly tracked
• Create eports,GanttchartsandfiguresasrequestedbytheDMPM
6.5 ProductOwner
A product owner is responsible for the quality and acceptance of a particular product. The
productownershallsignoff onthe equirementstobefulfilledineverydeliveryandtherefore
also on any descopes or enhancements. The product owner shall define tests which can be
runtoproveadeliverymeetsthe equirementsdueforthatproduct.
6.6 PipelinesScientist
Several DM products come together to form the LSST pipeline. The Pipelines Scientist is the
productownerfortheoverallpipeline.
ThePipelinesScientistshall:
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
24

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Provide guidance and test criteria for the full pipeline including how QA is done on the
products
• Keep the big picture of where the codes are going in view, predominantly with respect
tothealgorithms,butalsotheimplementationandarchitecture(aspartoftheSystems
EngineeringTeamSection7.2).
• Advise on how we should attack algorithmic problems, providing continuing advice to
subsystem product owners as we try new things.
• Advise on calibration issues, provide understanding of the detectors from a DM point of
view
• Advise on the overall (scientific) performance of the system, and how we’ll test it, think-
ing about all the small things that we have to get right to make the overall system good.
6.7 SystemsEngineer
With the Systems Engineering Team (Section 7.2) the Systems Engineer owns the DM entries
intherisk egisterandisgenerallyinchargeofthe
process
ofbuildingDMproducts.
As such, the Systems Engineer is responsible for managing equirements as they pertain to
DM.Thisincludes:
• Update and ensure traceability of the high level design & equirements documents:
DMSR(LSE-61),OSS(LSE-30),andLSR(LSE-29)
• Overseeworkonlowerlevel equirementsdocuments
• Ensure that the system is appropriately modeled in terms of e.g. drawings, design doc-
umentation, etc
• Ensure that solid verification plans and standards are established within DM
In addition, the Systems Engineer is responsible for the process to define & maintain DM
interfaces(internalandexternal)
•Defineandenforcestandardsforinternalinterfaces
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
25

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Direct the Interface Scientist’s (Section 6.8) work on external ICDs
The Systems Engineer shall chair the DM Change Control Board (Section 7.4)
• Organize DMCCB processes so that the change control process runs smoothly
• Identify RFCs equiring DMCCB attention
• Shepherd RFCs through change control
• CallandchairDMCCBmeetings,ensuringthatdecisionsaremadeand ecorded
Finally,theSystemsEngineer epresentsDMontheLSSTCCB.
6.8 DM InterfaceScientist(DMIS)
TheDM InterfaceScientistisresponsibleforallexternalinterfacestotheDMSubsystem.This
includes ensuring that appropriate tests for those interfaces are defined. This is a responsi-
bilitydelegatedfromtheDMSystemsEngineer(Section6.7).
6.9 SoftwareArchitect
The Software Architect is responsible for the overall design of the DM
software
system. Spe-
cificresponsibilitiesinclude:
• Define the overall architecture of the system and ensuring that all products integrate to
formacoherentwhole
• Selectandadvocateappropriatesoftwareengineeringtechniques
• Choose the technologies which are used within the codebase
• Minimize the exposure of DM to volatile external dependencies
The Software Architect will work closely with the Systems Engineer (Section 6.7) to ensure
that processesare inplace fortracing equirements tothe codebaseand providinghooks to
ensure that equirement verification is possible.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
26

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
6.10 OperationsArchitect
The DM Operations Architect is responsible for ensuring that all elements of the DM Sub-
system,includingoperationsteams,infrastructure,middleware,applications,andinterfaces,
come together to form an operable system.
Specificresponsibilitiesinclude:
• Setupandcoordinateoperations ehearsals
•EnsurereadinessofproceduresandpersonnelforOperations
•Setstandardsforoperationse.g.procedurehandlingandoperatorlogging
• Participate in stakeholder and end user coordination and approval processes and re-
views
• Serve as a member of the LSST Systems Engineering Team
6.11 ReleaseManager(RM)
TheDMReleaseManager(RM)isresponsibleformaintainingandapplyingthereleasepolicy.
Specifically,theDMReleaseManagerwill:
• Develop and maintain the DM Release Policy as a change controlled document;
•Managethesoftwarereleaseprocessanditscompliancewithdocumentedpolicy;
• Define the contents of releases, in conjunction with the product owners, the DM Sub-
systemScientist,andthetechnicalmanagers;
• Ensurethateachreleaseisaccompaniedbyanappropriatedocumentationpack,includ-
ing user manuals, test specifications and eports, and release notes;
• Ensure the release is delivered to NCSA for acceptance;
• Work with technical managers to coordinate bug fixes and maintenance of long-term
supportreleases;
• ServeasamemberoftheDMCCB(Section7.4).
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
27

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
6.12 Lead InstitutionSeniorPositions
Each Lead Institution (as defined in Section 8; see also Table 2) has a T/CAM and Scientific or
EngineeringLead,whojointlyhaveoverallresponsibilityforabroadareaofDMwork,typically
a Work Breakdown Structure (WBS) Level 2 element. They are supervisors of the team at their
institution,withrolesbroadlyanalogoustothoseoftheDMProjectManagerandSubsystem
Scientist.
6.12.1 Technical/Control Account Manager (T/CAM)
Technical/ControlAccountManagershavemanagerialandfinancialresponsibilityfortheen-
gineering teams within DM. Each T/CAM is responsible for a specific set of WBS elements.
Theirdetailedresponsibilities include:
• Develop, esourceload,andmaintaintheplanforexecutingtheDMconstructionproject
within the scope of their WBS
• SynchronizetheconstructionschedulewithdevelopmentinWBSelementsmanagedby
otherT/CAMs
• Maintain the budget for their WBS and ensuring that all work undertaken is charged to
thecorrectaccounts
• Work with the relevant Science Leads and Product Owners (Section 6.5) to develop the
detailedplanforeachcycleandsprintas equired
• Work with the DM Project Controller (Section 6.4) to ensure that all plans and milestones
arecapturedintheLSSTProjectControlssystem
• Perform day-to-day management of staff within their WBS
•Performtheroleof“scrum-master”duringagiledevelopment
• Reportactivitiesas equired,includingprovidinginputformonthlystatus eports.
6.12.2 Institutional Science/Engineering Lead
The InstitutionalScience/EngineeringLeadsserveasproductowners(Section6.5)forthema-
jor components of the DM System (Alert Production, Data Release Production, Science User
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
28

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
Interfaceetc).
Inaddition,theyprovidescientificandtechnicalexpertisetotheirlocalengineeringteams.
They work with the T/CAM who has managerial responsibility for their product to define the
overallconstructionplanandthedetailedcycleplansforDM.
Institutional science leads are members of the DM System Science Team (Section 7.1) and, as
such, eporttotheDMSubsystemScientist(Section6.3).
6.13 DMScienceValidationScientist
TheDMScienceValidationScientistleadstheScienceValidationteam(Section7.5).Thisindi-
vidualhasprimaryresponsibilityforplanning,executingandanalyzingtheresultsofscience
validationactivities,asdefinedinLDM-503;typically,thisincludeslarge-scaledatachallenges.
The Science Validation Scientist is responsible for End to End Science validation and eports
totheDMSubsystemScientist.

Back to top


7 DataManagementGroups/Bodies
Since the DM team is distributed in terms of geography and responsibility across the LSST
partnerandleadinstitutions,mechanismsareneededtoensurethattheprojectremainson
track at all times. There are five primary coordinating bodies to ensure the management,
technical,andqualityintegrityoftheDMSubsystem.
7.1 SystemScienceTeam
Members of the DM System Science Team (SST) work together to define, maintain, and com-
municate to the DM Systems Engineering team a coherent vision of the LSST DM system re-
sponsivetotheoverallLSSTProjectgoals,aswellasscientificallyvalidatetheas-builtsystem
(LDM-503,Section9.).
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
29

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
7.1.1 OrganizationandGoals
TheSystemScienceTeamincludes:
• DM Subsystem Scientist (chair)
• DM Science Validation Scientist
• DM Institutional Science Leads
• DM System Science Analysts
• DM Science Pipelines Scientist
The System Science Team has been chartered to:
• Support the DM Subsystem Scientist (as the overall DM Product Owner) in ensuring that
DataManagementSubsystem’sinitiativesprovidesolutionsthatmeettheoverallLSST
sciencegoals.
• Support the Institutional Science Leads in their roles as Product Owners for elements of
theDMsystemtheirrespectiveinstitutionshavebeentaskedtodeliver.
• SupporttheDMScienceValidationScientist,whoorganizesandcoordinatesthescience
validation efforts (LDM-503).
• Guide the work of System Science Analysts, who generally lead and/or execute studies
needed to support SST work.
• Provide a venue for communication with the Science Pipelines Scientist, who broadly
advises on topics related to the impact of science pipelines on delivered science and
viceversa(Section6.6).
ThemembersoftheSystemScienceTeam eporttotheDMSubsystemScientistandshare
thefollowingresponsibilities:
• Communicate with the science community and internal stakeholders to understand
their needs, identifying the aspects to be satisfied by the DM Subsystem.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
30

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
•Liaisewiththesciencecollaborationstounderstandandcoordinateanyconcurrentsci-
enceinvestigationsrelevanttotheDMSubsystem.
• Develop, maintain, and articulate the vision of DM-delivered LSST data products and
services that is responsive to stakeholder needs, balanced across science areas, well
motivated,andscientificallyandtechnologicallycurrent.
• Work with the DM Project Manager and DM Technical Managers to communicate and
articulate the DM System vision and equirements to the DM engineering team.
• Identify, develop, and champion new scientific opportunities for the LSST DM System,
as well as identify risks where possible.
• Developchangeproposalsand/orevaluatethescientificimpactofproposedchangesto
DMdeliverablesdrivenbyschedule,budget,orotherconstraints.
• Lead the Science Verification of the deliverables of the DM subsystem.
7.1.2 Communications
DMSystemScienceTeamcommunicationmechanismsaredescribedontheSSTConfluence
pageat
http://ls.st/sst.
7.2 DMSystemsEngineeringTeam
The Systems Engineering Team is led by the DMPM (Section 6.1) and looks after all aspects
of systems engineering. It is comprised of not only the Systems Engineer (Section 6.7) but
alsotheSoftwareArchitect(Section6.9),OperationsArchitect(Section6.10),PipelineScientist
(Section6.6)and InterfaceScientist(Section6.8).
While the product owners (Section 6.5) help DM to create products which are fit for purpose,
theSystemsEngineeringTeammustensurewedoitcorrectly.Thisgroupconcernsitselfwith
(sub)systemwidedecisionsonarchitectureandsoftwareengineering.
The specific tasks of this group include:
• Formalize the product list for DM
3
3I
n this sense, “products” are the software and systems which produce data products, ather than the data
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
31

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Formalize the documentation tree for DM, defining which documents need to be pro-
ducedforeachproduct
• Agree the process for tracing the baseline equirements verification and validation sta-
tus.
• Agreetheformalversionsofdocumentsandsoftwarewhichformthetechnicalbaseline,
individualitemswillgothroughtheCCBforformalapproval.
Someofthesetasksarewillbedelegatedtoindividualgroupmembers.Theseindividualsalso
are the conduit to/from the rest of the DM team to raise ideas/issues with the engineering
approach.
7.2.1 Communications
The Systems Engineering Team will only physically meet to discuss specific topics: there will
notbea egularmeetingofthegroupoutsideoftheonetoonemeetingswiththeDMproject
manager for the individuals in the group. Discussions will be held via email until in person
talksare equired.
7.3 DMLeadershipTeam
The purpose of the DM Leadership Team (DMLT) is to assist the DMPM establish the scope
of work and esource allocation across DM and ensure overall project management integrity
acrossDM.ThefollowingmandateestablishedtheDMLT:
• Charter/purpose
Maintainscopeofworkandkeepwithin esourceallocationacrossDM
EnsureoverallprojectmanagementintegrityacrossDM
Ensure Earned Value management equirements are met
• Membership
Co-chaired by the DM Project Manager (Section 6.1) and DM Project Scientist (Sec-
tion6.3)
productsthemselves.Seealso5.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
32

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
Lead InstitutionTechnical/ControlAccountManagers(T/CAMs;Section6.12.1)
InstitutionalScienceorEngineeringLeads(Section6.12.2)
MembersoftheDMSystemsEngineeringTeam(Section7.2)
• Responsibilities
Preparesallbudgets,schedules,plans
Meetseveryweektotrackprogress,addressissues/risks,adjustworkassignments
andschedules,anddisseminate/discussgeneralPMcommunications
The DM Leadership Team and the DM Systems Engineering Team (Section 7.2) work in syn-
chrony. The DMLT makes sure the equirements and architecture/design are estimated and
scheduled in accordance with LSST Project equired budgets and schedules.
7.3.1 Communications
A mailing list
4
exists for DMLT related messages. On Mondays the DMLT hold a brief (30 to
45 minutes) telecon. This serves to:
• Allow the Project manager and DM Scientist to pass on important project level informa-
tionandgeneralguidance.
• Raise any blocking or priority issues across DM — this may result in calling a splinter
meetingtofurtherdiscusswithrelevantparties.
• Inform all team members of any change requests (LCRs) in process at LSST level which
maybeofinteresttoorhaveanimpactonDM
• Check on outstanding actions on DMLT members
Face to Face meetings of DM are held twice a year; these are opportunities to:
• Discuss detailed planning for the next cycle
4
lsst-dmlt@listserv.lsstcorp.org
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
33

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Discuss technical topics in a face to face environment
• Work together on critical issues
• HelpmakeDMfunctionasateam
7.4 DMChangeControlBoard
The DMCCB has responsibility for issues similar to those of the LSST Change Control Board,
butwithitsscope estrictedtotheDMSubsystem. TheDMCCBreviewsandapproveschanges
to all baselines in the Subsystem, including proposed changes to the DM System Require-
ments (DMSR), eference design, sizing model, i.e. any LDM-series document. The Technical
Baseline,includingsoftware/hardwareanddocumentation,iswrittenbyDMandcontrolled
bytheDMCCB.DMCCBvalidatesthattheformandcontentoftheTechnicalBaselineisconsis-
tent with LSST project standards such as the Systems Engineering Management Plan (SEMP)
LSE-17.
• Charter/purpose
Ensure that the DM Technical Baseline (LDM-xxx) documents are baselined and
subsequentlychangedonlywhennecessaryandaccordingtoLSSTandDMconfig-
urationcontrolprocesses
• Membership
ChairedbytheSystemsEngineer
MembersincludetheDMSoftwareArchitect(§6.9),DMOperationsArchitect(§6.10),
DM System Interfaces Scientist (§6.8), DM SQuaRE T/CAM (§6.12.1), DM Release
Manager(§6.11)andDMProjectManager(§6.1)
For on-line virtual meetings, if a consensus or quorum or is not reached within one
week, the DM Project Manager will make a unilateral decision
• Responsibilities
Determineswhenspecificationanddeliverablesareofsufficientmaturityandqual-
itytobebaselined(placedunderconfigurationcontrolledstatus)orreleased.
Reviewsandapproves/rejectsproposedchangestobaselineditems
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
34

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
7.5 DMScienceValidationTeam
TheDMScienceValidationTeamguidesthedefinitionof,andreceivestheproductsof,science
validation and dress ehearsal activities, following the long-term roadmap described in LDM-
503. Decisions on the strategic goals of these activities are made in conjunction with the DM
SubsystemScientistandProjectManager.
TheDMScienceValidationTeamischairedbytheDMScienceValidationScientist(Section6.13).
ItsmembershipincludestheDMPipelinesScientist(Section6.6)andthevarious Institutional
Science/EngineeringLeads(Section6.12.2).Dependingontheactivitiescurrentlybeingexe-
cuted, other members of the System Science Team (Section 7.1), the wider DM Construction
Project,and/orexternalexpertsmaybetemporarilyaddedtotheteam.

Back to top


8 LeadinstitutionsinDM
8.1 LSSTTucson
The LSST Project Office in Tucson hosts the DM Project Manager (Section 6.1) and the Sys-
tems Engineer (Section 6.7). In addition, it is home to the Science Quality and Reliability Engi-
neering(SQuaRE)groupandLSST InternationalCommunicationsandBaseSite(ICBS)groups,
describedbelow.
8.1.1 Science Quality and Reliability Engineering
The SQuaRE group is primarily charged with providing technical feedback to the DM Project
Manager that demonstrates that DM is fulfilling its responsibilities with egard to quality — of
bothscientificdataproductsandsoftware—softwareperformance,andreliability.Assuch,
areasofactivityinclude:
• Development of algorithms to detect and analyze quality issues with data
5
• Infrastructuredevelopmenttosupportthegeneration,collection,andanalysisofdata
qualityandperformancemetrics
5
This may overlap with work carried out by the Science Pipelines groups (§§8.3.1 & 8.2.1). In some instances
this will involve sharing code; in others, it may merit duplicating a metric to ensure that it is correct.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
35

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• DM developer support services to ensure DM is using appropriate tools to aid software
quality
• DMdocumentationsupport,toincludedefiningstandardsandprovidingtoolingfordoc-
umentationaswellassomedocumentwriting
• Supportofpubliclyreleasedsoftwareproducts,includingportinganddistributingthem
accordingtothescientificcommunity’sneeds
In the event that SQuaRE identifies issues with the performance or future maintainability of
theDMcodebase,itwillbringthemtotheattentionoftheDMSoftwareArchitect. Inthe
event that SQuaRE identifies issues with the quality of the data or algorithmic performance,
it will bring them to the attention of the DM Subsystem Scientist.
8.1.2 LSST International Communications and Base Site
The ICBS group spans both Tucson and La Serena, and is responsible for the design, procure-
ment,installation,deployment,verification,andoperatingsupportduringconstructionand
commissioning of all data communications networks at the Summit and Base sites, as well as
links between all the LSST Sites, with two exceptions: the Summit Network (WBS 1.04C.12.5)
and the Archive External Network (1.02C.07.04.06). In the case of the exceptions, there are
technicalandmanagerialinterfacesbetweenthe ICBSandtheresponsibleparties,aswellas
overlaps of staff. The LSST Network Engineering Team (NET) spans all of these networking
assignees and is chaired by the ICBS staff.
The ICBS group is also jointly responsible with the Data Facility Team at NCSA for procure-
ment,installation,deployment,verification,andoperatingsupportduringconstructionand
commissioningofthecomputingandstorageinfrastructureattheBaseSite.
Since a large majority of the ICBS work involves procurement and contracted services, the
group works in close cooperation with AURA procurement and contracts, as well as with the
followingmajorsub-awardeesandtheirsubcontractors:
•REUNA:ChileanNationalNetworks
• Florida InternationalUniversity/AmLight: InternationalNetworksconnectingChileand
theUnitedStates,andUSNationalNetworks.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
36

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
8.2 PrincetonUniversity
PrincetonUniversityhoststhePipelinesScientist(Section6.6)andtheDataReleaseProduc-
tiongroup,describedbelow.
8.2.1 DataReleaseProduction
The Data Release Production (DRP) group has three major areas of activity within DM.
• Definition and implementation of the scientific algorithms and pipelines which will be
usedtogenerateLSST’sannualdatareleases;
• Definition and implementation of the algorithms and pipelines which will be used to
producethe“calibrationproducts”(forexample,flatfields,characterizationofdetector
effects, etc) which will be used as inputs to the photometric calibration procedure in
both nightly and annual data processing. This includes the development of the spec-
trophotometricdatareductionpipelinefortheAuxiliaryTelescope;
• Development, in conjunction with the Alert Production team (AP; Section 8.3.1), of a
libraryofre-usablesoftwarelibrariesandcomponentswhichformthebasisofboththe
AP and DRP pipelines and which are made available to science users within the LSST
SciencePlatform.
Development of software in support of annual data releases and of reusable software com-
ponents are carried out under the direction of the DRP Science Lead, who acts as product
owner for this part of the system. The DRP Science Lead is ultimately responsible to both the
PipelinesScientist(Section6.6)andDMSubsystemScientist(Section6.3).
TheproductownerforthecalibrationproductsistheLSSTCalibrationScientist(whodoubles
as the Pipelines Scientist, Section 6.6). The Calibration Scientist liaises with other LSST sub-
systemsandwiththeproductsownersoftheannualandnightlydataprocessingpipelinesto
ensurethatappropriatecalibrationproductsareavailabletothosepipelinestoenablethem
tomeetspecifications.
Management of the group is the responsibility of the Science Pipelines T/CAM, eporting to
theDMProjectManager(Section6.1).
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
37

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
The DRP group is responsible for delivering software which adheres to the architectural and
testingstandarddefinedbytheSoftwareArchitect(Section6.9). Inaddition,theDRPgroupis
responsiblefortestingeachmajorproductdeliveredtodemonstrateitsfitnessforpurpose,
and working with the DM Subsystem Scientist and DM System Science Team (Section 7.1) to
define, run and analyze “data challenges” and other large scale tests to validate the perfor-
manceofthedatareleaseproductionsystem.
8.3 TheUniversityofWashington
8.3.1 AlertProduction
The Alert Production (AP) group has 4 major areas of activity within DM.
• Definition and implementation of the scientific algorithms and pipelines which will be
usedtogeneratealertsfromLSST’simagestream.Thiswillserveasthealertgeneration
pipeline;
• Definitionandimplementationascalableandreliablesystemfortransmittingthealerts
generated by the alert generation pipeline including a mechanism for applying simple
filters to the stream. This is the alert distribution and filtering system;
• Definition and implementation of a system for identifying moving objects in our solar
system and fitting their physical properties. This is the Moving Objects Processing Sys-
tem(MOPS);
• Development,inconjunctionwiththeDataReleaseProductionteam(DRP;Section8.2.1),
ofalibraryofre-usablesoftwarelibrariesandcomponentswhichformthebasisofboth
the AP and DRP pipelines and which are made available to science users within the LSST
SciencePlatform.
Development of software in support of the alert generation pipeline, alert distribution sys-
tem, MOPS and of reusable software components are carried out under the direction of the
AP Science Lead, who acts as product owner for this part of the system. The AP Science
LeadisultimatelyresponsibletoboththePipelinesScientist(Section6.6)andDMSubsystem
Scientist(Section6.3).
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
38

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
Management of the group is the responsibility of the Science Pipelines T/CAM, eporting to
theDMProjectManager(Section6.1).
The AP group is responsible for delivering software which adheres to the architectural and
testing standard defined by the Software Architect (Section 6.9). In addition, the AP group is
responsiblefortestingeachmajorproductdeliveredtodemonstrateitsfitnessforpurpose,
and working with the DM Subsystem Scientist and DM System Science Team (Section 7.1) to
define, run and analyze “data challenges” and other large scale tests to validate the perfor-
manceofthedatareleaseproductionsystem.
8.4 California Institute of Technology/IPAC
IPAC hosts the DM Interface Scientist (Section 6.8) and the Science User Interface and Tools
(SUIT)groupdescribedbelow.
8.4.1 ScienceUser InterfaceandTools
The Science User Interface and Tools (SUIT) group has four major areas of activity within DM:
Design and develop the Firefly Web-based visualization and data exploration framework,
based upon the the same software already in operations in other NASA archive services (i.e.
IRSA’s WISE Image Service) . The Firefly framework provides three basic components – im-
age display and manipulation, tabular table display and manipulation, and 2D plotting – all
of which work together to provide different views into the same data. Firefly also provides
JavaScript and Python APIs to enable developers to easily use the components in their own
WebpagesorJupyternotebooks.
Develop the interfaces needed to connect Firefly to the other LSST Science Platform com-
ponents, e.g., connect to authentication and authorization, DAX services, user workspace,
flexiblecomputesystem.DevelopvisualizationsoftheobjectsintheLSSTDataProductsdata
model, and support their metadata; e.g., Footprint, HeavyFootprint, WCS models. Provide
basicaccesstoFireflyfromtheLSSTstackviaafw.display.
DesignandimplementthePortalAspectoftheLSSTSciencePlatformforDataAccessCenter,
basedonFirefly,providingscientistsaneasytouseinterfacetosearch,visualize,andexplore
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
39

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
LSSTdata. Theportalwillenableuserstodoasmuchdatadiscoveryandexplorationas
possiblethroughcomplexsearchesandfacilitatedataassessmentthroughvisualizationand
interaction. The Portal will assist users in understanding the semantic linkages between the
various LSST data products. The Portal will guide users to documentation on the Science
Platform itself, the LSST data products, and the processing that generated them. Support
linkage between the Portal and Notebook aspects of the Science Platform, enabling users
to switch between the aspects easily by providing tools to make data selected in the Portal
readilyavailableforfurtheranalysisinusernotebooks.
Design and develop the LSST Alert Subscription web portal to enable scientists to access the
alert system. The subscription service will enable users to egister filters and destinations
for alerts matching their interests. The Alert portal will also provide basic capabilities for
searchingalertshistoryandforexploringlinkagebetweenalertsandotherdataproducts.
8.5 SLAC
SLAC hosts the DM Software Architect (Section 6.9) and the Science Data Archive and Data
AccessServicesgroupdescribedbelow.
8.5.1 ScienceDataArchiveandDataAccessServices
TheScienceDataArchiveandDataAccessServices(DAX)grouphasthefollowingmajorareas
ofactivitywithinDM:
• Provides software to support ingestion, indexing, query, and administration of DM cat-
alogandimagedataproducts,dataprovenance,andotherassociatedmetadatawithin
theLSSTDataAccessCenters;
• Provides implementations of data access services (including IVOA services), as well as
associated client libraries, to be hosted within the LSST Data Access Centers, which fa-
cilitate interaction between LSST data products and tools provided by both other parts
oftheLSSTprojectandbytheastronomical esearchcommunityatlarge;
• Provides a Python framework (the “Data Butler”), used by the LSST science pipelines,
tofacilitateabstractpersistence/retrievalofin-memoryPythonobjectsto/fromgeneric
archivesofthoseobjects;
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
40

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Provides a Python framework (“SuperTask”) which serves as an interface layer between
pipelineorchestrationandalgorithmiccode,andwhichallowspipelinestobeconstructed,
configured, and run at the level of a single node or a group of tightly-synchronized
nodes;
• Provides support for various middleware and infrastructure toolkits used by DM which
wouldotherwisehavenoauthoritativehomeinstitutionwithinDM(e.g.loggingsupport
library,sphericalgeometrysupportlibrary).
ManagementofthegroupistheresponsibilityoftheDAXT/CAM, eportingtotheDMProject
Manager(Section6.1).
The DAX group is responsible for delivering software which adheres to the architectural and
testingstandarddefinedbytheSoftwareArchitect(Section6.9). Inaddition,theDAXgroupis
responsiblefortestingeachmajorproductdeliveredtodemonstrateitsfitnessforpurpose,
and running and analyzing large scale tests to validate the performance of the science data
archiveanddataaccesssystems.
8.6 NCSA
NCSAhoststheLSSTProjectOffice InformationSecurityOfficerandComputerSecuritygroup,
as well as the DM group responsible for construction and integration of the LSST Data Facility
(LDF),describedbelow.
The LSST Data Facility group has the following major areas of activity within DM:
1. Construction of services, including software and operational methods, supporting ob-
servatoryoperationsandnightlydataproduction(Level1Services).Level1Servicesin-
gestrawdatafromallObservatorycamerasandtheEngineeringandFacilitiesDatabase
(EFD)intothecentralarchive;provideadedicatedcomputingservicecontrollablebythe
ObservatoryControlSystem(OCS)forpromptgenerationofnightlycalibrationassess-
ments,scienceimageparameters,andtransientalerts;andprovidecomputingservices,
data access, and a QA portal for Observatory staff.
2. Construction of services, including software and operational methods, for bulk batch
data production. Batch Production Services execute processing campaigns, using re-
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
41

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
sources at NCSA and satellite computing centers, to produce data release products,
generate templates and calibrations, and perform scaled testing of science pipelines
toassessproductionreadiness.
3. Construction of services, including software and operational methods, for hosting and
operatingdataaccessservicesforcommunityusers.TheseserviceshosttheSUITportal,
managetheJupyterLabenvironment,providecomputinganddatastoragefortheData
AccessCenters,enablebulkdataexport,andhosttheLSSTlimitedalert-filteringservice
andfeedstocommunity-providedbrokers.
4. Construction of services, including software and operational methods, for the Data
Backbone. Data Backbone Services provide ingestion, management, distribution, ac-
cess, integrity checking, and backup and disaster recovery for files and catalog data in
theLSSTcentraldataarchive.
5. Construction and operation of services for LSST staff. Staff Services provide specific
testing and integration platforms (e.g., a Prototype Data Access Center) and general
computinganddataservicesforLSSTdevelopers.
6. ProvisioningandmanagementofhardwareinfrastructureatNCSAandtheChileanBase
Centerforallservicesdescribedabove,aswellasinfrastructureforproject-widenetwork-
basedcomputersecurityservicesandauthenticationandauthorizationservices.
7. Construction and operation of a service management framework and methods to mon-
itor operations of service elements in accordance with service level agreements, track
issues,manageserviceavailability,andsupportchangemanagement.
8. Operation of services and IT systems during construction to support on-going develop-
ment,integration,andcommissioningactivities.
TheLDFgroupisresponsiblefordeliveringinstantiatedproductionservices,whichintegrate
software and hardware components developed across DM. The LDF group performs large-
scaleteststointegrateandverifyproductionreadinessofallcomponents.

Back to top


9 DevelopmentProcess
DM is essentially a large software project; in particular we are developing scientific software
with the uncertainties that brings with it. An Agile process [27] is particularly suited to sci-
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
42

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
entific software development. The development follows a six month cyclical approach and
DM products are under continuous integration using the Jenkins tool. All code is developed
intheGitHubopensource epositoryunderanopensourcelicense. Releasesfollowasix-
month cadence but the code on the master branch is intended to be always working with the
continuousintegrationsystemensuringthis.
How this fits with the Earned Value System is described in DMTN-020.
9.1 Communications
The main stories for the six-month planning period are discussed at the DMLT F2F meeting
near the beginning of the cycle (See Section 7.3).
The T/CAMs of each of the institutions meet via video on Tuesdays and Fridays for a short
standup
meeting to ensure that any cross-team issues are surfaced and resolved expedi-
tiously.ThismeetingischairedbytheDeputyProjectManager.EachT/CAMnotesanysignif-
icant progress of interest to other teams and any problems or potential problems that may
arise.
9.2 Conventions
Codingguidelinesandconventionsaredocumentedonlinein
https://developer.lsst.io
9.3 Reviews
The DM Project Manager and Subsystem Scientist will periodically convene internal reviews
(following LSE-159) of major DM components as necessary to assess progress and maintain
theintegrityoftheoverallsystem. PlannedDMreviewswillbelistedattheLSSTProject
Review Hub
(https://project.lsst.org/reviews/hub/).

Back to top


10 Data Management Problem/Conflict Resolution
Theaboveorganizationalstructureallocatessignificantresponsibilitytoleadinstitutions.As
such,whenproblemsarisethatcannotbesolvedwiththeresponsibilityandscopeallocated
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
43

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
to an institution, the path of escalation and resolution of such problems must be clear.
Anyinter-institutionalproblemshouldbebroughtasearlyaspossibletotheDMProjectman-
ager. The PM will attempt to mediate a resolution. The PM will consult with DMLT, DM System
Science Team and DM Systems Engineering if there are Scientific or technical impacts to be
considered.
Should an issue need to be escalated the PM will bring it up in the weekly LSST project man-
agers meeting. In that forum a way forward will be agreed with the LSST project manager and
othersubsystemmanagers.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
44

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19

Back to top


A DMProductList
WBS
Product
Description
Manager
Owner
Packages
1.02C
DataManagement DataManagementSystem
WilliamOMullane MarioJuric
.
DataAccessCenter DACSoftware
BulkDistrib
BulkDistributionSystem
JoelPlutchak
ProposalManager ProposalManager
JoelPlutchak
ResourceManager DACResourceManager
JoelPlutchak
.
DataBackbone
DataBackboneSystem
1.02C.06.02.01 DataButlerClient DataButlerdataaccessclientlibrary
FritzMueller
daf_persistence/db/daf_fmt_*
Tansport/Repl
Fileanddatabasetransportandreplication
withcachingendpoints
JoelPlutchak
.
ScienceCatArchive Sciencecatalogarchive
1.02C.06.02.03 QservDBMS
Qservdistributeddatabasesystem
FritzMueller
qserv/partition/scisql
.
ScienceCatDBs
Sciencecatalogdatabases
1.02C.03.03
AlertDB
Alertdatabase
SimonKrughoff EricBellm
1.02C.06.01.01 L1CatalogDB
L1catalogdatabase
FritzMueller
cat
1.02C.06.01.01 L2CatalogDB
L2catalogdatabase
FritzMueller
cat
.
Science ImgArchive Scienceimagearchive
1.02C.06.02.05 GlobalMetadata
Globalmetadataservice
FritzMueller
1.02C.06.01.01 ProvenanceDB
Provenancedatabase
FritzMueller
.
Developerservices Developerservices
1.02C.10.02.03.01 Build/UnitTest
Buildandunittestservice
FrossieEconomou
sconsUtils/ base/ lsstsw/
lsst_build
1.02C.10.02.03.04 DevelCommTools Developercommunicationtools
FrossieEconomou
1.02C.10.02.03.03 Doc Infrastructure Documentationinfrastructure
FrossieEconomou
lsst-texmf/ templates/ lsstDoxy-
gen
1.02C.10.02.03.01 SWVersionControl Softwareversioncontrolsystem
FrossieEconomou
1.02C.10.02.03.05 IssueTracking
Issue(ticket)trackingservice
FrossieEconomou
1.02C.10.02.03.02 Packaging/Distrib Packaginganddistribution
FrossieEconomou
lsst/
shebangtron/
lsst_dm_stack_demo
IdentityManager Identity (Authentication and Authorization)
Manager
JoelPlutchak
.
Infrastructure
Infrastructure Software Systems
BatchProc
BatchProcessingSystem
JoelPlutchak
InfraSystems
Filesystems/ provisioning/monitoring sys-
temsandsystemmanagement
JoelPlutchak
.
TaskExecution
Taskexecutionframework
1.02C.06.03
ActivatorBases
ActivatorbaseandCommandLineActivator FritzMueller
1.02C.06.03
PipelineConfig
Pipelineconfiguration
FritzMueller
pex_config
1.02C.06.04.01 Logging
Logging
FritzMueller
log
1.02C.06.03
Multi-CoreTask
Multi-coreTaskAPI
FritzMueller
1.02C.06.03
Multi-NodeTask
Multi-nodeTaskAPI
FritzMueller
pipe_base/ctrl_pool
1.02C.06.03
SuperTask
SuperTask
FritzMueller
pipe_supertask/ pipe_base/
pex_exceptions
.
IntegrationTest
Integrationandtest
1.02C.10.02.03.01 Contin Integration Automatedintegrationandtestservices
FrossieEconomou
Jenkins
1.02C.01.01
PrecursorData
Precursordatafordevelopmentandtesting MarioJuric
obs_*/ validation_data_*/ test-
data_*/ afwdata
.
ITEnvironments
Computing and Storage Infrastructure in-
cludingprovisioning
.
Archive IT
Archive ITEnvironments
ArchiveCenterEnv ArchiveProductionCenterenvironment
JoelPlutchak
ArchiveDACEnv
ArchiveDACenvironment
JoelPlutchak
DAC IntegEnv
DAC Integrationenvironment(PDAC)
JoelPlutchak
ArchiveDBBEnv
ArchiveDataBackboneendpointsandstor-
age
JoelPlutchak
DBB IntegEnv
DataBackbone Integrationenvironment
JoelPlutchak
DevEnv
Developerenvironment
JoelPlutchak
L1 IntegrationEnv Level1 Integrationenvironment
JoelPlutchak
L2 IntegrationEnv Level2 Integrationenvironment
JoelPlutchak
SatelliteEnv
Satellitecomputeenvironment
JoelPlutchak
ScienceValidEnv Archivesciencevalidationenvironment
JoelPlutchak
.
Base IT
Base ITEnvironments
BaseCenterEnv
BaseProductionCenterenvironment
JoelPlutchak
BaseDACEnv
BaseDACenvironment
JoelPlutchak
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
45

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
BaseDBBEnv
BaseDataBackboneendpointsandstorage JoelPlutchak
CommClusterEnv CommissioningClusterenvironment
JoelPlutchak
1.02C.03
Level1System
Level1System
SimonKrughoff EricBellm
1.02C.03.03
AlertDistribution Alertdistributionservice
SimonKrughoff EricBellm
1.02C.03.03
AlertFiltering
Alertfilteringservice
SimonKrughoff EricBellm
L1OfflineProc
L1OfflineProcessingSystem
JoelPlutchak
.
L1 OCS Compo-
nents
Level 1 Online (OCS-connected) compo-
nents
Archiver
ArchivingCommandableSALComponent JoelPlutchak
FelipeMenanteau ctrl_iip
CatchupArchiver Catch-upArchivingCommandableSALCom-
ponent
JoelPlutchak
FelipeMenanteau ctrl_iip
EFDTanform
EFD Tansformation Commandable SAL
Component
JoelPlutchak
FelipeMenanteau
HeaderGenerator HeaderGeneratorCommandableSALCom-
ponent
JoelPlutchak
FelipeMenanteau
OCSBatchProc
OCS-Driven Batch Processing Command-
ableSALComponent
JoelPlutchak
FelipeMenanteau ctrl_iip
PointingPublisher Pointing Prediction Publishing Command-
ableSALComponent
JoelPlutchak
FelipeMenanteau
PromptProc
PromptProcessingCommandableSALCom-
ponent
JoelPlutchak
FelipeMenanteau ctrl_iip
TelemGateway
Telemetry Gateway Commandable SAL
Component
JoelPlutchak
FelipeMenanteau ctrl_iip
.
L1SciencePayloads L1sciencepayloads
.
OfflineAlertProd OfflineAlertProductionpayload
1.02C.03.03
OfflineAlertGen
Offlinealertgenerationpipeline
SimonKrughoff EricBellm
1.02C.03.06
MovingObject
Movingobjectpipeline
SimonKrughoff EricBellm
mops_daymops
1.02C.03.04
Precovery
Precoveryandforcedphotometrypipeline SimonKrughoff EricBellm
1.02C.03.01
OfflineSFP
Offlinesingleframeprocessingpipeline
SimonKrughoff EricBellm
.
PromptAlertProd PromptProcessingAlertProductionpayload
1.02C.03.03
AlertGenPipe
Alertgenerationpipeline
SimonKrughoff EricBellm
1.02C.03.01
SingleFramePipe Singleframeprocessingpipeline
SimonKrughoff EricBellm
pipe_drivers
1.02C.04.02
AuxTelSpecPipe Offline Auxiliary Telescope spectrograph
pipeline
JohnSwinbank
JimBosch
1.02C.04.02
DailyCalibration
OCS-Controlled batch daily calibration up-
datepayload
JohnSwinbank
JimBosch
1.02C.04.02
OfflineCalibration Offline calibration single frame processing
pipeline
JohnSwinbank
JimBosch
pipe_drivers
1.02C.04.02
PromptCalibration Prompt Processing raw calibration valida-
tionpayload
JohnSwinbank
JimBosch
pipe_drivers
1.02C.04.02
CBPControl
OCScontrolscriptsforcollimatedbeampro-
jectorcontrol
JohnSwinbank
JimBosch
1.02C.03
L1QualityControl L1QCmeasurementgenerators
SimonKrughoff EricBellm
.
Level1Services
Level1Services
.
ArchivingServices ImageandEFDArchivingServices
AuxTelArchiver
AuxiliaryTelescopeArchivingService
ComCamArchiver ComCamArchivingService
LSSTCamArchiver LSSTCamArchivingService
ComCamCatchup ComCamCatchupArchivingService
LSSTCamCatchup LSSTCamCatchupArchivingService
EFDTransfService EFDTansformationService
PointingService
PointingPredictionPublishingService
.
PromptServices
PromptProcessingServices
ComCamPrompt ComCamPromptProcessingService
LSSTCamPrompt LSSTCamPromptProcessingService
.
Level2System
Level2System
1.02C.04.07
L2QualityControl L2QCmeasurementgenerators
JohnSwinbank
JimBosch
validate_drp/ verify_metrics/
ci_hsc
.
L2SciencePayloads L2sciencepayloads
1.02C.04.02
CPPQualityControl CPPQCmeasurementgenerators
JohnSwinbank
JimBosch
1.02C.04.02
PeriodicCalProd PeriodicCPPpayload
JohnSwinbank
JimBosch
1.02C.04.02
AnnualCalProd
AnnualCPPpayload
JohnSwinbank
JimBosch
.
DataReleaseProd Annualmini-DRPandDRPpayload
1.02C.04.04
CoaddandDiff
Imagecoadditionanddifferencing
JohnSwinbank
JimBosch
pipe_drivers
1.02C.04.05
CoaddProcessing Coaddprocessing
JohnSwinbank
JimBosch
pipe_drivers
1.02C.04.06
DRPPostprocessing DRPPostprocessing
JohnSwinbank
JimBosch
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
46

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.02C.04.03
ImageCharandCal Imagecharacterizationandcalibration
JohnSwinbank
JimBosch
pipe_drivers
1.02C.04.06
ObjectChar
Multi-epochobjectcharacterization
JohnSwinbank
JimBosch
1.02C.04.05
OverlapResolution Overlapresolution
JohnSwinbank
JimBosch
1.02C.06.01.01 DRP-InternalDB
DRP-internaldatabase
FritzMueller
daf_ingest
1.02C.03.04
TemplateGenProd Templategenerationpayload
SimonKrughoff EricBellm
.
ProductionExec
ProductionExecutionSystem
CampaignManager CampaignManager
JoelPlutchak
JobActivator
JobActivator
JoelPlutchak
Pre-FlightActivator Pre-flightActivator
JoelPlutchak
WorkflowManager WorkflowManager/Orchestrator
JoelPlutchak
ctrl_orca/
ctrl_platform_*/
ctrl_execute/
ctrl_stats/
ctrl_provenance
WorkloadManager WorkloadManager
JoelPlutchak
.
SciencePlatform
LSSTSciencePlatform
.
LSPJupyterLab
LSSTSciencePlatformJupyterLabAspect
1.02C.10.02.02.05 JupyterLabActivator JupyterLabActivators
FrossieEconomou
1.02C.10.02.02.06 JupyterLabDeploy JupterHubdeployment
FrossieEconomou
1.02C.10.02.02.01 JupyterLabEnv
BasicJupyterLabenvironment
FrossieEconomou
1.02C.05.07
JupyterLabSUIT Intf JupyterLabvisualizationwidgetsandother
JupterHub/Portalbridges
XiuqinWu
1.02C.10.02.02.04 JupyterLabSWEnv JupyterLabsoftwareenvironments
FrossieEconomou
.
LSPPortal
LSSTSciencePlatformPortalAspect
1.02C.05.07
FireflyPythonAPIs Low-levelPythonAPI toFirefly
XiuqinWu
firefly_client
1.02C.05.06
Firefly
LSST-independentFireflyframeworkandvi-
sualizationcapabilities
XiuqinWu
firefly
1.02C.05.09
SUI Alert Interfaces Portalalertinterfacestoconfigurealertsub-
scriptions
XiuqinWu
1.02C.05.08
PortalApplications Webapplication(s)implementingthePortal XiuqinWu
userworkspace Portal Interfaces
InterfacestoDAX
identity manage-
ment
1.02C.05.07
XiuqinWu
1.02C.05.07
Visualizers
FireflycomponentstovisualizeLSSTScience
Pipelinesdataobjects
XiuqinWu
.
LSPWebAPIs
LSSTSciencePlatformWebAPIsAspect
1.02C.06.02.05 CatalogAccess
Catalogaccess
FritzMueller
dax_dbserv
1.02C.06.02.05 CatMetadataAcc Catalogmetadataaccess
FritzMueller
dax_metaserv
1.02C.06.02.05 ImgMetadataAcc Imagemetadataaccess
FritzMueller
dax_metaserv
1.02C.06.02.04 ImageAccess
Imageaccess
FritzMueller
dax_imgserv
1.02C.06.02.02 WebFramework
Webservicesframework
FritzMueller
dax_webserv/
dax_webservcommon
.
DMNetworks
DataManagementProvidedNetworks
1.02C.07.08.06 ArcExtlNet
ArchiveExternalNetwork
DonPetravick
DWheeler
1.02C.07.08.03
(moving to
1.02C.08)
BaseNetwork
BaseLocalAreaNetwork
Don Petravick
(moving to Jeff
Kantor)
Jeff Kantor/Don
Petravick
.
ChileanNat
Summit-GatehouseLaSerena-Gatehouse/
La Serena - Santiago Networks DWDM
Equipment
1.02C.08.03.01.03 Summit-AURA
Summit-AURAGatehouseNetwork
Jeff Kantor
Jeff Kantor
1.02C.08.03.01.04 DWDMEquipment DWDMEquipment
Jeff Kantor
Jeff Kantor
1.02C.08.03.01.01A LaSerena-AURA LaSerena-AURAGatehouseNetwork
Jeff Kantor
Jeff Kantor
1.02C.08.03.01.01 LaSer-Santi
LaSerena-SantiagoNetwork
Jeff Kantor
Jeff Kantor
.
Int/USWANs
InternationalWAN/USWAN
1.02C.08.03.02.01 SCL-MIA100Gbps Santiago-Miami100GbpsRing
Jeff Kantor
Jeff Kantor
1.02C.08.03.02.02 NetworkMgmt
NetworkManagement
Jeff Kantor
Jeff Kantor
1.02C.08.03.02.03 SCL-BRSpectrum Santiago-BocaRatonSpectrum
Jeff Kantor
Jeff Kantor
1.02C.08.03.02.01 USNationalWAN USNationalWAN
Jeff Kantor
Jeff Kantor
1.02C.08.03
Long-HaulNets
Summit-Base/Base-Archive/USNetworks Jeff Kantor
Jeff Kantor
.
ScienceAlgorithms Commonsciencealgorithmiccomponents
1.02C.04.06
ApertureCorr
Aperturecorrection
JohnSwinbank
JimBosch
1.02C.03.01
ArtifactDetection Artifactdetection
SimonKrughoff EricBellm
meas_algorithms
1.02C.03.01
Artifact Interp
Artifactinterpolation
SimonKrughoff EricBellm
1.02C.04.06
Association/Match Associationandmatching
JohnSwinbank
JimBosch
1.02C.03.07
AstrometricFit
Astrometricfitting
SimonKrughoff EricBellm
jointcal/
meas_astrom/
meas_mosaic
1.02C.04.03
BackgroundEstim Backgroundestimation
JohnSwinbank
JimBosch
meas_algorithms
1.02C.04.03
BackgroundRef
Background eference
JohnSwinbank
JimBosch
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
47

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.02C.03.02
DIAObjAssociation DIAObjectassociation
SimonKrughoff EricBellm
1.02C.03.04
DCRTemplateGen DCR-correctedtemplategeneration
SimonKrughoff EricBellm
1.02C.04.05
Deblending
Deblending
JohnSwinbank
JimBosch
meas_deblender
1.02C.03.04
ImgDecorrelation Imagedecorrelation
SimonKrughoff EricBellm
ip_diffim
1.02C.04.04
ImageCoaddition Imagecoaddition
JohnSwinbank
JimBosch
coadd_utils/coadd_chisquared
1.02C.03.01
ISR
ISR
SimonKrughoff EricBellm
pipe_tasks/ip_isr
1.02C.04.06
Measurement
Measurement
JohnSwinbank
JimBosch
meas_base/ meas_algorithms/
meas_extensions_*/
meas_modelfit
.
Orbit/Ephemeris
Orbittools
1.02C.03.06
Attribution/Precov Attributionandprecovery
SimonKrughoff EricBellm
mops_daymops
1.02C.03.06
EphemerisCalc
Ephemeriscalculation
SimonKrughoff EricBellm
mops_night
1.02C.03.06
OrbitFitting
Orbitfitting
SimonKrughoff EricBellm
1.02C.03.06
OrbitMerging
Orbitmerging
SimonKrughoff EricBellm
1.02C.03.06
TrackletGen
Trackletgeneration
SimonKrughoff EricBellm
mops_daymops
1.02C.03.07
PhotometricFit
Photometricfitting
SimonKrughoff EricBellm
jointcal/meas_mosaic
1.02C.03.05
ProperMotion
Propermotionandparallax
SimonKrughoff EricBellm
1.02C.04.03
PSFEstimLarge
PSFestimation(visit)
JohnSwinbank
JimBosch
1.02C.03.01
PSFEstimSmall
PSFestimation(1CCD)
SimonKrughoff EricBellm
meas_algorithms
1.02C.04.04
PSFMatching
PSFmatching
JohnSwinbank
JimBosch
RawMeasCal
Rawmeasurementcalibration
JohnSwinbank
JimBosch
1.02C.03.01
ReferenceCatalogs Referencecatalogs
SimonKrughoff EricBellm
meas_algorithms
1.02C.03.02
ReferenceMatch
Matchingto eferencecatalogs
SimonKrughoff EricBellm
1.02C.03.01
SpatialModels
Spatialmodels
SimonKrughoff EricBellm
afw
1.02C.04.05
SourceDetection Sourcedetection
JohnSwinbank
JimBosch
1.02C.04.06
Star/GalaxySep
Star/galaxyclassification
JohnSwinbank
JimBosch
1.02C.03.04
TemplateStorage Differencetemplatestorage/retrieval
SimonKrughoff EricBellm
1.02C.03.04
VariabilityChar
Variabilitycharacterization
SimonKrughoff EricBellm
.
SciencePrimitives Sciencesoftwareprimitives
1.02C.03.05
CameraDescr
Cameradescriptions
SimonKrughoff EricBellm
afw
1.02C.03.05
ChromaticityUtils Chromaticityutilities
SimonKrughoff EricBellm
afw
1.02C.04.01
Convolution
Convolutionkernels
JohnSwinbank
JimBosch
afw
1.02C.03.05
Approx2-DFields Interpolation and approximation of 2-D
fields
SimonKrughoff
EricBellm
afw
1.02C.04.01
Footprints
Footprints
JohnSwinbank
JimBosch
afw
1.02C.03.05
FourierTansforms Fouriertransforms
SimonKrughoff EricBellm
afw
1.02C.03.05
CommonFunctions Commonfunctionsandsourceprofiles
SimonKrughoff EricBellm
afw
.
Geometry
Geometryprimitives
1.02C.03.05
CartesianGeom
Cartesiangeometry
SimonKrughoff EricBellm
1.02C.03.05
CoordT ansforms Coordinatetransformations
SimonKrughoff EricBellm
afw/astshim
1.02C.06.04.03 SphericalGeom
Sphericalgeometry
FritzMueller
sphgeom/skypix/skymap/geom/
afw
1.02C.04.01
Images
Images
JohnSwinbank
JimBosch
afw
1.02C.04.01
MCSampling
MonteCarlosampling
JohnSwinbank
JimBosch
afw
1.02C.04.01
Num Integration
Numericalintegration
JohnSwinbank
JimBosch
afw
1.02C.04.01
NumOptimization Numericaloptimization
JohnSwinbank
JimBosch
afw
1.02C.04.01
PhotoCalRepr
Photometriccalibration epresentation
JohnSwinbank
JimBosch
afw
1.02C.06.02.01 Property/Metadata Multi-typeassociativecontainers
FritzMueller
daf_base
1.02C.03.05
Point-SpreadFuncs Point-spreadfunctions
SimonKrughoff EricBellm
meas_algorithms/shapelet
1.02C.04.01
RandomNumbers Randomnumbergeneration
JohnSwinbank
JimBosch
afw
1.02C.04.01
ScienceTools
Sciencetools
JohnSwinbank
JimBosch
afw/utils
1.02C.04.01
BasicStatistics
Basicstatistics
JohnSwinbank
JimBosch
afw
1.02C.04.01
Tables
Tables
JohnSwinbank
JimBosch
afw
1.02C.03.05
TreeStructures
Treestructures(forsearching)
SimonKrughoff EricBellm
afw
1.02C.04.01
Warping
Warping
JohnSwinbank
JimBosch
afw
.
QCDashboard
QC measurement collection/storage/dash-
boardservice
1.02C.10.02.01.04 AlertQC
AlertstreamQCharness
FrossieEconomou
1.02C.10.02.01.01 QCHarness
QCharness
FrossieEconomou
validate_base
1.02C.10.02.01.02 QCNotifications
QCthresholdnotificationframework
FrossieEconomou
1.02C.10.02.01.03 QCReports
QCverification eporting
FrossieEconomou
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
48

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19

Back to top


B Proposed FY2018 WBS: 1.02C: Data Management Construction
This WBS element provides the complete LSST Data Management System (DMS). The DMS
hasthesemainresponsibilitiesintheLSSTsystem:
• Process the incoming stream of images generated by the Camera Subsystem during
observingtogenerateandarchivetheLSSTnightlydataproducts.
• Provide real-time information on data quality to the Observatory Control System (OCS)
duringobserving.
•Processtheentiresurveydataeachyeartoproducedeepcatalogsofobjectsandprecise
measurementsofthoseobjects.
• CaptureandprocesscalibrationimagesfromtheCameraSubsystem. Incorporatepipeline
improvementsandcorrecterrors.
•ProvideaVO-compliantinterfacethatmakespubliclyavailableallgenerateddataprod-
ucts.
The following products (per Section 5) are defined at this level of WBS:
• Data Management System
1.02C.01: SystemManagement
This WBS element provides all activities related to the management and administration of
the Data Management WBS elements. This includes all activities and support to maintain a
core team responsible for the execution of all Data Management Tasks. Effort in this task
insures compliance with Project level controls, documentation, and eporting. This also in-
cludes overseeing the management of DMS physical assets (hardware, software, facilities,
documents) and support for travel and communications during the Construction Phase, as
wellaspreparingforthesameactivitiesduringtheOperationsPhase.Thisincludes:
• Project management of esources, schedules, tasks, and deliverables.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
49

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
•QualityassuranceoftheDMproject,includingScienceDataQualityAssessment.
• Configuration control of all DMS elements and asset management for all DMS physical
assets.
No products are defined at this level of the WBS.
1.02C.01.01: Project Management
This WBS element includes Project Management staff and supporting tools for the project.
The primary deliverables are the Data Management sections of overall LSST PMCS-based
project plans, schedules, budgets, and eports. Communications and collaboration tools for
teammanagementandcoordinationarealsowithinthisWBSelement.
The following products (per Section 5) are defined at this level of WBS:
• Precursor data for development and testing
1.02C.01.02:[Legacy]ScienceDataQuality IntegrationandTest
ThisWBSelementwasusedearlyinconstruction,andismaintainedonlyforarchivalaccount-
ing purposes. No work is scheduled here; all budget has be been transferred to 1.02C.10.
No products are defined at this level of the WBS.
1.02C.02: SystemsEngineering
ThisWBSelementincludesallsystemsengineeringactivitiesfortheLSSTDataManagement
subsystem, including all subsystem level modeling and simulation development as well as
subsystemandsystemleveltechnicalreviewsofdeliverables.
No products are defined at this level of the WBS.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
50

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.02C.02.01: DataManagementScience
ThisWBSelementprovidesforthescientificleadershipoftheDataManagementSubsystem.
Specifically,theactivitiescoveredbythisWBSelementinclude:
• Ultimate ownership of all science-related DM products (per §5);
• Coordinationofthe InstitutionalScienceLeads;
• Communicationwiththeexternalscientificcommunityandinternalstakeholderstoun-
derstand their needs, and, where applicable, ensure they are satisfied with by the DM
Subsystem;
• Liaisonwithsciencecollaborations;
• Resources to identify, develop and champion new scientific opportunities for the LSST
DM System, as well as identify risks where possible;
• Leadership of the DM Science Validation effort (day to day responsibility for this is dele-
gatedtotheDMScienceValidationScientist(1.02C.09).
No products are defined at this level of the WBS.
1.02C.02.02: DMSystemArchitecture
The Architecture Team is a subset of the DM Systems Engineering Team that implements
decisionsofthatteambycreating,maintaining,disseminating,andensuringadherencetoa
common,consistentsystemarchitecturefortheDataManagementSystem.
TheArchitectureTeammonitorsconstructionactivitiestoensureconsistencywiththedefined
architectureandperformsinvestigationsneededtosupportitscoreresponsibilities.
No products are defined at this level of the WBS.
1.02C.02.02.01: System Architecture Definition
This WBS element includes all activities
related to documenting the high-level architecture of the LSST Data Management System.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
51

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
This includes writing and maintaining documents that define and describe the DMS’s high-
level components and their interfaces, both internal and external, as well as how they work
together and are operated to meet the DM System Requirements. This work is performed
in conjunction with the technical leadership of the DM teams as well as that of other LSST
subsystems.
No products are defined at this level of the WBS.
1.02C.02.02.02: System Architecture Oversight
This WBS element includes all activities
relatedtoensuringthattheconstructedLSSTDataManagementSystem,includingthecom-
puting and storage systems, the processing systems, and the science pipelines, adheres to
itsarchitecturalprinciplesandstandardsandthattheDataManagementdevelopmentpro-
cesses are followed. It involves tracking software development; leading, advising, and edu-
cating during design, code, sprint, and other reviews; contributing to the completeness of
verificationtesting;maintainingtheDMRiskRegister;andcommunicatingtheDMSarchitec-
ture internally and externally. This WBS element also involves making decisions on design
and process changes to ensure emergent properties of the system such as usability, reliabil-
ity,understandability,andmaintainability. TheArchitectureTeamprovidesinputtodecision-
makingpersonnelandbodiesbutdoesnotsupervise,directlycontrol,orexerciseavetoover
development work except where explicitly delegated that role. One such delegation is the
ReleaseManagerrolewhichoverseesandcoordinatesthepreparationforeachsoftwarere-
lease.ArchitectureTeaminputaboutlow-levelcodeisconveyedtoindividualdevelopersdur-
ing reviews. Input about refinement of designs is conveyed to technical leads and the NCSA
Steering Committee. Input about revisions to designs or plans is conveyed to technical man-
agers and the NCSA Steering Committee for incorporation into prioritization. Interactions
withLSSTSystemEngineering,OperationsPlanning,RiskManagement,andChangeControl
are contained within this WBS, as is architectural epresentation in the DM Systems Engineer-
ingTeamandChangeControlBoard.
No products are defined at this level of the WBS.
1.02C.02.02.03: Architecture Investigation
This WBS element includes all activities related
to obtaining the data necessary to make architectural decisions, including literature esearch,
prototyping,andmodel-building.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
52

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
No products are defined at this level of the WBS.
1.02C.03: AlertProduction
This WBS element covers three broad areas of work:
• The development of scientific algorithms and pipelines which will be used to process
theLSSTimagestreamtoidentifytransients,variablesandmovingobjects;
• The rapid production and dissemination of alerts describing sources detected by LSST
inthedifferenceimage;
• The development of a system for identifying moving solar system objects and fitting
theirphysicalproperties;
• The development of reusable algorithmic and software primitives which will be used in
theconstructionofbothnightlyandannualdataprocessingpipelines.
The following products (per Section 5) are defined at this level of WBS:
• Level1System
• L1 QC measurement generators
1.02C.03.00:Management,Leadership&OtherCosts
This WBS element covers project management and scientific leadership of the Alert Produc-
tion group. It includes effort to develop the overall plan for the work of the group, to schedule
that work, to perform day-to-day technical and control account management of the team, to
coordinatedevelopmentwithotherpartsoftheLSSTWBS,andtocontributetotheoperation
oftheDMSubsystemScienceTeam. Italsoincludeseffortfromallteammemberswhichdoes
notdirectlycontributetoaspecificdeliverable(forexample,attendanceatteammeetings).
No products are defined at this level of the WBS.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
53

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.02C.03.01: SingleFrameProcessing
This WBS element covers the construction of the software, together with its associated con-
figurationfiles,testsuitesanddocumentation,usedtoproducethesingleframeprocessing
pipelineThispipelineproducescalibratedimagesfromrawimages.Thefocusisonapipeline
that produces science frames ready for image differencing, but is expected to result in many
pieces that can be re-used in the data release production system.
The following products (per Section 5) are defined at this level of WBS:
•Singleframeprocessingpipeline
• Offlinesingleframeprocessingpipeline
•Referencecatalogs
• ISR
• Artifact detection
• Artifactinterpolation
• Spatial models
• PSFestimation(1CCD)
1.02C.03.02:CatalogAssociationforAlertProduction
This WBS element covers the construction of the software, together with its associated con-
figuration files, test suites and documentation, which is used to associate DIASources with
other entities equired for alert packet construction. Specifically, this will include association
withsolarsystemobjectsandpreviouslyconstructedDIAObjects.
This will also cover the work necessary to carry out the updating of DIAObjects with the addi-
tionofanotherDIASource.
The following products (per Section 5) are defined at this level of WBS:
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
54

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Matchingto eferencecatalogs
• DIAObjectassociation
1.02C.03.03: AlertDistributionSystem
ThisWBSelementcoverstheconstructionofthesoftware,togetherwithitsassociatedconfig-
urationfiles,testsuitesanddocumentation,whichwillmakeupthealertdistributionsystem.
DIAObjects and ancillary data necessary for alert packet assembly will be delivered to this
system.Therearethreepartstothealertdistributionsystem:
• Robust, redundant message queue – DIAObjects and ancillary data will be delivered to
themessagequeuebythealertgenerationpipeline.
• Flexiblestreamfilteringsystem – Willoperateonthepackagedalertstream. Thiswill
provide the interface to both community broker and to the minimal LSST provided fil-
teringsystem.
• Alert database – All alerts will be dumped (possibly verbatim) to a database that can be
replayed later from any point in the stream.
The following products (per Section 5) are defined at this level of WBS:
•Alertgenerationpipeline
•Offlinealertgenerationpipeline
•Alertdistributionservice
• Alert filtering service
• Alert database
1.02C.03.04: AlertGenerationPipeline
This WBS element covers the construction of the software, together with its associated con-
figurationfiles,testsuitesanddocumentation,whichisusedtoproduceallinformationnec-
essary to build the LSST alert stream. This will include work in the areas of:
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
55

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Template optimization – An example area of optimization is that of reducing false pos-
itivesduetothepresenceofdifferentialchromatic efractioninthedatausedtobuild
thetemplates.
• Imagedifferencing – Productionofalgorithmsneededtoproduceoptimalimagediffer-
ences in all contexts LSST expects to operate in (i.e. both low and high galactic latitude).
• Differenceimagemeasurement – Dipole,pointsource(positiveandnegative),andtrailed
sourcemeasurementareallneeded.
The following products (per Section 5) are defined at this level of WBS:
• Precoveryandforcedphotometrypipeline
• Templategenerationpayload
• Differencetemplatestorage/retrieval
• DCR-correctedtemplategeneration
• Imagedecorrelation
• Variabilitycharacterization
1.02C.03.05:ToolsforSciencePipelines
This WBS element covers the construction of the software, together with its associated con-
figuration files, test suites and documentation, which are used to provide tooling, software
primitives,andsoftwareupkeepnecessarytoexecutethesciencepipelinessuccessfully.
The following products (per Section 5) are defined at this level of WBS:
• Proper motion and parallax
• Cartesian geometry
• Coordinatetransformations
• Chromaticityutilities
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
56

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Interpolationandapproximationof2-Dfields
• Common functions and source profiles
• Cameradescriptions
• Point-spreadfunctions
•Fouriertransforms
•Treestructures(forsearching)
1.02C.03.06:MovingObjectProcessingSystem(MOPS)
This WBS element covers the construction of the software, together with its associated con-
figurationfiles,testsuitesanddocumentation,whichareusedtoproducethemovingobject
processing system. This system is responsible for producing high quality orbital properties
forsolarsystemobjectsobservedbyLSST.
The following products (per Section 5) are defined at this level of WBS:
• Moving object pipeline
• Ephemeriscalculation
• Trackletgeneration
•Attributionandprecovery
• Orbit fitting
• Orbit merging
1.02C.03.07: T ansform fitting on stacks of images
ThisWBSelementcoverstheconstructionofthesoftware,togetherwithitsassociatedconfig-
uration files, test suites and documentation, which will allow for high precision photometric
andastrometriccalibrationofobjectsusingmultipleapparitionsofeachobject.
Major features of this work will be:
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
57

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Photometric zeropoint fitting on scales larger than a chip.
• Support photometric zeropoints which vary on scales smaller than a chip.
• Support fitting composeable astrometric models that can epresent distortions on a
varietyofscalesfrommultipledifferentsources.
• The above will allow the distortions from the optical system to be separated from the
distortionsimprintedbytheatmosphere.
The following products (per Section 5) are defined at this level of WBS:
• Astrometricfitting
•Photometricfitting
1.02C.03.08: Integration
This WBS element covers the construction of the software, together with its associated con-
figurationfiles,testsuitesanddocumentation,necessaryfortheintegrationofthealertpro-
duction system and MOPS. This includes pipeline assembly, integration with the workflow
system,definitionandimplementationofinterfaceswithothersubsystems,anddocumenta-
tion of the system to a level where it can be operated by non-AP team members.
No products are defined at this level of the WBS.
1.02C.04:DataReleaseProduction
This WBS element covers three broad areas of work:
• The development of scientific algorithms and pipelines which will be used to generate
LSST’sannualdatareleases;
• The development of algorithms and pipelines which will be used to generate the cali-
bration products equired in both nightly and annual data processing;
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
58

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• The development of reusable algorithmic and software primitives which will be used in
theconstructionofbothnightlyandannualdataprocessingpipelines.
No products are defined at this level of the WBS.
1.02C.04.00:Management,Leadership&OtherCosts
This WBS element covers project management and scientific leadership of the Data Release
Production group. It includes effort to develop the overall plan for the work of the group, to
schedulethatwork,toperformday-to-daytechnicalandcontrolaccountmanagementofthe
team, to coordinate development with other parts of the LSST WBS, and to contribute to the
operation of the DM Subsystem Science Team. It also includes effort from all team members
which does not directly contribute to a specific deliverable (for example, attendance at team
meetings).
No products are defined at this level of the WBS.
1.02C.04.01: Software Primitives
This WBS element covers the construction of low-level, re-usable software primitives which
form the core libraries underlying the LSST Science Pipelines. It includes the production of
testsuitesdemonstratingthecorrectoperationoftheseprimitivesandtechnical,developer-
focuseddocumentationdescribingtheiruse.
The following products (per Section 5) are defined at this level of WBS:
• Images
• Tables
• Footprints
• Basic statistics
• Photometric calibration epresentation
• Convolutionkernels
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
59

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Numericalintegration
• Random number generation
• Numericaloptimization
• Monte Carlo sampling
• Warping
• Science tools
1.02C.04.02: Calibration Products
This WBS element covers the construction of the software, together with its associated con-
figurationfiles,testsuitesanddocumentation,whichisusedtoproducetheLSSTcalibration
products.CalibrationproductsareusedintheLSSTSciencePipelines(bothAlertProduction
andDataReleaseProduction)to:
• Characterizedetectoranomalies;
• Correct for sensor cross-talk;
• PerformphotometriccalibrationthroughunderstandingthethroughputoftheLSSTsys-
temandthetransmissivityoftheatmosphere.
The following products (per Section 5) are defined at this level of WBS:
• PromptProcessingrawcalibrationvalidationpayload
• OCS control scripts for collimated beam projector control
• OfflineAuxiliaryTelescopespectrographpipeline
• Offlinecalibrationsingleframeprocessingpipeline
• OCS-Controlledbatchdailycalibrationupdatepayload
• Periodic CPP payload
• Annual CPP payload
• CPP QC measurement generators
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
60

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.02C.04.03: ImageCharacterization
This WBS element covers the construction of the software, together with its associated con-
figuration files, test suites and documentation, which is used to characterize and calibrate
eachexposureaspartoftheDataReleaseProductionprocessing.Thiswillinclude:
• Estimation of the point spread function;
• Modeling the background;
• Developingastrometricandphotometriccalibrationsolutions.
The following products (per Section 5) are defined at this level of WBS:
• Imagecharacterizationandcalibration
• Backgroundestimation
• Background eference
• PSF estimation (visit)
1.02C.04.04: Coaddition
This WBS element covers the construction of the software, together with its associated con-
figuration files, test suites and documentation, which are used to generate co-added and
differencedimagesaspartoftheDataReleaseProductionprocessing.
The following products (per Section 5) are defined at this level of WBS:
• Imagecoadditionanddifferencing
• PSFmatching
• Image coaddition
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
61

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.02C.04.05:Detection&Deblending
ThisWBSelementcoverstheconstructionofthesoftware,togetherwithitsassociatedconfig-
urationfiles,testsuitesanddocumentation,whichareusedtodetectsourcesonastronomi-
calimages,andtodecomposedetectionswhichconsistofmultipleoverlappingastronomical
objectsintotheirconstituentparts(“deblending”). Italsoincludesfunctionalitytomergere-
dundantprocessingcarriedoutintheoverlappingregionsoftheLSSTskytessellation.
The following products (per Section 5) are defined at this level of WBS:
• Coadd processing
•Overlapresolution
• Source detection
• Deblending
1.02C.04.06: Characterization & Measurement
This WBS element covers the construction of the software, together with its associated con-
figurationfiles,testsuitesanddocumentation,whichareusedtocharacterizeobjectsdetect
inLSSTimages.Thisincludes:
• Applicationoffundamentalmeasurementalgorithms;
• Fitting of source models to data;
• Multi-epochmeasurement;
• Forcedmeasurement;
• Star/galaxyclassification.
The following products (per Section 5) are defined at this level of WBS:
• Multi-epochobjectcharacterization
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
62

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
•DRPPostprocessing
•Measurement
• Aperturecorrection
• Star/galaxyclassification
• Association and matching
1.02C.04.07: Maintenance,Quality&Documentation
This WBS element covers holistic documentation, verification and maintenance tasks that
pertain to the pipelines and algorithms developed in the other 02C.04 WBS elements and
elsewhere in the project. It includes work to construct and maintain an end-to-end Data
ReleaseProductiontestsystemandongoingmaintenancetoadaptthepipelinestochanges
elsewhere in the system. It also includes providing high-level, scientist and operator facing
documentation describing the the system as delivered. Finally, it includes the construction
of QC measurement generators which will be used to verify that pipelines are functioning
correctly during operations.
The following products (per Section 5) are defined at this level of WBS:
• L2 QC measurement generators
1.02C.05:ScienceUser InterfaceandTools
This WBS element covers the work performed by the Science User Interface and Tools group.
Itincludesthefollowing:
• TheFireflyframeworktosupportinfrastructureandcommonfunctionalitiesforallSUIT
applications,includinglow-levelPythonAPI andJavaScriptAPI;
• Componentstoprovideconnectionbetweenthecoredatasearch/visualizationcompo-
nents and the rest of the “LSST Science Platform”;
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
63

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• The web portal aspect of the LSST Science Platform, to enable the science user commu-
nitytoaccess,discover,explore,analyze,anddownloadLSSTdata;
• A web-based user interface to alert subscription and searches.
No products are defined at this level of the WBS.
1.02C.05.00:Management,Leadership,&OtherCosts
ThisWBSelementcoversProjectManagement,ControlAccountManagement,andScientific
Leadership of the Science User Interface and Tools group. It includes effort to develop the
overall schedule and cost for the work, perform day-to-day technical management of the
team, coordinate with other parts of the WBS, provide support for invoices, and contribute
totheoperationoftheDMSubsystemScienceTeam. Italsoincludeseffortfromallteam
memberswhichdoesnotdirectlycontributetoaspecificdeliverable(forexample,attendance
atteammeetings).
No products are defined at this level of the WBS.
1.02C.05.06: Client-serverQuery&VisualizationFramework
ThisWBSelementcoverstheconstructionoflow-level,re-usablesoftwarecomponentswhich
form the core libraries (Firefly) underlying both the client and server sides of the LSST Sci-
ence User Interface Portal. It includes software packaging, releases, and deployment, the
production of test suites demonstrating the correct operation of the components, and the
developmentoftechnical,developer-focuseddocumentationdescribingtheiruse.
Fireflycomprisesavarietyofcapabilities,includingthefollowing:
• Librariesfordatadisplayandvisualizationfortabulardataandastronomicalimages,in-
cludingvarious2Dcharts,andashareddatamodelsupportingdataoverlaysonimages,
andbrushingandlinkingamongrelateddisplays;
• Abstractsearchprocessorinterface;
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
64

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Libraries for data query, etrieval, and export, including an abstract search processor
interface as well as implementations providing common astronomical archive search
formsandinterfacestostandard(includingVO)dataqueryAPIs;
• Support for various common formats for astronomical tabular and image data;
•JavaScriptandlow-levelPythonAPIs,usedinternallyaswellasprovidingforusercontrol
andcustomization;
• Identityandpreferencesmanagement;
• Load balancing.
The following products (per Section 5) are defined at this level of WBS:
• LSST-independentFireflyframeworkandvisualizationcapabilities
1.02C.05.07:LSSTSciencePlatform Interfaces
ThisWBSelementcoverstheconstructionofsoftwarecomponentsthatprovidetheconnec-
tionbetweenthecoredatasearch/visualizationcomponentsandtherestofthe“LSSTScience
Platform”: supporting the LSST data model, data-access services, and compute and storage
esourceaccess. Itincludesthedevelopmentofassociateddocumentationandtestsuites.
Itincorporatesthefollowing:
• InterfacetoLSST-specificuseridentityandmanagementservices;
• Access to LSST user workspace;
•SearchprocessorsmatchingtheDAXinterfaces;
• SearchprocessorsfortheEngineeringandFacilitiesDatabase;
• InterfaceforinvokingLSSTstackPythoncodetoperformservicesneededbythePortal;
• SupportforreadinganddisplayingLSST-specificdataformats(suchasafw.table-format
files)anddataobjects(e.g.,masks,Footprints,PSFmodels);
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
65

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Support for the afw.display interface to Firefly.
The following products (per Section 5) are defined at this level of WBS:
• JupyterLabvisualizationwidgetsandotherJupterHub/Portalbridges
• Low-level Python API to Firefly
• InterfacestoDAX,userworkspace,SuperTask,identitymanagement
•FireflycomponentstovisualizeLSSTSciencePipelinesdataobjects
1.02C.05.08: Applications
This WBS element covers the construction of the Portal Aspect of the LSST Science Platform
software, including online help, deployment instructions, and other documents. The Portal
uses the core Firefly components and the LSST-specific software interfaces in 1.02C.05.07,
relying on infrastructure and data access services provided under other WBS elements, to
deliveraportalforthesciencecommunitytoaccess,discover,explore,analyze,anddownload
the LSST data. It covers the following:
• The overall user interface layout and structure;
• Basic access to all LSST catalog and image data (Level 1, Level 2, user/Level 3, calibration,
andEngineeringandFacilitiesDatabase);
• Scientifically motivated workflows guiding users to the available data and illuminating
connectionsamongtablesandbetweentablesandimages;
• All-sky displays allowing exploration of the image data and of maps of the properties of
the LSST survey across the sky;
• Access to the user workspace, supporting data sharing and and collaboration;
• Deploymentpackagingandconfigurationmanagement.
The following products (per Section 5) are defined at this level of WBS:
• Webapplication(s)implementingthePortal
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
66

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.02C.05.09: Alert Interfaces
This WBS element covers the construction of a Web-based user interface for alert subscrip-
tionandsearches. ItisbasedonthecoreFireflypackageandusesinterfacesprovidedby
SUIT,DAX,andAP. Itcovers:
• Alertsubscription,settingfiltersandalertstreamdestination;
• Access to user management system;
• Alert searches and filtering.
The following products (per Section 5) are defined at this level of WBS:
• Portalalertinterfacestoconfigurealertsubscriptions
1.02C.05.10: Integration&Test
This WBS element covers the integration and higher-level testing of all the deliverables from
the 02C.05 sub-WBSs, including manual UI tests, tracking and eporting bugs in the software
fromotherteams,possibleautomatedUI testsetupandinstructions,anddeploymentofthe
softwareattheLSSTDataAccessCentersincollaborationwithNCSA.
No products are defined at this level of the WBS.
1.02C.06:ScienceDataArchiveandApplicationServices
TheScienceDataArchiveandDataAccessServicesprovidestheabilitytoingest,index,feder-
ate,query,andadministerDMdataproductsondistributed,heterogeneousstoragesystems
anddataservers.Allserviceswillbeimplementedtoprovidereasonablefault-toleranceand
autonomousrecoveryintheeventofsoftwareandhardwarefailures.
No products are defined at this level of the WBS.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
67

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.02C.06.00:Management,Leadership,&OtherCosts
ThisWBSelementcoversProjectManagement,ControlAccountManagement,andtechnical
leadership of the Science Data Archive and Data Access Services group. It includes effort to
develop the overall schedule and cost for the work, perform day-to-day technical manage-
ment of the team, coordinate with other parts of the WBS, and provide support for invoices.
It also includes effort from all team members which does not directly contribute to a specific
deliverable(forexample,attendanceatteammeetings).
No products are defined at this level of the WBS.
1.02C.06.01:ScienceDataArchive
This WBS element is a summary element that includes software programs, database tables,
configuration files, unit tests, component integration tests, and documentation that imple-
mentstructuresandtoolstomanagetheLSSTDataProductsindatabasesandfiles,including
defining schemas and ingesting tables and files and their metadata and provenance into the
archive.
No products are defined at this level of the WBS.
1.02C.06.01.01: Database Catalogs, Alerts, and Metadata
This WBS element includes
software programs, database tables, configuration files, unit tests, component integration
tests,anddocumentationthatimplementDatabaseCatalogs,Alerts,andMetadatacapabili-
ties.
Itimplementsalldatabasecatalogs:L1AlertProductionandUserDatabase,L2 InternalDRP,
L2DataReleaseCatalogs,Level3Catalogs,CalibrationDatabase,RestructuredEngineering
Facilities Database, and Deep Drilling Database. It includes schemas and structures (parti-
tioning,replication,distributionmodels,L1production/user,L2swap/release),andtoolsfor
manipulating the catalogs, such as managing ingest, replication, hot swap, recovery and im-
port/export.
Itimplementsdata-product-specificmetadataandprovenanceforallLSSTdataproductcat-
alogsandimages. Itincludesschemasandstructures(partitioning,replication,distribution
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
68

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
models),andtoolsformanipulatingthemetadata,suchasmanagingingest,replication,hot
swap, recovery and import/export. Global metadata that spans multiple data products is
handled through 02C.06.02.05.
It implements structure for alerts, as well as tools for persisting and efficiently etrieving
them.
The following products (per Section 5) are defined at this level of WBS:
• L1 catalog database
• L2 catalog database
• Provenancedatabase
• DRP-internaldatabase
1.02C.06.01.02: Image and File Archive
This WBS element includes software programs,
databasetables,configurationfiles,unittests,componentintegrationtests,anddocumenta-
tion that implement the Image and File Archive, including tools for managing image and files
(ingestion,import/export).
No products are defined at this level of the WBS.
1.02C.06.02:DataAccessServices
This WBS element is a summary element that includes software programs, database tables,
configuration files, unit tests, component integration tests, and documentation that imple-
mentDataAccessServicecapabilities.
No products are defined at this level of the WBS.
1.02C.06.02.01: Data Access Client Framework
This WBS element includes software pro-
grams,databasetables,configurationfiles,unittests,componentintegrationtests,anddoc-
umentationthatimplementoverallClientFrameworkforDataAccessServices.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
69

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
Itimplementscapabilitytostoreand etrieveLSSTDataProductsintermsoftheirapplication
level“astronomical”semantics,mappingthosesemanticstophysical,persistentversionsof
thosedataproductsindatabasesandfiles.
It provide capabilities to run user data analysis close to the data.
The following products (per Section 5) are defined at this level of WBS:
• Data Butler data access client library
• Multi-typeassociativecontainers
1.02C.06.02.02: Web Services Framework
This WBS element includes software programs,
databasetables,configurationfiles,unittests,componentintegrationtests,anddocumenta-
tionthatimplementanoverallframeworkforrunningdatabase,metadataandimagecutout
services.
Theworkinvolvesproviding IVOAstandardserviceinterfaceswhereapplicable.
The following products (per Section 5) are defined at this level of WBS:
• Web services framework
1.02C.06.02.03: QueryServices
ThisWBSelementincludesworkneededtocomeupwith
a DBMS that meets LSST user query analysis needs. Such DBMS should include standard
off-the-shelfDBMScapabilitiesincludingadvancedfeaturessuchasscalabilitytopetabytes,
incremental scaling, parallel queries, shared scans, fault tolerance, esource management, as
wellasLSST-specificfeaturessuchasefficientsupportforspatialandtemporaldataatscale.
The work involves customizing, optimizing, improving and gluing together relevant compo-
nents, building missing features, configuration files, unit tests, integration tests, and doc-
umentation. It also includes building tools for maintaining, configuring, and administering
suchsystem.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
70

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
The following products (per Section 5) are defined at this level of WBS:
•Qservdistributeddatabasesystem
1.02C.06.02.04: Image and File Services
This WBS element includes software programs,
databasetables,configurationfiles,unittests,componentintegrationtests,anddocumenta-
tion that implement image and file services. Image and File Services provide the capability to
access and manipulate image and file-based data, manage file caches, and ecreate images
ondemand.
The following products (per Section 5) are defined at this level of WBS:
• Imageaccess
1.02C.06.02.05: Catalog Services
This WBS element includes software programs, database
tables,configurationfiles,unittests,componentintegrationtests,anddocumentationneeded
to build web services on top of all LSST database products (all levels, all metadata).
It includes work on global metadata structures for all LSST data products, including all data
releases, L3 user data and all images. Data-product-specific metadata is handled through
02C.06.01.01.
The following products (per Section 5) are defined at this level of WBS:
• Global metadata service
• Catalog access
• Imagemetadataaccess
• Catalog metadata access
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
71

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.02C.06.03: TaskFramework
ThisWBSelementincludessoftwareprograms,databasetables,configurationfiles,unittests,
componentintegrationtests,anddocumentationneededtobuildtheTaskFramework.
The Task Framework is a Python class library that provides a structure (standardized class
entrypointsandconventions)toorganizelow-levelalgorithmsintopotentially-reusablealgo-
rithmic components (Tasks; e.g. dark frame subtraction, object detection, object measure-
ment),andtoorganizetasksintobasicpipelines(SuperTasks;e.g.,processasinglevisit,build
a coadd, difference a visit). The Task Framework allows the pipelines to be constructed, con-
figured,andrunatthelevelofasinglenodeoragroupoftightly-synchronizednodes. In
additiontomulti-nodeTasks,italsoallowsforsub-nodeparallelizationacrossmultiplecores.
Pipelineconfigurationincludesconfiguringparametersforscientificalgorithms,allowingover-
rides of defaults based on camera/survey, computing environment, or user choice. It also
includesconfigurationofdebuggingcapabilitiesusedduringpipelinedevelopment.
TheTaskFrameworkservesasaninterfacelayerbetweenorchestrationandthealgorithmic
code. It exposes a standard interface to “activators” (command-line runners as well as the
orchestration layer and QA systems), which use it to execute the code wrapped in tasks. The
TaskFrameworkexposestotheorchestrationsystemneedsandcapabilitiesoftheunderlying
algorithmic code (e.g., the number of cores needed, expected memory-per-core, expected
needfordata). Itmayalsoreceivefromtheorchestrationlayertheinformationonhowto
optimallyruntheparticulartask(i.e.,whichlevelofintra-nodeparallelizationisbedesired).
ThisWBSincludesconstructionofbasicimplementationsforthesecomponents.Morecom-
plex (or custom) implementations and alternative backends for the APIs and components
above(e.g.,aspecialbackendto etrieveaconfigurationfromacentraldatabase,oraback-
end to send logs to a database instead of files, or a MultiCore API backend that’s better aware
oflocalmachinearchitecture)areoutofscope.
The following products (per Section 5) are defined at this level of WBS:
• SuperTask
• Activator base and Command Line Activator
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
72

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Pipelineconfiguration
• Multi-node Task API
• Multi-core Task API
1.02C.06.04: Middleware, Infrastructure,andToolkits
This WBS element is a summary element that includes software programs, database tables,
configuration files, unit tests, component integration tests, and documentation that imple-
mentbasicmiddlewareinfrastructureprimitives.
No products are defined at this level of the WBS.
1.02C.06.04.01: Logging
This WBS element includes software programs, database tables,
configuration files, unit tests, component integration tests, and documentation that imple-
ment a set of classes/functions enabling tasks to log diagnostic messages about their execu-
tion.
The following products (per Section 5) are defined at this level of WBS:
• Logging
1.02C.06.04.02: Daf_base and Utilities
This WBS element is a summary element that in-
cludes software programs, database tables, configuration files, unit tests, component inte-
gration tests, and documentation that implement small common utility classes for memory
management,key/valuestorage,datesandtimes,RA/declinationformatting,etc.
No products are defined at this level of the WBS.
1.02C.06.04.03: Sphgeom
This WBS element is a summary element that includes software
programs,databasetables,configurationfiles,unittests,componentintegrationtests,and
documentationpertainingtothesphgeomsphericalgeometrylibrary.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
73

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
The following products (per Section 5) are defined at this level of WBS:
• Spherical geometry
1.02C.07:LSSTDataFacility
This WBS element primarily supports the construction of the LSST data facility, a distributed
facility centered at NCSA, with a goal of providing services in Construction to Observing Op-
erations and Science Operations. Services range from acquisition of pixels from the LSST
instruments, faithful retention of LSST data, batch production (including executing data re-
lease processing, and providing data access centers). Construction responsibilities include
providing computing esources, integration of the center, and providingservices to the con-
struction project. The LSST data facility uses esources at the base center, NCSA, CC-IN2P3,
andcommercialproviders.
No products are defined at this level of the WBS.
1.02C.07.05:LSSTDataFacilityManagement,ServiceArchitecture,andProjectControls
This element of the WBS contains the work to oversee and manage the LSST Data Facility’s
performance and strategy, design and interface controls, and project controls and eporting.
The work includes all cross-cutting elements of the Data Facility: line management, gover-
nance and oversight, overall engineering and design, planning for operations, service man-
agement, and project eporting.
1.ManagementandOversight
2. ServiceArchitectureandManagement
3.ProjectControlsandReporting
No products are defined at this level of the WBS.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
74

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.02C.07.06.01: LDF-offered Services
This element of the WBS contains the work to instantiate and run LSST Data Facility (LDF)
production services, which each satisfy a specific use case, in order to achieve LSST science
equirements.
The work includes integration of all service components, development of verification and
validationtests,readinesstesting,service-leveldocumentation,integrationintoserviceman-
agement and service monitoring systems (including feeding status and quality metrics for
display),integrationwithsecuritycontrols,configurationofcomponentsandintegrationwith
reliantservices,deploymentintoproduction,earlylifesupport,andoperationforconstruc-
tionandcommissioningusecases(includingmanagement,configuration,upgrading,mon-
itoring, request response, problem management, and first-order quality assurance of data
productsandscientificandtechnicalaspectsoftheproductionservices).
1. ServicesforObservatoryOperations
2. ServicesforDesignatedOfflineCampaignProcessing
3. Data Access Services for Authorized Users
4. Services for General Staff
5. Data Facility Service Desk
No products are defined at this level of the WBS.
1.02C.07.06.02: Reusable Production Services
ThiselementoftheWBScontainstheworktoinstantiateandrunproject-orientedproduction
servicesthatarereusedtosupportmanydevelopment,integrationandproductionusecases.
These services understand the operational relationships of the service dependencies and
componentsandareawareof epresentativeusecasesoftheservice.
Theworkincludesintegrationofallservicecomponents,readinesstesting,service-leveldocu-
mentation,integrationintoservicemanagementandservicemonitoringsystems,integration
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
75

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
with security controls, configurations of components and integration with reliant services,
deploymentintoproduction,earlylifesupport,andoperationforconstructionandcommis-
sioningusercases(includingmanagement,configuration,upgrading,monitoring,requestre-
sponse, problem management, and first-order quality assurance of scientific and technical
aspectsofproductionservices).
1. PromptProcessingService
2. InternalTransientEventHandlingService
3. TelemetryGatewayingService
4. Master Batch Job Scheduling Service
5. QA Portal Hosting Service
6. ImplementationofFileManagementPoliciesandHigh-levelDataMovementWorkflows
7. Management of End-user Data Rights
8.CentralElementsofWorkflows,Reports,and Interactive InformativeDisplaysbasedon
IdMServiceEndpoints
No products are defined at this level of the WBS.
1.02C.07.07:Data,Computeand ITSecurityServices
This element of the WBS contains the work to instantiate and run general IT services that sup-
portallproject-facingservicesdescribedintheprecedingWBSelements.Thislayerachieves
the functionality of storing files and data within the Data Backbone and providing access at
all service endpoints with the equired quality of service.
Theworkincludesintegrationofallsoftwareandhardwarecomponentsintoaservice,readi-
nesstesting,service-leveldocumentation,integrationintoservicemanagementandservice
monitoring systems, integration with security controls, configurations of components, de-
ployment into production, early life support, and operation for construction and commis-
sioningusercases(includingmanagement,configuration,upgrading,monitoring,requestre-
sponse, problem management, and first-order quality assurance of scientific and technical
aspectsofproductionservices).
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
76

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.File-orientedServiceswithintheDataBackbone
2.ManagedDatabaseServices
3.BackupandDisasterRecoveryServices
4.BatchComputingandDataStagingEnvironmentServices
5. ContainerizedApplicationManagementServices
6. ITServiceManagementandMonitoringSupportServices
No products are defined at this level of the WBS.
1.02C.07.08:LDFServiceSoftware
This element of the WBS contains the work to construct, test, and maintain software for LSST
DataFacilityServices.
1. Level 1 Services Software
2. BatchProductionServicesSoftware
3.DataBackboneServicesSoftware
4. MiscellaneousFacilityServicesSoftware
The following products (per Section 5) are defined at this level of WBS:
• Base Local Area Network
• Archive External Network
1.02C.07.09: ITCandFacilities
This element of the WBS contains the work to provide ITC and supporting facility elements
fortheUSArchiveCenterandtheChileanBaseCenter. ITCincludeslocalandwide-area
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
77

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
networking, file storage esources and file systems, disaster recovery esources, database
hardware,computesystems,and ITCmanagementinfrastructure.
The work includes planning, provisioning, operation, and decommissioning of physical re-
sources at Chile and NCSA, as well as the work to construct and operate ITC configuration
managementtools(e.g.,Puppet),coordinationtools(e.g.,ticketsystems),and ITCprocesses
(e.g.,incidentresponse). ITCisorganizedbysecurityenclave,eachwithspecificadministrative
controls.
1. MasterProvisioningEnclaves ITC
2. Networking
3. Developmentand IntegrationEnclave ITC
4. L1Enclave ITC
5.GeneralProductionEnclave ITC
6. General Base Enclave ITC
7. USDACEnclave ITC
8. Chilean DAC Production Enclave ITC
9. Data Backbone Enclave ITC
No products are defined at this level of the WBS.
1.02C.08: InternationalCommunicationsandBaseSite
This WBS element is a summary element that includes the infrastructure for the Base Center
andthenationalandinternationalnetworksconnectingtheMountainSummit,Base,Archive,
andHeadquarterssites.
No products are defined at this level of the WBS.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
78

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.02C.08.01:BaseCenter
This WBS element is a summary element that includes the infrastructure that receives data
from the camera DAQ subsystem and the Observatory Control System, store a copy of that
data, and forwards the data on to the Archive Center for processing. It also includes a cluster
dedicated to Commissioning activities. The Base Center is hosted in the Base Facility, which
is provided by the Telescope and Site WBS.
No products are defined at this level of the WBS.
1.02C.08.02:ChileanDataAccessCenter
ThisWBSelementisasummaryelementthatincludestheinfrastructurefortheChileanData
Access Center. The Chilean Data Access Center provides a community access point for all
LSST data, including computing and storage esources dedicated to end user processing to
createLevel3dataproductsenablingtheLSSTscience.
No products are defined at this level of the WBS.
1.02C.08.03:LongHaulNetworks
TheLSSThigh-speednetworkplanconsistsoftwoprinciplesegmentsandseveralsub-segments:
1. Chilean National WAN
• LaSerena–Santiago(principalandsecondarypaths)
• La Serena–AURA Gatehouse
• AURAGatehouse–Summit
2. InternationalChile–USWAN
• Santiago–U.S (100 Gbps Ring)
• Santiago–US(Spectrum)
• USNational
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
79

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
Implementation of these two primary segments were assigned to REUNA (Chilean) and FI-
U/AmLight(International). Subsequently,itwasdeterminedthattheAURAGatehouse–Summit
subsegmentwouldbeadirectcontractfromAURAtoTelefonicaforinstallation,andoperated
byAURA/REUNA.
TheChileannetworkimplementationinvolvestheexecutionoffiveseparatecontracts.These
contracts are embedded in the Work Breakdown Structure and details are provided for each
one
6
.
Similarly,the Internationalnetworkimplementationinvolvestheexecutionoffourseparate
contracts. These contracts are embedded in the Work Breakdown Structure, and details are
providedforeachone
7
.
The following products (per Section 5) are defined at this level of WBS:
• Summit-Base/Base-Archive/USNetworks
• La Serena - Santiago Network
• La Serena - AURA Gatehouse Network
• Summit - AURA Gatehouse Network
• DWDM Equipment
• Santiago-Miami100GbpsRing
• Network Management
• Santiago - Boca Raton Spectrum
• USNationalWAN
1.02C.09:SystemLevelTesting&ScienceValidation
This WBS element covers oversight and management of integration and test activities. Each
WBS has its own elements of the integration and test for specific items. It includes:
6
Contractualdetailshavebeenelidedfromthesummaryinthisdocument.
7
Ditto.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
80

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
• Support for the activities of the DM Validation Scientist and the management of the
ScienceValidationteam.
•Maintenanceoftheoverallsubsystemtestingplan(LDM-503).
•Participationinlargescaletests,withspecificresponsibilityforgatheringoftestresults.
• Curation of fixed data sets and associated tests assembled to provide a rich set of test
datafortheSoftware&ScienceQualityControlService(1.02C.10.02.01)and,wherenec-
essary,forvalidationactivities.
No products are defined at this level of the WBS.
1.02C.10:ScienceQualityandReliabilityEngineering
ScienceQualityandReliabilityEngineering(SQuaRE)deliversservicesthatensurethequality
ofDMsoftware,dataproductsandreliabilityofservices.
This WBS element covers three broad areas of work:
• AutomatedSoftwareandScienceQualityControl(Verification)Services;
•SciencePlatformNotebookEnvironmentforQA,Commissioning&UserScience;
• Developer Infrastructure,SoftwareDistribution,DocumentationTooling,Communica-
tionTooling.
No products are defined at this level of the WBS.
1.02C.10.01:Management,Leadership,&OtherCosts
This WBS element consists of function associated with the project, technical and scientific
management of the 1.02C.10 WBS, including planning, eporting, presentations, meetings,
staffing and other functions associated with organizing delivery of the WBS. It also includes
SQuaRE staff participation in meetings and events equiring their presence, such as the regu-
larLSSTJointTechnicalMeetingsandProject&CommunityWorkshops.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
81

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
No products are defined at this level of the WBS.
1.02C.10.02: QualityControl,DataspaceServices,andDeveloper Infrastructure
This WBS element consists of software, services, unit tests, integration tests, configuration
anddeploymentautomation,availabilitymonitoringanddocumentationfor:qualityanalysis,
enablingofscienceanalysis,automatedqualitycontrol,verificationanddeveloperservices.
No products are defined at this level of the WBS.
1.02C.10.02.01: Automated Software and Science Quality Control Service
This WBS el-
ement consists of software and services that support the implementation of Data Manage-
ment’s plan to ensure the quality of the DM Pipelines.
1.02C.10.02.01.01: SQuaSH
A harness for executing prepared tests automatically and con-
tinuously to characterize the algorithmic performance of the code, key aspects of the
performance of the facility that are apparent in the data, its verification status, and un-
cover egressions to aid development.
1.02C.10.02.01.02: Monitoring
A system for notifying when values for SQuaSH metrics ex-
ceednotifiablelimits.
1.02C.10.02.01.03:VerificationReportsTooling
UsingdataproducedbySQuaSHtocreate
verification eports and software release characterizations.
1.02C.10.02.01.04: Alert QA harness
A harness to perform QA tests on the alert stream.
The following products (per Section 5) are defined at this level of WBS:
• QCharness
• QCthresholdnotificationframework
• QCverification eporting
• Alert stream QC harness
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
82

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.02C.10.02.02:SciencePlatformNotebookEnvironmentforQA,Commissioning&User
Science
1.02C.10.02.02.01: Jupyter Notebook & Templates
A set of notebooks, and templates for
makingthem,thatdemonstratekeyfeaturesofthecapabilitiesofthesystem.
1.02C.10.02.02.02: JupyterLabDeployment
Architecture,orchestrationanddeploymentcon-
figurationfortheSciencePlatformNotebookserviceforcommissioning.
1.02C.10.02.02.03:CustomPortals/Notebooks
ThisWBSelementcoverssupportingthepor-
tals delivered by the SUIT team (1.02C.05.07) post-delivery where they relate to QA and
commissioningactivitiesasnecessary.
1.02C.10.02.02.04:NotebookSoftwareEnvironments
Productionofenvironments(e.g.con-
tainers)suitablefortheexecutionofcustomportals/notebooks.
1.02C.10.02.02.05: Notebook Execution
The process to scale notebook execution so they
can execute over a large dataset. This involves an interface to the batch workflow sys-
tem.
1.02C.10.02.02.06: Dataspace packaging
The packaging and configuration equired to de-
ploy the dataspace on a platform that is design-matched to the compute and filespace
elements of the Archive Center dataspace (e.g. if the DAC compute is based on an Open-
Stackarchitecture,thedeliverableofthisWBSarethepackages,configuration,automa-
tion deployment and instructions that would allow a Data Access Center at an interna-
tional partner to deploy a Dataspace service on top of their open OpenStack compute
fortheirownusers).
The following products (per Section 5) are defined at this level of WBS:
• BasicJupyterLabenvironment
• JupyterLabsoftwareenvironments
• JupyterLabActivators
• JupterHubdeployment
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
83

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
1.02C.10.02.03: Developer Infrastructure, Software Distribution, Documentation Tool-
ing, Communication Tooling
This WBS element consists of services that support a large
distributed software team and its product. It includes systems that support current best
practicesinsoftwareengineeringsuchascontinuousintegration,releasemanagement,soft-
warepackaginganddistribution,documentationstandards,andinfrastructureandcommu-
nication tooling supporting development and team culture. All these services are oriented
towards developers, and some are also oriented towards users of the DM software outside
DM.WhiletheseareEVMdeliverables,workisplannedinsuchawayto eserveeffortforad-
hocdeveloper-drivenrequests,sincethesearecustomer-orientedservicesthatbenefitfrom
continuousimprovement.
1.02C.10.02.03.01: SoftwareDevelopmentServices
Continuous Integrationservice(s),repos-
itorymanagement,codelinters,softwaredevelopmentenvironments.
1.02C.10.02.03.02:ReleaseEngineering
WorkwiththeReleaseManager(1.02C.02.02.02)to
provideportabilitytesting,binaryandcontainerizeddistribution,buildtooling.
1.02C.10.02.03.03:DocumentationTooling
Documentation standards, documentation lin-
ters,softwareandtechnicaldocumentationproductionandpublication,developerguide,
userguide,tutorials,documentdiscoveryservices.
1.02C.10.02.03.04: CommunicationTooling
Communityforum,ChatOps.
1.02C.10.02.03.05: Bug/TrackingHelpdesk
BugTracking,Helpdesk,CommunityManagement
The following products (per Section 5) are defined at this level of WBS:
•Softwareversioncontrolsystem
• Buildandunittestservice
•Packaginganddistribution
• Documentationinfrastructure
• Developercommunicationtools
• Issue(ticket)trackingservice
•Automatedintegrationandtestservices
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
84

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19

Back to top


C DMDiscussionandDecisionMakingProcess
The Escalation process only occurs when the issue cannot be resolved within the DM, i.e.
when the following internal discussion and decision making process has failed to yield a de-
cision.
C.1 Empowerment
All DM team members are empowered by the DM Project Manager (PM) and DM Subsystem
Scientist(SS)tomakedecisionsonanyDM-internalmatter,includingtechnical/algorithmis-
sues,processimprovements,toolchoices,etc.,when:
1. theyarewillingandabletodotheworktoimplementthedecisionorwithpeoplewho
agree with the team member,
2. they (collectively) are willing and able to fix any problems if it goes wrong, and
3. they believe that all affected parties (including your immediate manager) would not se-
riouslyobjecttoyourdecisionandimplementation.
C.2 RFCProcess
If the above three criteria are not met, perhaps because the team member doesn’t know all
the affected parties or because they don’t know their positions, the team member should
publish the proposed decision and implementation as a JIRA issue in the Request For Com-
ments (RFC) project with a component of “DM.”
It is usually difficult to determine all the affected parties for published package interfaces.
Changestointerfacesshouldthustypicallygothroughthisprocess.
It’s a good idea to contact any known affected parties before starting this process to check
that the resolution is sensible. The institutional technical manager is always affected, as she
orheisresponsiblefortrackingtheworkschedule. Ifworkforothersisbeingproposed,
they are obviously affected. The institutional scientist, the DM Software Architect (SA), the
DM Interface Scientist (IS), and the DM Subsystem Scientist (SS) are also valuable esources
fordeterminingaffectedparties.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
85

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
The purpose of an RFC is to inform others about the existence and content of the proposed
decision and implementation in order to allow them to evaluate its impact, comment on it,
refineitifnecessary,andagree(implicitlyorexplicitly)orobject(explicitly)toitsexecution.
ThediscussionoftheRFCtakesplaceinthemediumofthe equestor’schoosing(e.g.,aspe-
cific mailing list, the RFC JIRA issue itself, a Slack Channel, a convened videocon, some combi-
nationofthose,etc.),butthe equestorshouldbeopentoprivatecommunicationsaswell.
In the RFC process, the opinions of those who will be doing the work (and fixing any problems
if something goes wrong) are given more weight. In some cases, this may mean that the RFC
issue’sAssigneepassestosomeoneelse.Theopinionsofmoreseniorpeopleorpeoplemore
experienced in the area should also be given more weight and may also result in the Assignee
changing.
TheAssigneeisresponsiblefordeterminingwhennoseriousobjectionsremain. Inparticular,
there is no need to call for a formal vote on the (refined) resolution. If no explicit objections
havebeenraisedwithin,typically,72hoursfor“ordinary”issuesand1weekfor“major”issues,
the Assignee should assume that there are none. This is known as “lazy consensus.” When
thisstatehasbeenreached,theAssigneeisresponsibleforensuringthatthefinalconsensus
hasbeen ecordedintheRFCissuebeforeclosingitandproceedingwithimplementationof
thedecision.
The equestor must be especially careful about not making irreversible changes in the “lazy
consensus”timeperiodunlesstheyareabsolutelycertainthere’sageneralagreementonthe
statedcourseofaction. Ifsomethingisbroken,the equestormustbebereadytofixit. It
is critical to apply sound reasoning and good judgment about what may be acceptable and
whatmightbenot. Mistakeswillhappen;acceptthatoccasionallytherewillbea equirement
to revert an action for which it was thought agreement existed.
C.3 ExceptionsandAppeals
Some proposed resolutions may equire changes to one or more of the baselined, change-
controlleddocumentsdescribingtheDataManagementsystem(thoseinDocuSharewithan
LDM- handle or marked as change-controlled in Confluence). Note that major changes to
budget or scope will almost certainly affect one or more LDM- documents. In this case only,
the DM Configuration Control Board (DMCCB) (Section 7.4) may empanel an ad hoc commit-
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
86

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
tee including the lead author of the document and other relevant experts. This committee or
theCCBitselfmust
explicitly
approvethechange.
Change-controlleddocumentswithotherhandles,suchasLSE-orLPM-,includinginter-subsystem
interfaces, have project-wide change control processes. Please consult the DM PM, SA, or IS
for more information. At least one member of the DM CCB will read each RFC to determine if
itmightaffectachange-controlleddocument.
If the DM team can’t converge on a resolution to an RFC that has no serious objections but
the equestorstillfeelthatsomethingmustbedone,therequestwillbeescalated. Inmost
non-trivial cases, they will, with the advice of the SA, empanel a group of experts to which
they will delegate the right to make the decision, by voting if need be.
C.4 Formalities
For project management purposes, RFCs are formally proposals made to the DM PM and PS
who by default are responsible for everything in DM (they “own” all problems). As owners,
they have the final word in accepting or rejecting all proposals. Functionally, they delegate
that ownership, the right and responsibility to make decisions – to others within the team
(e.g. the SA, IS, group leads, etc.) who are expected to delegate it even further. Notifying the
institutional technical manager about an RFC serves to inform the DM PM.
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
87

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19

Back to top


D TraceabilitymatrixofDMSR equirementstoOSSRequirements
DMS
OSS
DMS-REQ-0002 Transient Alert Distribu-
tion
OSS-REQ-0127Level1DataProductAvailability
OSS-REQ-0184Transient AlertPublication
DMS-REQ-0004 Nightly Data Accessible
Within 24 hrs
OSS-REQ-0127Level1DataProductAvailability
DMS-REQ-0006 Timely Publication of
Level2DataReleases
OSS-REQ-0134Level2DataProductAvailability
DMS-REQ-0008PipelineAvailability
DMS-REQ-0009SimulatedData
OSS-REQ-0351DifferenceSourceSpuriousProbabilityMetric
OSS-REQ-0353DifferenceSourceSpuriousnessThreshold-
Transients
OSS-REQ-0354DifferenceSourceSpuriousnessThreshold-
MOPS
DMS-REQ-0010DifferenceExposures OSS-REQ-0129Exposures(Level1)
DMS-REQ-0018RawScience ImageData
Acquisition
OSS-REQ-0114AcquisitionofScienceSensordata
DMS-REQ-0020 Wavefront Sensor Data
Acquisition
OSS-REQ-0316WavefrontSensorData
DMS-REQ-0022CrosstalkCorrectedSci-
ence ImageDataAcquisition
OSS-REQ-0114AcquisitionofScienceSensordata
OSS-REQ-0127Level1DataProductAvailability
DMS-REQ-0024Raw ImageAssembly OSS-REQ-0114AcquisitionofScienceSensordata
OSS-REQ-0129Exposures(Level1)
DMS-REQ-0029 Generate Photometric
ZeropointforVisit Image
OSS-REQ-0056SystemMonitoring&Diagnostics
OSS-REQ-0152Level1PhotometricZeroPointError
DMS-REQ-0030 Generate WCS for Visit
Images
OSS-REQ-0162Level2CatalogAccuracy
OSS-REQ-0149Level1CatalogPrecision
DMS-REQ-0032 ImageDifferencing
OSS-REQ-0121OpenSource,OpenConfiguration
OSS-REQ-0129Exposures(Level1)
DMS-REQ-0033 Provide Source Detec-
tionSoftware
OSS-REQ-0121OpenSource,OpenConfiguration
OSS-REQ-0130Catalogs(Level1)
OSS-REQ-0137Catalogs(Level2)
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
88

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
DMS
OSS
DMS-REQ-0034AssociateSourcestoOb-
jects
OSS-REQ-0339Level2Source-ObjectAssociationQuality
DMS-REQ-0042 Provide Astrometric
Model
OSS-REQ-0149Level1CatalogPrecision
OSS-REQ-0153WorldCoordinateSystemAccuracy
OSS-REQ-0160 Level 1 Difference Source - Difference Object
AssociationQuality
OSS-REQ-0162Level2CatalogAccuracy
DMS-REQ-0043ProvideCalibratedPho-
tometry
OSS-REQ-0137Catalogs(Level2)
OSS-REQ-0130Catalogs(Level1)
OSS-REQ-0275 Calibration Processing Performance Alloca-
tions
DMS-REQ-0046 Provide Photometric
RedshiftsofGalaxies
DMS-REQ-0047ProvidePSFforCoadded
Images
OSS-REQ-0153WorldCoordinateSystemAccuracy
OSS-REQ-0316WavefrontSensorData
OSS-REQ-0136Co-addedExposures
DMS-REQ-0052EnableaRangeofShape
MeasurementApproaches
OSS-REQ-0137Catalogs(Level2)
DMS-REQ-0059BadPixelMap
OSS-REQ-0129Exposures(Level1)
OSS-REQ-0271Supported ImageTypes
DMS-REQ-0060BiasResidual Image OSS-REQ-0271Supported ImageTypes
OSS-REQ-0046Calibration
DMS-REQ-0061CrosstalkCorrectionMa-
trix
OSS-REQ-0349DataReleaseProductionCrosstalkCorrection
DMS-REQ-0062 Illumination Correction
Frame
OSS-REQ-0271Supported ImageTypes
OSS-REQ-0046Calibration
DMS-REQ-0063 Monochromatic Flatfield
DataCube
OSS-REQ-0271Supported ImageTypes
OSS-REQ-0046Calibration
DMS-REQ-0065 Provide Image Access
Services
OSS-REQ-0180DataProductsQueryandDownloadAvailabil-
ity
OSS-REQ-0181 Data Products Query and Download Infras-
tructure
OSS-REQ-0176DataAccess
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
89

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
DMS
OSS
DMS-REQ-0068 Raw Science Image
Metadata
OSS-REQ-0122Provenance
OSS-REQ-0171EngineeringandFacilitiesData
DMS-REQ-0069ProcessedVisit Images OSS-REQ-0129Exposures(Level1)
DMS-REQ-0070 Generate PSF for Visit
Images
OSS-REQ-0056SystemMonitoring&Diagnostics
DMS-REQ-0072 Processed Visit Image
Content
OSS-REQ-0129Exposures(Level1)
DMS-REQ-0074DifferenceExposureAt-
tributes
OSS-REQ-0122Provenance
DMS-REQ-0075CatalogQueries
OSS-REQ-0176DataAccess
DMS-REQ-0077MaintainArchivePublicly
Accessible
OSS-REQ-0186AccesstoPreviousDataReleases
DMS-REQ-0078CatalogExportFormats OSS-REQ-0176DataAccess
DMS-REQ-0089 Solar System Objects
AvailableWithinSpecifiedTime
OSS-REQ-0127Level1DataProductAvailability
DMS-REQ-0094 Keep Historical Alert
Archive
OSS-REQ-0128Alerts
DMS-REQ-0096 Generate Data Quality
ReportWithinSpecifiedTime
OSS-REQ-0131NightlySummaryProducts
DMS-REQ-0097 Level 1 Data Quality Re-
port Definition
OSS-REQ-0131NightlySummaryProducts
DMS-REQ-0098 Generate DMS Perfor-
manceReportWithinSpecifiedTime
OSS-REQ-0131NightlySummaryProducts
DMS-REQ-0099Level1PerformanceRe-
port Definition
OSS-REQ-0131NightlySummaryProducts
DMS-REQ-0100GenerateCalibrationRe-
portWithinSpecifiedTime
OSS-REQ-0131NightlySummaryProducts
DMS-REQ-0101 Level 1 Calibration Re-
port Definition
OSS-REQ-0131NightlySummaryProducts
DMS-REQ-0102 Provide Engineering &
FacilityDatabaseArchive
OSS-REQ-0132EngineeringandFacilityDatabaseArchive
DMS-REQ-0103 Produce Images for EPO OSS-REQ-0136 Co-added Exposures
DMS-REQ-0106 Coadded Image Prove-
nance
OSS-REQ-0122Provenance
DMS-REQ-0119 DAC esource allocation
forLevel3processing
OSS-REQ-0143ResourceAllocation
DMS-REQ-0120Level3DataProductSelf
Consistency
OSS-REQ-0120Consistency
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
90

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
DMS
OSS
OSS-REQ-0118 Consistency and Completeness
DMS-REQ-0121 Provenance for Level 3
processingatDACs
OSS-REQ-0122Provenance
DMS-REQ-0122Accesstocatalogsforex-
ternalLevel3processing
OSS-REQ-0140Production
OSS-REQ-0180DataProductsQueryandDownloadAvailabil-
ity
DMS-REQ-0123Accesstoinputcatalogs
forDAC-basedLevel3processing
OSS-REQ-0140Production
DMS-REQ-0124Federationwithexternal
catalogs
OSS-REQ-0140Production
DMS-REQ-0125Softwareframeworkfor
Level3catalogprocessing
OSS-REQ-0121OpenSource,OpenConfiguration
OSS-REQ-0122Provenance
DMS-REQ-0126 Access to images for ex-
ternalLevel3processing
OSS-REQ-0140Production
OSS-REQ-0180DataProductsQueryandDownloadAvailabil-
ity
DMS-REQ-0127 Access to input images
forDAC-basedLevel3processing
OSS-REQ-0140Production
DMS-REQ-0128Softwareframeworkfor
Level3imageprocessing
OSS-REQ-0121OpenSource,OpenConfiguration
OSS-REQ-0122Provenance
DMS-REQ-0130 Calibration Data Prod-
ucts
OSS-REQ-0129Exposures(Level1)
OSS-REQ-0194CalibrationExposuresPerDay
OSS-REQ-0271Supported ImageTypes
DMS-REQ-0131Calibration ImagesAvail-
ableWithinSpecifiedTime
OSS-REQ-0021BaseSite
OSS-REQ-0046Calibration
OSS-REQ-0194CalibrationExposuresPerDay
DMS-REQ-0132Calibration ImageProve-
nance
OSS-REQ-0122Provenance
OSS-REQ-0123Reproducibility
DMS-REQ-0155ProvideDataAccessSer-
vices
OSS-REQ-0176DataAccess
DMS-REQ-0156 Provide Pipeline Execu-
tionServices
OSS-REQ-0037ObservatoryControlSystemDefinition
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
91

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
DMS
OSS
OSS-REQ-0117AutomatedProduction
DMS-REQ-0158 Provide Pipeline Con-
structionServices
DMS-REQ-0160 Provide User Interface
Services
OSS-REQ-0057 Image Visualization
DMS-REQ-0161OptimizationofCost,Re-
liabilityandAvailabilityinOrder
DMS-REQ-0162PipelineThroughput OSS-REQ-0127Level1DataProductAvailability
OSS-REQ-0020UsableObservingTime
DMS-REQ-0163Re-processingCapacity OSS-REQ-0134Level2DataProductAvailability
DMS-REQ-0164 Temporary Storage for
CommunicationsLinks
DMS-REQ-0165 InfrastructureSizingfor
"catchingup"
OSS-REQ-0050SummitPowerGridLoss
OSS-REQ-0051 Summit-Base Connectivity Loss
OSS-REQ-0052SummitDataBuffer
DMS-REQ-0166 Incorporate Fault-
Tolerance
DMS-REQ-0167 IncorporateAutonomics
DMS-REQ-0168 Summit Facility Data
Communications
OSS-REQ-0002TheSummitFacility
DMS-REQ-0170 Prefer Computing and
StorageDown
DMS-REQ-0171SummittoBaseNetwork OSS-REQ-0003 The Base Facility
OSS-REQ-0127Level1DataProductAvailability
DMS-REQ-0172SummittoBaseNetwork
Availability
OSS-REQ-0373 Unscheduled Downtime Subsystem Alloca-
tions
DMS-REQ-0173SummittoBaseNetwork
Reliability
OSS-REQ-0373 Unscheduled Downtime Subsystem Alloca-
tions
DMS-REQ-0174SummittoBaseNetwork
SecondaryLink
OSS-REQ-0049Degraded Operational States
DMS-REQ-0175SummittoBaseNetwork
OwnershipandOperation
OSS-REQ-0036LocalAutonomousAdministrationofSystem
Sites
DMS-REQ-0176BaseFacility Infrastruc-
ture
OSS-REQ-0003TheBaseFacility
DMS-REQ-0177BaseFacilityTemporary
Storage
OSS-REQ-0053 Base-Archive Connectivity Loss
OSS-REQ-0054BaseDataBuffer
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
92

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
DMS
OSS
DMS-REQ-0178BaseFacilityCo-Location
withExistingFacility
OSS-REQ-0006Sites
DMS-REQ-0180 Base to Archive Network OSS-REQ-0053 Base-Archive Connectivity Loss
OSS-REQ-0055BaseUpdatingfromArchive
DMS-REQ-0181BasetoArchiveNetwork
Availability
OSS-REQ-0053 Base-Archive Connectivity Loss
DMS-REQ-0182BasetoArchiveNetwork
Reliability
OSS-REQ-0053 Base-Archive Connectivity Loss
DMS-REQ-0183BasetoArchiveNetwork
SecondaryLink
OSS-REQ-0049Degraded Operational States
DMS-REQ-0185ArchiveCenter
OSS-REQ-0004TheArchiveFacility
DMS-REQ-0186ArchiveCenterDisaster
Recovery
OSS-REQ-0176DataAccess
DMS-REQ-0187 Archive Center Co-
LocationwithExistingFacility
OSS-REQ-0022ArchiveSite
DMS-REQ-0188 Archive to Data Access
CenterNetwork
DMS-REQ-0189 Archive to Data Access
CenterNetworkAvailability
DMS-REQ-0190 Archive to Data Access
CenterNetworkReliability
DMS-REQ-0191 Archive to Data Access
CenterNetworkSecondaryLink
DMS-REQ-0193DataAccessCenters OSS-REQ-0004TheArchiveFacility
DMS-REQ-0194 Data Access Center Si-
multaneousConnections
DMS-REQ-0196DataAccessCenterGeo-
graphicalDistribution
OSS-REQ-0021BaseSite
OSS-REQ-0022ArchiveSite
DMS-REQ-0197 No Limit on Data Access
Centers
OSS-REQ-0021BaseSite
OSS-REQ-0022ArchiveSite
DMS-REQ-0265GuiderCalibrationData
Acquisition
OSS-REQ-0194CalibrationExposuresPerDay
DMS-REQ-0266ExposureCatalog
OSS-REQ-0130Catalogs(Level1)
DMS-REQ-0267SourceCatalog
OSS-REQ-0137Catalogs(Level2)
DMS-REQ-0268Forced-SourceCatalog OSS-REQ-0137Catalogs(Level2)
DMS-REQ-0269DIASourceCatalog
OSS-REQ-0130Catalogs(Level1)
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
93

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
DMS
OSS
DMS-REQ-0270 Faint DIASource Mea-
surements
OSS-REQ-0166AlertCompletenessandPurity
DMS-REQ-0271DIAObjectCatalog
OSS-REQ-0130Catalogs(Level1)
DMS-REQ-0272DIAObjectAttributes OSS-REQ-0130Catalogs(Level1)
DMS-REQ-0273SSObjectCatalog
OSS-REQ-0130Catalogs(Level1)
DMS-REQ-0274AlertContent
OSS-REQ-0128Alerts
DMS-REQ-0275ObjectCatalog
OSS-REQ-0137Catalogs(Level2)
DMS-REQ-0276 Object Characterization OSS-REQ-0137 Catalogs (Level 2)
DMS-REQ-0277CoaddSourceCatalog OSS-REQ-0137Catalogs(Level2)
DMS-REQ-0278 Coadd Image Method
Constraints
OSS-REQ-0136Co-addedExposures
DMS-REQ-0279 Deep Detection Coadds OSS-REQ-0136 Co-added Exposures
DMS-REQ-0280TemplateCoadds
OSS-REQ-0136Co-addedExposures
DMS-REQ-0281Multi-bandCoadds
OSS-REQ-0136Co-addedExposures
DMS-REQ-0282DarkCurrentCorrection
Frame
OSS-REQ-0271Supported ImageTypes
OSS-REQ-0046Calibration
DMS-REQ-0283 Fringe Correction Frame OSS-REQ-0271 Supported Image Types
OSS-REQ-0046Calibration
DMS-REQ-0284Level-1ProductionCom-
pleteness
OSS-REQ-0052SummitDataBuffer
DMS-REQ-0285 Level 1 Source Associa-
tion
OSS-REQ-0159Level1MovingObjectQuality
OSS-REQ-0160 Level 1 Difference Source - Difference Object
AssociationQuality
OSS-REQ-0130Catalogs(Level1)
DMS-REQ-0286SSObjectPrecovery
OSS-REQ-0159Level1MovingObjectQuality
DMS-REQ-0287DIASourcePrecovery OSS-REQ-0130Catalogs(Level1)
DMS-REQ-0288 Use of External Orbit
Catalogs
OSS-REQ-0159Level1MovingObjectQuality
DMS-REQ-0289 Calibration Production
Processing
OSS-REQ-0170 Calibration Data
OSS-REQ-0004TheArchiveFacility
DMS-REQ-0290Level3Data Import
OSS-REQ-0140Production
DMS-REQ-0291QueryRepeatability OSS-REQ-0181 Data Products Query and Download Infras-
tructure
DMS-REQ-0292 Uniqueness of IDs
AcrossDataReleases
OSS-REQ-0137Catalogs(Level2)
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
94

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
DMS
OSS
OSS-REQ-0130Catalogs(Level1)
DMS-REQ-0293SelectionofDatasets OSS-REQ-0118ConsistencyandCompleteness
OSS-REQ-0176DataAccess
DMS-REQ-0294ProcessingofDatasets OSS-REQ-0117AutomatedProduction
OSS-REQ-0118 Consistency and Completeness
OSS-REQ-0119Completeness
OSS-REQ-0120Consistency
DMS-REQ-0295T ansparentDataAccess OSS-REQ-0176DataAccess
DMS-REQ-0296 Pre-cursor, and Real
Data
DMS-REQ-0297 DMS Initialization Com-
ponent
OSS-REQ-0041SubsystemActivation
OSS-REQ-0307Subsystem Initialization
OSS-REQ-0121OpenSource,OpenConfiguration
OSS-REQ-0122Provenance
DMS-REQ-0298 Data Product and Raw
DataAccess
OSS-REQ-0176DataAccess
DMS-REQ-0299DataProduct Ingest OSS-REQ-0141Storage
OSS-REQ-0004TheArchiveFacility
DMS-REQ-0300BulkDownloadService OSS-REQ-0178DataDistribution
DMS-REQ-0301 Control of Level-1 Pro-
duction
OSS-REQ-0044StandardOperatingStates
DMS-REQ-0302 Production Orchestra-
tion
OSS-REQ-0004TheArchiveFacility
OSS-REQ-0038ScopeofControl
OSS-REQ-0117AutomatedProduction
DMS-REQ-0303ProductionMonitoring OSS-REQ-0004TheArchiveFacility
OSS-REQ-0038ScopeofControl
OSS-REQ-0034System Control
DMS-REQ-0304 Production Fault Toler-
ance
OSS-REQ-0117AutomatedProduction
DMS-REQ-0305TaskSpecification
OSS-REQ-0121OpenSource,OpenConfiguration
OSS-REQ-0122Provenance
DMS-REQ-0306TaskConfiguration
OSS-REQ-0121OpenSource,OpenConfiguration
OSS-REQ-0122Provenance
DMS-REQ-0307UniqueProcessingCov-
erage
OSS-REQ-0120Consistency
OSS-REQ-0118 Consistency and Completeness
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
95

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
DMS
OSS
DMS-REQ-0308SoftwareArchitectureto
EnableCommunityRe-Use
OSS-REQ-0121OpenSource,OpenConfiguration
DMS-REQ-0309RawDataArchivingReli-
ability
OSS-REQ-0111Science ImageArchivingReliability
DMS-REQ-0310Un-ArchivedDataProd-
uctCache
OSS-REQ-0130Catalogs(Level1)
DMS-REQ-0311 Regenerate Un-archived
DataProducts
OSS-REQ-0129Exposures(Level1)
DMS-REQ-0312 Level 1 Data Product Ac-
cess
OSS-REQ-0127Level1DataProductAvailability
OSS-REQ-0185TransientAlertQuery
DMS-REQ-0313 Level 1 & 2 Catalog Ac-
cess
OSS-REQ-0186AccesstoPreviousDataReleases
DMS-REQ-0314 Compute Platform Het-
erogeneity
OSS-REQ-0177DataAccessEnvironment
OSS-REQ-0124 Software Development Standards
DMS-REQ-0315 DMS Communication
withOCS
OSS-REQ-0003TheBaseFacility
DMS-REQ-0316CommissioningCluster
DMS-REQ-0317 DIAForcedSource Cata-
log
OSS-REQ-0130Catalogs(Level1)
DMS-REQ-0318 Data Management Un-
scheduledDowntime
OSS-REQ-0373 Unscheduled Downtime Subsystem Alloca-
tions
DMS-REQ-0319CharacterizingVariability
DMS-REQ-0320ProcessingofDataFrom
SpecialPrograms
LSR-REQ-0075SurveyTimeAllocation
DMS-REQ-0321 Level 1 Processing of
SpecialProgramsData
DMS-REQ-0322 Special Programs
Database
DMS-REQ-0323CalculatingSSObjectPa-
ameters
DMS-REQ-0324MatchingDIASourcesto
Objects
DMS-REQ-0325 Regenerating L1 Data
Products During Data Release Process-
ing
OSS-REQ-0135Uniformlycalibratedandprocessedversions
of Level 1 Data Products
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
96

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
DMS
OSS
DMS-REQ-0326 Storing Approximations
ofPer-pixelMetadata
DMS-REQ-0327 Background Model Cal-
culation
OSS-REQ-0056SystemMonitoring&Diagnostics
DMS-REQ-0328 Documenting Image
Characterization
DMS-REQ-0329 All-Sky Visualization of
DataReleases
OSS-REQ-0136Co-addedExposures
DMS-REQ-0330BestSeeingCoadds OSS-REQ-0136Co-addedExposures
DMS-REQ-0331 Computing Derived
Quantities
DMS-REQ-0332 Denormalizing Database
Tables
DMS-REQ-0333 Maximum Likelihood
ValuesandCovariances
DMS-REQ-0334 Persisting Data Products OSS-REQ-0136 Co-added Exposures
DMS-REQ-0335 PSF-Matched Coadds
DMS-REQ-0336RegeneratingDataProd-
uctsfromPreviousDataReleases
LSR-REQ-0049DataProductArchiving
DMS-REQ-0337Detectingfaintvariable
objects
OSS-REQ-0136Co-addedExposures
DMS-REQ-0338TargetedCoadds
OSS-REQ-0136Co-addedExposures
LSR-REQ-0040DataQualityMonitoring
DMS-REQ-0339 Tracking Characteriza-
tionChangesBetweenDataReleases
LSR-REQ-0040DataQualityMonitoring
DMS-REQ-0340AccessControlsofLevel
3DataProducts
OSS-REQ-0142Access
OSS-REQ-0176DataAccess
OSS-REQ-0187 InformationSecurity
DMS-REQ-0341 Providing a Precovery
Service
DMS-REQ-0342AlertFilteringService LSR-REQ-0025TransientFiltering
DMS-REQ-0343 Performance Require-
mentsforLSSTAlertFilteringService
OSS-REQ-0184Transient AlertPublication
DMS-REQ-0344 Constraints on Level 1
SpecialProgramProductsGeneration
DMS-REQ-0345 Logging of catalog
queries
OSS-REQ-0134Level2DataProductAvailability
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
97

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
DMS
OSS
DMS-REQ-0346DataAvailability
OSS-REQ-0004TheArchiveFacility
OSS-REQ-0167DataArchiving
OSS-REQ-0313 Telemetry Database Retention
DMS-REQ-0347 Measurements in cata-
logs
DMS-REQ-0348 Pre-defined alert filters LSR-REQ-0026 Predefined Transient Filters
DMS-REQ-0349Detectingextendedlow
surfacebrightnessobjects
DMS-REQ-0350 Associating Objects
acrossdatareleases
DMS-REQ-0351ProvideBeamProjector
CoordinateCalculationSoftware
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
98

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19

Back to top


E TraceabilitymatrixofOSS equirementstoDMSR equirements
OSS
DMS
LSR-REQ-0025TransientFiltering
DMS-REQ-0342AlertFilteringService
LSR-REQ-0026 Predefined Transient Fil-
ters
DMS-REQ-0348Pre-definedalertfilters
LSR-REQ-0040DataQualityMonitoring DMS-REQ-0338TargetedCoadds
DMS-REQ-0339 Tracking Characterization Changes Between
DataReleases
LSR-REQ-0049DataProductArchiving DMS-REQ-0336 Regenerating Data Products from Previous
DataReleases
LSR-REQ-0075SurveyTimeAllocation DMS-REQ-0320ProcessingofDataFromSpecialPrograms
OSS-REQ-0002TheSummitFacility
DMS-REQ-0168SummitFacilityDataCommunications
OSS-REQ-0003TheBaseFacility
DMS-REQ-0315DMSCommunicationwithOCS
DMS-REQ-0171SummittoBaseNetwork
DMS-REQ-0176 Base Facility Infrastructure
OSS-REQ-0004TheArchiveFacility
DMS-REQ-0299DataProduct Ingest
DMS-REQ-0302ProductionOrchestration
DMS-REQ-0303ProductionMonitoring
DMS-REQ-0185ArchiveCenter
DMS-REQ-0193DataAccessCenters
DMS-REQ-0289CalibrationProductionProcessing
DMS-REQ-0346 Data Availability
OSS-REQ-0006Sites
DMS-REQ-0178BaseFacilityCo-LocationwithExistingFacility
OSS-REQ-0020UsableObservingTime DMS-REQ-0162PipelineThroughput
OSS-REQ-0021BaseSite
DMS-REQ-0196DataAccessCenterGeographicalDistribution
DMS-REQ-0197NoLimitonDataAccessCenters
DMS-REQ-0131Calibration ImagesAvailableWithinSpecified
Time
OSS-REQ-0022ArchiveSite
DMS-REQ-0187 Archive Center Co-Location with Existing Fa-
cility
DMS-REQ-0196DataAccessCenterGeographicalDistribution
DMS-REQ-0197NoLimitonDataAccessCenters
OSS-REQ-0034SystemControl
DMS-REQ-0303ProductionMonitoring
OSS-REQ-0036 Local Autonomous Ad-
ministrationofSystemSites
DMS-REQ-0175SummittoBaseNetworkOwnershipandOp-
eration
OSS-REQ-0037ObservatoryControlSys-
temDefinition
DMS-REQ-0156ProvidePipelineExecutionServices
OSS-REQ-0038ScopeofControl
DMS-REQ-0302ProductionOrchestration
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
99

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
OSS
DMS
DMS-REQ-0303ProductionMonitoring
OSS-REQ-0041SubsystemActivation DMS-REQ-0297DMS InitializationComponent
OSS-REQ-0044 Standard Operating
States
DMS-REQ-0301ControlofLevel-1Production
OSS-REQ-0046Calibration
DMS-REQ-0131Calibration ImagesAvailableWithinSpecified
Time
DMS-REQ-0060BiasResidual Image
DMS-REQ-0282DarkCurrentCorrectionFrame
DMS-REQ-0063MonochromaticFlatfieldDataCube
DMS-REQ-0062 Illumination Correction Frame
DMS-REQ-0283FringeCorrectionFrame
OSS-REQ-0049 Degraded Operational
States
DMS-REQ-0174SummittoBaseNetworkSecondaryLink
DMS-REQ-0183BasetoArchiveNetworkSecondaryLink
OSS-REQ-0050 Summit Power Grid Loss DMS-REQ-0165 Infrastructure Sizing for "catching up"
OSS-REQ-0051 Summit-Base Connectiv-
ityLoss
DMS-REQ-0165 InfrastructureSizingfor"catchingup"
OSS-REQ-0052SummitDataBuffer
DMS-REQ-0165 InfrastructureSizingfor"catchingup"
DMS-REQ-0284 Level-1 Production Completeness
OSS-REQ-0053 Base-Archive Connectiv-
ityLoss
DMS-REQ-0177BaseFacilityTemporaryStorage
DMS-REQ-0180BasetoArchiveNetwork
DMS-REQ-0181BasetoArchiveNetworkAvailability
DMS-REQ-0182BasetoArchiveNetworkReliability
OSS-REQ-0054BaseDataBuffer
DMS-REQ-0177BaseFacilityTemporaryStorage
OSS-REQ-0055 Base Updating from
Archive
DMS-REQ-0180BasetoArchiveNetwork
OSS-REQ-0056 System Monitoring & Di-
agnostics
DMS-REQ-0327 Background Model Calculation
DMS-REQ-0029GeneratePhotometricZeropointforVisit Im-
age
DMS-REQ-0070GeneratePSFforVisit Images
OSS-REQ-0057 ImageVisualization
DMS-REQ-0160ProvideUser InterfaceServices
OSS-REQ-0111Science ImageArchiving
Reliability
DMS-REQ-0309RawDataArchivingReliability
OSS-REQ-0114 Acquisition of Science
Sensordata
DMS-REQ-0024Raw ImageAssembly
DMS-REQ-0018RawScience ImageDataAcquisition
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
100

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
OSS
DMS
DMS-REQ-0022 Crosstalk Corrected Science Image Data Ac-
quisition
OSS-REQ-0117AutomatedProduction DMS-REQ-0156ProvidePipelineExecutionServices
DMS-REQ-0302ProductionOrchestration
DMS-REQ-0304ProductionFaultTolerance
DMS-REQ-0294ProcessingofDatasets
OSS-REQ-0118 Consistency and Com-
pleteness
DMS-REQ-0307UniqueProcessingCoverage
DMS-REQ-0293SelectionofDatasets
DMS-REQ-0294ProcessingofDatasets
DMS-REQ-0120Level3DataProductSelfConsistency
OSS-REQ-0119Completeness
DMS-REQ-0294ProcessingofDatasets
OSS-REQ-0120Consistency
DMS-REQ-0307UniqueProcessingCoverage
DMS-REQ-0294ProcessingofDatasets
DMS-REQ-0120Level3DataProductSelfConsistency
OSS-REQ-0121OpenSource,OpenCon-
figuration
DMS-REQ-0308SoftwareArchitecturetoEnableCommunity
Re-Use
DMS-REQ-0032 Image Differencing
DMS-REQ-0033ProvideSourceDetectionSoftware
DMS-REQ-0297 DMS Initialization Component
DMS-REQ-0306TaskConfiguration
DMS-REQ-0305TaskSpecification
DMS-REQ-0125 Software framework for Level 3 catalog pro-
cessing
DMS-REQ-0128 Software framework for Level 3 image pro-
cessing
OSS-REQ-0122Provenance
DMS-REQ-0297DMS InitializationComponent
DMS-REQ-0306TaskConfiguration
DMS-REQ-0305TaskSpecification
DMS-REQ-0121ProvenanceforLevel3processingatDACs
DMS-REQ-0125 Software framework for Level 3 catalog pro-
cessing
DMS-REQ-0128 Software framework for Level 3 image pro-
cessing
DMS-REQ-0068RawScience ImageMetadata
DMS-REQ-0074 Difference Exposure Attributes
DMS-REQ-0106Coadded ImageProvenance
DMS-REQ-0132 Calibration Image Provenance
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
101

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
OSS
DMS
OSS-REQ-0123Reproducibility
DMS-REQ-0132Calibration ImageProvenance
OSS-REQ-0124 Software Development
Standards
DMS-REQ-0314 Compute Platform Heterogeneity
OSS-REQ-0127 Level 1 Data Product
Availability
DMS-REQ-0312Level1DataProductAccess
DMS-REQ-0162PipelineThroughput
DMS-REQ-0171SummittoBaseNetwork
DMS-REQ-0002 Transient Alert Distribution
DMS-REQ-0089SolarSystemObjectsAvailableWithinSpeci-
fiedTime
DMS-REQ-0022 Crosstalk Corrected Science Image Data Ac-
quisition
DMS-REQ-0004NightlyDataAccessibleWithin24hrs
OSS-REQ-0128Alerts
DMS-REQ-0094KeepHistoricalAlertArchive
DMS-REQ-0274AlertContent
OSS-REQ-0129Exposures(Level1)
DMS-REQ-0032 ImageDifferencing
DMS-REQ-0311RegenerateUn-archivedDataProducts
DMS-REQ-0024Raw ImageAssembly
DMS-REQ-0069ProcessedVisit Images
DMS-REQ-0072ProcessedVisit ImageContent
DMS-REQ-0010 Difference Exposures
DMS-REQ-0130CalibrationDataProducts
DMS-REQ-0059BadPixelMap
OSS-REQ-0130Catalogs(Level1)
DMS-REQ-0033ProvideSourceDetectionSoftware
DMS-REQ-0043 Provide Calibrated Photometry
DMS-REQ-0310Un-ArchivedDataProductCache
DMS-REQ-0292Uniquenessof IDsAcrossDataReleases
DMS-REQ-0285Level1SourceAssociation
DMS-REQ-0287DIASourcePrecovery
DMS-REQ-0266 Exposure Catalog
DMS-REQ-0269 DIASource Catalog
DMS-REQ-0271 DIAObject Catalog
DMS-REQ-0272DIAObjectAttributes
DMS-REQ-0273 SSObject Catalog
DMS-REQ-0317DIAForcedSourceCatalog
OSS-REQ-0131 Nightly Summary Prod-
ucts
DMS-REQ-0096 Generate Data Quality Report Within Speci-
fiedTime
DMS-REQ-0098 Generate DMS Performance Report Within
SpecifiedTime
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
102

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
OSS
DMS
DMS-REQ-0100GenerateCalibrationReportWithinSpecified
Time
DMS-REQ-0097Level1DataQualityReportDefinition
DMS-REQ-0099Level1PerformanceReportDefinition
DMS-REQ-0101Level1CalibrationReportDefinition
OSS-REQ-0132 Engineering and Facility
DatabaseArchive
DMS-REQ-0102 Provide Engineering & Facility Database
Archive
OSS-REQ-0134 Level 2 Data Product
Availability
DMS-REQ-0345Loggingofcatalogqueries
DMS-REQ-0163Re-processingCapacity
DMS-REQ-0006TimelyPublicationofLevel2DataReleases
OSS-REQ-0135Uniformlycalibratedand
processedversionsofLevel1DataProd-
ucts
DMS-REQ-0325 Regenerating L1 Data Products During Data
ReleaseProcessing
OSS-REQ-0136Co-addedExposures DMS-REQ-0334PersistingDataProducts
DMS-REQ-0279DeepDetectionCoadds
DMS-REQ-0280TemplateCoadds
DMS-REQ-0281 Multi-band Coadds
DMS-REQ-0330BestSeeingCoadds
DMS-REQ-0337Detectingfaintvariableobjects
DMS-REQ-0338TargetedCoadds
DMS-REQ-0278Coadd ImageMethodConstraints
DMS-REQ-0047ProvidePSFforCoadded Images
DMS-REQ-0103Produce ImagesforEPO
DMS-REQ-0329All-SkyVisualizationofDataReleases
OSS-REQ-0137Catalogs(Level2)
DMS-REQ-0033ProvideSourceDetectionSoftware
DMS-REQ-0043 Provide Calibrated Photometry
DMS-REQ-0052 Enable a Range of Shape Measurement Ap-
proaches
DMS-REQ-0292Uniquenessof IDsAcrossDataReleases
DMS-REQ-0267SourceCatalog
DMS-REQ-0275ObjectCatalog
DMS-REQ-0276ObjectCharacterization
DMS-REQ-0277CoaddSourceCatalog
DMS-REQ-0268Forced-SourceCatalog
OSS-REQ-0140Production
DMS-REQ-0122 Access to catalogs for external Level 3 pro-
cessing
DMS-REQ-0126AccesstoimagesforexternalLevel3process-
ing
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
103

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
OSS
DMS
DMS-REQ-0123AccesstoinputcatalogsforDAC-basedLevel
3processing
DMS-REQ-0127AccesstoinputimagesforDAC-basedLevel3
processing
DMS-REQ-0124Federationwithexternalcatalogs
DMS-REQ-0290Level3Data Import
OSS-REQ-0141Storage
DMS-REQ-0299DataProduct Ingest
OSS-REQ-0142Access
DMS-REQ-0340AccessControlsofLevel3DataProducts
OSS-REQ-0143ResourceAllocation
DMS-REQ-0119 DAC esource allocation for Level 3 process-
ing
OSS-REQ-0149 Level 1 Catalog Precision DMS-REQ-0042 Provide Astrometric Model
DMS-REQ-0030GenerateWCSforVisit Images
OSS-REQ-0152Level1PhotometricZero
PointError
DMS-REQ-0029GeneratePhotometricZeropointforVisit Im-
age
OSS-REQ-0153WorldCoordinateSystem
Accuracy
DMS-REQ-0042ProvideAstrometricModel
DMS-REQ-0047ProvidePSFforCoadded Images
OSS-REQ-0159 Level 1 Moving Object
Quality
DMS-REQ-0285Level1SourceAssociation
DMS-REQ-0286SSObjectPrecovery
DMS-REQ-0288UseofExternalOrbitCatalogs
OSS-REQ-0160Level1DifferenceSource
-DifferenceObjectAssociationQuality
DMS-REQ-0042ProvideAstrometricModel
DMS-REQ-0285Level1SourceAssociation
OSS-REQ-0162Level2CatalogAccuracy DMS-REQ-0042ProvideAstrometricModel
DMS-REQ-0030GenerateWCSforVisit Images
OSS-REQ-0166 Alert Completeness and
Purity
DMS-REQ-0270 Faint DIASource Measurements
OSS-REQ-0167DataArchiving
DMS-REQ-0346DataAvailability
OSS-REQ-0170CalibrationData
DMS-REQ-0289CalibrationProductionProcessing
OSS-REQ-0171EngineeringandFacilities
Data
DMS-REQ-0068RawScience ImageMetadata
OSS-REQ-0176DataAccess
DMS-REQ-0155ProvideDataAccessServices
DMS-REQ-0298DataProductandRawDataAccess
DMS-REQ-0065Provide ImageAccessServices
DMS-REQ-0075 Catalog Queries
DMS-REQ-0078CatalogExportFormats
DMS-REQ-0186ArchiveCenterDisasterRecovery
DMS-REQ-0293SelectionofDatasets
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
104

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
OSS
DMS
DMS-REQ-0295TansparentDataAccess
DMS-REQ-0340AccessControlsofLevel3DataProducts
OSS-REQ-0177 Data Access Environment DMS-REQ-0314 Compute Platform Heterogeneity
OSS-REQ-0178DataDistribution
DMS-REQ-0300BulkDownloadService
OSS-REQ-0180DataProductsQueryand
DownloadAvailability
DMS-REQ-0065Provide ImageAccessServices
DMS-REQ-0122 Access to catalogs for external Level 3 pro-
cessing
DMS-REQ-0126AccesstoimagesforexternalLevel3process-
ing
OSS-REQ-0181DataProductsQueryand
Download Infrastructure
DMS-REQ-0065Provide ImageAccessServices
DMS-REQ-0291QueryRepeatability
OSS-REQ-0184 Transient Alert Publica-
tion
DMS-REQ-0002 Transient Alert Distribution
DMS-REQ-0343PerformanceRequirementsforLSSTAlertFil-
teringService
OSS-REQ-0185TransientAlertQuery DMS-REQ-0312Level1DataProductAccess
OSS-REQ-0186 Access to Previous Data
Releases
DMS-REQ-0313 Level 1 & 2 Catalog Access
DMS-REQ-0077MaintainArchivePubliclyAccessible
OSS-REQ-0187 InformationSecurity DMS-REQ-0340AccessControlsofLevel3DataProducts
OSS-REQ-0194 Calibration Exposures
Per Day
DMS-REQ-0131Calibration ImagesAvailableWithinSpecified
Time
DMS-REQ-0265GuiderCalibrationDataAcquisition
DMS-REQ-0130CalibrationDataProducts
OSS-REQ-0271Supported ImageTypes DMS-REQ-0130CalibrationDataProducts
DMS-REQ-0059BadPixelMap
DMS-REQ-0060BiasResidual Image
DMS-REQ-0282DarkCurrentCorrectionFrame
DMS-REQ-0063MonochromaticFlatfieldDataCube
DMS-REQ-0062 Illumination Correction Frame
DMS-REQ-0283FringeCorrectionFrame
OSS-REQ-0275 Calibration Processing
PerformanceAllocations
DMS-REQ-0043 Provide Calibrated Photometry
OSS-REQ-0307 Subsystem Initialization DMS-REQ-0297 DMS Initialization Component
OSS-REQ-0313 Telemetry Database Re-
tention
DMS-REQ-0346 Data Availability
OSS-REQ-0316WavefrontSensorData DMS-REQ-0020WavefrontSensorDataAcquisition
Continuedonnextpage
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
105

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
OSS
DMS
DMS-REQ-0047ProvidePSFforCoadded Images
OSS-REQ-0339Level2Source-ObjectAs-
sociationQuality
DMS-REQ-0034AssociateSourcestoObjects
OSS-REQ-0349DataReleaseProduction
CrosstalkCorrection
DMS-REQ-0061CrosstalkCorrectionMatrix
OSS-REQ-0351DifferenceSourceSpuri-
ousProbabilityMetric
DMS-REQ-0009SimulatedData
OSS-REQ-0353DifferenceSourceSpuri-
ousnessThreshold-Transients
DMS-REQ-0009SimulatedData
OSS-REQ-0354DifferenceSourceSpuri-
ousnessThreshold-MOPS
DMS-REQ-0009SimulatedData
OSS-REQ-0373 Unscheduled Downtime
SubsystemAllocations
DMS-REQ-0318DataManagementUnscheduledDowntime
DMS-REQ-0172SummittoBaseNetworkAvailability
DMS-REQ-0173SummittoBaseNetworkReliability
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
106

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19

Back to top


F References
[1]
[LSE-159]
, Angeli, G., 2013,
Reviews Definitions, Guidelines, and Procedures
, LSE-159, URL
https://ls.st/LSE-159
[2]
[Document-11920]
, Angeli, G., McKercher, R., 2013,
Document Cover Page and Style
Guide
, Document-11920, URL
https://ls.st/Document-11920
[3]
[Document-9224]
, Angeli, G., McKercher, R., 2013,
Change Controlled Document Cover
Page and Style Guide
, Document-9224, URL
https://ls.st/Document-9224
[4]
[LDM-141]
, Becla, J., Lim, K.T., 2013,
Data Management Storage Sizing and I/O Model
, LDM-
141, URL
https://ls.st/LDM-141
[5]
[DMTN-020]
, Becla, J., Economou, F., Gelman, M., et al., 2016,
Project Management Guide
,
DMTN-020, URL
https://dmtn-020.lsst.io/
[6]
[LPM-191]
, Calabrese, D., 2017,
Travel Policy
, LPM-191, URL
https://ls.st/LPM-191
[7]
[Document-13760]
, Carlson, E., 2017,
Travel Request Instructions for AURA Employees
,
Document-13760, URL
https://ls.st/Document-13760
[8]
[Document-13762]
,Carlson,E.,2017,
LSSTTravelSummaryReportTemplate
, Document-
13762, URL
https://ls.st/Document-13762
[9]
[LSE-17]
, Claver, C., Angeli, G., Selvy, B., 2016,
Systems Engineering Management Plan
,
LSE-17, URL
https://ls.st/LSE-17
[10]
[LSE-29]
,Claver,C.F.,TheLSSTSystemsEngineering IntegratedProjectTeam,2016,
LSST
System Requirements
, LSE-29, URL
https://ls.st/LSE-29
[11]
[LSE-30]
,Claver,C.F.,TheLSSTSystemsEngineering IntegratedProjectTeam,2016,
LSST
System Requirements
, LSE-30, URL
https://ls.st/LSE-30
[12]
[LSE-75]
, Dubious-Felsmann, G., 2011,
Control System Interfaces between the Telescope
and Data Management
, LSE-75, URL
https://ls.st/LSE-75
[13]
[LSE-76]
, Dubious-Felsmann, G., 2011,
Infrastructure Interfaces between Summit Facility
and Data Management
, LSE-76, URL
https://ls.st/LSE-76
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
107

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
[14]
[LSE-77]
, Dubious-Felsmann, G., 2013,
Infrastructure Interfaces between Base Facility and
Data Management
, LSE-77, URL
https://ls.st/LSE-77
[15]
[LSE-72]
, Dubious-Felsmann, G., Schumacher, G., Selvy, B., 2014,
OCS Command Dictio-
nary for Data Management
, LSE-72, URL
https://ls.st/LSE-72
[16]
[LSE-69]
, Dubois-Felsmann, G., 2014,
Interface between the Camera and Data Manage-
ment
, LSE-69, URL
https://ls.st/LSE-69
[17]
[LSE-130]
, Dubois-Felsmann, G., 2015,
Support-Data Exchanges between Data Manage-
ment and Camera
, LSE-130, URL
https://ls.st/LSE-130
[18]
[LSE-68]
, Dubois-Felsmann, G., 2015,
Camera Data Acquisition Interface
, LSE-68, URL
https://ls.st/LSE-68
[19]
[LSE-140]
, Dubois-Felsmann, G., 2016,
Auxiliary Instrumentation Interface between Data
Management and Telescope
, LSE-140, URL
https://ls.st/LSE-140
[20]
[LSE-61]
, Dubois-Felsmann, G., 2016,
LSST Data Management Subsystem Requirements
,
LSE-61, URL
https://ls.st/LSE-61
[21]
[LDM-144]
, Freemon, M., Pietrowicz, S., Alt, J., 2016,
Site Specific Infrastructure Estimation
Model
, LDM-144, URL
https://ls.st/LDM-144
[22]
[LSE-131]
, Jacoby, S., Emmons, B., Selvy, B., 2017,
Interface between Data Management
and Education and Public Outreach
, LSE-131, URL
https://ls.st/LSE-131
[23]
[LSE-163]
, Juric, M., etal., 2017,
LSSTDataProductsDefinitionDocument
, LSE-163, URL
https://ls.st/LSE-163
[24]
[LDM-148]
, Kantor, J., Axelrod, T., 2013,
Data Management System Design
, LDM-148, URL
https://ls.st/LDM-148
[25]
[LDM-138]
,Kantor,J.,Axelrod,T.,Lim,K.T.,2013,
DataManagementComputeSizingModel
,
LDM-138, URL
https://ls.st/LDM-138
[26]
[LPM-20]
,Krabbendam,V.,Selvy,B.,2015,
Risk&OpportunityManagementPlan
,LPM-20,
URL
https://ls.st/LPM-20
[27]
Larman,C.,Basili,V.R.,2003,Computer,36,47,doi:10.1109/MC.2003.1204375
[28]
[LPM-98]
, Long, K.E., 2016,
LSST Project Controls System Description
, LPM-98, URL
https:
//ls.st/LPM-98
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
108

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
[29]
[LPM-51]
, McKercher, R., 2013,
Document Management Plan
, LPM-51, URL
https://ls.
st/LPM-51
[30]
[LPM-43]
, McKercher, R., 2016,
WBS Structure
, LPM-43, URL
https://ls.st/LPM-43
[31]
[LPM-44]
, McKercher, R., 2016,
WBS Dictionary
, LPM-44, URL
https://ls.st/LPM-44
[32]
[LDM-503]
, O’Mullane, W., Jurić, M., Economou, F., 2017,
Data Management Test, Verifica-
tion and Validation Plan
, LDM-503, URL
https://ls.st/LDM-503
[33]
[LDM-493]
, Sick, J., 2016,
Data Management Documentation Architecture
, LDM-493, URL
https://ls.st/LDM-493
[34]
[LPM-55]
, Sweeney, D., McKercher, R., 2013,
Project Quality Assurance Plan
, LPM-55, URL
https://ls.st/LPM-55

Back to top


G Acronyms
Thefollowingtablehasbeengeneratedfromtheon-lineGaiaacronymlist:
Acronym Description
AP
AlertsProduction
API
ApplicationProgramming Interface
AURA AssociationofUniversitiesforResearchinAstronomy
CCB
ChangeControlBoard
CMDB ConfigurationManagementDataBase
DAC
DataAccessCenter
DAX
Dataaccessservices
DDMPM DataManagementDeputyProjectManager
DM
DataManagement
DMCCB DMChangeControlBoard
DMIS DM InterfaceScientist
DMLT DMLeadershipTeam
DMPM DataManagementProjectManager
DMSR DMSystemRequirements
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
109

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
DMSS DMSubsystemScientist
DMTN DMTechnicalNote
DRP DataReleaseProduction
EFD
EngineeringFacilitiesDatabase
ICBS InternationalCommunicationsandBaseSite
ICD
InterfaceControlDocument
IPAC Nolongeranacronym
IS
InterfaceScientist
IT
IntegrationTest
ITC
InformationTechnologyCenter
IVOA InternationalVirtual-ObservatoryAlliance
JIRA issue tracking product (not an acronym, but a truncation of Gojira, the
JapanesenameforGodzilla)
LCR
LSSTChangeRequest
LDF
LSSTDataFacility
LDM LightDataManagement
LPM LSSTProjectManagement(DocumentHandle)
LSE
LSSTSystemsEngineering(DocumentHandle)
LSST Large-apertureSynopticSurveyTelescope
LaTeX (Leslie)LamportTeX(documentmarkuplanguageanddocumentprepara-
tionsystem)
MOPS MovingObjectPipelines
NASA NationalAeronauticsandSpaceAdministration(USA)
NCSA NationalCenterforSupercomputingApplications
NET
NotEarlierThan
NSF
NationalScienceFoundation
OCS
ObservatoryControlSystem
OSS
ObservatorySystemSpecifications
PDF
PortableDocumentFormat
PM
ProjectManager
PMCS ProjectManagementControlSystem
PS
ProjectScientist
PST
ProjectScienceTeam
QA
QualityAssurance
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
110

LARGESYNOPTICSURVEYTELESCOPE
DMPMP
LDM-294
LatestRevision2017-07-19
RFC
RequestforComments
RM
ReconfigurationModule
SA
ScienceAlert(s)
SAT
ScienceArchivesTeam(atESAC)
SEMP SystemsEngineeringManagementPlan
SLA
ServiceLevelAgreement
SLAC StanfordLinearAcceleratorCenter
SS
SubsystemScientist
SST
SpaceSurveillanceTelescope
SUI
ScienceUser Interface
SUIT ScienceUser InterfaceandTools
TCT
TechnicalControlTeam(Obsolete-nowDMCCB)
US
UnitedStates
WBS WorkBreakdownStructure
WCS WorldCoordinateSystem
WISE Wide-fieldSurveyExplorer
ThecontentsofthisdocumentaresubjecttoconfigurationcontrolbytheLSSTDMTechnicalControlTeam.
111

Back to top