1. Project Status
  2. Current Photos
  3. Risk Management
  4. Detailed Project Progress and Status

Data Management Summary Monthly Report

January 2015
Project Status

The Winter 2015 release was completed with accomplishments in all areas of applications and middleware. However, slower than planned staff ramp up during this cycle prevented all functions from being delivered and some were slipped into Summer 2015 (see Risk Management section below). Planning for Summer 2015 was completed with all teams identifying the next round of features and tests against the LDM-240 Key Progress Metrics.

The LSST Chilean Networks Acquisition Review was conducted and passed on February 24. The review confirmed the overall Chilean networks acquisition plan as a series of 5 contracts with a total value of $9.6M. The panel recommended we defer implementing path diversity on Mountain - Base link as a possible upscope.


The IPAC, SLAC, Princeton, NCSA, UW, and REUNA MREFC contracts are signed. The FIU and UCD sub award contracts have completed final round of comments with AURA contracts and institution counterparts and are awaiting signature. The total value of these contracts is $102M, with the FY15 authorization a total of  $9.8M.
The terms of the amendment to the NCSA agreement for the procurement of hardware were agreed to and the amendment is being drafted. The Memorandum of Agreement with IN2P3 for data processing operations was completed and is ready for anticipated signature on March 5 in an executive meeting at IN2P3.
Recruiting and hiring activities continued across all DM institutions: four positions were opened, twenty-two interviews were conducted, seven offers were made, two positions were filled. Fifteen positions have been filled to date since the MREFC award, while nine positions are currently open. Intensive training activities continue to integrate the new personnel and to regain full productivity.


Back to top

Current Photos
The DM SQuaRE team recently migrated the LSST open source software stack to the most widely used open source repository, github. Here is an image of the LSST github repository page:

DM's Science User Interface (SUI) team at Caltech IPAC is prototyping the UI for LSST using their existing Firefly package. A major requirement is to be able to access the UI from within the python programming environment, to allow researchers to use the LSST stack in an interactive mode. This image shows Firefly rendering images from within the ipython environment.


Back to top

Risk Management
There were no new DM risks reported this month and no significant change to estimated Exposure. Having now completed the first cycle of development in Construction, while ramping up staff, we have triggered the risk DM-043 DM organization is understaffed compared to plan.
We are currently behind in hiring by a significant factor and this has led to the slipping of some Winter 2015 deliverables into Summer 2015. We had enough float to accommodate some of the slip in the Summer 15 schedule, but there will be some ripple effect beyond Summer 15 in the next several cycles. This is being addressed by having each Technical/Control Account Manager (T/CAM) at each institution:

·  update the FY15 column of LDM-240 to accurately reflect the W15/S15 scope;
·  perform a comprehensive re-evaluation of the scope of all pre-FY18 releases in LDM-240 (Data Management Roadmap);
·  replan their staffing profile using the underspend accumulated to date to add an additional FTE (either a 3-year postdoc or a full-term scientist/engineer) starting in FY16;
·  perform a comprehensive review of the risks in their respective WBS areas.

The results of these analyses will be captured in change requests to the DM PMCS baseline.
In addition, the AURA/LSST underspend was used to open and fill a new Deputy System Architect position. This will enable more rapid turn-around of critical technical analyses and design/code reviews.
Had there been no slippages, there still would have been a re-evaluation of LDM-240 since the prior versions were done before the T/CAMs were on board.
The delay in the signing of the contracts has resulted in virtually no invoicing by the DM institutions since the start of construction. The DM Project Manager and the LSST Project Controls Specialist have been estimating actuals in this time frame, so there may be some difference in the amount of estimated underspend and actuals, but this is expected to be a relatively small % of the estimate.

Back to top

Detailed Project Progress and Status

DM Project Management and Control

Current accomplishments:

The DM Project Manager:

·   Continued to support NSF Cooperative Agreement risk/contingency negotiations. Submitted risk correlations for DM plan in PMCS and 9 upscopes for LPM-72 update.
·   Interviewed and hired Alex Withers of NCSA as the Information Security Officer for LSST, implementing the key position in the IT Security Program (aka Cyber-security plan). Alex will work 25% on LSST.
·   Continued recruiting and hiring activities: filled the DM Deputy System Architect (internal posting) and DM SQuaRE Software/Documentation Engineer positions.
·   Continued work with AURA on MREFC contracts for all DM lead institutions. The IPAC, SLAC, Princeton, NCSA, UW, and REUNA contracts are signed. The FIU sub award contract has completed final round of comments with AURA contracts and institution counterparts and is awaiting signature.
·   The Memorandum of Agreement with IN2P3 for data processing operations was completed and is ready for anticipated signature on March 5 in an executive meeting at IN2P3.
·   Continued tracking progress of the MOA for Brazilian contributions to operations (Networks). The MOA has been briefed to the responsible agencies, the Ministry of Science and Technology (MCTI) and the Sao Paolo Research Foundation (FAPESP). A letter of support enabling the MOA and allowing for signature by the Brazilian RENs RNP and ANSP is in process.
·   Completed the Summer 2015 detailed plan.
·   Attended and led several sessions at the DM All Hands Meeting at SLAC.
·   Planned the next Data Management All Hands Meeting at NCSA May 18 - 20.
·   Participated in the feedback session at the AURA Board meeting.
·   Briefed DM status at the LSST Corporation Board meeting.

Planned activities:

The DM Project Manager will:

·   Continue to support NSF Cooperative Agreement Risk/Contingency negotiations.
·   Continue recruiting and hiring: open DM SQuaRE technical position.
·   Complete work with AURA on MREFC contracts for FIU contract, and amendment to NCSA contract covering LSST equipment procurements.
·   Continue supporting process to move draft Memoranda of Agreement with Brazil to signature.
·   Conduct Technical/Cost Account Manager (T/CAM) training in Tucson on March 25 - 26, covering PMCS, Risk Management, Travel and Administrative Systems, and JIRA Agile/PMCS/LDM-240 integration.
·   Participate as panel member in the Square Kilometer Array Science Data Processing Preliminary Design Review at Jodrell Bank March 18 - 19.
·   Conduct T/CAM training in Tucson March 25 - 26.

DM Science

Current accomplishments:
Staffing up continued to dominate February activities, in addition to travel to support DM and project-wide activities such as the Project Science Team meeting, the DM All-Hands Meeting and the Dark Energy Science Collaboration meeting.
The position for DM SQuaRE Science lead remains unfilled. The candidate to whom the offer has been made has turned it down (due to a competing, tenure-track, counter-offer from their current institution).  Another prospective candidate has withdrawn from consideration (again, after a pre-emptive counter-offer by their current institution). The search committee is discussing whether to interview more candidates, or develop alternative strategies to ensure adequate scientific leadership.
An offer has been made for the DM Analyst scientist postdoc position (renamed to LSST Data Systems Fellow) to Colin Slater. Colin accepted, and is expected to begin in September 2015. Unfortunately, the attempt to hire two people into that position failed because sufficient non-project funds were not available to cover both hires.
The DM Project Scientist took part in the PST meeting at SLAC, as well as presented a talk on LSST DM at the DESC meeting, and also participated at the first DM All Hands meeting at SLAC, contributing to a variety of discussions.
Planned activities:
In March, the DM Project Scientist will focus on completion of hiring activities, as well as increasingly more activity in updating the key science/technical baseline documents.
DM System Engineering

Current accomplishments:

Activities completed by the DM System Architect include:

·   Helped conduct DM All Hands Meeting.
·  Started refining Base Center design.
·  Discussed status of Sizing Model and noted action items.
·  Refreshed team on development practices and tools.
·  Assisted with deep dive into Data Release algorithms and dataflows.
·   Helped with design and implementation of REST API for data access services.
·   Commented on and resolved many RFCs.
·  Coding standards updates.
·  Package interface changes.
·   Gave input to SUI design.
·   Gave input to Base Center design.
·   Helped with middleware planning for Summer2015 and beyond.
·   Provided DM-based use cases for Technical Operations Working Group.
·   Interviewed and selected candidate for Deputy System Architect position.
·   Attended OCS Middleware Workshop to define details of OCS communication.
·   Worked to define details of Camera DAQ protocols (LSE-68/69).
·   Delivered talk on LSST data movement at CrossConnects Cosmology workshop.
·   Participated in Chilean Networks Acquisition review.
·   Provided information about speeds&feeds for NCSA Infrastructure hiring.
·   Fixed some problems in eups.
·   Continued work on New Butler.
·   Assisted with preparations for XLDB-2015.

Planned activities:

The DM System Architect will:

·   Update LDM-240 roadmap.
·   Interview candidates for SLAC position.
·   Continue work on Camera DAQ protocols (LSE-68/69).
·   Continue to give input to Base Center design.
·   Educate new Security Officer, new Deputy System Architect, SLAC group.
·   Define DM-related personnel for Technical Operations Working Group.

DM Science Quality and Reliability Engineering (SQuaRE)

Current Accomplishments:

02C.01.02 Management / LoE

·  Recruitment for Documentation Engineer position - offer made and accepted

·  DM All Hands meeting

·  S15 EVM Planning

·  JH attended the SCALE conference

02C.01.02.02 SDQA Toolkit aka PipeQA

·  A kickoff meeting exploring closer collaboration between this WBS and the SUI effort at IPAC was had

02C.01.02.03 L3 Toolkit aka Productized Stack

·  Documentation was updated for the Stack v10_0 release

02C.07.03.01 Software Development Tools aka Developer support

·  The migration of the DM primary software development to Github was carried out. Only some data-heavy repos remain on the Gitolite server

·  The JIRA Team component was re-organized to match the DM T/CAM structure

Planned Activities:

·  Release engineering activities / v11 (aka W15) release

DM Applications, Middleware, and Infrastructure

Current accomplishments:

·  Completed Winter 2015 applications and middleware and closed out stories and epics in JIRA, ready for start of Summer 2015.

·  Passed LSST Chilean Networks Acquisition Review on February 24. Review confirmed overall Chilean networks acquisition plan as a series of 5 contracts. Recommended defer implementing path diversity on Mountain - Base link as possible upscope.

Planned activities:

·  Execute Summer 2015 plan for March.
·  Complete amendment to NCSA contract for first acquisitions of Development and Integration clusters.
·  Complete Base Data Center requirements and prepare statement of work for design consultant to support AE contract.

Back to top