1. Data Management Extended Monthly Report
  2. March 2015
    1. LSST Program Office
    2. University of Washington
    3. Princeton University and University of California, Davis
    4. IPAC / California Institute of Technology
    5. SLAC / Stanford University
      1. 02C.06.02.05 Catalog Services
      2. · Defined JSON results for Web Services (SLAC and IPAC teams agreed on the format for results and errors, it is documented)
      3. · Added support for DDL in MetaServ (design and implementation). The Webserv now supports structures that define schema and plus additional custom fields, e.g. ucds, units. Some of that work is ahead of schedule.
      4. Qserv 2015_03 Release notes
    6. NCSA / University of Illinois
    7. Current accomplishments:
    8. NOAO
    9. Current accomplishments:

Data Management Extended Monthly Report

Back to top

March 2015

LSST Program Office

DM Project Management and Control

Current accomplishments:

The DM Project Manager:

·   Continued recruiting and hiring activities. We made an offer for the DM SQuaRE Scientist position.
·   Continued work with AURA on MREFC contracts for all DM lead institutions. The IPAC, SLAC, Princeton, NCSA, UW, UCD, and REUNA MREFC contracts are signed. The FIU sub award contract has completed final round of comments with AURA contracts and institution counterparts and is awaiting signature. The total value of these contracts is $102M, with the FY15 authorization a total of $9.8M.
·   The Memorandum of Agreement with IN2P3 for data processing operations was signed on March 5 in an executive meeting at IN2P3.
·   Continued tracking progress of the MOA for Brazilian contributions to operations (Networks). The MOA has been briefed to the LSSTC committee on international partnerships and is in revision based on their comments.
·   Continued planning and coordination for the next Data Management Leadership Team Meeting at NCSA May 18 - 21.
·   Prepared briefing on DM status, planning process, and upcoming plans for the AMCL April meeting. Total of 100 slides prepared covering all aspects of DM.
·   Conducted Technical/Cost Account Manager (T/CAM) training in Tucson on March 25 - 26, covering PMCS, Risk Management, Travel and Administrative Systems, and JIRA Agile/PMCS/LDM-240 integration.
·   Participated as panel member in the Square Kilometer Array Science Data Processing Preliminary Design Review at Jodrell Bank March 18 - 19.

Planned activities:

The DM Project Manager will:

·   Continue recruiting and hiring.
·   Complete work with AURA on MREFC contracts for FIU contract, and the amendment to the NCSA contract covering LSST equipment procurements.
·   Continue supporting process to move draft Memorandum of Agreement with Brazil to signature.
·   Complete planning and coordination for the next Data Management All Hands Meeting at NCSA May 18 - 21.
·   Conduct T/CAM training in Tucson April 8 - 9.
·   Attend the LSST First Stone, AMCL, and LSST Corporation Board meetings in Chile April 13 - 17.
·   Represent LSST at the South American Astronomy Coordination Committee (SAACC) meeting in Chile April 20.

DM Science

Current accomplishments:
Staffing continued to be the focus of March activities, with interviews for Level 1 DM lead search at UW, and visits to DM group in Tucson. Another promising candidate was interviewed for Level 1 DM science lead position at University of Washington. The position has informally been offered, and we're waiting to hear back whether the candidate is interested. Another offer has also been made for the SQuaRE science lead position in Tucson; we're awaiting for the response from that candidate as well.
The DM Project Scientists has begun work on the update of the Galactic star counts model for the LSST sizing model, with the help of a graduate student.
 The DM Project Scientist also had a productive working two-day meeting with Eric Tollerud (AstroPy coordinator) on understanding the architecture and opportunities for collaborations with the AstroPy project. The key results have been discussed with the DM System Architect. They resulted in an increase of probability of occurrence of DM-065 risk ("LSST DM software architecture incompatible with de-facto community standards") from 25-50% to 50-75% due to divergence between DM's current C++-centric model and the pure Python model of AstroPy.
Finally, the DM Project Scientist contributed patches to EUPS and the lsstsw build and continuous integration system, resulting in speed-ups of builds of approximately ~2.3x, and also contributed materials for the presentation on the state of DM for the April AMCL meeting.
Planned activities:
In April, the DM Project Scientist expects a continued focus on completion of hiring activities, as well as increasingly more activity in updating the key science/technical baseline documents.
DM System Engineering

Current accomplishments:

Activities completed by the DM System Architect include:

·  Updated LDM-240 roadmap.
·  Interviewed candidates for SLAC position.
·  Continued work on Camera DAQ protocols (LSE-68/69).
·  Continued to give input to Base Center design.
·  Educated new Security Officer, new Deputy System Architect, SLAC group.
·  Defined DM-related personnel for Technical Operations Working Group.

Planned activities:

The DM System Architect will:

·  Complete LDM-240 updates and review document
·  Take vacation

The DM System Interfaces Scientist will:

·   Produce drafts for CCB review of revisions to LSE-68, LSE-69, and LSE-75.
·   Produce description of summit-base-link-severed use cases for the Technical Operations Working Group
·   Analyze the DM-related dependencies in the PMCS during the Commissioning period and ensure that they are up to date, and synchronized with corresponding Camera integration activities.

DM Science Quality and Reliability Engineering (SQuaRE)

Current Accomplishments:

The W15 stack release (v10_1) process started. Two release candidates were prepared in March. The release was blocked on a number of bugs that spilled into April. A full release report will be provided next month.


A new system was released for beta-testing that allows for the automation of clean-slate builds on a variety of operating systems provided by cloud back ends. It is based on Vagrant and Puppet and eventually will support a number of APIs including OpenStack, AWS, and Digital Ocean.

The beta testing version supports Digital Ocean as the most available provider for now. This allows us with a few commands to have a brand new instance of, say, RedHat 7 to be provisioned with the stack pre-requisites, and verify the user-facing experience (via newinstall.sh from our distribution server). This will allow release testing to speed up considerably and for us to be able to verify the stack’s availability on a wider range of platforms.

Work started on a Jenkins-based CI system to replace buildbot. The new system will allow for an arbitrary number of build slaves to be be instantiated on demand from a cloud or virtualised back end. A proof of concept of running the stack build script in a Jenkins slave was completed.

Planned Activities:

·  Release engineering activities for v10_1 release

·  CI system to prototype stage

DM Applications, Middleware, and Infrastructure

Current accomplishments:

·  Executed Summer 2015 plan for March (refer to individual institution reports in DM Extended Monthly Progress Report for details).
·  Continued work on updating the DM Sizing Model unit quotes in preparation the first acquisitions of the Development and Integration clusters.
·  Completed Base Data Center requirements and prepared statement of work for design consultant to support AE bid process.
·  Completed work on making IPAC Firefly an open source package, accessible through Github. Firefly will be the foundation of the first generation Science User Interface. The SUI group has also conducted meetings to support visualization needs for SQuaRE and the Camera team.

·  DM team members participated in the LSST Technical Operations Working Group (TOWG). They developed use cases to help with specifications for the buildings in La Serena, justifying the required office space based on the necessary level of staff.

·  The base data center requirements were revised with updates to square footage, average rack weights, and power requirements. These revisions were reviewed with the Chilean site team.

Planned activities:

·  Execute the Summer 2015 plan for April (refer to individual institution reports in DM Extended Monthly Progress Report for details).
·  Complete the update to the DM Development Roadmap (LDM-240).
·   Complete the DM Sizing Model update.

University of Washington

Current accomplishments:

02C.03.00 -- Alert Production Management Engineering and Integration

Simon Krughoff (SK) attended the simulations team meeting as a DM representative to keep track of simulations progress as it pertains to DM concerns. SK hosted John Parejko, Ian Sullivan, and David Reiss on UW site visits as part of the hiring process for the DM team at UW. SK also hosted Erik Tollerud (Astropy) in order to talk about the differences and similarities between how LSST and Astropy have approached similar problems: table like data, coordinates, quantities (scalar values with units), world coordinate systems.

SK also attended the Open Science Grid (OSG) all hands meeting at Northwestern. The OSG is interested in LSST workflows and how users will use OSG in the era of LSST.

02C.03.04 -- Image Differencing Pipeline

Because of the multiple meetings and visitors SK made very little progress on cleaning up the diffim task.

02C.03.05 -- Application Framework for Exposures

Russell Owen (RO) worked on making SWIGed interfaces work better when passed numpy types. It turns out that numpy scalars are harder to deal with. It was decided to leave scalar interfaces alone and only pass native python types. The array interfaces were converted to accept numpy arrays.

02C.03.08 -- Astrometric Calibration Pipeline

RO ran into some issues integrating the astrometric solver into the new framework. The original plan was to bring over the solver from HSC since it is battle tested. However, it did not pass the current unit tests without major revisions. It was decided to use the version of a similar solver already existing in the LSST stack since it is better documented and tested.

Planned activities:

02C.03.00 -- Alert Production Management Engineering and Integration

SK will attend the T/CAM training in Tucson. SK will host one more candidate for the DM team job.

02C.03.01 -- Single Frame Processing

Yusra AlSayyad (YA) will work on making ISR tasks runnable without an an opaque input object.

02C.03.04 -- Image Differencing Pipeline

SK will continue work held over from March on bringing the difference imaging pipelines up to standard.

02C.03.08 -- Astrometric Calibration Pipeline

There is some holdover work due to the issues encountered by RO. These will need to be resolved early in the next month as a new production release is expected on short timescales.


Princeton University and University of California, Davis

This report covers work carried out in FM5 of FY15 in the Data Release Production group (staff at Princeton plus Price and Gee working remotely).

Current accomplishments:

02C.04.00 Data Release Production Management Engineering and Integration


This month was the first of the Summer 2015 development cycle. The plans developed over the preceding two months are therefore locked in to the project management system and we have commenced executing them.

Some work planned for the final days of Winter 2015 was not complete on schedule as it turned out to be more complex than anticipated. This work will be continued in Summer 2015 under the DM-1769 (“Measurement Framework Improvements”) epic. This epic has therefore increased in scope somewhat over our original planning.

Together with Krughoff (UW), Bosch & Swinbank completed a reworking of the LDM-240 Software Development Roadmap for all years through project construction. There are a number of changes since the previous version, although these are mainly for clarity and to better reflect when we expect manpower to be available to address issues, rather than any fundamental change to the plan. We expect to iterate on this in the short term to incorporate changes in other WBS elements, ensuring dependencies are mapped correctly.

After the current round of LDM-240 changes have been agreed, we plan to embark on a larger scale and longer term effort to identify each entry on LDM-240 with the requirements for specific outputs specified by LSST’s Data Products Definition and use this information to produce another revision of the plan. We do not anticipate results from this before the end of the Summer 2015 cycle at the earliest.

02C.04.01 - Application Framework for Catalogs

A design for the new Footprint API was completed. This represents a substantial overhaul of the old codebase, for reasons of performance and maintainability and to enable the new functionality which will be required for upcoming work on the deblender. This design is now undergoing review; if and when it is accepted, we expected implementation to take place over the rest of this cycle. [DM-1126]

The afw_extensions_rgb package was merged into the afw package following an RFC. This reduces code duplication and means that the functionality can be entirely implemented in Python. [RFC-32, DM-2343, DM-2363]

A bug which caused a segmentation fault when attempting to write images with a certain header layout to disk. [DM-882]

A bug which caused the creation of extremely large images to result in a segmentation fault was resolved by porting and updating code from HyperSuprimeCam (HSC). [DM-527]

Minor cleanups to the codebase included:

·  Removal of old and unused code [DM-2332];

·  Refactoring the measurement framework to avoid compiler warnings [DM-2131];

·  The way we compare celestial coordinates for equality was redesigned to make it more consistent and transparent [DM-2347].

02C.04.06 - Object Characterization Pipeline

As described above, a number of minor tasks spilled over into this cycle from previous work and were addressed this month. In particular:

·  Old and unused code was removed from the meas_algorithms package, as the modern equivalent is now available in meas_base [DM-420, DM-1073];

·  The test suite was updated to avoid obsolete code [DM-1076];

·  A number of algorithms — SDSS-like shape and centroid measurements, and Kron photometry — were refactored for clarity and maintainability [DM-1161, DM-1364, DM-1953].

An update for the deblender which resolved an issue with incorrect substitution of noise in images was ported from the HSC stack. This issue resolved a long-standing but previously unexplained bug. [DM-1738, DM-1525]

Fixes for bugs in the RGB image generating code were also copied from HSC. [DM-2436]

Planned activities:

02C.04.01 - Application Framework for Catalogs

After the new Footprint design has been reviewed, we expect implementation to start during April.

02C.04.06 - Object Characterization Pipeline

Work will start on measurements of shear precision in galaxy fitting. This work will build on the GREAT3 codebase to develop a library of simulated galaxy images and PSFs, then establish the optimal parameters for measuring them in the LSST stack. This provides insight into the ultimate computational burden which galaxy fitting will impose.

Work will continue on porting all the multi-band deblender work from HSC to LSST.

Measurement transformation and calibration functionality will be made available for the three major measurement types: fluxes, centroids and shapes.


IPAC / California Institute of Technology

Current accomplishments:

02C.05.00 Science User Interface Management Engineering and Integration

Xiuqin and David worked on the first version of LDM-240 update for SUI. We feel there are more places need to be explained and defined.

Several meetings between the SUI and the Camera team on the revision of LSE-68, reaching a number of decisions on clarifications and changes to the interface for buffered data.

Gregory has been talking to Frossie about SDQA needs of visualization.

02C.05.01 Basic Archive Access Tools

SUI team met at least twice weekly to discuss the SUI requirements and design issues. The minutes have been captured on confluence page.

We have continued our weekly meeting with SLAC on data access APIs. Other topics also came up, like VO TAP API, database schema browser (moved from SLAC to IPAC), data access through butler for SUI. Those topics will need further design discussion.

Firefly package has been migrated to GWT2.7, enabling us to use the new JS interop capabilities. Some simple input forms in the Firefly have been written using React framework.

02C.05.02 Data Analysis and Visualization Tools

Firefly is officially an open source package, accessible through Github

Trey and Gregory went to SLAC to work with Camera team Tony Johnson and others for two days. Trey gave a demo of Firefly visualization components. He also helped the team to set up a version and they started development using it almost immediately.

SUI team hosted Robert Lupton to discuss the pipeline needs of visualization. Robert gave a demo of his current usage of ds9 and Trey gave a demo on ho wot use Firefly http APIs to show and control image display.

SUI team will take Camera team needs and pipeline needs as additional driver in designing the visualization tools.

Planned activities:
Xiuqin will attend the T/CAM training.
LDM-240 road map revamp for Y16-Y20, revision.
Frossie will be visiting IPAC to discuss SDQA visualization needs and L3v3l 3 eta products.
Continue the discussion of data access APIs with SLAC group.
K-T will visit to discuss butler and DM architecture with us.
Continue the development on the JavaScript APIs and Python APIs for Firefly visualization components.
Update and add new the stretch algorithm in Firefly for image visualization.


SLAC / Stanford University

Current accomplishments:

02C.06.00 Science Data Archive and Application Services Management Engineering and Integration

·  Coordinated March Sprint for the Data Access Team

·  Attended 2-day T/CAM training session in Tucson

·  Attended 3-day Tools for Big Data in Astronomy workshop in Tucson and presented keynote about Qserv to raise awareness about Qserv in the community. NOAO is interested in using it.

·  Provided SLAC-related slides for AMCL presentation

·  Made minor updates to the long term planning for the Data Access Team

·  Improved the script for building html-based version of LDM-240, and moved it to official LSST-DM git repo managed by SQUARE

·  Interviews: Full on-site: George Kremenek. Full on-site: Ralph Seguin. Short on-site: Jeremy McCormick (slac employee)

·  Brought to the team two SLAC employees part-time (Brian @40%, Tony @10%), on-boarding Brian

·  Organized weekly Qserv and Data Access meetings

·  Continued preparing for XLDB 2015 Conference

·  SLAC-related:

·  met with DOE CTO (Peter Tseronis)

·  Working with the Computing Team on using a small number off Open Stack virtual machines for Qserv tests, we now have access to several VMs and we use them for experimenting with Data Distribution for Qserv

·  Internal discussions about X-SWAP proposal, which will attempt to simulate Qserv interactions with hardware

02C.06.01.01 Catalogs, Alerts and Metadata

·   No updates here. We planned to revisit ingest code (DM-210). This is now moved to April. It is a low-priority item

02C.06.01.02 Image and File Archive

· Synchronized metaserv schema structure with the front-end of the form for collecting user input data. The work on the form is done by Tony, the person we are adding to the group. He is now ready to do the “final” implementation.
02C.06.02.01 Data Access Client Framework
· No updates here, nothing was scheduled for March here
02C.06.02.02 Web Services
· Fine-tuned Data Access Interfaces (DM-1916) – the latest version reflects the latest thinking and decisions made at Data Access meetings
· Ported dbserv and metaserv to the db module – the web service now uses the db module (better code reuse)
· Implemented basic db pool to avoid problems with mysql timeout
· As a result, the WebServ is now ready for testing by the IPAC team, however the format of results returned is still not “final”, so after basic testing, they stopped and are waiting for the “proper” formatting of the results.
02C.06.02.03 Query Services
· Built Qserv Release 2015_03. Release notes are pasted below. Few highlights:
· have new worker management system – we can now manage worker nodes through ssh
· have much better unit test for one of the core complex components: query
02C.06.02.04 Image Services
· Implemented image stitching (DM-2013)
· Discussions with Robert Lupton about advanced features of image cutout service

02C.06.02.05 Catalog Services

·   Defined JSON results for Web Services (SLAC and IPAC teams agreed on the format for results and errors, it is documented)

·   Added support for DDL in MetaServ (design and implementation). The Webserv now supports structures that define schema and plus additional custom fields, e.g. ucds, units. Some of that work is ahead of schedule.

Qserv 2015_03 Release notes

Code improvements:

·  DM-202: Qserv: unit testing (query execution)

·  DM-2387: Build testQDisp.cc on ubuntu

·  DM-2390: Errors need to be checked in UserQueryFactory from QuerySession objects

New features:

·  DM-1900: Worker management service – design

·  DM-2176: Worker management service – impl

·  DM-998: Adapt integration test to multi-node setup v1

Bug fixes:

·  DM-1841: Fix query error on case03: "SELECT scienceCcdExposureId FROM Science_Ccd_Exposure_Metadata"

·  DM-854: duplicate column name when running near neighbor query

·  DM-2451: Fix interface between QservOss and new cmsd version

·  DM-2417: Data loader script crashes trying to create chunk table

·  DM-2294: Unable to start cmsd on Qserv worker node

Documentation improvements:

·  DM-2190: Documentation for data loader

·  DM-2309: Update dev quick-start guide to new git repositories

Planned activities:

02C.06.00 Science Data Archive and Application Services Management Engineering and Integration

·  Organize weekly Qserv and Data Access meetings

·  Rework 02C.06 4th level WBS structure

·  Participate remotely in parts of DM T/CAM training session

·  Search for candidates for remaining open position

·  Attend MySQL User Conference

·  Continue preparing for XLDB 2015 Conference

02C.06.01.01 Catalogs, Alerts and Metadata

·   Implement Data Ingest (DM-210)

02C.06.01.02 Image and File Archive

·   No plans in this area in March

02C.06.02.01 Data Access Client Framework

·   No plans in this area in March

02C.06.02.02 Web Services

·  Implement error handling

·  Implement API versioning

02C.06.02.03 Query Services

·   Preparing for running large scale tests

·   Continue designing data distribution and replication (DM-1060), start lightweight prototyping

·   Continue work on Qserv Refactoring (DM-1707) – unit testing for controller module, query killing, supporting unlimited results

·   Continue work on DM & Table Mgmt (DM-1703) – worker mgmt service: data loading, authentication, structures for deleting tables

·   Add support for transmitting VAR/Binary data

·   Finish work on Multi-node Multi-query integration testing harness

·   Improve Qserv documentation and transfer knowledge from Daniel (who is leaving the team 5/11) to other members of the team

02C.06.02.04 Image Services

·  Implement image stitching across tract boundaries

02C.06.02.05 Catalog Services

·  Implement properly structured results



NCSA / University of Illinois

Current accomplishments:

02C.07.00 Processing Control and Site Infrastructure Management

NCSA’s management activities during the month of March included drafting requirements for three new positions in the areas of system architecture and workflows, signing the MOU with IN2P3 and appointing Don Petravick as head of the Joint Oversight Committee, and revising the milestones for FY15 in LDM-240. Additionally, the management team began considering options for better internal tracking and reporting of effort that will streamline with existing tools and processes at NCSA and the UIUC.

02C.07.01 Processing Control

In March most effort was placed on finishing activities that were not completed at the end of the last cycle (rollover from W15). More work was done on building state diagrams to show job progress throughout a nightly computing run. The goal was to improve the diagrams so that they demonstrate that the Data Management Control System (DMCS) will always be in a specific known state. Among those drafted or revised were an Archive DMCS diagram, a Base DMCS diagram, a Distributor Node diagram, a Replicator Job diagram, and a Worker Job diagram. There has been a detailed discussion and review of the components and potential states of the DMCS.

Simplifying and refactoring the Event Services API, another activity worked on extensively in the W15 cycle, was an additional focus of effort in March. After integrating and testing changes from the API code review (which were mostly minor and aesthetic), this activity was completed. The completion of this refactoring allowed for work on the Apache logging framework to progress. Unit tests were created in preparation for writing the DM message appender class.

NCSA team members participated in the LSST Technical Operations Working Group (TOWG). They developed use cases to help with specifications for the buildings in La Serena, justifying the required office space based on the necessary level of staff.  

02C.07.02 Infrastructure Services

In March, the new Information Security Officer (ISO) began researching the project’s existing requirements and security policies. He reviewed SCADA documents, the data management design, the operations plan, and the observatory site requirements to gain a high-level understanding of the project’s systems and architecture. He investigated identity management systems that could be implemented for users to access scientific data. One such system in Shibboleth.

Work began on infrastructure configuration management. This will simplify managing the various servers and VMs at NCSA. Initial work was done prototyping with Puppet, configuration management tool.

After an internal discussion with the NCSA team about information classification, a document was developed describing storage structure and policies for the LSST DM developers environment. It identifies storage categories based on the data classification category (LPM-122), responsible manager of the data in that category, disk quota, purge policy, and backup and archive policies.

A major area of focus for this cycle is file systems and management. Since the LSST team at NCSA still lacks the ability to procure goods, we have been working closely with the Innovative Systems Lab (ISL) at NCSA, an organizational vehicle used to investigate pre-production technologies at NCSA. In March we evaluated and commented on an ISL proposal to investigate the CEPH file system for its properties as an alternative to the LSST baseline file system GPFS.

Research began on file management technologies, specifically iRODs functionality. This work was accomplished using resources on the NCSA ISL OpenStack. After setting up and configuring servers and host clients, testing was done on data replication. The configurations were also installed in Docker.

One activity related to understanding commercial file systems between data centers was not planned to begin until May, but was worked on in March due to the timely visit of vendors from DataDirect Network (DDN), a provider of big data storage. The NCSA team discussed our system needs and DDN presented on a product called WOS. This is a commercial object store which has interfaces for NFS, GPFS and Swift. Notes from this visit are captured in Jira ticket DM-2230.

02C.07.03 Environment and Tools

No activities are planned in this area for the S15 cycle.

02C.07.04 Site Infrastructure

Discussions began between NCSA and the qserv and SUI teams about the requirements for setting up a qserv prototype. We established “short-term” and “long-term” system needs (VMs, memory, disk space, ports, etc.). Implementation of this prototype is not scheduled to begin until July, though it will require NCSA to purchase new equipment and will thus be highly dependent on finalizing the pending procurement contract.

The base data center requirements were revised with updates to square footage, average rack weights, and power requirements. These revisions were circulated around with the Chilean site team.

No activities dependent on purchasing new hardware have been planned until mid-summer due to pending procurement contract.

Several LOE tasks emerged during March and were addressed. Two user accounts were created and one was disabled since the user has left the project. Some issues with the existing systems at NCSA were fixed, including a Crashplan failure, a GNU library issue, and a drive failure on one of the LSST machines at NCSA (lsst-dbdev2). The NCSA infrastructure team deployed NAGIOS Puppet agent checks and active bandwidth network tests that would maximize bandwidth from a user perspective. Some packet loss was detected, indicating we are not getting the bandwidth we would like.

At the end of the month several users reported slowness with the SSD on one of the development machines; this is being investigated.

Planned activities:
02C.07.00 Processing Control and Site Infrastructure Management
Anticipated management work in the month of April includes finalizing the position profiles for new hires, updating the development roadmap in LDM-240 with milestones in FY16 and beyond, and preparing slides for the DM presentation at the AMCL meeting. Additionally, the Cost Account Manager will travel to Tucson for a week to attend a training meeting about the LSST PMCS and risk register. NCSA managers plan to work more closely with the UIUC Business Office, NCSA financial department, and NCSA IT personnel to continue integrating our reporting, planning, and invoicing processes.
02C.07.01 Processing Control
In April, work will continue on drafting the job progress state diagrams and writing the handler for the logging framework. We expect both to be complete by the end of the month. We will also work extensively on the Alert Production Simulator, refactoring, creating long-running pilot Condor jobs, integrating OCS, and implementing APIs to transfer files to the replicator.
02C.07.02 Infrastructure Services
In April, the ISO will continue his investigation. The focus will be on the SCADA enclave and interfacing with the Telescope and Site team. Work will begin to draft a SCADA security plan.
More work will be done with the Puppet configuration management tool to prototype standard configurations on a set of test servers. Improvements will be made to the proposed storage policies, and work will be done to develop storage procedures and implementation based on these policies. Researching and prototyping file systems will continue with tuning and implementing ZFS on our own servers, as well as potentially beginning to look at optimizing NFS or replacing it with Ceph.
We will continue researching the capabilities of iRODS, especially in registering data virtually and into a cache, and in handling file corruption and data loss. We will also investigate the current usage of iRODS and its development track, as well as the LSST Datacat in the context of managing files at NCSA.
02C.07.03 Environment and Tools
No activities are planned in this area for the S15 cycle.
02C.07.04 Site Infrastructure
In April, NCSA will look into on-boarding staff to help with the base site requirements. It is believed that facility manager at the National Petascale Computing Facility will be available to participate as a consultant to the project.
No activities dependent on purchasing new hardware have been planned until mid-summer due to pending procurement contract. In April we anticipate another draft from AURA and move towards finalizing this contract.
We will continue to investigate the user reports of slowness on the development machines at NCSA.




Current accomplishments:
02C.08.03 Long-Haul Networks

·  Fernando Liello was paid the first $25K of $240K for his consultancy work for the Chilean Networking. (This includes work for the National link and Summit to Base and end equipment purchases)
·  There was planned with Telefonica a trip to Pachon and Tololo to re-quote the fiber cable run from AURA Gatehouse to LSST. Telefonica cancelled. Trying to reschedule next visit as soon as possible.
·  Not exactly an LSST issue but AMlight and Reuna provisioned a third diverse path to the US. This could be useful for the LHN end to end testing in the event of failures. We also configured the end gateway routers to switch paths in 2-3 seconds instead of the current 3-4 minutes.
·  Two meeting held for LHN end2end testing and perfsonar. Still some work to do before we are able to actually perform any testing.
·  We received the first design deliverable from Reuna. This covers only the Chilean National network. It is in compliance with my expectations.
·  Attended a workshop in Miami for SDN/Openflow testbeds. This technology could play a role in LSST’s network links and/or Data or Computer Centers.
·  Spent some time learning about new Cisco technologies via youtube tutorials. This is mainly for updating the technologies and architectures in the Data and Computing centers at the summit and base.
·  Reuna do not want their first payment from AURA until the details are completed with their contract with Telefonica and that payment due.

Planned Activities
02C.08.03 Long-Haul Networks

·  Attend TCAM meeting in Tucson
·  Meet with Cisco in Tucson to discuss ACI (Application Centric Infrastructure) and SDN (Software Defined Networks)
·  Next month we will go with Telefonica to Pachon and Tololo to obtain the updated quote of the fiber cable as per the advice of the Network review panel in February.
·  Attend Pachon first stone ceremony. April 14th.
·  Continue working with Reuna and by default Telefonica to close the contract for the summit-base link.
·  Meet with Chip Cox for an update on the International and US links.
·  Start working on the RFQ to send to potential DWDM providers. This will be done alongside Reuna with some consultancy from Fernando.


Back to top