Prototype Data Access Center: User Report

LargeSynopticSurveyTelescop

PrototypeDataAccessCenter:

Report

K.Suberlak,Ž.Ivezić,andthePDAC

DMTR-22

LatestRevision:2017-07-11

Abstract

AreportonuserexperienceofthePrototypeDataAccess

qualityandeaseofaccesstothedata.PDACwillpavethe

InterfaceandTools(SUIT).Weemploybothin-detailstudy

astatisticalstudyofanensembleofobjects.Weevaluate

currentinterface,andmakerecommendationsforitsfuture

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

ChangeRecord

VersionDateDescriptionOwnername

1.02017-05-28Reportcomplete.KrzysztofSuberlak

1.12017-07-11ReleaseasDMTR-22T.Jenness

Documentsourcelocation:

https://github.com/lsst-dmsst/DMTR-22

LARGE SYNOPTIC SURVEY TELESCOPE

PDAC report

DMTR-22

Latest Revision 2017-07-11

Contents

1 Introduction

2 Overview of performed tests

3 User Interface: what we see

3.1 UI overview

...........................

4 Infrastructure: what is available and how to get

4 it

4.1 Overview

............................

4.2 Tests Performed

..........................

4.2.1 Identification numbers in DeepSource

.......

and DeepForcedSource

4.2.2 Postage Stamp

.....................

Miniatures

4.2.3 Database linkage: .................

obtaining magnitudes

4.2.4 External

........................

Images

4.2.5 External

........................

Catalogs

4.3 Time Series:.......................

periodogram

5 Database Ingestion: is what we get what we expected

to get?

5.1 Single forced photometry

....................

light curve

5.2 Positional comparison:

.....................

box query

5.3 Light curve comparison:.................

multiple cone queries

6 Conclusions

A Appendix: SQL queries

A.1 Single source cone query,

...................

calibrated flux

A.2 Single source cone query,

.................

calibrated magnitude

A.3 Box query,

..........................

i<23.5

iii

LARGE SYNOPTIC SURVEY TELESCOPE

PDAC report

DMTR-22

Latest Revision 2017-07-11

Prototype Data Access Center: User

1 Introduction

This is a document to report on the user experience testing

Center on macOS Sierra Version 10.12.3 (16D32), using the

The Large Scale Synoptic Telescope (LSST) will produce a

dented data stream poses new challenges to provide an easy

that they can quickly find what they need, and thus be able

they would like to achieve. The detailed description of technical

the Science User Interface and Tools

LDM-130

(SUIT

outlined

requirements)

in documents

and

LDM-492

(SUIT Vision)

, as well as confluence

. Current

pages

Prototype Data

Center implementation of SUIT is described on the Guide to

An online tool that can serve as an interface between the

idea. There is indeed a number of websites that serve such

, Sloan

Digital Sky Survey (SDSS) Sci Server including CasJobs

, NASA/IPAC InfraRed Science

(IRSA)

, Mikulsky NASA

, NASA

Archive

Extragalactic

.Databased

These allow

(NED)

a user

to query for data (either via SQL query, or with the interactive

data table. Some user interfaces (eg. IRSA) have some rudimentary

stands at the forefront of user interface development, that

time series, spectrum, and other static information, but

on the cloud. For instance, a jupyter notebook could be employed

models, and use the output to further select objects based

National Optical Astronomy Observatory (NOAO) DataLab

is an example of a tool that

help explore the data in an interactive way, using allocated

developments are advocated at the MIT Astroinformatics

9,4].

These documents were recently superseded by

LDM-554

(Science Platform Requirements)

LSE-319

(Sci-

and

ence Platform Vision).

https://confluence.lsstcorp.org/display/DM/Science+User+Interface+and+Tools

https://confluence.lsstcorp.org/display/DM/Guide+to+PDAC+version+1

http://aladin.u-strasbg.fr

http://www.sciserver.org/tools/

https://irsa.ipac.caltech.edu/frontpage/

https://archive.stsci.edu

https://ned.ipac.caltech.edu

http://datalab.noao.edu

LARGE SYNOPTIC SURVEY TELESCOPE

PDAC report

DMTR-22

Latest Revision 2017-07-11

This report details tests performed on the PDAC implementation

shots and data-based plots. A shorter summary of monthly

at the github repository of the

https://github.com/lsst-

LSST System Science Team:

DMTR-22

2 Overview of performed tests

We test a variety of aspects of PDAC: the user interface,

tion, focusing on the Sloan Digital Sky Survey Stripe 82

dataset. In

3we

Section

describe the functionality available through

tion

4we describe the structure of available data: both datasets

NCSA (internal catalogs) , and data that is available from

tion we also provide an overview of query and analysis methods

User Interface, as well as through

5we consider

SQL. Finally,

the quality

in Section

ingestion, answering the question of how well was a given

ular we compare the S82 dataset, an outcomeDocument-

of the Summer 2013

15097

], to the same data stored locally at the University of

3 User Interface: what we see

3.1 UI overview

In order to access PDAC we follow the

directions

that include logging to NCSA via

https:

//vpn.ncsa.illinois.edu/

using Cisco AnyConnect Secure Mobile Client,

web

browser

http://lsst-sui-proxy01.ncsa.illinois.edu/suit

This opens the main inter-

face screen, which allows to select the database, and perform

Currently, PDAC v1, in the upper-left corner of 1

the

)

interface,

includes the Summer 2013 DM-stack reprocessed

sdss_stripe82_00

SDSS),

Stripe

hosted at the NCSA on the LSST prototype (”integration cluster”)

Dubois-Felsmann, priv.comm. 02-20-2017, slack]. The only

of March 2017), is WISE catalog, that is not yet accessible

be queried as

wise_00

Data,Base

with catalogs ’Object’ containing objects

in S82 above), and ’ForcedSource’ containing forced photometry

LARGE SYNOPTIC SURVEY TELESCOPE

PDAC report

DMTR-22

Latest Revision 2017-07-11

S82)).

The upper-left corner of the interface also leads to ’External

The Catalogs are all NASA/IPAC

Infrared Science Archive(IRSA) publicly

including Gaia, WISE, 2MASS,

).SPITZER, etc. (see Fig.

3.1.0.1 Range of input accepted by ‘Name or Position’ box

Description:

We test what is the range of RA, dec values and types

’Name or Position’

1 box (see Fig.

Input:

As input we use a set of coordinates, and names: first

∘<RA<180∘convention

(ra,dec)=∘(-7.530128,-1.171239∘),thenthesamebutin0

∘<RA<360∘convention:(ra,dec)

(352.469872∘,-1.171239∘).FinallyweuseanobjectId=216471849679198456,

theDeepSourcetablewhenqueryingthislocationwith

DeepSourcetablewith2

′′searchradius.

Results:NegativeRAisnotresolved(Fig.3).UnlessweuseanamefromNED/Simbad,

objectIdisnotresolved,eventhoughitispresentin

Date:2/15/2017

3.1.0.2Areallsearchoptionsavailablein‘MethodSearch’?

Description:The’MethodSearch’dialogboxcontains’Cone’,’Elliptical’,

Object’,’AllSky’options.Wetestwhethereachmethod

Input:Weemploycoordinates(ra,dec)=(352.469872,-1.171239)

use’Cone’radius2arcsec,’Elliptical’semi-major

’Polygon’defaultvertices(352.48041-1.18156,352.45985

352.48040-1.16073),’Multi-Object’:alistoftwora,

formatra,dec:(352.469872,-1.171239|342.469872

SQLconstraintid=216471849679198456.Forallsearch

catalog.

10InfraredProcessingandAnalysisCenter,

http://www.ipac.caltech.edu/project/lsst

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Results:’Cone’,’Elliptical’,’Box’,and’Polygon’search

foragivensearchregion.’Multi-Object’providedwith

ror’Failtoloadtable.Error:edu.caltech.ipac.firefly.server.query.DataAccessException:

DataAccessException:ERROR:CouldnotdoMultiObject

wrong.:tableshouldbeapostsearchnotagetfrom:unknown’.

formativeifaninformationaboutunavailabilityofthis

pageratherthanreturningaresultafterauseruploaded

The’AllSky’methodreturnscorrectoutput(alistof

=216471849679198456).

Date:3/28/2017

Figure1:ThemainuserinterfaceofPDACver.1.AsofApril2017,

queriesarenotavailable.The’NameorPosition’onlyresolves

∘<RA<360∘),

whileusingdirectSQLqueryresolvesbothpositiveandnegative

∘<RA<180∘).

Currentlythisisaninconsistencythatwerecommendtobe

thermore,thenamesresolvedhavetobeconsistentwiththose

databases-anyid’sfromthedatabasequeried(eg.’id’in

RunDeepForcedSource)arenotyetresolved.

4Infrastructure:whatisavailableandhow

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure2:IPAC-hostedcatalogs,accessibleviaIRSA.

Figure3:Testingrangeofinputacceptedbythe’NameorPosition’

notresolved,eventhoughdirectSQLqueryacceptsbothpositive

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

4.1Overview

AswedescribedinSection3,themainuserinterfaceallowsaccessto

NCSA)SDSSStripe82datareprocessedduringtheSummer2013

11aspartofDataChallange

withthecontinuouslydevelopedLSSTStack

12.

Thereprocessingincluded:

•coaddingthedatafromallepochsineachoftheugriz

oncoadds(perobject)areavailableasRunDeepSourcetable,accessibleviaCatalogs

’DeepSource’.Thesingle-bandcoaddedimageswithMariaDB

asDeepCoaddtable,accessiblaviaImages–>’DeepCoadd’.

•usingi-banddetectionstoseedforcedphotometryon

sultsofphotometryareavailableasRunDeepForcedSourcetable,accessibleviaCatalogs

–>’DeepForcedSource’.

•Forreference,theindividualcalibratedsingleepochScience_Ccd_Exposure

table,accessibleviaImages–>’ScienceCCDExposure’

AdditionaldetailsoftheschemaarealsooutlinedintheLDM-

226,DMTN-035],andtheLSSTDatabaseSchema[LDM-153]Browser13.

SpatialqueriesthatcanbedirectlyexecutedfromthePDAC

includecone,box,ellipticalandpolygon(SeeFig.1).Spatialqueriesallowtochoose

regionoftheskybytheobjectra,deccoordinates.Cone,

objectsinaregionoftheskyboundbyageometricalshape

(ra,dec).Coneisthemostusefultypeofquery,allowing

fromthecoordinatequery.Ellipticalsearchallowsto

givensemi-majoraxis,positionangleandtheaxisratio.

querycoordinates,withagivensidesize.Apolygonallows

between3and15coordinatepairs(verticesofthepolygon).

inthedrop-downmenu,buthasnotyetbeenimplemented(March

allowtheusertouploadalistofra,decandsearchradii,

11https://confluence.lsstcorp.org/display/DM/Properties+of+the+2013+SDSS+Stripe+82+reprocessing

12https://pipelines.lsst.io/index.html

13http://ls.st/8g4

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

catalog.AnAll-Skyoption(nospatialconstraints)has

database.

Anyqueryreturnsalistofallobjectswithinthegivenregion5).

4.2TestsPerformed

4.2.1IdentificationnumbersinDeepSourceandDeepForcedSource

AcertainlimitationofthemainUIisinabilitytoresolve

Fig.1).Indeed,theonlywaytofindwhichobjectshavebeen

regioninDeepSourcecoadds,anddownloadlightcurves

ForcedSourceforcedphotometrycatalog,istouseanSQL

formedconequeryagainstDeepSourcetableforra,dec=

∘,1.178522∘,′′2search

radius(thisistheRRLyraeID=13350alsoinvestigated4.3).Limitingtheresults

[id,coord_ra,coord_decl,flux_psf,coadd_id,coadd_filter_id],wefindthatthere

isacoaddforeachfilter(denotedwithcoadd_filter_id).Theidentificationini-band

(coadd_filter_id=3)isid=3588818166880604.NotethatwhileDeepSource

idforacoaddineachband,onlyid’sfori-bandcoaddare

catalog.TheDeepSource.id==DeepForcedSource.objectId,

standsforforcedphotometrydetectionid,whichisunique

gleobjecthasoneDeepSource.id,equaltoDeepForcedSource.objectId,

ForcedSource.id-werecommendtohighlightthisinthe

forconfusion.Theonlywaytocurrentlyrecoveralight

ForcedSourceistofirstselectthedetectionidinDeepSource,

whenusingconequeryonDeepForcedSource(seeFig.4)

4.2.1.1Easeofselectingalightcurveforasingleobject

Description:wetesthowdifficultitistoselectforcedphotometry

object.

Input:ConequeryDeepForcedSourcecatalogforra,dec=0.283437

∘,1.178522∘,with2′′search

radius.Selectforcedphotometrydataproductsfora

dius.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Result:ThereisnooptiontoshowwhichobjectId’sarepresent

query,andsomehowselectonlyone.Thesimplestworkaround

againstgivencoordinates,addingSQLconstraint'coadd_filter_id=3'toreturn

onlyi-bandid’s(becauseid’scorrespondingtocoadds

offorcedphotometry,andonlyfori-bandthereisacorrespondence

=DeepForcedSource.objectId).Thei-bandDeepSource.id

thenqueryDeepForcedSourceagainstthesamecoordinates,

'objectId'=3588818166880604'.Thisisquiteconfusing(idmeans

entforDeepSourceandDeepForcedSource),andmaycause

willingtodownloadforcedphotometryforonlyoneobject

Date:4/3/2017

4.2.2PostageStampMiniatures

Wecomparedthepostagestampminiaturesshowingtheoverview

agivenquerywasperformed.Wefindthattheminiatureimage

thecatalogwequeryagainst.Infact,the”coverage”image

WISE-thesurveyischosendependingonthesizeoftheregion

Wu,priv.comm.,2017].Indeed,asthequeryregionisincreased,

unexpectedlyfromDSStoIRASorWISE,withoutissuingarelevant

recommendationistodisplayinformationabouttheorigin

4.2.2.1ComparingtheminiatureimagestoSDSSDR13Sky

Description:wetesthowwelldoesacoverageimagereflectthequeried

Input:performconequeryagainsttheDeepSourcetable,using

23h30m57.31s,+1d1m13.8s(or352.73878

∘,1.02049∘),withsearchradiiof2′′,10′′,

100′′,or1000′′.Eachtime,comparethecoveragetotheSDSShttp:

//skyserver.sdss.org/dr13/en/tools/chart/navi.aspx

Result:weexpectedthatthecoverageimagewouldbecentered

at2

′′,10′′,100′′searchradii.However,at1000′′radius(andlarger),theimage

drasticallyswitchestousingadifferentimagingsurvey

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure4:Thecorrectwaytoselectalightcurveforaparticular

tometryDeepForcedSourcecatalog.Herewefirstqueriedagainst

tofindid’sforobjectsdetectedincoaddsinasmallregionwithin

′′fromra,dec=0.283437∘,

1.178522∘.Fori-bandthereisonlyoneid:3588818166880604.Since

ForcedSource.objectId,werequireobjectIdtobeequalto

abletoaccesforcedphotometryforpreciselyoneobject.Otherwise,

fromadirectspatialqueryofDeepForcedSourcewouldprovide

jectsdetectedincoaddswithinthesearchradius,whichmay

analysisofTimeSeries.Werecommendthattheresultofspatial

ForcedSourceshouldcontainasummaryofwhichuniqueobjectId’s

abilitytoselectonlyoneobject(withmulti-bandphotometry),

thesearchregion.Otherwiseitbecomesalong-windedprocess

detectedincoadds(DeepSource),tothenselectidfori-band

correspondingtothatobjectIdinRunDeepForcedSource.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

muchmoreblurrythanbefore,impossibletorecognize

atthatscaleascompatedtotheSDSSDR13SkyServer.

Date:3/7/2017

4.2.2.2ComparingtheminiatureimagestoSDSSDR13SkyServer

Description:wetesthowwelldoesacoverageimagereflectthe

tendedsource(agalaxy)

Input:performconequeryagainsttheDeepSourcetable,using

ra,dec=40.433

∘,0.449∘′′.Eachtime,comparethecoveragetotheSDSS

http://skyserver.sdss.org/dr13/en/tools/chart/navi.aspx

Result:weexpectedthatthecoverageimagewouldbecentered

at2

′′,10′′,100′′searchradii.However,at1000′′radius(andlarger),theimage

drasticallyswitchestousingadifferentimagingsurvey

muchmoreblurrythanbefore,impossibletorecognize

atthatscaleascomparedtotheSDSSDR13SkyServer.

Date:3/7/2017

4.2.3Databaselinkage:obtainingmagnitudes

Auserqueryingadatabasewouldbeverylikelyinterested

Wetesttheeaseofobtainingcoaddmagnitudesforsources

taininglightcurvesforaparticularobject.

4.2.3.1Obtainingthecalibratedmagnitudesforcoaddimages

Description:wetesthoweasyitistoaccesswiththeuserinterface

forcoaddsforsourceswithinacertainregion.

Input:performconequeryagainsttheDeepSourcetable,using

23h30m57.31s,+1d1m13.8s(or352.73878

∘,1.02049∘),withsearchradiiof10′′.Seek

toselectmagnitudesfromtheavailablefield.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure5:ExampleconequeryagainstDeepSourcetable,returning

withinaradiusof10

′′fromthepositionra=352.469872∘,dec=-1.171239∘.Notethatthe

backgroundimage(postagestampminiature)doesnotshowthe

particularfeatureisdescribedfurtherinSec.4.2.2

Result:negative.Weexpectedtofindafield’gmagnitude’,or

agivenfilter.However,suchfieldisnotpresent.Currently,

tudesviaadirectSQLquery

Date:4/27/2017

4.2.3.2Obtainingthecalibratedmagnitudesforforced

Sourcecatalog)

Description:wetesthoweasyitistoaccesswiththeuserinterface

forforcedphotometrylightcurvesforsourceswithin

Input:performconequeryagainsttheDeepSourcetable,using

23h30m57.31s,+1d1m13.8s(or352.73878

∘,1.02049∘),withsearchradiiof10′′.Seek

toselectmagnitudesfromtheavailablefield.

14https://confluence.lsstcorp.org/display/DM/PDAC+sample+queries+and+test+cases

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Result:negative.Weexpectedtofindafield’gmagnitude’,or

agivenfilter.However,suchfieldisnotpresent.Asin

onlyobtainmagnitudesviaadirectSQLquery

Date:4/27/2017

4.2.4ExternalImages

ApartfromStripe82reprocesseddata,theusercanalsoa

datafrom2MASS,WISE,SDSS,MSX,DSS,IRAS(seeFig.6)

4.2.4.1AretheminiaturesfromExternalImagesproperly

Description:wetesthowwelltheobjectisrenderedwhenquerying

ternalImagescatalogs

Input:performaNEDresolvedqueryforM3globularcluster

’CreateNewPlot’.QuerySDSSu-band.

Result:weexpectedthattheimagewouldbecenteredonM3.In

offthecenter(seeFig.11).

Date:4/25/2017

4.2.4.2AretheminiaturesfromExternalImagesproperly

Description:wetesthowwellthelarge-scaleextendedobjectis

animagefromvariousExternalImagescatalogs

Input:performaNEDresolvedqueryforM81inExternalImages.

QueryWISE,2MASS,SDSSandDSSsurveys.

Result:weexpectedthatallminiatureswouldbecenteredon

approximatelycentered,andtheSDSSimageisrotated

Adifferentangularscaleisseeminglyshownintiledview.7).Update:selecting

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

WCSoption(seeFig.8)showsthatimagesareapproximatelyproperly

stillnotorientedinthesamefashion.

Date:4/23/2017

4.2.4.3Doestheoptionofcreatingnewcolorimagefrom

properly?

Description:wetesttheeaseofuseofExternalImages-’Create

interface.

Input:inExtendedImages:CreateNewPlot-3Colors,query

bandstostandforred,green,bluecolors.

Result:theframesareproperlyaddedandtheabilitytoselect

(suchasinfrared)isreallyuseful.However,theSDSS

Fig.9and10)

Date:4/24/2017

4.2.5ExternalCatalogs

TheexistinginterfaceallowsaccesstodatafromExternal

presentattheNCSA,butparsedthroughIRSA.Wetestthe

dataforsimpleNED-resolvedlocations.

4.2.5.1CanweobtaindatafromGaiaforaregionspecified

aNED-resolvedobject?

Description:wetestwhetherthedataobtainedfromGaiamatches

bytheuser.WechooseM81asourtargetfortheeaseof

imageshownintheresults.

Input:performaNEDresolvedqueryforM81galaxyinExternal

searchradius.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure6:ExternalImagesUserInterface.Theradiobuttonsonthe

image,addanotherpaneltoanimagecreatedbytheprevious

image.Inthisexample,describedinSec.4.2.4,wequeryforanimageofM81,with

expectedsizeoftheimageof0.139degrees.

Figure7:ExternalImagesUserInterface:asuccessionofqueries

andDSScatalogsforanimageminiatureofM81.TheSDSSminiatures

respecttotheothercatalogs.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure8:ExternalImagesUserInterface:afollow-uptoFig.7,showingtilesofWISE,2MASS,

SDSSu,DSSimages.Selectingthe’WCSmatch’option,andclicking

totheimage,andshowthedirectionsofEquatorialJ2000North

imagesarenotallorientedinthesamefashion.Usingtheruler

distanceoneachtiletogiveasenseofscale.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure9:ExternalImages:’CreateNewPlot-3Colors’main

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure10:ExternalImages:theresultofqueryforM81inSDSSR,G,U

RGBcolors.

Result:positve.WeseedetectionsaroundM81,andtheminiature

displaysthequeriedregion(seeFig.13).

Date:5/3/2017

4.2.5.2CanweobtainWISEdatafromExternalCatalogsfor

Description:wetesthowtheWISEdatacanbeaccessedviatheUI.

Input:performaNEDresolvedconequeryforM33inExtended

SourceCatalog,and100arcsecsearchradius.

Result:positive.Theminiatureimageisapproximatelycentered

infrared(aswecanseefromcomparingFig.14to2MASSimagefromSDSSSkyServer

onFig.15

Date:5/3/2017

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure11:ExternalImages:queryingforanSDSSimageofM3Globular

offtheimagecenter.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure12:ExternalCatalogs:mainmenu.Theusercanselectone

Fig.2):Gaia,WISE,2MASS,SPITZER,IRAS<Planck,MSX,COSMOS,

DENIS,HERSCHEL,PTF,andothers.Someprojectshavemorethan

(egWISEincludesAllWISE,NeoWISE,etc).

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure13:ExternalCatalogs:theresultofqueryingforGaiadetections

ofM81.

Figure14:ExternalCatalogs:theresultofqueryingAllWISESource

aroundlocationofM33-theTriangulumGalaxy.Notethatthe

infrared,whichiswhythespiralfeaturesarenoteasilydiscernible15-the

2MASScutoutfromSDSSSkyServer).

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure15:SDSSDR13SkyServer:imageshowingthe2MASSimageof

positionofM33galaxy:ra,dec=23.46204,30.66022.Compare

locationofM33forAllWISEdetectionsshownonFig.14.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

4.3TimeSeries:periodogram

Abilitytoviewthetimeseriesofanyobjectisveryuseful

ofaqueryintoForcedDeepSourcetableallowsasimple2D

thatcouldincludetimevsflux.However,theadvantageof

UserInterfaceisthatitalsoperformsthecross-matchagainst

tofindthecalibratedmagnitudes,andallowscalculating

thebestfittingperiod.

HerewetesttheoverallnavigationexperienceintheTime

findingthebestperiodicitywiththeLomb-Scargleperiodogram,

acceptedperiod,andchangingthebandinwhichthelight

SeriesViewonknownRRLyraestars[10].

4.3.0.1CanweaccesstheTimeSeriesVieweasily?

Description:wetesttheaccesstotheTimeSeriesView:isthebutton

wegetthesameresultifqueryingtwice?

Input:conequerytheDeepSourcecatalogforaknownRRLyrae

of[10]),atra,dec=0.283437

∘,1.178522∘,with2arcsecsearchradius,to

tIdsatthislocationini-band(sinceonlyi-bandcoadd

tometryinallbands).Toselectonlyi-bandcoaddwe

coadd_filter_id=3.WefindthattheonlyobjectIdini-bandis’id=3588818166880604’.

WeconequeryRunDeepForcedSourcetableatthesame

addinganSQLconstraint’objectId=3588818166880604’

RunDeepForcedSource.objectId).Wethenselect’View16).

Result:positive.InmostcasestheTimeSeriesViewappears,

missinganyplots:amessage’PlotFailed=Couldnotcreate17).

Itisnotclearwhatthethreewindowsweremeanttodisplay.

curveinadifferentbandforthesameimagewiththeradio

corner.

Date:4/7/2017

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

4.3.0.2CanwerecovertheperiodofanRRLyraestarusing

odogramwithdefaultsettings?

Description:thetoolbecomesusefulifitcanprovidemeaningful

ispossibletofindthetrueperiodofaknownRRLyraestar

Input:asinthe’canweaccesstheTimeSeriesVieweasily?’

Sourcetableforra,dec=0.283437

∘,1.178522∘,with2arcsecsearchradius,

constraint’objectId=3588818166880604’.Click’View

(thedefaultisu),click’FindPeriod’.SelectPeriodogram

Method’FixedFrequency’.LeaveNumberofPeaksat50.

(Fig.18).

Result:negative.Thedefaultgridstretchestoomanypossible

coverthetrueperiodof0.546days(seeFig.19).Werecommendimplementing

odogramsforMultibandAstronomicalTimeSeries(asin15])toutilizethepresence

near-simultaneousmulti-bandobservations.Another

differentpresentgridspacingsettings,eg.AstroML-like,

Date:4/7/2017

4.3.0.3CanwerecovertheperiodofanRRLyraestarusing

odogramwithcustomsettings?

Description:wetestwhetheritispossibletofindthetrueperiod

usingthecustomsettings.Givenourknowledgeofthe

weconstrainthemaximumandminimumperiods(frequency)

odogramcalculation.

Input:asinthe’canweaccesstheTimeSeriesVieweasily?’

Sourcetableforra,dec=0.283437

∘,1.178522∘,with2arcsecsearchradius,

constraint’objectId=3588818166880604’.Click’View

(thedefaultisu),click’FindPeriod’.SelectPeriodogram

Method’FixedFrequency’.LeaveNumberofPeaksat50,

PeriodMax=0.998[days].Click’PeriodogramCalculation’.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Result:positive.Oneofthemajorperiodogrampeakscorresponds

0.546days(seeFig.20).Byjudiciouslyconstrainingtheminimum

odssearched(basedonourknowledgeofperiodsforthis

recoverthecorrectperiod.However,themethodisvery

grid-smallchangeinboundsleadstodramaticimprovement

nessofperiodogram(seeFig.21).Wealsorecommendthatthemethod

Frequency’or’FixedPeriod’)shouldbedisplayedinthe

Date:4/7/2017

4.3.0.4Isthereadocumentationdescribingthealgorithms

odogram?

Description:welookfordocumentationsothattheusercanbetter

ofLomb-Scarglealgorithmisused,sinceitwouldaffect

Input:searchtheTimeSeriesview,ortheresultsoftheperiodogram

documentation.

Result:negative.Wefindnoreferencetowhichspecificalgorithms

PDACteamwefoundthatthebackendoftheperiodogram

ExoplanetArchive[XiuqinWu2017,priv.comm.].The

internallytestedtocomplywiththatoriginaltoolset,

ofNASAExoplanetArchiveisavailableathttp://exoplanetarchive.ipac.caltech.edu/

docs/pgram/pgram_parameters.htmlWerecommendthattherebealinkto

vidingtheuserwiththiscrucialinformation.

Date:5/4/17

4.3.0.5Doestheperiodselectionbarcorrectlyupdate

Description:wetestinselectingminimumandmaximumperiodsin

lationwhetherthex-ticksoftheperiodselectionslider

itiseasytochangethevaluesmanually.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Input:asin”canwerecovertheperiodofanRRLyraestarusing

odogramwithcustomsettings?”.Checkwhetherthex-ticks

(selectedminimumonthefarleftofthesliderbar,and

right).Enteradifferentminimumvalue:0.4insteadof

Result:negative.AsinFig.19,theminimumandmaximumofthesliderbar

365days.Furthermore,asinFig.20,weseethattheperiodsliderbar

correctlyupdatetothecustom-chosenperiodrange.

Date:5/4/17

4.3.0.6Isiteasytofoldthelightcurveonanaccepted

Description:wetestwhetheritispossibletofoldthelightcurve

withoneband(eg.u)butusingthedatafromadifferent

becausethereshouldbeapproximatelythesameperiod

formanysources(eg.RRLyrae).

Input:followthestepsasin”canwerecovertheperiodofan

Scargleperiodogramwithcustomsettings?”.Choosethe

Click’Acceptperiod’inthelowerrightcorner.Inthe

buttoncorrespondingtoadifferentband(eg.r).Observe

foldedontheacceptedperiod.

Result:negative.Oncetheperiodisaccepted,wearebrought

showsthelightcurvefoldedintheoriginalbandonan

r-bandinsteadoffoldingthelightcurveinr-bandon

showstherawphotometry,withnowayoffoldingitonany23).Further-

more,theactualvalueoftheacceptedperiodisnotdisplayed

window.Werecommend:a)oncetheuserclicks’accept

windowonradiobuttonsofu,g,r,iorzbandwouldsimply

datafromtheselectedbandonthechosenperiod.b)the

playedsomewhereinthe’Viewer’window,eg.ontopofthe

=....[days]’.c)thewindowsonthebottomdonotshow

beusefuliftheuserwasinformedwhataretheysupposed

tersusedforperiodogramcalculation(minimum,maximum

withinasingleobjectTimeSeriesView.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Date:5/3/17

4.3.0.7Istheperiodogramtoolverysensitivetoboundary

Description:wetestwhetherthecurrentimplementationofthe

tivetotheboundaryvalueschosenforperiod.

Input:asin”canwerecovertheperiodofanRRLyraestarusing

odogramwithcustomsettings?”.ConequerytheDeepForced

=0.283437∘,1.178522∘,with2arcsecsearchradius,addingSQL

3588818166880604’.Click’ViewTimeSeries’,select

’FindPeriod’.SelectPeriodogramType’Lomb-Scargle’,

PeriodStepMethodto’FixedFrequency’.EnterPeriod

[days].Click’PeriodogramCalculation’.Tryalso0.3-0.9,

to’FixedPeriod’,usingthesameperiodbounds(0.22-0.9,

Result:positive.Theperiodogramtoolisverysensitiveto

Figs.20and21:0.229-0.998vs0.29-0.9.Wefindthatwiththe

stepmethodyieldalsodifferentresults:comparepairs

is’FixedFrequency’,rightis’FixedPeriod’).Surprisingly,

wemaybelesssensitivetothetrueperiod:compare0.29-0.921and

24)-thelatterhasamoredensegrid,andyettheperformance

recommendtowarntheuserthatatbesttheperiodogram

period,andthatthedefaultperiodogramnumberofpeaks

Date:5/5/17

Finally,wecomparethePDACTimeSeriesUserInterfaceto

Periodogram15(see26Fig.).UsingfewRRLyraePDACg-bandlightcurves,

allocatedatimeslotofapproximately15seconds.Alsosee2forasummaryofresults.

5DatabaseIngestion:iswhatwegetwhatwe

15http://exoplanetarchive.ipac.caltech.edu/cgi-bin/Pgram/nph-pgram

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure16:AresultofconequeryagainstRunDeepForcedSourcetable

data),with(ra,dec,radius=0.283437,1.178522,0.00055

Series’buttonthatlinkstotheTimeSeriesUIshownonFig.17.

Figure17:TimeSeriesViewerforanobjectId=3588818166880604,

∘,

1.178522∘.Notethatinitiallyonthebottomtherearethreeempty

tonsintheupperleftcornerallowintuitiveselectionofthe

calculation.Wechooseg-band,andclick’FindPeriod’,marked

Lomb-Scargleperiodogramforthatband(thistakestheuser18)

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure18:CalculatingLomb-ScargleperiodogramforanobjectId=3588818166880604,

ra,dec=0.283437∘,1.178522∘.Thesliderintheupperleftcornerallows

curveonachosenperiod.Clicking’CalculatePeriodogram’

dow’Periodogram’.CurrentlyitcontainsonlytheLomb-Scargle

StepMethodincludeFixedFrequencyorFixedPeriod,similar

odogramTool(Fig.26).Ifwedon’tchooseanythingformaximumandminimum

calculationwillproceedwithdefaults,whichforthisRRLyrae

(Fig.19).Ifwechoosetheminimumandmaximumperiodsknowing

foragivenclassofobject,wearemorelikelytodetectthe20),althoughthe

methoditselfisverysensitivetofrequencygrid.Clicking

ceedswithevaluatingLomb-Scargleperiodogramwithchosen

defaultfrequencygrid(seeFig.19)

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure19:CalculatingLomb-ScargleperiodogramforanobjectId=3588818166880604,

dec=0.283437∘,1.178522∘.Usingthedefaultsettingsdoesnotrecover

periodof0.547987days.SeeFig.20foramoreappropriatechoiceofperiodrange.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure20:CalculatingLomb-ScargleperiodogramforanobjectId=3588818166880604

dec=0.283437∘,1.178522∘.Whenweappropriatelyconstrainthefrequency

thepowersofperiodogramareevaluated,werecovertheperiod

of0.547987days.Onthisfigurewechoosefixedfrequencymethod,??????=0.229and

??????=0.998days,whichare90%ofthesmallestand110%ofthelargest

in[10]sample.NotethatasofApril2017,theminimumandmaximum

allowingtointeractivelyfoldthelightcurveonanyperiod

usedinthePeriodogramsearch.However,slightchangeoffrequency

improvethecalculation:seeFig21.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure21:CalculatingLomb-ScargleperiodogramforanobjectId=3588818166880604

dec=0.283437∘,1.178522∘.Slightchangeoffrequencygridcanheavily

hereweusefixedfrequency,withperiodchosenbetween0.29

Period’takestheusertoFig.22.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure22:Theresultofacceptingtheperiodfoundbylimitingthe

??????=0.29and??????=0.9days,foranRRLyraeID=13350(objectId=3588818166880604

ra,dec=0.283437∘,1.178522∘).Itisnotclearwhattheminiaturesshow:

Thelightcurveiscorrectlyfoldedong-banddata.Asurprising

adifferentband(eg’r’),insteadoffoldingthelightcurve

therawdataisdisplayed(Fig.23)

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure23:Theresultofclickingontheradiobuttonforther-band

ther-bandlightcurvefoldedontheacceptedperiod,wesee

the’FindPeriod’buttondoesnot’remember’theresultofthe

data.

Figure24:CalculatingLomb-ScargleperiodogramforanobjectId=3588818166880604

dec=0.283437∘,1.178522∘.Boundingperiodbetween0.22-0.9days,the

’FixedFrequency’,andtherightpanel’FixedPeriod’.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure25:CalculatingLomb-ScargleperiodogramforanobjectId=3588818166880604

dec=0.283437∘,1.178522∘.Boundingperiodbetween0.3-0.9days,theleft

Frequency’,andtherightpanel’FixedPeriod’.

5.1Singleforcedphotometrylightcurve

Wecompareaforcedphotometrylightcurveforasingleobject

thePDACcopyandthelocallystoredversion(attheUniversity

anobjectId=216172782516437336,inpatchg00_22,with72forcedphotometryepochs

g-band.Fromavailabledetectiondata(analoguoustoDeep

dec=359.974019436∘,-1.25626927667∘,extendedness=1.

5.1.0.1CanwequeryDeepSourcetablebyobjectId?

Description:wehaveboththera,deccoordinatesandobjectId

possibletoquerysolelybyobjectId,withouthavingto

Input:conequerytheDeepSourcecatalogforanobjectId=

dec=359.974019436∘,-1.25626927667∘,searchradius3arcsec.AddanSQL

coadd_filter_id=3,objectId=216172782516437336.Trytoquerywithoutspecifying

ra,dec.

Result:negative.Itisnecessarytospecifyra,dec,evenif

findthesoughtobject.WerecommendthatqueryingbyobjectId

nextreleaseoftheUI.

Date:5/22/2017

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure26:ThesameobjectasFig.39,andFig42,usingtheSDSSdatafrom[10].Thehighest

significancefrequencypeak(power21.58)correspondstoaperiod

thesecondinsignificancepeak(power20.62)correspondsto

[10].Notethebottom-leftcorner:thecalculationtook15secs

tofewmilisecondsofAstroMLcodenaivesingle-sinusoidapproach

forthisparticularobject)

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

5.1.0.2IsthenumberofphotometricpointsbetweenUWand

identical?

Description:wecomparethelocalcopyoftheg-bandphotometry

vsPDAChostedversion.

Input:conequerytheDeepForcedSourcecatalogforanobjectId

ra,dec=359.974019436,-1.25626927667,searchradius

id=216172782516437336andexposure_filter_id=1.

Result:positive.Thisqueryyields73epochsascomparedto

amination1epochofthePDACresult(exposureid6518110430,exposure_time_mid=

54040.248144284444)hasflux_psf_errflux_psf=None.Recommendation:forced

fluxmeasurementswherefluxisnotavailableshouldnot

outwarning,oranappropriateinformation(eg.’for

wherethereportedflux_psfisNone,availablehere’).Afterremoving

thenumberofepochsforforcedphotometrymeasurements

isidenticaltothelocalcopy.

Date:5/24/2017

5.1.0.3Arethetimestampsforeachforcedphotometryepoch

Description:wecomparethelightcurveresultingfromtestabove,

timeperexposure:arethetimesidenticalbetweenPDAC

Input:lightcurvefromPDACandlightcurvefromUWforobjectId

Result:negative.TheUWlightcurvehasameanoffsetof26.95

curveforthesameobject:approximatelyhalfofthe

recordsthebeginningoftheexposure,whereasPDAC:

AlthoughmetadataforUWdoesnotallowtoguessthat

umn’mjd’),thePDACcolumnnamemakesthisdistinction

'exposure_time_mid'.Recommendation:giventhatitcausesasmall

incidencemeasurements,itwouldbeprudenttoperhaps

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

oftheexposureandthemiddleoftheexposure,unless

ningisaverynonstandardpractice.Notethatthemjd

significantdigits(12-13inPDACdatacomparedto6-7

Date:5/25/2017

5.1.0.4AretheunitsforfluxmeasurementsbetweenPDAC

Description:wetestwhetherthefluxunitsreturnedbyPDACquery

SourcetableallowsfordirectcomparisonwiththeUW

possibletomakethemcomparableusingonlytheUI.

Input:queryagainsttheobjectId=216172782516437336,using

Result:negative.DataatUWisalreadycalibratedforzero-point,

whereasthePDACdataisstoredinADUs.Itiscurrently

fluxesusingthePDACUI:anexampleofanSQLquerytoobtain

ingajoinbetweenRunDeepForcedSourcetable,forthe

ualobservations,andtheScience_Ccd_Exposuretable(foraper-visit-image

zeropoint),isonthePDACSampleQueries.Recommendation:

RunDeepForcedSourceandofferoptionstoreturnuncalibrated

acalibratedfluxmakingajoinquerywithScience_Ccd_Exposureinthebackground.

Date:5/27/2017

5.1.0.5ArethecalibratedfluxesfromPDACidenticalas

Description:wecomparethecalibratedfluxesobtainedbySQL

ForcedSourceandScience_Ccd_Exposuretableing-magnitudefortheobjectId

andtheUWhostedforcedphotometrydata.

Input:calibratedfluxlightcurveforobjectId=216172782516437336,scisql_dnToFlux

andscisql_dnToFluxSigmafunctionstoconvertrawtocalibratedfluxesA.1for

theSQLquery).

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Result:positive.Ifweremovethemissingphotometrydatapoint

anidenticalnumberofepochs.Themeanresidualdifference

calibratedfluxdataiszero,andallthedifferencecan

metic(differentaccuracyofdatastoredatUWandPDAC).27forthehistogram

ofdifferencebetweenUWandPDACfluxes.Meanfluxratio

OnFig.28weplotthePDACandUWdataastimeseries,withthe

fluxdifferencebetweenthetwosourcesofdata.

Date:5/27/2017

Figure27:HistogramoffluxdifferencebetweendatastoredatPDAC

216172782516437336.Thedifferenceislessthan1E-35,i.e.

error.Thisprovesthatoncefluxesarecorrectlycalibrated

fromScience_Ccd_Exposuretable),lightcurvesatPDACrepresentapproximately

informationasthosestoredatUW.

Forcedphotometrymeasurementsmayreportfluxvaluesthat

noisy(S/N<2).Forinstance,forobjectId=216172782516437336,

developedapipeline

16thatimposesauniformprioroneachfluxmeasurement

withthenon-physicalityofnegativefluxvalues.Thuswe

(S/N<2)measurement,andamagnitudeforeachforced-photometry

16https://github.com/suberlak/Faint_pipeline_report

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure28:CalibratedfluxforobjectId=216172782516437336,same27.Lightcurves

atPDACandUWareidenticaloncebroughttoacommonunitsystem.

showsthedifferencebetweenPDACandUWlightcurve-thevalues

showthedifference(inunitsof1E-29ergs/sec/cm2/Hz,sothat

is1E-34).

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

thatwiththePDACscisql_dnToAbMagfunctionwhichsimplyremovesthepoints

fluxfromthelightcurve.Theimpactofthispipelineonthe29

Figure29:LightcurvefromUWforobjectId=216172782516437336,

points,withS/N<2.Thegreenpointsshowtheeffectofapplying

prioronforcedphotometry.

Giventhataswehaveshown,thefluxmeasurementscontained

UWareidentical,weshowtheimpactofcorrectionoffaint

onFig.30.

Giventheimpactofcorrectiononflux,magnitudeswould

impactofcorrected-fluxmagnitudesvsthescisqlfunction

whereS/N>2,butforfaintpoints,thecorrectedfluxesyield31

wejuxtaposethePDACSQL-computedmagnitudes(usingSQLA.2),vs.magni-

tudesbasedonPDACSQL-computedfluxes,correctedusing

Inconclusion,basedonthecaseofobjectId=216172782516437336,

PDACareidenticaltotheprecisionaffordedbythefloating

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure30:SameasFig.29,butusingPDACdata.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure31:AsectionoflightcurveforobjectId=216172782516437336,

tudesforPDACdatacalculatedwiththebuilt-inscisql_dnToAbMagandscisql_dnToAbMagSigma

functions(orange),andthemagnitudesbasedonPDACcalibrated

faintfluxespipeline(blue).NotethatincaseswherePDACflux

scisqlpipelineignoresthesepointsinmagnitudecalculation,

fornegativeflux.UsingourBayesianpipeline,weputabetter

ofthefluxmeasurement.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

thePDACcalibratedmagnitudesisnontrivial:werecommend

ingajointquerybetweenDeepForcedSourceandScience_CCD_Exposuretoreturncalibrated

fluxesandmagnitudesbeaccessibleviatheUI.Inaddition,

putingmagnitudesfromfluxesshouldallbedisplayedand

zeropointmagnitudesarestoredtocalculatecalibrated

valuesareusedtocalculatemagnitudes,whatformulae

wouldgreatlyimprovetheeducationalvalueofSUIT.Furthermore,

significantlyimproveifitwerepossibletoqueryeither

rectlybytheobjectId,withouttheneedtospecifyra,dec.

5.2Positionalcomparison:boxquery

WecomparethesourcedensitybetweenthePDACandUWS82

testthatcanhelpconfirmwhetherthetwodatasetsareidentical,

irregularitiesandinformationonhowtheydifferwould

PDACS82datasetthatshouldbebroughttotheattentionof

thiscomparisonalsohighlightdirectwaystoimprovethe

deepsourcecoaddcatalog,limitedati<23.5,containing

PDACRunDeepSourcecatalogcontainscoaddsineachofthe

inmagnitudedepth.

5.2.0.1Canweapplythemagnitudecutoffinaboxquerywithin

Description:limitingthemaximummagnitudeofsourcesreturned

functionality-giventheunderstandingaboutthelimiting

usermaywishtoselectagoodqualitymeasurementsby

magnitudebelowthelimitingmagnitudeofthesurvey.

thesurveydepth,thereisalotoffaintsourcesforwhich

forscientificpurpose.Themagnitudecutoffalsospeeds

ofsourcesgrowsexponentiallywithmagnitude.

Input:queryRunDeepSourceusingboxquerywithra,declimited

325.1).UseSQLconstraintcoadd_filter_id=3.Seektolimitthei-bandmagnitude

23.5.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Result:negative.Werecommendthatinthefutureversionof

fluxesandmagnitudesforobjectsinbothDeepSourceand

beavailableviatheUI.Anadditionalfeaturecouldbe

nitudeofreturnedobjects.ThisiscurrentlyonlypossibleA.3,

usingHAVINGi<23.5clause,whereimagnitudeistheoutcomeofjoinbetween

ForcedSourceandScience_CCD_Exposure.

Date:5/30/2017

5.2.0.2IsthesourcedensitythesamebetweenUWandPDAC

Description:wecomparethesourcedensityintheS82datasetusing

catalogandthelocallyhosted(UW)DeepSourcecatalog.

23.5mag.Thetwoshouldagreeonthenumberofsources

Input:UseboxqueryagainstthePDACS82data.Thequeryboxinqserv_areaspec_box

functionrequiresminimumandmaximumra,decbounds.

∘,325+D∘,and∘-0.7,

-0.7+D∘,withD=[0.1,0.2,0.4,0.6,0.8,1,1.2,1.4].This

ingarea(seeFig.32).Measurethequeryspeedincludingthedownload

objectid,i-bandcoaddmagnitudeanderror.Limitthe

theUWdata.Selectonlyi-bandcoaddbyspecifyingRunDeepSource.coadd_filter_id

MeasurethetimeittakestoquerytheQservanddownload

ofuniqueidinthequery.Comparetothenumberofobjects

UWS82DeepSourcecatalog.

Result:negative.NumberofsourcesinPDACperchosenarea,

astheUWcatalog,is≈15%higher(seeTab.1fordetailedresults).This

ofchoosingwhethertoincludedeblenderparents.In

extrasourcesatthesamelocationassourcesmatched

’parentDeepSourceId’intheUWcatalog.Werecommend

totheuserinmoredetail,inparticularthemeaningofparent,ordeblend_n_childcol-

umnnames.Otherwisesearchesforobjectswithinacertain

results.Aspredicted,themeasuredquerytimeincreases

area(Fig.33),allowingwithinlessthanaminutetoinvestigate

degrees.SeeSec.A.3foranexampleofanSQLqueryused.

Date:5/29/2017

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure32:TheareaofcomparisonbetweenthePDACandUWS82Deep

thebackgroundtheS82sourcecountsfromtheUWcatalog.Each

thequeryarea.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure33:ResultofaboxqueryagainstPDACS82datasetwithincreasing

plotqueryareainsquaredegreesandthenumberofsources

areaagainstquerytime.Thereisascatterinquerytimecorresponding

workspeed,thuswefitthepolynomialtothemedianquerytime

trend.Thisalsoemphasizestheimportanceofallowingthe

tude:otherwisethenumberofsourcesresultingfromaquery

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure34:Comparingsourcesinaverysmallarea(1.44squarearcminutes)

andUWhostedDeepSourcecatalogs,boundedbyaPolygonwith

-0.7,325.02-0.68,325.0-0.68).Bluedotsrepresentsources

catalogs.Orangedotsrepresentsourceswithidonlypresent

log.Thesearechildsourceswithaparentatthesamelocation.

sourcesid=217571363454454701and217571363454457757,

secfromra,dec=325.008953599611,-0.6820032359336851

23.5mag,but217571363454454701istheparent,andalsohas

(deblend_nchild=5for217571363454454701),thatarefainterthan

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Table1:Comparisonofthenumberofsourcesinthesameareabetween

hostedversionsoftheDeepSourceCoaddcatalogs.Thenumber

tobeaproxyforthenumberofsources.Ifweexcludeobjects

sourcesatthesamelocation,wecomeclosertoresolvingthe

where’id’isthesameasa’parent’ofanotherobject).However,

mayormaynothaveanymatchintheUWcatalog.Thosecouldbe

sourcesthataretoofainttobeincludedinthequery(i<23.5

Area[sq.deg]N(PDAC)N(UW)N(PDAC)noparents

0.01731562583

0.04302022732350

0.161232091709533

0.36258541915419909

0.64448803307434371

1.00693745107752989

1.44988717287675506

1.9613391699007102630

5.3Lightcurvecomparison:multipleconequeries

WeassesswhethertheLomb-Scargleperiodogramcanbesafely

forthedatasimilarincadenceandsparsenesstotheS82

presetsettingsforthePeriodSearchtoolintheTimeSeries

’true’periodthebest-fitresultsfromdetailedtemplate10].Bothfitparameters

SDSSDR2lightcurvesusedin[10]arepubliclyaccessibleintheonline

WeobtaintheS82S13calibratedg-magnitudelightcurves

neousconequeryofDeepForcedSourcecatalogatthegiven

againstScienceCCDExposure,currentlyonlyavailable37,

only343of483RRLyraeareatlocationswithinthePDACS82

usingtheastroMLpythonmodule[13],wecalculatetheLomb-Scargleperiodogram

ersonauniformfrequencygridof5000frequenciesspanning??????=0.9(2??/????????),

??????=1.1(2??/????????),where??????and??????correspondto

90%ofthesmallestand110%ofthe

largestperiodsinthecatalog.Thusweareconstraining

whereweexpectthemtobe.Forperiodogramweassessthe

performing500bootstrapresamplings

17.WeusethegeneralizedLomb-Scargle

culationmode,followingthedefaultsinastroML(seealsoEq.20in[16],andSection10.3.2

[5]).

17http://www.astroml.org/book_figures/chapter10/index.html

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Table2:ComparisonofRRLyraeperiodsobtainedwithdifferent

’true’,isthe’groundtruth’-periodresultingfromdetailed10].Second

column-DR2LS,istheperiodcorrespondingtothemostprominent

ScargleperiodogramcomputedontheSDSSDR2lightcurvefrom10].Thirdcolumn-DR2

EXO,showstheperiodfoundforSDSSDR2dataof[10]withtheNASAExoplanetArchive

Periodogramservice.Fourthcolumn-PDACLS,istheperiod

periodogramonPDACS82S13data.

IDtrueDR2LSDR2EXOPDACLS

40990.6417540.2808270.641750.280827

133500.5479870.5471610.353650.547969

4709940.3467940.5316670.346790.531667

Lomb-Scargleperiodogramdoesnotalwaysfindthe’true’

sampling,aliasing,andnecessityofchoosingwellthefrequency

powersareevaluated(see[14]forarecentoverview).Wenevertheless

sanitychecks:

1.DoesthePDAClightcurvefoldedonthe’true’periodlook

2.UsingnaiveLomb-Scargle,canwefindthisperiodusing

usedby[10]?

3.UsingnaiveLomb-Scargle,canwefindthisperiodusing

WeshowthatevenwithsuchadenselysampledLomb-Scargle

thanthedefault50powerscalculatedinthePDACTimeSeries

propriatefrequencyrangeweareabletorecoverthetrue

testedstars(seeFig.36)

WesummarizetheresultsofLomb-ScarglecalculationonSDSS

wherewiththeLSwefindthesameperiod(Fig.39),asmallerperiod(Fig.40),,orabigger

period(Fig.41)thanthegroundtruth.ThisprovesthatwithSDSS

tousesolelyLSforperiodfinding.

UsingthePDACS82S13datawealsocalculateLSperiodogram

42,43and44.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure35:DistributionofRRLyraeperiodsfor483objectsin[10].Notethebimodaldis-

tribution,reflectingtwomainRRLyraetypes:309RRab(right)

Fig.16in[10]).

Figure36:Thedistributionoftheratioof???????to??????.???????=2??/?????????,isthefrequencycor-

respondingtothe’true’periodasfoundbySesar+2010(see10]).??????=2??/??

????

correspondstothehighestpeakintheLomb-Scargleperiodogram

data-SDSSDR2.???????/??????isapproximatelyequalto1,whenthenaiveLS

torecoverthe’true’period.Whenthisratioissmalleror

periodrecoveredfromtheLSmethodisrespectivelyshorter

Thismaybecausedbytheinherentsimplicityofthesimplesingle-term

Indeed,someRRLyraelightcurvesmayhaveshapesthatare

singlesinusoid(asonFig.10.18in[5]).SeeFigs.39,40and41fordetailsofevaluatingLS

SDSSDR2lightcurves.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure37:Resultsofpositionalqueryagainst483RRLyraestars10],usingtheir

RA,Dec.Bluedotsare343starsthathaveamatchinthePDACS82

sec,andredcrossesare140starsthatdidnot.Increasingthe

notalterthisresult.

Figure38:ComparisonofRRLyrID=4099from[10](redcrosses),andPDAC(bluecrosses).

Thetwolightcurveshavedifferentlength:59vs162points,

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure39:AnexampleoftheastroMLLombScargleperiodogramperformance,

forRRLyrID=13350inSDSSgband(followingTable2in[10]),usingtheSDSSdatafrom

[10].Ittook18.6milisecondsonalaptoptocalculatethisperiodogram.

depictstherawdata.Theupperrightpanelshowsthephased

the’true’periodof0.547987days(’???????’).ThelowerleftpanelshowstheLomb

odogram,wheretheorangeandmagentaverticallinesmarkthe

riodogrampeak,andthefrequencybasedonthereportedperiod???????=2??/?????????).Thelower

rightpanelshowsthephasedlightcurveconstructedwiththe

periodof0.547161days,correspondingtothehighestpeak,??????=2??/??

????.Thehorizontal

redandgreenlinesmarkthe5%and1%significancelevelsforthehighestpeak,

from500bootstrapresamplings(See).Thesameobject,butusingthePDACS82

isshownonFig.42

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure40:AnexampleofafailureofnaivesingleLombScargleperiodogram

-theratioof???????/??????=0.437InthesefourpanelsweuseSDSSDR2data

=4099from[10].Upper-leftpanel:rawdata.Upper-rightpanel:

period.Bottom-leftpanel:Lomb-Scargleperiodogramwith

panel:therawdatafoldedontheLSperiodcorrespondingto

periodfrom[10]is0.641754days,whereasthenaiveLomb-Scargle

yieldsthe’fit’periodof0.280827days.Here??????and???????significantlydifferforthis

andthe’true’frequencyappearsasonlyoneofmanyinsignificant

showthePDACS82S13datausedforthisobjectonFig.43).EverythingelseasonFig.39.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure41:SameasFig.40,usingtheSDSSDR2datafrom[10].ThisRRLyrID=470994,has

acitedperiodof0.346794days(’???????’),whereastheperiodderivedfromthe

periodogramis0.531667days(’??????’).Thus???????/??????=1.53Itmaybeagoodexampleof

aliasing.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure42:ThesameobjectasFig.39,butusingdatadownloadedusingPDAC.

data,theRRLyrID=13350hasabest-fitperiodof0.547969days,

periodof0.547987from[10].PanelsthesameasonFig.40

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure43:ThesameobjectasFig.40,butusingdatadownloadedusingPDAC.

anaiveLSperiodogramusingPDACdataforRRLyrID=4099we

quencywithhighestpower)of0.280827days,almostidentical

periodogramontheSDSS[10]dataof0.280827days.Botharediscrepant

the’true’periodof0.641754daysfrom[10].PanelsthesameasonFig.39

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure44:ThesameobjectasFig.41,butusingdatadownloadedfromPDAC.Calculating

anaiveLSperiodogramusingPDACdataforRRLyrID=470994

(frequencywithhighestpower)of0.531667days,almosttwice

0.346794daysfrom[10].Forthisstarwegetanidenticalperiodifwe

onSDSSdatafrom[10]asopposedtoPDAC.PanelsthesameasonFig.39

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

Figure45:Comparisonoftheoriginal[10]lightcurves(green)againstdata

objectspulledfromPDAC(blue).Foreachofthe383lightcurves-band,without

anypre-processingorclipping,wecalculatedthemedian,weighted??ℎ??2??????.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

6Conclusions

ThePrototypeDataAccessCenterreadilyimplementsmany

linedintheLDM-554document.Wefindthattheoverallstructure

intuitive,buildingonthestrengthsofNASA/IPACIRSA

definingthequeryregion,andresolvingnamesofobjects

data,andaddingSQLconstraintsmakesitpossibletoreduce

formation.AninnovativeTimeSeriesView,relyingonthe

oplanetArchivePeriodogramTool,hasagreatpotential

data.Easyaccesstoexternalimagecatalogsallowsoverlaying

ofwavelengths,includingaverywelldesignedRGB-making

complicatedSQLqueries,andscisqlfunctionshelptoexecute

logstoobtainneededinformation.

Thereisanumberofareaswithpossibleimprovements,including

face,aswellastheSQLqueryengine.First,certaininformation,

magnitudes,isonlyavailableviaTimeSeriesvieworadirect

interestedinaccessingthisinformationmoredirectly.

alogbyanobjectId,withouttheneedtospecifythecoordinates,

informationforaparticularobject.Searchcapabilities

andMultipleobjectqueriesaresupported.Abilitytoselect

wouldaidefficiency,avoidingtheneccessitytodisplay

desirable,measurementsorobjects.Wealsorecommendexpanding

oftheuserinterface:providingmoreexplanatorymetadata

theavailabledatatotheimageprocessingpipeline,and

usedincalculationofcalibratedfluxesandmagnitudes,

(like[14]).OntheleveloftheQservengine,errorscurrently

dationoftheerrormessagequality,anditwouldbevery

errortotheuserinteractingwithPDAC.Thiswouldmake

andstreamlinethedataaccess.

Thisreportoutlinedasystemthatisinactivedevelopment.

basedontheuserfeedbackarecapturedintheepicDM-10432.

sultingfromthisworkincludeDM-7990,DM-10477,DM-10431,

10465,DM-10466.

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

AAppendix:SQLqueries

A.1Singlesourceconequery,calibratedflux

curl−ocalib_lightcurve.json−d’query=\

SELECT\

id,fsrc.exposure_time_mid,\

scisql_dnToFlux(fsrc.flux_psf,exp.fluxMag0

scisql_dnToFluxSigma(fsrc.flux_psf,fsrc.f

exp.fluxMag0,exp.fluxMag0Sigma)ASpsfFluxEr

FROM\

RunDeepForcedSourceASfsrc,

Science_Ccd_ExposureASexp\

WHERE\

exp.scienceCcdExposureId=fsrc.exposure_id\

ANDfsrc.exposure_filter_id=1\

ANDobjectId=216172782516437336\

ORDERBYexposure_time_mid’\

http://lsst−qserv−dax01.ncsa.illinois.edu:5000/db/v0/

A.2Singlesourceconequery,calibratedmagnitude

curl−ogmag_lightcurve.json−d’query=\

SELECT\

objectId,fsrc.exposure_time_midASmjd,\

scisql_dnToAbMag(fsrc.flux_psf,exp.fluxMag0

scisql_dnToAbMagSigma(fsrc.flux_psf,fsrc.f

exp.fluxMag0,exp.fluxMag0Sigma)ASgErr\

FROM\

RunDeepForcedSourceASfsrc,\

Science_Ccd_ExposureASexp\

WHERE\

exp.scienceCcdExposureId=fsrc.exposure_id\

ANDfsrc.exposure_filter_id=1\

ANDobjectIdIN(216172782516437336)’\

http://lsst−qserv−dax01.ncsa.illinois.edu:5000/db/v0

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

A.3Boxquery,i<23.5

curl−odeepSourceBox.json−d’query=\

SELECT\

dsrc.parentASparent,dsrc.idASid,\

scisql_dnToAbMag(dsrc.flux_psf,coadd.fluxMa

scisql_dnToAbMagSigma(dsrc.flux_psf,dsrc.f

coadd.fluxMag0,coadd.fluxMag0Sigma)ASiErr\

FROM\

RunDeepSourceASdsrc,\

DeepCoaddAScoadd\

WHERE\

qserv_areaspec_box(325,−0.7,325.1,−0.6)\

ANDdsrc.coadd_id=coadd.deepCoaddId\

ANDdsrc.coadd_filter_id=3\

HAVINGi<23.5’\

http://lsst−qserv−dax01.ncsa.illinois.edu:5000/db/

References

[1][LDM-153],Becla,J.,2013,LSSTDatabaseBaselineSchema,LDM-153,URLhttps://ls.st/

LDM-153

[2][LDM-554],Ciardi,D.,Dubois-Felsmann,G.,2017,SciencePlatformRequirements,LDM-

554,URLhttps://ls.st/LDM-554

[3][LDM-492],Ciardi,D.R.,Wu,X.,Dubois-Felsmann,G.,2016,AVisionfortheScience

InterfaceandTools,LDM-492,URLhttps://ls.st/LDM-492

[4]Coster,A.,Pankratius,V.,Lind,F.,Erickson,P.,Semeter,

27thInternationalTechnicalMeetingofTheSatellite

gation(IONGNSS+2014),1213–1221,URLhttps://www.ion.org/publications/abstract.

cfm?articleID=12273

[5]Ivezić,Ž.,Connolly,A.J.,VanderPlas,J.T.,Gray,Statistics,DataMining,

chineLearninginAstronomy,PrincetonUniversityPress

[6][DMTN-035],Jurić,M.,Becker,A.,Shaw,R.,Krughoff,K.S.,Winter2013

LARGESYNOPTICSURVEYTELESCOPE

PDACreportDMTR-22LatestRevision2017-07-11

LSSTDMDataChallengeReleaseNotes,DMTN-035,URLhttps://dmtn-035.lsst.io

LSSTDataManagementTechnicalNote

[7][LSE-319],Jurić,M.,Ciardi,D.,Dubois-Felsmann,G.,2017,LSSTSciencePlatform

Document,LSE-319,URLhttps://ls.st/LSE-319

[8][Document-15097],Lim,K.T.,2013,LSSTDataChallengeReport:Summer,Document-

15097,URLhttps://ls.st/Document-15097

[9]Pankratius,V.,Li,J.,Gowanlock,M.,etal.,2016,

doi:10.1109/MIS.2016.60

[10]Sesar,B.,Ivezić,Ž.,Grammer,S.H.,etal.,2010,arXiv:0910.4611),

doi:10.1088/0004-637X/708/1/717,ADSLink

[11][LDM-226],Shaw,R.A.,Jurić,M.,Becker,A.,etal.,2013,LSSTDataChallengeReport:

mer2012/early-Winter2013,LDM-226,URLhttps://ls.st/LDM-226

[12][LDM-130],Unknown,2017,LSSTScienceUserInterfaceandToolsRequirements,LDM-130,

URLhttps://ls.st/LDM-130

[13]VanderPlas,J.,Connolly,A.J.,Ivezić,Ž.,Gray,A.,

DataUnderstanding,47–54,doi:10.1109/CIDU.2012.6382200

[14]VanderPlas,J.T.,2017,ArXive-prints(arXiv:1703.09824),ADSLink

[15]VanderPlas,J.T.,Ivezić,Ž.,2015,ApJ,812,18(arXiv:1502.01344),doi:10.1088/0004-

637X/812/1/18,ADSLink

[16]Zechmeister,M.,Kürster,M.,2009,A&A,496,577(arXiv:0901.2573),doi:10.1051/0004-

6361:200811296,ADSLink