Add sndata as affiliate package

[djperrefort]

SNData provides access to data releases published by a variety of supernova (SN) surveys. It is designed to support the development of scalable analysis pipelines that translate with minimal effort between and across data sets. This SEP proposes to make SNData an SNCosmo affiliate package and is motivated by previous work undertaken by Kyle Barbary on the sndatasets package.

[kbarbary]

I'm in favor of this 👍, but would like to hear any comments or concerns from others. After all, part of the benefit of moving to the sncosmo organization would be making it easier for a broader group to contribute.

[rbiswas4]

After @djperrefort talked about it at one of the SN meetings I checked it out.

• It is indeed nicely written and was easy to access the datasets advertized.
• I don't know much about scalability, since the datasets are not that huge etc., but even if that needs further development, the work of obtaining the data from servers is complementary.
• The main question I have is about OSC : While samples curated by surveys as accessed by sndata have their own importance, having data of the size of OSC (unfortunately with selection biases etc.) is also useful. We have a parallel proposal that will soon come on on EPS: (add all filter transmission functions from SVO sncosmo#268), which adds a multitude of instrument transmissions . @djperrefort , do you think this will make OSC easier to access from? Once this happens, can there be an additional method in sndata that achieves this?

[djperrefort]

@rbiswas4 I’ve thought about adding OSC before but initially didn’t because there were unfinished features it would have to build on. Fortunately, adding OSC now should be relatively straight forward. I envision adding a sndata.osc.OSCPhot and sndata.osc.OSCSpec class. Each of these would extend the base data parsing classes and could access object data in one of two ways (we will have to pick one to implement):

Our first option is to follow the current sndata design and have the download_module_data function download data from the OSC before users can access it. Downloading the entire OSC is probably not desired, so we could let the user specify what data to download. I’m not sure what the ideal implementation of this is.

Our second option is to have the get_data_for_id function fetch data directly from the OSC and cache it to local files as it goes. This would provide a gradual improvement in runtime and requires less work for the user. I am partial to this approach.

Adding additional filters would definitely help. One thing to note is that sndata follows a specific naming scheme for the filters used in each data release. For example, the CSP DR3 u band is called csp_dr3_u. This was done to account for different data releases using different versions of the same filters (e.g. SDSS has three that are used in the literature: The column averaged, column specific, and original manufacturer measurements). As part of this EP we should add these filters to sncosmo, but I’m not sure if the naming scheme should be maintained. @kbarbary might have input on this.

[kbarbary]

Sounds like whether to add OSC or not may be beyond the scope of this proposal (whether to move sndata under the sncosmo github organization). However, it sounds like the potential interplay between sncosmo filters and sndata datasets may be another good reason for the move.

[kbarbary]

Let's give folks another few days to comment on this and finalize it by the end of the week if nothing major comes up.

[kbarbary]

Want to add an accepted date and also add it to the table of accepted proposals in the top-level README?

[djperrefort]

@kbarbary I've updated the dates and status.