Module 1, Lesson 1 --> Presents an oversimplification of Stargazing Live / Exoplanet Explorers

[jessie-dotson]

I love that we're using the Stargazing Live / Exoplanet Explorers example as a way to demonstrate the power of open science. It's a fantastic example about the power of open science / citizen science! But the way it is currently described in the training material isn't technically accurate...

This experiment was a grand success that failed to answer it's key questions... The questions the team were trying to address were statistical demographics questions (as are listed in the current online training material). The team worked hard to design an experiment that was appropriate to address the demographics questions -- including injecting known signals into the dataset so they'd have a control sample that could be used to calibrate the detections from the citizen scientists. The citizen scientists were fantastic at identifying planets in the data -- and indeed identified a lot of planets (including a really awesome multi-planet system). Unfortunately the results were not appropriate for statistical demographics studies. In order to do demographics studies, you need to measure and balance completeness and reliability. If your sample is complete, but unreliable or if it's incomplete but reliable -- you can't get solid statistics out. As it turns out, the citizen scientists were very complete (found everything) and very unreliable (including things that weren't real) -- so the team wasn't able to answer the demographics questions they started with.

So -- how to make the training material more accurate? On one hand it'd be cool to have the time to get into this level of detail about designing and executing a citizen science project -- but that's out of scope for this training material. Here's some suggested changes to keep the example, make it technically correct, and not dig deep into the details...

• delete the three questions
• modify the second sentence after the Kepler graphic: "The hosts invited viewers to contribute to research by classifying solar systems..."
• modify the second sentence in the next paragraph: "On the second night, enough people had participated that the reserachers were able to share the planet candidates that had already been flagged and were undergoing additional analysis included 44 Jupiter-size planets, 72 Neptune-size planets, ..."

Perhaps this issue is only something that someone who works in this field would care about -- but it seemed worth documenting and suggesting a change...

[bressler95tops]

Thank you for submitting this suggestion @jessie-dotson. As we mentioned in our talk, we are actively working on a review process for OS101 and we should be reviewing these in the coming weeks. Stay tuned and feel free to keep submitting more suggestions in the meantime.

[bressler95tops]

Noting that we are tabling this for the next disposition.