Injected data and earth pointing

[dawsonri]

Tom:
Hi team,
I’ve been plating with injecting and recovering 1-year orbital period planets. I started using a dumb median filter but got stuck with detections always occurring after earth points through faulty filtering at data gaps.
I then tried using Gal’s detrender and am finding it causes massive spikes after every earth point. What am I doing wrong? I’ve attached a plot showing the a dumb median filter in red and Gal’s one in blue. I tried playing with the window size with little success.

By the way, the data files contain a column called SAP_QUALITY. The values in this column mean things, ie. there is a numerical value for ‘earth-point just happened’. I’ve started trying just throwing away the data for two days after every earth-point.

Cheers,
Tom

Bekki:
We're working on various approaches to avoid detections at data gaps. For example, what I'm doing now is:

1. Identify change points based on time gaps and/or big flux jumps
2. Detrend each segment between change points separately using running median filter with 3-day width
3. Delete data 3 days before and after each gap
4. Use wavelets to interpolate between gaps
   And I think others are working on different things. But basically for now most of us are sacrificing data near gaps.

Ruth is working with Gal's median filter too and I think she's implementing something to try to address this issue.

Tom:
Thanks for the update Bekki. I too am finding that sacrificing data near gaps is a necessary evil.
Why do you remove data before the gaps? These data shouldn’t be affected by the nasty ‘thermal ramps’, the spacecraft doesn’t know its about to do an Earth-point.
(I guess we should probably be having this conversation on Gibhub, oops)

Hope all is well there with you guys! It’s rainy here in SF!! in June!!

[dawsonri]

I'm removing the data before the gaps for a different reason: the running medium filter needs to stop before n_elements(segment)-wide/2
or else there isn't sufficient data on both sides of a given data point

Actually, as I write this out, I realize that the data before the gap only needs to be deleted 1.5 days before the gap.