BIFL scatter fit - incorrect scatter estimation?

[AndersBarth]

Hi Thomas,

when specifying a background curve and using the automatic estimation of background and fluorescence photons from the measurement durations, we noticed that we could not get a correct fit. To understand how things are implemented, I looked into the code and noticed that the model decay, before being multiplied by the estimated photon number, is not normalized to an area of 1.
Here are the respective lines in models/tcspc/lifetime.py

        # Calculate background curve from reference measurement
        if isinstance(
                background_curve,
                chisurf.curve.Curve
        ):
            if shift_bg_with_irf:
                background_curve = background_curve << self.convolve.timeshift

            bg_y = np.copy(background_curve.y)
            bg_y /= bg_y.sum()
            bg_y *= self.generic.n_ph_bg

            decay *= self.generic.n_ph_fl
            decay += bg_y

It turns out that the sum of the decay is > 1.

Is this intended behavior? Otherwise, I would suggest to normalize the decay to its sum before multiplying with the photon number, as done for the background pattern.

[AndersBarth]

Another minor bug: The IRF display disappears when a background curve is loaded.

[tpeulen]

Thank you a lot for opening this issue. I am aware that there is a problem. It makes much more sense to normalize the IRF. There is a version from 2013/14 where the BIFL scatter fit is implemented one would expect.

However,

1. The problem is/was that people got too attached to a scatter amplitude as a prefactor of an unnormalized IRF (that gets added to the model function, h(t) = f(t) + sc * IRF).

2. The used (fit23) BIFL scatter fit in fact does not really fix the scatter fraction (completly), as your estimates of the scatter fractions are always (slightly) off. Thus, the BIFL scatter fit optimizes using an fixed inital scatter fraction estimate all model parameters. Next the optimized parameters are used in a second fit where the scatter fraction is free. Not everyone may be aware of that. To find the actual used BIFL scatter implementation you would have to look at the code (see https://github.com/Fluorescence-Tools/fit2x/blob/master/src/fits2x.cpp)

In future releases I will change the meaning of "sc" to a scatter fraction as oposed to an prefactor.

[AndersBarth]

Thanks for the quick reply!
Just to be clear then: The automatic scatter estimate using a supplied reference background curve and measurement duration is then not working as intended and might give wrong results? Would my above suggestion work to fix this?