Save messages sent by bas.lm

[AlexanderLyNL]

Hi Merlise,

Sometimes Bas.lm prints messages such as

"There are 3 degenerate sampling probabilities (0 or 1); decreasing the number of models to 16"

or returns a warning

“Warning message: In BAS::bas.lm(simpleFormula, data = d, alpha = 0.125316, prior = "JZS",  : log marginals and posterior probabilities contain NA's.  Consider using `pivot=TRUE` if there are models that are not full rank”

Would it be possible to save these messages in the object? We can then print these messages as a footnote in JASP.

Cheers,
Alexander

[merliseclyde]

Hi Alexander,

The first could probably be eliminated or suppressed (particularly if variables are forced in) and added to the object/summary as described to distinguish between the case of estimates that are 1 or 0 and those that are forced to be 1 or 0.

For the second, I would like to have a general non-JASP user know that there are non-full rank models with the current default settings (pivot=FALSE). On the other hand, lm merrily reports output from non-full rank models without any warnings, but does issue warnings when using the objects for prediction with new data so at some point a warning should be issued. It would be helpful to have the warning or a flag that there are non-full rank models added to the output for perhaps printing the bas object or with summary. I'll need to add error checking for NA's in some of the subsequent code (i.e. plot, image, fitted, predict) that use the posterior probabilities. Since so much of the latter would fail, the early warning would be helpful for the general user.

For the JASP case, how would you use the output if there are non-full rank models that lead to NA's in probabilities with pivot=FALSE? Or is the goal to suppress warnings when running via JASP, but have a way to catch this if you use the default setting and rerun with pivot=TRUE?

I have not done any testing to see if the pivot option is significantly slower than the default yet (the code could be cleaned up to reduce some memory allocation which could speed it up)

In any case, having non-full rank models needs more thought with BMA due to dilution of posterior mass...

[AlexanderLyNL]

Hi Merlise,

I agree that messages and warnings should be printed in general. I’m not sure whether we should show these messages to JASP users, but saving them provides us an option to do so eventually.

Regarding pivoting, this is a bit of a mystery to me :) I tried to look up what pivot=TRUE does exactly, but I couldn’t figure it out. I see that pivoting relates to Gaussian elimination, is this what the code does? I’m not sure whether we should tell a JASP user to activate pivoting. Or that we should do this automatically for them. Do you have a preference?

Cheers,
Alexander

[merliseclyde]

Thanks; I'll try to see about adding messages to the output so that they can be printed later if they would be useful.

Yes; Gaussian elimination with a symmetric matrix leads to the Cholesky decomposition. Pivoting of the rows is used to improve stability with nearly singular matrices and it stops if the diagonal elements are 0 (singular).

R by default uses pivoting in lm so BAS could default to using pivoting. I would like to test more - both timing and results - before releasing into the wild. However, since all examples/vignettes were full rank before (as well as examples in Coursera) changing the default should not break earlier code in principle :-)

[merliseclyde]

dropped the warning message for cases with degenerate sampling probabilities

"There are 3 degenerate sampling probabilities (0 or 1); decreasing the number of models to 16"

[merliseclyde]

Some messages were being printed as information (more for me in some cases) using print or Rprintf from C. I am trying to eliminate all of those and leave the rest as warnings() or errors that the user should know about. (Let me know if you come across any that are printed when warnings are suppressed! The warnings could be captured for use in JASP after running the function I think.

Are you aware of other functions that save warnings back to the object? (would be helpful to see how they did it)

[AlexanderLyNL]

For the objects I make, I return NaN and add a string. No idea how other people do this :)