UltraNest support

[JohannesBuchner]

Would you be interested to add support for https://johannesbuchner.github.io/UltraNest/ ?
The interface should be very similar to pymultinest.

UltraNest is a very reliable tuning-parameter-free algorithm. I have published some examples where the UltraNest algorithm is unbiased while multinest's algorithm (implemented in pymultinest,nestle,dynesty) gives a different answer. UltraNest is a pure-python package and very easy to install with pip or conda.

UltraNest also supports resuming from disk and MPI parallelisation, if that is useful to you.

[sibirrer]

Hi @JohannesBuchner ! I am very interested to have UltraNest be supported! MPI and resuming from disk is a much appreciated (and needed) part for complex sampling. Most users currently are using emcee, but if UltraNest becomes available, that might change as well. Do you have any question for how to integrate the sampler? I am happy to also slightly adopt the design to make it happen. In general you are very free as long as a likelihood_module is provided in the class instance. Let me know if you have questions and thank you very much!

[JohannesBuchner]

Sorry, I will not have time in the next two months, but am happy to answer questions if you have a go at it yourself.

[sibirrer]

I see. So I leave this issue open and see whether someone else wants to make use of UltraNest in their application. Thanks for the pointer!

[ewoudwempe]

I added support for Ultranest some time ago (along with various other changes to the nested sampling code), but have not had the time to clean it up and submit it... The performance in terms of samples/second was quite bad compared to other samplers however, and I was unable to fix that, at the time I thought it had to do with the many bootstrap likelihood calls that are calculated (for clustering I think). I think it also had some issues with the plateaus that lenstronomy returns - things like if unphysical_realization: return -1e15 did not play well with Ultranest. Errors on the evidence are more realistic than the other packages say though, so that's good. Also, this was a last year, (v2.7.x I think), so the more recent changes may improve things quite a bit.

[sibirrer]

thanks @ewoudwempe for the info (and indeed I should have remembered you have this on your fork). Nested sampling is often more time-consuming but comes with the evidence, and possibly better mapping of complex posteriors in high dimensions. @ajshajib used nested sampling (not UltraNest) quite extensively 1.5 years ago. If the large punishing factors are generally a problem for nested sampling, we could make a fix there by smoothly (but still sharply) truncate the likelihood at the edges. @ajshajib might have done something like this, so I ping him in this issue. Perhaps he has an idea too.

[JohannesBuchner]

I'd like to know more why plateaus were problematic, this is a bit surprising to me. It is intended to work the way you say, if you cannot put the constraints in the prior.

There were some improvements since 2.7, so it may be worth trying again. In case you have many parameters, you may want to use a stepsampler.

I am not sure if there can be speed gains in this package from handling multiple proposed samples at once (vectorization)?