-
Notifications
You must be signed in to change notification settings - Fork 547
Inclusion of PRIMA solvers in Skit-Optimize #1182
Comments
Dear @zaikunzhang , As a user of this package I think the Let me know if you are interested to try this out. I have implemented genetic algorithm recently to compare against lbfgs. |
Hi Romain @Deathn0t , The solvers in PRIMA can solve black-box optimization problems without requiring derivatives (first-order information). They are quite capable in solving hyper-parameter tuning problems. So they are good solvers to be included in DeepHyper.
Sure. Whether it will outperform lbfgs or not, it depends on the properties of the problem. If your problem is a black-box optimization problem, the PRIMA solvers are the state of the art --- they definitely outperform genetic algorithms. However, it your problem can provide first-order information, then gradient-based algorithms will perform better. Thanks, |
Thanks for your reply @zaikunzhang . Also, I have the following questions:
Thanks, |
Hi,
Yes, you can wait for libprima/prima#52 .
No.
Not directly. Maybe you can implement that using a callback function, but I am not sure.
Since PRIMA solvers are intrinsically sequential (except for the initialization), I guess the answer is no. However, I do not think calling PRIMA will involve Python for loops --- it will be a single line of Python code, behind which is the Fortran code. Thank you. Zaikun |
Thank you for the detailed replies @zaikunzhang ! |
Dear Scikit-Optimize maintainers,
This is Dr. Zaikun Zhang from The Hong Kong Polytechnic University. Together with Professor N.I.M. Gould, I am responsible for maintaining the renowned derivative-free optimization solvers of the late Professor M.J.D. Powell, namely COBYLA, UOBYQA, NEWUOA, BOBYQA, and LINCOA. I am the author of PRIMA, which provides the reference implementation for these solvers. They are widely used by engineers and scientists. For instance, see Section 1 of a recent paper on Powell's solvers as well as the Google searches of COBYLA and BOBYQA.
Since your package is exactly oriented to derivative-free / black-box optimization, it might be desirable to include the PRIMA solvers. I will be happy to assist on the Fortran side if you would like to do so.
Note that, even though the old Fortran 77 implementation of the aforementioned solvers is truly a masterpiece, it contains many bugs (mostly due to the language itself), which can lead to segmentation faults or infinite loops. For example, see Section 4.4 of the above paper and many GitHub issues. It is strongly discouraged to use the Fortran 77 version of these solvers anymore.
Thanks and regards,
Zaikun ZHANG
Ph.D. and Assistant Professor
Dept. App. Math., Hong Kong Polytechnic University
The text was updated successfully, but these errors were encountered: