WIP/ENH: optimize.minimize_scalar: add method 'chandrupatla'?

[mdhaber]

Reference issue

gh-18829

What does this implement/fix?

This investigates what it would look like to add method='chandrupatla', an elementwise scalar minimizer slated for array API support, to minimize_scalar. I haven't updated all the documentation, because I'm not sure this is what we want to do.

import numpy as np
from scipy import stats
from scipy.optimize import minimize_scalar
from scipy.optimize._bracket import _bracket_minimum

# generate array of locations and scales for a normal distribution
rng = np.random.default_rng(435982409215)
m, n = 10, 10
loc = rng.normal(size=m)[:, np.newaxis]
scale = rng.uniform(size=n)
shape = np.broadcast_shapes(loc.shape, scale.shape)

# define the objective function for finding the mode - the negative of the pdf
def f(x, loc, scale):
    return -stats.norm(loc=loc, scale=scale).pdf(x)

# find the bracket
args = (loc, scale)
res = _bracket_minimum(f, xm0=np.zeros(shape), args=args)
bracket = res.xl, res.xm, res.xr

# find the minimizer
res = minimize_scalar(f, bracket, method='chandrupatla', args=args)

# the minimize should coincide with `loc`
loc = np.broadcast_to(loc, shape)
np.testing.assert_allclose(res.x, loc, atol=1e-7)

Additional information

The nice thing is that it could work. The downsides include:

• Method bounded accepts bounds but not bracket, and the other methods accept bracket but not bounds. I don't want to shove another method into this function if it's already cramming two interfaces into one function.
• Methods brent and golden don't require a three-point bracket, but I think chandrupatla should. It could attempt to find a bracket automatically, but I've seen a lot of confusion about brackets in the past (e.g. MAINT: optimize.bracket: don't fail silently #17704, DOC: optimize.minimize_scalar and friends: correct documentation about bracket #17705, and linked issues) and I think this could be resolved if we require users to find a bracket as a separate step. After all, we require them to find a two-point rootfinding bracket if they want to use a bracketing rootfinder. Also, if they can't find a bracket, they know they'll need to use a derivative-based rootfinder (e.g. via minimize, currently).
• The other methods support a disp option. I'm not going to put that into a new function.
• chandrupatal supports a callback; the other methods don't.
• The keyword tol is ambiguous. chandrupatla accepts four separate tolerances.
• chandrupatla does not provide a message attribute like the others because string messages are unweildy for an elementwise function. (It provides an integer status, instead.)
• chandrupatla works elementwise, and the others don't. Describing the differences would complicate the documentation a bit.

Should we continue with this or create a new function? Based on #7242 (comment) it already looks like we'll need a new function for elementwise rootfinding, so maybe it makes sense to have a new elementwise minimizer, too.

[mdhaber]

We discussed separately that this should go in a new function, too. I'll open a PR in a bit.