[DOC]: Fix compatibility with sphinx-gallery 0.16

[NirobNabil]

PR summary

I was following a thread on the same issue in sphinx-gallery . It seems every callables and classes need to be stringified. There is some work being done on sphinx-gallery side at 1289. But even with this PR the build fails, My assumption is that its because of the callable in subsection_order key which is not being handled on sphinx_gallery side. I was able to get the error for subsection_order fixed but had to modify some code within sphinx_gallery. I will ask for the subsection_order fix in that sphinx_gallery PR. but other than that i was able to build the doc without the not pickleable warning (using the small change i made in sphinx_gallery code for subsection_order) following the conf.py changes introduced at that PR.

PR checklist

• [N/A] "closes #0000" is in the body of the PR description to link the related issue
• [N/A] new and changed code is tested
• [N/A] Plotting related features are demonstrated in an example
• [N/A] New Features and API Changes are noted with a directive and release note
• Documentation complies with general and docstring guidelines

[tacaswell]

You will probably also have to un-pin sphinx to see if this works.

[tacaswell]

Traceback (most recent call last):
  File "/home/circleci/.local/lib/python3.9/site-packages/sphinx_gallery/gen_gallery.py", line 261, in _fill_gallery_conf_defaults
    scraper = import_module(scraper)
  File "/home/circleci/.pyenv/versions/3.9.19/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 981, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'sphinxext.util.matplotlib_reduced_latex_scraper'; 'sphinxext.util' is not a package

No module named 'sphinxext.util.matplotlib_reduced_latex_scraper'; 'sphinxext.util' is not a package

Configuration error:
Unknown image scraper 'sphinxext.util.matplotlib_reduced_latex_scraper', got:
No module named 'sphinxext.util.matplotlib_reduced_latex_scraper'; 'sphinxext.util' is not a package

This looks like it is trying to import the function as a module (not import the (sub) module sphinxext.utils and then get a function out of it.

In the removed code in that PR it looks like it used to always look for a particular name: https://github.com/sphinx-gallery/sphinx-gallery/pull/1289/files#diff-016e2fd897b8720412d190b7936fb295e31dc953c73fcb912ca1362ec8ca2191L261-L264

---

I think if you want to test mpl with that pr you need to:

• unpin sphinx (undoing DOC: exclude sphinx 7.3.* #28094 )
• install sphinx-gallery from the branch that PR is from

[NirobNabil]

I think if you want to test mpl with that pr you need to:

* unpin sphinx (undoing [DOC: exclude sphinx 7.3.* #28094](https://github.com/matplotlib/matplotlib/pull/28094) )

* install sphinx-gallery from the branch that PR is from

I did exactly that. Installed sphinx manually from the released source code at v7.3.6 and built sphinx_gallery by cloning from branch at larsoner's fork (which is the source of PR)

This is how i verified the versions and generated build logs

(matplotlib) nabil@pop-os:/media/nabil/workspace3/projects/gsoc24/matplotlib/doc$ python
Python 3.12.2 | packaged by Anaconda, Inc. | (main, Feb 27 2024, 17:35:02) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sphinx, sphinx_gallery
>>> sphinx.__version__
'7.3.6'
>>> sphinx_gallery.__path__
['/media/nabil/workspace3/projects/gsoc24/sphinx-gallery/sphinx_gallery']
>>> exit()
(matplotlib) nabil@pop-os:/media/nabil/workspace3/projects/gsoc24/matplotlib/doc$ make html > out 2>&1

The generated build logs is at out.txt

Note that i made some changes in the sphinx_gallery code

• changed line 447 in gen_gallery to

  sortkey = _get_class(gallery_conf, "subsection_order")

• and changed the _get_class function in gen_rst at line 1503 to

  if not inspect.isclass(val) and not hasattr(val, '__class__'):
          raise ConfigError(
              f"{what} must be 1) a fully qualified name (string) that resolves "
              f"to a class or 2) or a class itself 3) or an instance of a class,"
              f" got {val.__class__.__name__} ({repr(val)})"
          )

  so that it passes with both a class definition or an instance of a class

[tacaswell]

Sorry, I meant to do that in the CI configuration so that we can see CI run clean!

[NirobNabil]

I think documentation is building alright in current configuration. @tacaswell can you please check if its alright. I also couldn't figure out whats wrong with the two failed pytest steps or if these are even relevant.

[tacaswell]

You can put this detail is the requirements files (which means it will hit all of the CI that pull it and any developers who try to build locally will also get it.

[timhoffm]

On a general note: It's ok to use a non-released version for developing this fix. But this should not go into our main branch. We should wait with merging until an update of sphinx-gallery is released, and if necessary pin sphinx in the mean time.

[NirobNabil]

You can put this detail is the requirements files (which means it will hit all of the CI that pull it and any developers who try to build locally will also get it.

moved sphinx-gallery installation to doc-requirements and all the checks passed :D

[QuLogic]

I was looking into this since sphinx-gallery 0.16.0 is out now, and started working on it before I found this PR. The older version doesn't support using the strings AFAICT, so I'm going to push a change here that works for both.

[QuLogic]

I'm going to mark this as ready; it looks good to me, though I'm biased since I made some of the changes here.

[lumberbot-app]

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

git checkout v3.8.x
git pull

2. Cherry pick the first parent branch of the this PR on top of the older branch:

git cherry-pick -x -m1 a4243d9557da23d9a409b28695394704cf3cd6c9

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

git commit -am 'Backport PR #28103: [DOC]: Fix compatibility with sphinx-gallery 0.16'

4. Push to a named branch:

git push YOURFORK v3.8.x:auto-backport-of-pr-28103-on-v3.8.x

5. Create a PR against branch v3.8.x, I would have named this PR:

"Backport PR #28103 on branch v3.8.x ([DOC]: Fix compatibility with sphinx-gallery 0.16)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

[lumberbot-app]

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

git checkout v3.8.4-doc
git pull

2. Cherry pick the first parent branch of the this PR on top of the older branch:

git cherry-pick -x -m1 a4243d9557da23d9a409b28695394704cf3cd6c9

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

git commit -am 'Backport PR #28103: [DOC]: Fix compatibility with sphinx-gallery 0.16'

4. Push to a named branch:

git push YOURFORK v3.8.4-doc:auto-backport-of-pr-28103-on-v3.8.4-doc

5. Create a PR against branch v3.8.4-doc, I would have named this PR:

"Backport PR #28103 on branch v3.8.4-doc ([DOC]: Fix compatibility with sphinx-gallery 0.16)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

[ksunden]

Not going to worry about a manual backport for now

[ksunden]

@NirobNabil Thanks for this, congrats on your first PR merged, hope to hear from you again!

[NirobNabil]

@ksunden Sorry i was inactive for some days. Thanks for fixing it up and the merge and will surely be contributing more.
I also had another PR waiting for review for 2 months. Any update on that too will be really appreciated.

[story645]

Any update on that too will be really appreciated.

😬 sorry about that, feel free to ping me privately if I owe you a review.