Add support for Python 3.13 #8198
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix a segfault when we cannot allocate PyConfig structure due to unsupported python version.
Add our copy of PyConfig structure definition for python 3.13, and hook it into our abstraction API layer.
Starting with python 3.13, the global function calls (e.g., `foo()`) opcode sequence contains a `LOAD_NAME` followed by a `PUSH_NULL` opcode. For a bit of historical context: prior to python 3.11, only `LOAD_NAME` was emitted; with python 3.11, this changed to a `PUSH_NULL` followed by the `LOAD_NAME`. However, our regex for bytecode matching simply got away without matching this preceding `PUSH_NULL`. Now that the `PUSH_NULL` is emitted after the `LOAD_NAME`, we do need to somehow deal with it. However, instead of trying to explicitly deal with the `PUSH_NULL` opcodes (i.e., adjusting the regex and fixing the post-processing), we can simply filter them out.
`lib2to3` was removed in python 3.13.
It looks like the re-introduced functional API from importlib-resources 6.4.0 was also back-ported to python stdlib in 3.13b1, but the sub-module that we use to detect presence of this API was renamed from `functional` to `_functional`. Check for both names, as importlib-resources also seems to be planning to rename that module.
Looks like python 3.13 added two new attributes, `__firstlineno__` and `__static_attributes__` that needs to be accounted for by `assertNoMethods` and `assertHasExactMethods` helper methods, used by `TestNode.test_subclasses` in `test_modulegraph.py`. While we are at it, use `dict.pop()` to remove the elements insead of checking for their presence and removing them via `del`. Also move the clean-up part into common shared helper function.
It looks like that with python 3.13, the increase in number of opened file handles in `test_pipe_leakage` on Windows is down from six to three. Spawning the child process now seems to open only one handle instead of two. Opening python file objects on top of the file descriptors (that are in turn opened on top of the two pipe handles) does not seem to open any additional handles anymore.
Implement support for python builts with PEP703 free-threading enabled,
i.e., built with `--disable-gil` option (`Py_GIL_DISABLED`).
On build side, this involves detecting such a build, which is done via
`sysconfig.get_config_var('Py_GIL_DISABLED')`. Add an equivalent
`compat.is_nogil` flag to cache the obtained value.
In free-threading-enabled build, the python shared library has "t"
ABI suffix, so adjust `compat.PYDYLIB_NAMES` accordingly. We strictly
enforce this suffix (or the lack thereof) to prevent accidental mix-up
of shared libraries when multiple different builds can be discovered
in the library search path.
If free-threading is enabled, we add `pyi-python-flag Py_GIL_DISABLED`
run-time OPTION to PKG archive, and have bootloader parse this
option to recognize that free-threading is enabled.
On bootloader side, we need to keep two variants of `PyConfig` structure
for python 3.13; one with `enable_gil` field (`Py_GIL_DISABLED defined`)
and one without it (`Py_GIL_DISABLED` not defined). Unfortunately, this
field was added in the middle of the config structure, and we need to
access fields that come after it, so the layout matters. (Although
strictly speaking, this applies only to 32-bit builds; on 64-bit
plaftorms, presence/absence of this field makes no difference to the
layout of subsequent fields, due to structure alignment rules).
Based on presence or absence of afore mentioned OPTION passed from our
build process, the bootloader selects the correct `PyConfig` layout
to use. We do no need to interact with the `enable_gil` field; if
user wants to force-enable or force-disable GIL (the equivalent of
python's `-X gil=0` and `-X gil=1`, or `PYTHON_GIL` environment
variable), they can do so via `--python-option`, which already
supports pass-through for X-options. For the sake of completeness,
add an example to Specifying Python Interpreter Options documentation
section.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A glimpse into the horrors that near-future holds...