feat: add support for free-threaded (no-gil) Python 3.13 #1831

mayeut · 2024-05-18T17:56:49Z

Support has been added for Linux & Windows awaiting macOS installers (IMHO, not blocking to ship this experimental feature)

Users have to opt-in this feature using the new free-threaded-support option of the config file (probably preferable over the environment variable which is supported as well).

henryiii · 2024-05-19T03:02:33Z

Nice! Are we doing to go ahead and ship experimental support for Linux & Windows early, or wait on macOS builds with beta 2? I also think macOS might be tricky, as I'm not what it will look like installing both, I guess they aren't supposed to collide? (Maybe that's why it's taking till beta 2!)

cibuildwheel/windows.py

mayeut · 2024-05-19T08:30:48Z

Are we doing to go ahead and ship experimental support for Linux & Windows early, or wait on macOS builds with beta 2? I also think macOS might be tricky, as I'm not what it will look like installing both, I guess they aren't supposed to collide? (Maybe that's why it's taking till beta 2!)

I think we should ship experimental support without waiting on macOS.
The only thing that's missing is the opt-in mechanism & I think it should ship with the first version allowing free threaded builds (c.f. #1657 (comment))

As for how the macOS build will turn out, the only info I found in the cpython repo was python/cpython#112430 (comment) (didn't find anything on discourse, there might be some other threads out there).
If I understand correctly, it will be an installation option. This probably means we'll have to rework the macOS bit to force a re-install (only in CI of course) if the free-threaded build is missing in an existing installation. But it might get trickier:

Ideally, we'd want to allow both variants to be able to to be installed side-by-side; while that should mostly work today, there are some loose ends that would need to be taken care of involving path names and the like. If it turns out to be too disruptive to support co-existing installs, the installer package could simply inhibit the installation of the normal variant if the user selects the optional free-threaded variant.

mayeut · 2024-05-19T12:06:15Z

cibuildwheel/windows.py

@@ -519,6 +524,10 @@ def build(options: Options, tmp_path: Path) -> None:
                # check that we are using the Python from the virtual environment
                call("where", "python", env=virtualenv_env)

+                # TODO remove me once virtualenv provides pip>=24.1b1
+                if config.version.startswith("3.13."):
+                    call("python", "-m", "pip", "install", "--pre", "-U", "pip", env=virtualenv_env)


This does not work with call("python", "-m", "pip", "install", "pip>=24.1b1", env=virtualenv_env)
Maybe something to do with shell=True being used on Windows ?

henryiii · 2024-05-20T06:36:57Z

What if we swap the default, at least while it’s under a beta flag, then decide if we keep it opt-out instead of opt-in when the RC drops?

henryiii · 2024-05-20T06:39:06Z

@ned-deily, will it be possible to install both variants at the same time on macOS like on windows/linux?

henryiii · 2024-05-20T10:42:14Z

Should we call this 2.18.1 or 2.19.0? It's experimental and opt-in, so I'm slightly biased toward the first. Especially since we don't have macOS yet.

mayeut · 2024-05-20T10:51:14Z

with #1792, we might want to go with 2.19.0 anyway ?

henryiii · 2024-05-20T11:00:05Z

Yes, possibly. Though I think that's also only opt-in, something that was disallowed before is now allowed, but existing builds don't change.

It's sad that #1456 isn't quite ready (even if it was finished, still waiting on the pyodide release, which will likely be delayed I'd expect with some wheel tag changes planned - we were working on it at PyCon). GraalPy might not be too far away, etc. Minor version bumps require everyone to update their two-digit pins. In that sense, I'm not sure this is a minor bump, as nothing changed or is added without user changes.

mayeut · 2024-05-20T11:08:40Z

I'm not sure this is a minor bump, as nothing changed or is added without user changes.

Let's go with 2.18.1 then ! Maybe being too pedantic regarding semver is not a good idea here.

GraalPy might not be too far away

This one is waiting at least on pip 24.1

I haven't looked in the pyodide one in a long time but saw there were some commits.

henryiii · 2024-05-20T11:11:16Z

This one is waiting at least on pip 24.1

Well, technically free-threading was too. ;)

henryiii · 2024-05-20T11:11:59Z

Not going to bother with Travis, it's not even started the job.

mayeut · 2024-05-20T11:16:52Z

Well, technically free-threading was too. ;)

yes, that's true. but bumping pip to a beta version for cp 3.13 is definitely acceptable (all is in BETA) while bumping pip for python 3.10 is probably less so (and I don't want to add specific code for this, personal opinion)

ned-deily · 2024-05-20T11:35:01Z

install both variants at the same time on macOS

Yes

ned-deily · 2024-06-06T02:30:31Z

FYI: 3.13.0b2 is now released (https://discuss.python.org/t/3-13-0b2-now-available/55056) and the macOS installer provided for it on python.org now (finally!) includes an optional and experimental python3.13t free-threaded build along with the traditional build. We are using python/cpython#120098 to track macOS installer-specific issues.

henryiii · 2024-06-06T02:49:24Z

Thanks, great work!

Ohh, 10.13 minimum, now too! ❤️

henryiii · 2024-06-06T20:07:14Z

@ned-deily The builds are not visible at https://www.python.org/api/v2/downloads/release/?is_published=true yet (https://www.python.org/api/v2/downloads/release/?is_published=true&name=Python%203.13.0b2 is empty). It's also not showing up at https://www.python.org/downloads/macos/ for example. It does show up here: https://www.python.org/download/pre-releases/. We use the first link above to update our Python versions, so it would be nice if those update, is that planned? I'd rather not manually tinker on our update scripts if it's going to fix itself soon.

ned-deily · 2024-06-06T20:30:24Z

The builds are not visible at https://www.python.org/api/v2/downloads/release/?is_published=true yet (https://www.python.org/api/v2/downloads/release/?is_published=true&name=Python%203.13.0b2 is empty). It's also not showing up at https://www.python.org/downloads/macos/ for example. It does show up here: https://www.python.org/download/pre-releases/. We use the first link above to update our Python versions, so it would be nice if those update, is that planned? I'd rather not manually tinker on our update scripts if it's going to fix itself soon.

@Yhg1s ?

Yhg1s · 2024-06-06T22:21:00Z

@Yhg1s ?

Should be visible now.

henryiii · 2024-06-06T22:42:50Z

Superb, thanks!

mayeut commented May 19, 2024

View reviewed changes

cibuildwheel/windows.py Outdated Show resolved Hide resolved

This was referenced May 19, 2024

[FR] Make it easier to run cibuildwheel against unlisted Python builds #1718

Open

[request] support --disable-gil builds (PEP703) #1657

Closed

mayeut commented May 19, 2024

View reviewed changes

mayeut marked this pull request as ready for review May 19, 2024 14:17

mayeut added 4 commits May 20, 2024 10:10

feat: add support for free-threaded (no-gil) Python 3.13

d9eb9d5

feat: add option to opt-in free-threaded builds

46f76a7

chore: pass a PythonConfiguration to install_cpython

6087551

doc: add documentation for the CIBW_FREE_THREADED_SUPPORT option

037b274

mayeut force-pushed the cp313t branch from ce1ff7d to 037b274 Compare May 20, 2024 08:11

henryiii approved these changes May 20, 2024

View reviewed changes

henryiii merged commit 90dd476 into pypa:main May 20, 2024
19 of 20 checks passed

mayeut deleted the cp313t branch May 20, 2024 13:43

henryiii mentioned this pull request Jun 6, 2024

Min macOS version #1855

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add support for free-threaded (no-gil) Python 3.13 #1831

feat: add support for free-threaded (no-gil) Python 3.13 #1831

mayeut commented May 18, 2024 •

edited

henryiii commented May 19, 2024

mayeut commented May 19, 2024

mayeut May 19, 2024

henryiii commented May 20, 2024

henryiii commented May 20, 2024

henryiii commented May 20, 2024 •

edited

mayeut commented May 20, 2024

henryiii commented May 20, 2024 •

edited

mayeut commented May 20, 2024

henryiii commented May 20, 2024

henryiii commented May 20, 2024

mayeut commented May 20, 2024

ned-deily commented May 20, 2024

ned-deily commented Jun 6, 2024

henryiii commented Jun 6, 2024

henryiii commented Jun 6, 2024

ned-deily commented Jun 6, 2024

Yhg1s commented Jun 6, 2024

henryiii commented Jun 6, 2024

feat: add support for free-threaded (no-gil) Python 3.13 #1831

feat: add support for free-threaded (no-gil) Python 3.13 #1831

Conversation

mayeut commented May 18, 2024 • edited

henryiii commented May 19, 2024

mayeut commented May 19, 2024

mayeut May 19, 2024

Choose a reason for hiding this comment

henryiii commented May 20, 2024

henryiii commented May 20, 2024

henryiii commented May 20, 2024 • edited

mayeut commented May 20, 2024

henryiii commented May 20, 2024 • edited

mayeut commented May 20, 2024

henryiii commented May 20, 2024

henryiii commented May 20, 2024

mayeut commented May 20, 2024

ned-deily commented May 20, 2024

ned-deily commented Jun 6, 2024

henryiii commented Jun 6, 2024

henryiii commented Jun 6, 2024

ned-deily commented Jun 6, 2024

Yhg1s commented Jun 6, 2024

henryiii commented Jun 6, 2024

mayeut commented May 18, 2024 •

edited

henryiii commented May 20, 2024 •

edited

henryiii commented May 20, 2024 •

edited