Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Install fails on Python 3.12 due to missing distutils #181

Open
telotortium opened this issue Apr 26, 2024 · 5 comments
Open

Install fails on Python 3.12 due to missing distutils #181

telotortium opened this issue Apr 26, 2024 · 5 comments

Comments

@telotortium
Copy link

See https://docs.python.org/3.10/whatsnew/3.10.html#distutils-deprecated for more info.
@MahmoudAshraf97
Copy link
Owner

which package exactly causes the failure?

@telotortium
Copy link
Author

telotortium commented May 3, 2024 •
edited

I believe it's numpy 1.23.5:

$ source venv3.12/bin/activate
$ pip install -r requirements.txt
Collecting git+https://github.com/m-bain/whisperX.git@78dcfaab51005aa703ee21375f81ed31bc248560 (from -r requirements.txt (line 3))
  Cloning https://github.com/m-bain/whisperX.git (to revision 78dcfaab51005aa703ee21375f81ed31bc248560) to /private/var/folders/wn/0jmk5twd7j756pcyqwtnfd4r0000gn/T/pip-req-build-eej96ayq
  Running command git clone --filter=blob:none --quiet https://github.com/m-bain/whisperX.git /private/var/folders/wn/0jmk5twd7j756pcyqwtnfd4r0000gn/T/pip-req-build-eej96ayq
  Running command git rev-parse -q --verify 'sha^78dcfaab51005aa703ee21375f81ed31bc248560'
  Running command git fetch -q https://github.com/m-bain/whisperX.git 78dcfaab51005aa703ee21375f81ed31bc248560
  Running command git checkout -q 78dcfaab51005aa703ee21375f81ed31bc248560
  Resolved https://github.com/m-bain/whisperX.git to commit 78dcfaab51005aa703ee21375f81ed31bc248560
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting demucs (from -r requirements.txt (line 4))
  Cloning https://github.com/facebookresearch/demucs to /private/var/folders/wn/0jmk5twd7j756pcyqwtnfd4r0000gn/T/pip-install-gardeugs/demucs_ae033499040946c8be07b6a285bd9cbe
  Running command git clone --filter=blob:none --quiet https://github.com/facebookresearch/demucs /private/var/folders/wn/0jmk5twd7j756pcyqwtnfd4r0000gn/T/pip-install-gardeugs/demucs_ae033499040946c8be07b6a285bd9cbe
  Resolved https://github.com/facebookresearch/demucs to commit e976d93ecc3865e5757426930257e200846a520a
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting wget (from -r requirements.txt (line 1))
  Using cached wget-3.2.zip (10 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting nemo_toolkit==1.22.0 (from nemo_toolkit[asr]==1.22.0->-r requirements.txt (line 2))
  Using cached nemo_toolkit-1.22.0-py3-none-any.whl.metadata (15 kB)
Collecting deepmultilingualpunctuation (from -r requirements.txt (line 5))
  Using cached deepmultilingualpunctuation-1.0.1-py3-none-any.whl.metadata (4.0 kB)
Collecting huggingface-hub (from nemo_toolkit==1.22.0->nemo_toolkit[asr]==1.22.0->-r requirements.txt (line 2))
  Downloading huggingface_hub-0.23.0-py3-none-any.whl.metadata (12 kB)
Collecting numba (from nemo_toolkit==1.22.0->nemo_toolkit[asr]==1.22.0->-r requirements.txt (line 2))
  Using cached numba-0.59.1-cp312-cp312-macosx_11_0_arm64.whl.metadata (2.7 kB)
Collecting numpy<1.24,>=1.22 (from nemo_toolkit==1.22.0->nemo_toolkit[asr]==1.22.0->-r requirements.txt (line 2))
  Using cached numpy-1.23.5.tar.gz (10.7 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/commands/install.py", line 377, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 427, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 239, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 230, in _get_updated_criteria
    self._add_to_criteria(criteria, requirement, parent=candidate)
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
    return bool(self._sequence)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
    return any(self)
           ^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
    candidate = func()
                ^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 182, in _make_candidate_from_link
    base: Optional[BaseCandidate] = self._make_base_candidate_from_link(
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 228, in _make_base_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
                                       ^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 290, in __init__
    super().__init__(
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
    self.dist = self._prepare()
                ^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 222, in _prepare
    dist = self._prepare_distribution()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 301, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 525, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 640, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 71, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/distributions/sdist.py", line 54, in prepare_distribution_metadata
    self._install_build_reqs(finder)
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/distributions/sdist.py", line 124, in _install_build_reqs
    build_reqs = self._get_build_requires_wheel()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/distributions/sdist.py", line 101, in _get_build_requires_wheel
    return backend.get_requires_for_build_wheel()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/utils/misc.py", line 745, in get_requires_for_build_wheel
    return super().get_requires_for_build_wheel(config_settings=cs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 166, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 321, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
    obj = import_module(mod_path)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/private/var/folders/wn/0jmk5twd7j756pcyqwtnfd4r0000gn/T/pip-build-env-4ezcfsc3/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 10, in <module>
    import distutils.core
ModuleNotFoundError: No module named 'distutils'

@transcriptionstream
Copy link
Contributor

Have you tried using pip to install numpy and distutils in the venv prior to running the install?

@MahmoudAshraf97
Copy link
Owner

I guess the final solution to this dependency hell is to fork nemo and remove all the bloated requirements that aren't related to this project

@MahmoudAshraf97
Copy link
Owner

The requirements were updated, can you try again?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@telotortium @MahmoudAshraf97 @transcriptionstream