Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

spack fails to concretize if there are built packages when using develop packages #44160

Open
3 tasks done
robertu94 opened this issue May 13, 2024 · 4 comments
Open
3 tasks done

spack fails to concretize if there are built packages when using develop packages #44160

robertu94 opened this issue May 13, 2024 · 4 comments
Labels
bug Something isn't working triage The issue needs to be prioritized

Comments

@robertu94
Copy link
Contributor

Steps to reproduce

spack env activate .
spack develop
spack install
# edit the spack.yaml in a way that is should not cause concretization failures (e.g. turning on a cuda variant)
spack concretize -f

Error message

spack --debug concretize -f
==> [2024-05-13-13:07:10.673418] Reading config from file /home/runderwood/git/contrib/spack/etc/spack/defaults/config.yaml
==> [2024-05-13-13:07:10.687594] Reading config from file /home/runderwood/.spack/config.yaml
==> [2024-05-13-13:07:10.701035] Reading config from file /home/runderwood/git/libpressio_spack/spack.yaml
==> [2024-05-13-13:07:10.717204] Using environment '/home/runderwood/git/libpressio_spack'
==> [2024-05-13-13:07:10.757803] Imported concretize from built-in commands
==> [2024-05-13-13:07:10.768996] Imported concretize from built-in commands
==> [2024-05-13-13:07:10.781669] Reading config from file /home/runderwood/git/libpressio_spack/spack.yaml
==> [2024-05-13-13:07:10.793352] Deactivated environment '/home/runderwood/git/libpressio_spack'
==> [2024-05-13-13:07:10.797987] Using environment '/home/runderwood/git/libpressio_spack'
==> [2024-05-13-13:07:10.798928] Reading config from file /home/runderwood/git/contrib/spack/etc/spack/defaults/concretizer.yaml
==> [2024-05-13-13:07:10.807318] DATABASE LOCK TIMEOUT: 60s
==> [2024-05-13-13:07:10.807375] PACKAGE LOCK TIMEOUT: No timeout
==> [2024-05-13-13:07:10.807626] Reading config from file /home/runderwood/git/contrib/spack/etc/spack/defaults/packages.yaml
==> [2024-05-13-13:07:10.842494] Reading config from file /home/runderwood/git/contrib/spack/etc/spack/defaults/linux/packages.yaml
==> [2024-05-13-13:07:10.845749] Reading config from file /home/runderwood/.spack/packages.yaml
==> [2024-05-13-13:07:10.849744] Reading config from file /home/runderwood/git/contrib/spack/etc/spack/defaults/repos.yaml
==> [2024-05-13-13:07:10.851538] Reading config from file /home/runderwood/.spack/repos.yaml
==> [2024-05-13-13:07:10.952638] Reading config from file /home/runderwood/.spack/linux/compilers.yaml
==> [2024-05-13-13:07:10.965332] '/usr/lib64/ccache/clang' '-v' '/tmp/spack-implicit-link-infow4p1ww3e/main.c' '-o' '/tmp/spack-implicit-link-infow4p1ww3e/output'
==> [2024-05-13-13:07:11.007098] '/usr/lib64/ccache/gcc' '-v' '/tmp/spack-implicit-link-info2xoq5acz/main.c' '-o' '/tmp/spack-implicit-link-info2xoq5acz/output'
==> [2024-05-13-13:07:11.030403] '/home/runderwood/git/contrib/spack/opt/spack/linux-centos7-x86_64_v3/gcc-10.2.1/gcc-13.2.0-h5cwrnecxnpqef5ewgr7j6ttn7cjwj63/bin/gcc' '-v' '/tmp/spack-implicit-link-infoeolrtax6/main.c' '-o' '/tmp/spack-implicit-link-infoeolrtax6/output'
==> [2024-05-13-13:07:11.137088] DATABASE LOCK TIMEOUT: 60s
==> [2024-05-13-13:07:11.137160] PACKAGE LOCK TIMEOUT: No timeout
==> [2024-05-13-13:07:11.252959] DATABASE LOCK TIMEOUT: 60s
==> [2024-05-13-13:07:11.253019] PACKAGE LOCK TIMEOUT: No timeout
==> [2024-05-13-13:07:11.341573] Reading config from file /home/runderwood/git/contrib/spack/etc/spack/defaults/mirrors.yaml
==> [2024-05-13-13:07:11.343099] Reading config from file /home/runderwood/.spack/mirrors.yaml
==> [2024-05-13-13:07:11.344087] Checking existence of https://binaries.spack.io/develop/developer-tools-manylinux2014/build_cache/index.json
==> [2024-05-13-13:07:11.344478] certs: relative path not allowed: $SSL_CERT_FILE
==> [2024-05-13-13:07:11.910707] DATABASE LOCK TIMEOUT: Nones
==> [2024-05-13-13:08:20.205470] SolverError: Spack concretizer internal error. Please submit a bug report and include the command, environment if applicable and the following error message.
[qcat, libpressio-tools+adios2+dctz+error_injector+hdf5+mpi+nvcomp+opt+rcpp+tthresh ^libpressio+bitgrooming+blosc+blosc2+bzip2+clang+cuda+docs+fpzip+hdf5+json+libdistributed+lua+magick+matio+mgard+mpi+ndzip+netcdf+openmp+petsc+python+sz+sz3+szx+unix+zfp] is unsatisfiable, errors are:
internal_error("Reuse version weight used for build package")
internal_error("must choose a single version to satisfy version constraints")
internal_error("version weights must exist and be unique"). You could consider setting concretizer:unify to when_possible or false to allow multiple versions of some packages.
==> [2024-05-13-13:08:20.205526] Error: Spack concretizer internal error. Please submit a bug report and include the command, environment if applicable and the following error message.
[qcat, libpressio-tools+adios2+dctz+error_injector+hdf5+mpi+nvcomp+opt+rcpp+tthresh ^libpressio+bitgrooming+blosc+blosc2+bzip2+clang+cuda+docs+fpzip+hdf5+json+libdistributed+lua+magick+matio+mgard+mpi+ndzip+netcdf+openmp+petsc+python+sz+sz3+szx+unix+zfp] is unsatisfiable, errors are:
internal_error("Reuse version weight used for build package")
internal_error("must choose a single version to satisfy version constraints")
internal_error("version weights must exist and be unique"). You could consider setting concretizer:unify to when_possible or false to allow multiple versions of some packages.
Traceback (most recent call last):
File "/home/runderwood/git/contrib/spack/lib/spack/spack/main.py", line 1069, in main
return _main(argv)
^^^^^^^^^^^
File "/home/runderwood/git/contrib/spack/lib/spack/spack/main.py", line 1022, in _main
return finish_parse_and_run(parser, cmd_name, args, env_format_error)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runderwood/git/contrib/spack/lib/spack/spack/main.py", line 1052, in finish_parse_and_run
return _invoke_command(command, parser, args, unknown)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runderwood/git/contrib/spack/lib/spack/spack/main.py", line 649, in _invoke_command
return_val = command(parser, args)
^^^^^^^^^^^^^^^^^^^^^
File "/home/runderwood/git/contrib/spack/lib/spack/spack/cmd/concretize.py", line 44, in concretize
concretized_specs = env.concretize(force=args.force, tests=tests)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runderwood/git/contrib/spack/lib/spack/spack/environment/environment.py", line 1447, in concretize
return self._concretize_together(tests=tests)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runderwood/git/contrib/spack/lib/spack/spack/environment/environment.py", line 1566, in _concretize_together
concrete_specs: List[spack.spec.Spec] = spack.concretize.concretize_specs_together(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runderwood/git/contrib/spack/lib/spack/spack/concretize.py", line 749, in concretize_specs_together
return _concretize_specs_together_new(*abstract_specs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runderwood/git/contrib/spack/lib/spack/spack/concretize.py", line 757, in _concretize_specs_together_new
result = solver.solve(
^^^^^^^^^^^^^
File "/home/runderwood/git/contrib/spack/lib/spack/spack/solver/asp.py", line 3829, in solve
result, _, _ = self.driver.solve(
^^^^^^^^^^^^^^^^^^
File "/home/runderwood/git/contrib/spack/lib/spack/spack/solver/asp.py", line 925, in solve
result.raise_if_unsat()
File "/home/runderwood/git/contrib/spack/lib/spack/spack/solver/asp.py", line 468, in raise_if_unsat
raise SolverError(constraints, conflicts=conflicts)
spack.solver.asp.SolverError: Spack concretizer internal error. Please submit a bug report and include the command, environment if applicable and the following error message.
[qcat, libpressio-tools+adios2+dctz+error_injector+hdf5+mpi+nvcomp+opt+rcpp+tthresh ^libpressio+bitgrooming+blosc+blosc2+bzip2+clang+cuda+docs+fpzip+hdf5+json+libdistributed+lua+magick+matio+mgard+mpi+ndzip+netcdf+openmp+petsc+python+sz+sz3+szx+unix+zfp] is unsatisfiable, errors are:
internal_error("Reuse version weight used for build package")
internal_error("must choose a single version to satisfy version constraints")
internal_error("version weigh

Information on your system

❯ spack debug report
* **Spack:** 0.22.0.dev0 (83945f9c550722a4256c5e6ed66ad603a462f40c)
* **Python:** 3.12.3
* **Platform:** linux-fedora40-icelake
* **Concretizer:** clingo

General information

  • I have run spack debug report and reported the version of Spack/Python/Platform
  • I have searched the issues of this repo and believe this is not a duplicate
  • I have run the failing commands in debug mode and reported the output
@robertu94 robertu94 added bug Something isn't working triage The issue needs to be prioritized labels May 13, 2024
@robertu94
Copy link
Contributor Author

Here is the envionment:

spack:
  # add package specs to the `specs` list
  config:
    ccache: true
  specs:
  - libpressio-tools+adios2+error_injector+hdf5+mpi+opt+rcpp+tthresh+nvcomp+dctz ^libpressio+bitgrooming+blosc+blosc2+bzip2+clang+cuda+docs+fpzip+hdf5+json+libdistributed+lua+magick+matio+mgard+mpi+ndzip+netcdf+openmp+petsc+python+sz+sz3+szx+unix+zfp
  - qcat
  view: true
  concretizer:
    unify: true
  develop:
    libpressio:
      spec: libpressio@git.master=0.99.4
    sz:
      spec: sz@git.master
    sz3:
      spec: sz3@git.master
    mgard:
      spec: mgard@git.master
    zfp:
      spec: zfp@git.staging
    libpressio-adios2:
      spec: libpressio-adios2@git.main
    libpressio-tools:
      spec: libpressio-tools@git.master
    libstdcompat:
      spec: libstdcompat@git.master
    libpressio-opt:
      spec: libpressio-opt@git.develop
    libpressio-adios1:
      spec: libpressio-adios1@git.main
    libpressio-errorinjector:
      spec: libpressio-errorinjector@git.master
    libpressio-sperr:
      spec: libpressio-sperr@git.master
    r-libpressio:
      spec: r-libpressio@git.master
    libpressio-jit:
      spec: libpressio-jit@git.main
    lc-framework:
      spec: lc-framework@git.main

using a branch of spack with: #44076

@alalazo
Copy link
Member

alalazo commented May 21, 2024 •
edited

I cannot reproduce (even trying a simpler environment based on the descrioption), but using your environment I got:

$ spack concretize -f
==> Error: Couldn't get the git repo for libpressio-adios2

Can you post a sequence of commands to reproduce the issue? I am on:

  • Spack: 0.23.0.dev0 (8020a11)
  • Python: 3.11.5
  • Platform: linux-ubuntu20.04-icelake
  • Concretizer: clingo

@robertu94
Copy link
Contributor Author

robertu94 commented May 21, 2024 •
edited

I just did this and it fails for me:

spack:
  # add package specs to the `specs` list
  config:
    ccache: true
  specs:
  - libpressio-tools ^libpressio+zfp
  view: true
  concretizer:
    unify: true
  develop:
    libpressio:
      spec: libpressio@git.master=0.99.4

mkdir /tmp/foo
cd /tmp/foo
spack env activate .
spack develop
spack install

and I get the error:

❯ spack install
==> Error: Spack concretizer internal error. Please submit a bug report and include the command, environment if applicable and the following error message.
    libpressio-tools ^libpressio+zfp is unsatisfiable, errors are:
    internal_error("Reuse version weight used for build package")
    internal_error("must choose a single version to satisfy version constraints")
    internal_error("version weights must exist and be unique")

❯ spack debug report
* **Spack:** 0.23.0.dev0 (94dea42eb4e6623ab102acb6fd651585cd6eed6f)
* **Python:** 3.12.3
* **Platform:** linux-fedora40-icelake
* **Concretizer:** clingo

I've attached the concretizer ASP log

@robertu94
Copy link
Contributor Author

debug.asp.txt.gz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working triage The issue needs to be prioritized
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@alalazo @robertu94