Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI Fixes #152

Merged
merged 1 commit into from
May 10, 2024
Merged

Conversation

IzerOnadimQC
Copy link
Collaborator

@IzerOnadimQC IzerOnadimQC commented May 9, 2024

CI failures are caused by the following package incompatibilities.

  1. Incompatibility with python 3.11.9 dask/dask#11038 Some versions of dask are incompatible with python 3.11.9. We were accidentally upgrading to this python version due to a --no-py-pin I added that we no longer need. Removing this fixes the issue, but this is not a true fix - perhaps we should pin dask to avoid the problematic versions (I don't know which versions are affected, only that the fix went into 2024.4.1, and has been back-ported to 2024.2.1).
  2. Pyarrow uses pandas.core.internals.DatetimeTZBlock, which will be removed in the next major release, and is already gone from the nightly builds. This is already handled in newer versions of pyarrow by checking the pandas API version, but causes failures for older versions of pyarrow. Perhaps this could be fixed by patching the repodata for pyarrow, but I'm uncertain whether this can be done for an optional dependency like pandas. For now, I've fixed this by testing the pyarrow and pandas nightly builds together, but an actual solution might be to drop support for older version of pyarrow when pandas 3 is released.

@IzerOnadimQC IzerOnadimQC marked this pull request as ready for review May 9, 2024 12:27
@xhochy
Copy link
Member

xhochy commented May 10, 2024

You can also repodata patch optional dependencies via constraints in conda-forge.

@xhochy xhochy merged commit 6e3d0cd into data-engineering-collective:main May 10, 2024
24 checks passed
@IzerOnadimQC IzerOnadimQC deleted the ci-failures branch May 12, 2024 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants