Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependencies: Update requirements for kiwipy and plumpy #5732

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

sphuber
Copy link
Contributor

@sphuber sphuber commented Oct 31, 2022

The new version kiwipy==0.8 and plumpy==0.22 provide compatiblity with newer versions of aio-pika==9.0 which comes with various connection stability improvements.

@sphuber sphuber force-pushed the fix/bump-engine-dependencies branch 5 times, most recently from 2088647 to d8e5693 Compare October 31, 2022 11:47
@sphuber sphuber force-pushed the fix/bump-engine-dependencies branch from d8e5693 to 31a152d Compare November 9, 2022 15:45
@sphuber sphuber force-pushed the fix/bump-engine-dependencies branch 2 times, most recently from 82d38fc to 1f22365 Compare December 7, 2022 11:31
@sphuber
Copy link
Contributor Author

sphuber commented Dec 7, 2022

There is a single bug that prevents this from being merged. Once the RmqCommunicator has opened a channel (which is done as soon as a process is run, for example a calcfunction) the close() call will hang. We have traced the source to the finished attribute of aio_pika.OneShotCallBack (which is an asyncio.Event) never being set. It is set when the OneShotCallBack is called (see the __call__ method which calls __task_inner whichs sets the finished event), but this is never called. We noticed that the channel, which is closed before the connection, had an exception during the close callbacks. We couldn't figure out which exception (seemed empty) or where it had been raised, but it may have a knock-on effect on the closing of the connection. Finally, if we disable nest_asyncio, the problem disappears, so it may be related to nest_asyncio as well. Unfortunately we currently cannot get rid of nest_asyncio as we need it to be able to call processes from within processes, i.e., to have a workfunction call a calcfunction.

@sphuber sphuber force-pushed the fix/bump-engine-dependencies branch from 1f22365 to 578044e Compare March 5, 2023 17:11
@sphuber sphuber force-pushed the fix/bump-engine-dependencies branch 3 times, most recently from 7af5355 to 6a31919 Compare March 13, 2023 23:05
@sphuber sphuber force-pushed the fix/bump-engine-dependencies branch from 6a31919 to a4ead79 Compare June 2, 2023 10:31
@sphuber sphuber force-pushed the fix/bump-engine-dependencies branch 7 times, most recently from 800a73c to fb1d2f6 Compare June 23, 2023 20:21
@sphuber sphuber force-pushed the fix/bump-engine-dependencies branch from fb1d2f6 to 4ba3e6a Compare July 13, 2023 13:30
@sphuber sphuber force-pushed the fix/bump-engine-dependencies branch 3 times, most recently from f55080f to 8229a7c Compare November 9, 2023 10:34
@sphuber sphuber force-pushed the fix/bump-engine-dependencies branch 6 times, most recently from f0d0b48 to e428cf3 Compare November 17, 2023 14:06
@sphuber sphuber force-pushed the fix/bump-engine-dependencies branch 3 times, most recently from 12d1fd3 to adfac17 Compare February 2, 2024 14:57
The new version `kiwipy==0.8` and `plumpy==0.22` provide compatiblity
with newer versions of `aio-pika==8.0` which comes with various
connection stability improvements.
The exception is caught and logged as a warning.
@sphuber sphuber force-pushed the fix/bump-engine-dependencies branch from c4186ee to 564f908 Compare March 25, 2024 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant