-
Notifications
You must be signed in to change notification settings - Fork 392
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Gracefully close realtime trigger when possible #3734
Comments
We could add a new In addition, I think that not properly stopping realtime triggers on worker shutdown there is a high risk of data loss. Futhermore, for the Kafka realtime trigger, if the consumer is not properly closed, this will delay the time when the currently assigned partitions will be available for consumption (default value is 5 minutes), could be the same for Pulsar. |
@fhussonnois I think you update all existing realtime trigger and I do this for Debezium so we can close it. |
Issue description
When possible, realtime triggers must be closed gracefully to be able to release resources.
Currently, realtime triggers uses an infinite loop and when killed/restarted their thread is interrupted which lead to resources like client connections to not be released properly.
For example, the Redis realtime trigger log this warning:
2024-05-15 10:01:48,986 WARN Finalizer i.l.c.r.DefaultClientResources io.lettuce.core.resource.DefaultClientResources was not shut down properly, shutdown() was not called before it's garbage-collected. Call shutdown() or shutdown(long,long,TimeUnit)
Ideally, all Realtime triggers must evolves so that:
There is a slight risk that a realtime trigger is "blocked" so we may investigate whether or not we want to do this with a timeout and interrupt the thread as a last resort.
The text was updated successfully, but these errors were encountered: