Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

K8s shutdown #294

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

K8s shutdown #294

wants to merge 4 commits into from

Conversation

SVyatoslavG
Copy link

Added environment variable to support k8s graceful shutdown. It is not a graceful shutdown by definition.
It would be useful for running gobetween behind Google Cloud Loadbalancer. Kubernetes Engine sends syscall.SIGTERM and at the same time stop sending requests to the pod before shutdown. Because those actions are done at the same time race condition occurs and the process needs to sleep before exiting.
Please accept simple pull request.

@yyyar
Copy link
Owner

yyyar commented Aug 18, 2020

Kubernetes Engine sends syscall.SIGTERM and at the same time stop sending requests to the pod before shutdown. Because those actions are done at the same time race condition occurs and the process needs to sleep before exiting.

Could you please clarify why the process needs to sleep before exiting? So that we make sure gobetween won't shutdown while Kubernetes still sending requests to it? And also what value should GOBETWEEN_SHUTDOWN_TIME have - are there some recommendations for those who use Google Cloud Loadbalancer?
Thanks!

@SVyatoslavG
Copy link
Author

Sorry for the delay in answer.
Yes, Kubernetes still sending requests to pods that are about to get deleting. SIGTERM & routing of requests happen at the same time and cause race conditions.
I am using GOBETWEEN_SHUTDOWN_TIME 20s in my production environment. It is extremely hight for safety (theoretically GKE doesn't route any requests after 1s but for safety, I set 20s in case there are long-running requests).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants