kubernetes - Manage terminationGracePeriodSeconds over 600 seconds - Stack Overflow

admin管理员组

文章数量:1334277

I'm using an autopilot cluster and have some workloads that can't be stopped under 10 minutes. I would like to set a termination grace period to X hours/days to prevent any interruption.

When I set manually a terminationGracePeriodSeconds over 600 seconds, it's automatically set back to 600 with a warning: Warning: autopilot-default-resources-mutator:The max supported TerminationGracePeriodSeconds is 600 seconds. Defaulting down from configured 1800 seconds to 600 seconds. deployment.apps/extended-pods configured

In Google documentation I saw this annotation to use to prevent this behavior cluster-autoscaler.kubernetes.io/safe-to-evict: "false" introduced in July, 2023 and documented here

But, when I'm using this annotation, I sill have the warning and the update of the TerminationGracePeriodSeconds.

Am I missing something?

I'm using an autopilot cluster and have some workloads that can't be stopped under 10 minutes. I would like to set a termination grace period to X hours/days to prevent any interruption.

When I set manually a terminationGracePeriodSeconds over 600 seconds, it's automatically set back to 600 with a warning: Warning: autopilot-default-resources-mutator:The max supported TerminationGracePeriodSeconds is 600 seconds. Defaulting down from configured 1800 seconds to 600 seconds. deployment.apps/extended-pods configured

In Google documentation I saw this annotation to use to prevent this behavior cluster-autoscaler.kubernetes.io/safe-to-evict: "false" introduced in July, 2023 and documented here

But, when I'm using this annotation, I sill have the warning and the update of the TerminationGracePeriodSeconds.

Am I missing something?

• kubernetes
• google-kubernetes-engine
• autopilot
• gke-autopilot

Share Improve this question Follow edited Nov 20, 2024 at 8:46 jps 22.6k1616 gold badges8888 silver badges106106 bronze badges asked Nov 20, 2024 at 8:43 AirmanbzhAirmanbzh 64511 gold badge55 silver badges1111 bronze badges 1

• By workload do you mean jobs or pods? Can't you implement graceful shutdowns and have the graceful shutdown hook to gracefully cleanup the workload within 10 mins and let the next pod or job to pick it up? – M B Commented Nov 20, 2024 at 11:33

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

As per this official document

In GKE For Autopilot clusters therterminationGracePeriodSeconds is limited to 10 minutes (600 seconds) for most Pods except for Spot Pods, which are limited to 25 seconds.

The Annotation which you applied prevents the cluster Autoscaler from evicting a pod during Scale down. But it does not override the termination grace period enforced by Autopilot.

As per this kodekloud blog You can also use a preStop hook in your pods lifecycle to cleanup tasks before termination.

Still you require a longer grace period to consider requesting extended run time of pods through google cloud support to inquire about specific feature requests for your use case.

本文标签： kubernetesManage terminationGracePeriodSeconds over 600 secondsStack Overflow