Der Shop ist online

Wir bauen Workstation AI-Maschinen für Websites, KI-Agenten und LLM-Training – online konfigurieren und kaufen.

Händlerpreise erhalten
KI-Produkte
KI-LabsOpenAI-AgentenClaude-AgentenGrok-BotWorkstation AI-natives CRMMarketingAlle Produkte
KI-Lösungen
Angewandte KIKI-AgentenGenerative KIKI-AutomatisierungKI-WeiterbildungKI-Pakete für KMUEnterprise AI LabKI-WorkstationsNVIDIA-Hardware & -SoftwareNVIDIA Accelerated ComputingPrivate KIGPU-Private-CloudGPU-ClusterEdge-KIKI nach Branche
KI-Services
PlattformmodernisierungDigital EngineeringDatengrundlagen & KIAutonomer BetriebKI-BeratungDevOps-AutomatisierungCybersicherheitSoftwareentwicklungAgentenentwicklungMLOps-EinrichtungGPU-LeasingGPU-Private-Cloud
Über uns
UnternehmenUnsere GeschichtePartnerKundengeschichten
KI-Rechenzentren
KI-RechenzentrenGPU-Private-CloudGPU-ClusterGPU-Leasing
KI-EinblickeJev-ReiheBenchmark LabModellauswahl
Technische Hinweise
Dokumentation
Linux-BetriebssystemeUbuntuRed Hat Enterprise LinuxSUSE LinuxAlpine LinuxDebian, Rocky & AlmaLinux
KI-Cloud-PlattformenAWS & Amazon BedrockGoogle Cloud & Vertex AIMicrosoft Azure & AI FoundryGPU-Clouds & Marktplätze
Private KI-RechenzentrenFür AnwaltskanzleienFür Steuer- und WirtschaftsprüfungskanzleienOn-Premises-PlattformenVollständig abgesicherte Cloud
ToolsRing PromoterJobshout
News
Blog
KontaktAnmelden

KI-Workstations, Multi-Agenten-KI-Software, GPU-Infrastruktur und intelligente Agentenlösungen für moderne Unternehmen.

Kontakt

KI-Lösungen

Angewandte KIKI-AgentenGenerative KIKI-AutomatisierungKI-WeiterbildungKI-Pakete für KMUEnterprise AI LabKI nach Branche

Produkte

Alle ProdukteWorkstation AI-natives CRM & ERPMarketingOpenAI-AgentenRing PromoterJobshout

Unternehmen

Über unsUnternehmenKI-RechenzentrenWarum WorkstationPartnerKundengeschichtenPreiseKontaktTenthMatrix

Ressourcen

KI-EinblickeTechnische HinweiseDokumentationLinux-BetriebssystemeKI-Cloud-PlattformenPrivate KI-RechenzentrenBlogSucheSitemap
Büro Belgien
Workstation SRL, Rue Vanderkindere 34, 1180 Uccle, BrusselsBE 0751.518.683Mo – Fr: 9:00 – 18:00 Uhr MEZ
+32 492 45 67 46
Hauptsitz Großbritannien
77-79 Marlowes, Hemel Hempstead HP1 1LFAnfahrt – Ausfahrt 20 der M25 (Outer London)Handelsregisternr.: 11641870Mo – Fr: 9:00 – 18:00 Uhr GMT
+44 7515 356 146
Büro Indien
#159 Sector 9, Pocket 1, DDA Flats, 110077 Dwarka, New Delhi
+91 98881 98841
Büro Taiwan
3F, No. 2-4, Aisi Rd., Ren'ai Dist., Keelung City 200, Taiwan基隆市仁愛區愛四路2之4號3樓Mo – Fr: 9:00 – 18:00 Uhr Taiwan-Zeit (GMT+8)
+886 926 892 701+32 492 45 67 46
Büro Estland
Aleksander Puškini tn 30-25, 20306 Narva, EstoniaMo - Fr: 9:00 - 18:00 Uhr estnischer Zeit (EET)
+32 492 45 67 46

© 2026 Workstation AI. Alle Rechte vorbehalten.

DatenschutzDatenlöschungCookiesNutzungsbedingungenWebsite-Sitemap

NVIDIA, das NVIDIA-Logo, RTX, RTX PRO, Blackwell, CUDA, NVLink, MGX, Omniverse, Spectrum-X, Quantum, BlueField, DGX und HGX sind Marken und/oder eingetragene Marken der NVIDIA Corporation in den USA und anderen Ländern. AMD, das AMD-Arrow-Logo, EPYC, Instinct, Ryzen und Threadripper sind Marken von Advanced Micro Devices, Inc. Dell und PowerEdge sind Marken der Dell Inc. oder ihrer Tochtergesellschaften. Intel und Xeon sind Marken der Intel Corporation oder ihrer Tochtergesellschaften. AMAX, AceleMax und LiquidMax sind Marken der AMAX Engineering Corporation. Alle anderen Produktnamen, Logos und Marken sind Eigentum ihrer jeweiligen Inhaber. Wir verwenden sie ausschließlich, um die Produkte zu kennzeichnen, die wir beschreiben, liefern oder unterstützen; ihre Verwendung bedeutet nicht, dass ihre Inhaber uns empfehlen.

Loading blog...

Startseite / Blogs
Engineering

Balancing Safety and Speed in Kubernetes Rolling Updates

JobShout AI Blog Writer Agent (opens in a new window)18. November 20268 min read
Cover

Optimizing deployments in Kubernetes is not merely about making things faster; it is about engineering a trade-off between safety margins and rollout velocity. In cloud-native environments, the goal is to design, develop, and deploy dependable, dynamic services that meet the demanding requirements of the industry [1]. However, achieving zero-downtime deployments while maintaining high throughput requires a deep understanding of how Kubernetes interacts with underlying infrastructure, load balancers, and build systems. This article explores the technical challenges of rolling updates, the performance optimizations embedded in recent Kubernetes versions, and the tools that help developers navigate the complexity of fast, reliable deployments.

The Challenge of Rolling Updates

A rolling update is designed to update a service's replicas without interrupting service availability. However, the process involves a complex interplay between the Kubernetes API server, the kubelet on the nodes, and the external load balancer. A critical challenge arises during the termination phase of an old pod.

When an application pod is scheduled for termination, the AWS Load Balancer (or similar LBs) continues to send new requests to the target for several seconds after the application has received the termination signal [2]. This behavior creates a race condition: the application may be in the process of shutting down or flushing connections, yet it is still receiving new traffic.

Consequently, determining the amount of time to wait for the endpoint change to fully propagate to the target group is a balancing act between safety margin and rollout speed [2]. If the wait is too short, you risk dropping in-flight requests. If it is too long, your deployment pipeline slows down significantly, increasing feedback loops for developers.

Diagram

Note: The sequence above illustrates the race condition where the Load Balancer may continue routing traffic to a pod that has already received a termination signal [2].

To mitigate this, engineers often attempt to use a sidecar container with a preStop lifecycle hook to introduce a deliberate delay before the container terminates. However, this approach has limitations. Specifically, adding a sidecar container with a preStop lifecycle hook does not work for distroless base images, which lack the necessary shell or init system to execute these commands [2]. This forces teams to rely on other mechanisms, such as graceful shutdown handling within the application code itself, or to accept the risk inherent in the load balancer's propagation delay.

Kubernetes Performance and Scalability

The ability to perform rapid rolling updates is underpinned by the core performance of the Kubernetes cluster. Since Kubernetes 1.1, significant performance tweaks and enhancements have made it possible to serve 1 million requests per second [1]. This baseline performance is crucial because rolling updates often involve scaling up new pods before terminating old ones, placing temporary load on the control plane.

In Kubernetes 1.2, density was further improved from a pods-per-node perspective, allowing up to 100 pods on a single node, compared to 30 in the 1.1 release [3]. This increase in pod density means that a single node can handle more concurrent workloads, which is beneficial during high-churn deployment windows.

Two specific optimizations in Kubernetes 1.2 are particularly relevant to the stability of rolling updates:

  1. PLEG (Pod Lifecycle Event Generator) Optimization: The PLEG component is responsible for polling the state of containers. In previous versions, this polling could be expensive. The introduction of concurrent polling elimination in Kubernetes 1.2 allowed the system to lower the steady-state CPU usage of the Kubelet and the container runtime by 4x [3]. Lower CPU contention on the node ensures that the kubelet can respond quickly to update signals and termination requests, reducing the time between "intended state" and "actual state."
  2. JSON Encoding/Decoding Speedups: Communication between the Kubernetes API server and the kubelets relies heavily on JSON serialization. The adoption of the ugorji codec library provided a 200% improvement in encoding and decoding JSON when using generated serializers [3]. This faster data transfer means that status updates and watch events propagate more quickly through the cluster, which is essential for coordinating the phased rollout of pods across many nodes.
Component Optimization in K8s 1.2 Impact on Rolling Updates
PLEG Eliminated concurrent polling Reduced Kubelet CPU usage by 4x, allowing faster reaction to pod lifecycle events [3]
JSON Codec Use of ugorji codec library 200% improvement in JSON encode/decode speed, accelerating API communications [3]
Pod Density Increased max pods per node Supports up to 100 pods/node (vs 30 in 1.1), allowing higher churn rates on single nodes [3]

The Role of AWS Load Balancers

As discussed in the challenges of rolling updates, the interaction between Kubernetes and cloud-native load balancers is a primary source of deployment friction. The AWS Load Balancer behavior is not a bug, but a design choice that prioritizes connection availability over immediate state synchronization.

The fact that the AWS Load Balancer keeps sending new requests to the target for several seconds after the application is sent the termination signal [2] means that preStop hooks must be carefully calibrated. For standard images, a sidecar container can be used to sleep for a specific duration before the main container stops, effectively holding the port open while the LB deregisters the target.

However, this solution is not universal. For distroless images, which are minimal and immutable, this sidecar approach fails because the necessary tools to execute the delay are missing [2]. In these cases, the application must implement its own graceful shutdown logic, such as:

  • Stopping accepting new requests.
  • Waiting for active requests to complete.
  • Exiting the process.

The application must exit before the termination grace period expires, but after the load balancer has removed it from the target group. Since the exact propagation time varies, developers must often add a buffer to the terminationGracePeriodSeconds setting. This buffer is the "balancing act" between safety and speed [2].

Tilt: A Faster Way to Update Kubernetes

While optimizing the cluster and load balancer behavior reduces downtime, it does not necessarily speed up the feedback loop for developers. Traditional deployment workflows involve building Docker images, pushing them to a registry, and then instructing Kubernetes to pull the new image. This process can take minutes, even for small changes.

Tilt addresses this by making Kubernetes updates fast—really fast. Instead of minutes, Tilt updates Kubernetes in seconds [4]. This is achieved through two key mechanisms:

1. Incremental Builds

In a standard Docker workflow, creating a new layer with the generated/compiled artifacts happens for every update [4]. Even if you only changed one file, Docker will create a new layer and start a build from scratch. Tilt's incremental build API makes it easy to use the build cache on subsequent builds, with realistic builds improving from 30 seconds to 1 second [4].

2. File Syncing via Synclet

Rather than always replacing the entire pod, Tilt can reuse existing pods. A "Synclet" runs on the same node as the pod. When you update files, the Synclet adds them to the existing pod and restarts the container [4]. This bypasses the image pull and container creation overhead entirely, providing a near-instant update experience that is "cloud-as-fast-as-laptop" fast [4].

Approach Mechanism Typical Update Time Use Case
Traditional Docker New layer, full build, image pull Minutes Production releases, full integration tests
Tilt Incremental build, file sync to pod Seconds Development, rapid iteration, local testing [4]

Note: Tilt reuses existing pods by adding updated files to the existing pod and restarting the container, which significantly reduces overhead compared to standard image updates [4].

Best Practices for Rolling Updates

To balance safety and speed effectively, developers should adopt a multi-layered strategy:

  1. Understand LB Propagation Delays: Acknowledge that AWS Load Balancers (and similar services) continue routing traffic for several seconds after termination signals [2]. Do not rely on preStop hooks alone if using distroless images, as these hooks may not function as expected [2].
  2. Leverage Kubernetes Performance Gains: Ensure your cluster is running on a version that benefits from PLEG optimizations and faster JSON serialization [3]. This ensures the control plane can handle the churn of rolling updates without becoming a bottleneck.
  3. Optimize the Development Loop: Use tools like Tilt for rapid iteration to reduce the cognitive load of waiting for builds [4]. By using incremental builds and file syncing [4], you can verify that your rolling update configuration works correctly in seconds, not minutes.
  4. Monitor Service States: Use tools that can track the state of services, such as Gitopper, which can track states like OK, FREEZE, ROLLBACK, BROKEN, and DIFF [5]. This allows you to roll back a service to a specific commit if a rolling update fails [5], providing a safety net when pushing for speed.

By combining an understanding of the underlying infrastructure behaviors (like LB propagation and Kubelet performance) with modern development tooling (like Tilt), teams can achieve deployments that are both fast and safe. The goal is not to eliminate downtime at all costs, but to manage the trade-offs intelligently, ensuring that the "balancing act" [2] results in reliable, high-velocity software delivery.

References

  1. One million requests per second: Dependable and dynamic distributed systems at scale | Kubernetes — blog.kubernetes.io (11 November 2015)
  2. Zero-Downtime Kubernetes Deployments on AWS with EKS | Glasskube — glasskube.dev (10 March 2025)
  3. 1000 nodes and beyond: updates to Kubernetes performance and scalability in 1.2 | Kubernetes — blog.kubernetes.io (28 March 2016)
  4. How Tilt updates Kubernetes in Seconds, not Minutes — medium.com (7 January 2019)
  5. miekg/gitopper — github.com (27 August 2024)