> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gcore.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Upgrade a GPU Kubernetes cluster

export const MethodSection = ({children}) => children ?? null;

export const MethodSwitch = ({children}) => {
  const tabs = React.Children.toArray(children).map(c => {
    if (!c || !c.props) return null;
    if (c.props.id) return c;
    const inner = c.props.children;
    if (inner && inner.props && inner.props.id) return inner;
    return null;
  }).filter(Boolean);
  const firstId = tabs.length > 0 ? tabs[0].props.id : "";
  const [active, setActive] = React.useState(firstId);
  React.useEffect(() => {
    try {
      const saved = localStorage.getItem("gcore_docs_method");
      if (saved && tabs.find(t => t.props.id === saved)) {
        setActive(saved);
      }
    } catch (_) {}
  }, []);
  React.useEffect(() => {
    try {
      document.querySelectorAll("h2[id], h3[id]").forEach(heading => {
        const visible = heading.offsetParent !== null;
        document.querySelectorAll(`a[href="#${heading.id}"]`).forEach(link => {
          if (link.closest("h1,h2,h3,h4,h5,h6")) return;
          const li = link.closest("li");
          if (li) li.style.display = visible ? "" : "none";
        });
      });
    } catch (_) {}
    window.dispatchEvent(new Event("scroll"));
  }, [active]);
  const handleClick = id => {
    setActive(id);
    try {
      localStorage.setItem("gcore_docs_method", id);
    } catch (_) {}
  };
  return <div>
      <div className="not-prose flex gap-0 border-b border-zinc-200 dark:border-zinc-800 mb-8 mt-2" role="tablist">
        {tabs.map(tab => {
    const isActive = active === tab.props.id;
    return <button key={tab.props.id} role="tab" aria-selected={isActive} onClick={() => handleClick(tab.props.id)} className={["px-4 py-2 text-sm font-medium border-b-2 -mb-px transition-colors cursor-pointer", isActive ? "border-primary text-primary" : "border-transparent text-zinc-500 hover:text-zinc-800 dark:hover:text-zinc-200"].join(" ")}>
              {tab.props.label}
            </button>;
  })}
      </div>

      {tabs.map(tab => <div key={tab.props.id} style={{
    display: active === tab.props.id ? "" : "none"
  }}>
          {tab.props.children}
        </div>)}
    </div>;
};

<MethodSwitch>
  <MethodSection id="portal" label="Customer Portal">
    <p>Kubernetes upgrades use a rolling update strategy: a new node on the target version is provisioned, workloads migrate to it, and the old node is removed. This repeats until all nodes run the new version, minimizing downtime.</p>

    <Warning>
      Upgrades are one minor version at a time. To move from v1.33 to v1.35, run two separate upgrades: v1.33 → v1.34, then v1.34 → v1.35. Downgrade is not supported.
    </Warning>

    ## Before upgrading

    <p>Check that the cluster control plane version and node versions match:</p>

    ```bash theme={null}
    kubectl version
    kubectl get nodes
    ```

    <p>Verify that all nodes show the same version as the control plane before starting the upgrade. Review the [changelog](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/) for deprecated APIs that may affect running workloads.</p>

    <p>Rolling upgrades require quota for one additional node of the same flavor during the process. For a 10-node pool, quota for 11 nodes is needed temporarily. To avoid requesting extra quota, scale the pool down by one node before starting the upgrade, then scale it back up after the upgrade completes.</p>

    ## Start the upgrade

    <p>When a newer Kubernetes version is available, an upgrade prompt appears in the cluster overview next to the **Kubernetes Version** field:</p>

    <Steps>
      <Step title="Click the upgrade prompt next to Kubernetes Version">
        <Frame>
          <img src="https://mintcdn.com/gcore/c2W3cwtGjSbFG_V1/images/docs/edge-ai/managed-kubernetes/manage-node-pools/kubernetes-version-upgrade.png?fit=max&auto=format&n=c2W3cwtGjSbFG_V1&q=85&s=46e5902ec867b71bf10dccd02f4832cb" alt="Kubernetes version upgrade prompt on the cluster overview page" width="1920" height="945" data-path="images/docs/edge-ai/managed-kubernetes/manage-node-pools/kubernetes-version-upgrade.png" />
        </Frame>
      </Step>

      <Step title="Select the target version (next minor version only)" />

      <Step title="Confirm the upgrade" />
    </Steps>

    <p>The upgrade cannot be paused or cancelled once started.</p>

    ## After upgrading

    <p>After the control plane upgrades, update the local kubectl client if the version skew exceeds one minor version:</p>

    ```bash theme={null}
    kubectl version
    ```

    **Update the Cluster Autoscaler**

    <p>The Cluster Autoscaler image must match the cluster's Kubernetes minor version. Find the latest release for the new version on the [releases page](https://github.com/kubernetes/autoscaler/releases), then update the image:</p>

    ```bash theme={null}
    kubectl -n kube-system set image deployment.apps/cluster-autoscaler \
      cluster-autoscaler=registry.k8s.io/autoscaling/cluster-autoscaler:v<X.XX.X>
    ```

    <p>Replace `<X.XX.X>` with the release that matches the upgraded cluster version. If the cluster is now v1.35, use the latest `1.35.x` Cluster Autoscaler release. A version mismatch causes the autoscaler to stop scaling correctly.</p>

    <p>Verify the rollout completed:</p>

    ```bash theme={null}
    kubectl -n kube-system rollout status deployment/cluster-autoscaler
    ```

    **Update the NVIDIA device plugin (GPU bare metal pools only)**

    <p>The NVIDIA device plugin DaemonSet must be updated after a cluster upgrade to maintain GPU resource scheduling. Find the latest compatible release on the [releases page](https://github.com/NVIDIA/k8s-device-plugin/releases), then apply it:</p>

    ```bash theme={null}
    kubectl apply -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/v<X.X.X>/deployments/static/nvidia-device-plugin.yml
    ```

    <p>Replace `<X.X.X>` with the target plugin version. Verify the DaemonSet is running on all GPU nodes:</p>

    ```bash theme={null}
    kubectl get daemonset -n kube-system nvidia-device-plugin-daemonset
    ```

    <p>All pods in the DaemonSet should show `READY` equal to `DESIRED` before scheduling GPU workloads.</p>
  </MethodSection>

  <MethodSection id="api" label="REST API">
    <p>Check available upgrade versions and trigger a rolling upgrade to the next Kubernetes minor version.</p>

    <Info>
      An [API token](/account-settings/api-tokens) is required, along with a
      [project ID](/api-reference/cloud/projects/list-projects)
      and a [region ID](/api-reference/cloud/regions/list-regions).
    </Info>

    <p>Set these environment variables before running the examples:</p>

    ```bash theme={null}
    export GCORE_API_KEY="{YOUR_API_KEY}"
    export GCORE_CLOUD_PROJECT_ID="{YOUR_PROJECT_ID}"
    export GCORE_CLOUD_REGION_ID="{YOUR_REGION_ID}"
    export GCORE_CLUSTER_NAME="{YOUR_CLUSTER_NAME}"
    ```

    ## Upgrade the cluster

    <p>Upgrades the cluster to the next available Kubernetes minor version using a rolling update. Check available versions before starting, then trigger the upgrade.</p>

    <Warning>
      Upgrades move exactly one minor version at a time. The upgrade cannot be paused or cancelled once started. Verify that the cluster has quota for one additional node of the same flavor during the rolling update.
    </Warning>

    <Tabs>
      <Tab title="Python SDK">
        ```python theme={null}
        import os
        import time
        from gcore import Gcore

        client = Gcore(api_key=os.environ["GCORE_API_KEY"])
        PROJECT = int(os.environ["GCORE_CLOUD_PROJECT_ID"])
        REGION = int(os.environ["GCORE_CLOUD_REGION_ID"])
        CLUSTER = os.environ["GCORE_CLUSTER_NAME"]

        # Step 1. Check available upgrade versions
        versions = client.cloud.k8s.clusters.list_versions_for_upgrade(
            cluster_name=CLUSTER, project_id=PROJECT, region_id=REGION
        )
        if not versions.results:
            print("No upgrades available.")
            exit()
        target = versions.results[0].version
        print(f"Upgrading to {target}")

        # Step 2. Start the upgrade
        task_list = client.cloud.k8s.clusters.upgrade(
            cluster_name=CLUSTER,
            project_id=PROJECT,
            region_id=REGION,
            version=target,
        )
        task_id = task_list.tasks[0]

        # Step 3. Poll until finished
        while True:
            task = client.cloud.tasks.get(task_id)
            if task.state == "FINISHED":
                print("Upgrade complete.")
                break
            if task.state == "ERROR":
                raise RuntimeError(f"Upgrade failed: {task}")
            print(f"Upgrade state: {task.state}")
            time.sleep(30)
        ```
      </Tab>

      <Tab title="Go SDK">
        ```go theme={null}
        package main

        import (
        	"context"
        	"fmt"
        	"os"
        	"strconv"
        	"time"

        	gcore "github.com/G-Core/gcore-go"
        	"github.com/G-Core/gcore-go/cloud"
        	"github.com/G-Core/gcore-go/option"
        )

        func main() {
        	projectID, _ := strconv.ParseInt(os.Getenv("GCORE_CLOUD_PROJECT_ID"), 10, 64)
        	regionID, _ := strconv.ParseInt(os.Getenv("GCORE_CLOUD_REGION_ID"), 10, 64)
        	clusterName := os.Getenv("GCORE_CLUSTER_NAME")

        	client := gcore.NewClient(option.WithAPIKey(os.Getenv("GCORE_API_KEY")))

        	// Step 1. Check available upgrade versions
        	versions, err := client.Cloud.K8S.Clusters.ListVersionsForUpgrade(
        		context.TODO(),
        		clusterName,
        		cloud.K8SClusterListVersionsForUpgradeParams{
        			ProjectID: gcore.Int(projectID),
        			RegionID:  gcore.Int(regionID),
        		},
        	)
        	if err != nil {
        		panic(err)
        	}
        	if len(versions.Results) == 0 {
        		fmt.Println("No upgrades available.")
        		return
        	}
        	target := versions.Results[0].Version
        	fmt.Printf("Upgrading to %s\n", target)

        	// Step 2. Start the upgrade
        	taskList, err := client.Cloud.K8S.Clusters.Upgrade(
        		context.TODO(),
        		clusterName,
        		cloud.K8SClusterUpgradeParams{
        			ProjectID: gcore.Int(projectID),
        			RegionID:  gcore.Int(regionID),
        			Version:   target,
        		},
        	)
        	if err != nil {
        		panic(err)
        	}
        	taskID := taskList.Tasks[0]

        	// Step 3. Poll until finished
        	for {
        		task, err := client.Cloud.Tasks.Get(context.TODO(), taskID)
        		if err != nil {
        			panic(err)
        		}
        		if task.State == "FINISHED" {
        			fmt.Println("Upgrade complete.")
        			break
        		}
        		if task.State == "ERROR" {
        			panic(fmt.Sprintf("Upgrade failed: %+v", task))
        		}
        		fmt.Printf("Upgrade state: %s\n", task.State)
        		time.Sleep(30 * time.Second)
        	}
        }
        ```
      </Tab>

      <Tab title="curl">
        ```bash theme={null}
        # Step 1. Check available upgrade versions
        curl "https://api.gcore.com/cloud/v2/k8s/clusters/$GCORE_CLOUD_PROJECT_ID/$GCORE_CLOUD_REGION_ID/$GCORE_CLUSTER_NAME/upgrade_versions" \
          -H "Authorization: APIKey $GCORE_API_KEY"
        ```

        Response:

        ```json theme={null}
        {"count": 1, "results": [{"version": "v1.36.1"}]}
        ```

        ```bash theme={null}
        # Step 2. Start the upgrade
        curl -X POST "https://api.gcore.com/cloud/v2/k8s/clusters/$GCORE_CLOUD_PROJECT_ID/$GCORE_CLOUD_REGION_ID/$GCORE_CLUSTER_NAME/upgrade" \
          -H "Authorization: APIKey $GCORE_API_KEY" \
          -H "Content-Type: application/json" \
          -d '{"version": "v1.36.1"}'
        ```

        Response:

        ```json theme={null}
        {"tasks": ["d3b8f2a1-4c5e-6789-abcd-ef0123456789"]}
        ```

        <p>Run <code>GET /cloud/v1/tasks/{task_id}</code> every 30 seconds until `state` is `FINISHED`. GPU bare metal node upgrades take longer than standard VM upgrades.</p>
      </Tab>
    </Tabs>
  </MethodSection>
</MethodSwitch>