学习 Kubernetes 基础知识

任务

Edit This Page

将 kubeadm 集群从 v1.12 升级到 v1.13

本页介绍了如何将 kubeadm 创建的 Kubernetes 集群从 1.12.x 版本升级到 1.13.x 版本，以及从版本 1.13.x 升级到 1.13.y ，其中 y > x。

准备开始

附加信息

附加信息

Warning: The command join --experimental-control-plane is known to fail on single node clusters created with kubeadm v1.12 and then upgraded to v1.13.x. This will be fixed when graduating the join --control-plane workflow from alpha to beta. A possible workaround is described here.

Determine which version to upgrade to

  1. Find the latest stable 1.13 version: 
apt update
apt-cache policy kubeadm
# find the latest 1.13 version in the list
# it should look like 1.13.x-00, where x is the latest patch

yum list --showduplicates kubeadm --disableexcludes=kubernetes
# find the latest 1.13 version in the list
# it should look like 1.13.x-0, where x is the latest patch

升级控制平面

  1. 在主节点上，升级 kubeadm： 
# replace x in 1.13.x-00 with the latest patch version
apt-mark unhold kubeadm && \
apt-get update && apt-get install -y kubeadm=1.13.x-00 && \
apt-mark hold kubeadm

# replace x in 1.13.x-0 with the latest patch version
yum install -y kubeadm-1.13.x-0 --disableexcludes=kubernetes

  1. 验证下载是否有效并且是预期的版本

    kubeadm version

  1. 在主节点上，运行：

    kubeadm upgrade plan

    您应该可以看到与下面类似的输出：

    [preflight] Running pre-flight checks.
[upgrade] Making sure the cluster is healthy:
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[upgrade] Fetching available versions to upgrade to
[upgrade/versions] Cluster version: v1.12.2
[upgrade/versions] kubeadm version: v1.13.0

Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT   CURRENT       AVAILABLE
Kubelet     2 x v1.12.2   v1.13.0

Upgrade to the latest version in the v1.12 series:

COMPONENT            CURRENT   AVAILABLE
API Server           v1.12.2   v1.13.0
Controller Manager   v1.12.2   v1.13.0
Scheduler            v1.12.2   v1.13.0
Kube Proxy           v1.12.2   v1.13.0
CoreDNS              1.2.2     1.2.6
Etcd                 3.2.24    3.2.24

You can now apply the upgrade by executing the following command:

        kubeadm upgrade apply v1.13.0

_____________________________________________________________________
此命令检查您的集群是否可以升级，并可以获取到升级的版本。

  1. 选择要升级到的版本，然后运行相应的命令。 例如：

    kubeadm upgrade apply v1.13.0
您应该可以看见与下面类似的输出：

<!-- TODO: output from stable -->

```shell
[preflight] Running pre-flight checks.
[upgrade] Making sure the cluster is healthy:
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[upgrade/apply] Respecting the --cri-socket flag that is set with higher priority than the config file.
[upgrade/version] You have chosen to change the cluster version to "v1.13.0"
[upgrade/versions] Cluster version: v1.12.2
[upgrade/versions] kubeadm version: v1.13.0
[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y
[upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler etcd]
[upgrade/prepull] Prepulling image for component etcd.
[upgrade/prepull] Prepulling image for component kube-controller-manager.
[upgrade/prepull] Prepulling image for component kube-scheduler.
[upgrade/prepull] Prepulling image for component kube-apiserver.
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-etcd
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-etcd
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler
[upgrade/prepull] Prepulled image for component etcd.
[upgrade/prepull] Prepulled image for component kube-apiserver.
[upgrade/prepull] Prepulled image for component kube-scheduler.
[upgrade/prepull] Prepulled image for component kube-controller-manager.
[upgrade/prepull] Successfully prepulled the images for all the control plane components
[upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.13.0"...
Static pod: kube-apiserver-ip-10-0-0-7 hash: 4af3463d6ace12615f1795e40811c1a1
Static pod: kube-controller-manager-ip-10-0-0-7 hash: a640b0098f5bddc701786e007c96e220
Static pod: kube-scheduler-ip-10-0-0-7 hash: ee7b1077c61516320f4273309e9b4690
map[localhost:2379:3.2.24]
[upgrade/staticpods] Writing new Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests969681047"
[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to     "/etc/kubernetes/tmp/kubeadm-backup-manifests-2018-11-20-18-30-42/kube-apiserver.yaml"
[upgrade/staticpods] Waiting for the kubelet to restart the component
[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)
Static pod: kube-apiserver-ip-10-0-0-7 hash: 4af3463d6ace12615f1795e40811c1a1
Static pod: kube-apiserver-ip-10-0-0-7 hash: bf5b045d2be93e73654f3eb7027a4ef8
[apiclient] Found 1 Pods for label selector component=kube-apiserver
[upgrade/staticpods] Component "kube-apiserver" upgraded successfully!
[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to     "/etc/kubernetes/tmp/kubeadm-backup-manifests-2018-11-20-18-30-42/kube-controller-manager.yaml"
[upgrade/staticpods] Waiting for the kubelet to restart the component
[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)
Static pod: kube-controller-manager-ip-10-0-0-7 hash: a640b0098f5bddc701786e007c96e220
Static pod: kube-controller-manager-ip-10-0-0-7 hash: 1e0eea23b3d971460ac032c18ab7daac
[apiclient] Found 1 Pods for label selector component=kube-controller-manager
[upgrade/staticpods] Component "kube-controller-manager" upgraded successfully!
[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to     "/etc/kubernetes/tmp/kubeadm-backup-manifests-2018-11-20-18-30-42/kube-scheduler.yaml"
[upgrade/staticpods] Waiting for the kubelet to restart the component
[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)
Static pod: kube-scheduler-ip-10-0-0-7 hash: ee7b1077c61516320f4273309e9b4690
Static pod: kube-scheduler-ip-10-0-0-7 hash: 7f7d929b61a2cc5bcdf36609f75927ec
[apiclient] Found 1 Pods for label selector component=kube-scheduler
[upgrade/staticpods] Component "kube-scheduler" upgraded successfully!
[uploadconfig] storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.13" in namespace kube-system with the configuration for the kubelets in the cluster
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.13" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "ip-10-0-0-7" as an annotation
[bootstraptoken] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstraptoken] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstraptoken] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.13.0". Enjoy!

[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.
```

  1. 手动升级软件定义网络（SDN）。

    您的容器网络接口（CNI）应该提供了程序自身的升级说明。 检查 addons 页面以 查找您 CNI 所提供的程序，并查看是否需要其他升级步骤。

  1. 升级控制平面节点上的 kubelet: 
# 将 1.13.x-00 中的 x 替换为最新的版本
apt-mark unhold kubelet && \
apt-get update && apt-get install -y kubelet=1.13.x-00 && \
apt-mark hold kubelet

# 将 1.13.x-00 中的 x 替换为最新的版本
yum install -y kubelet-1.13.x-0 --disableexcludes=kubernetes

升级所有节点上的 kubectl

  1. 升级所有节点上的 kubectl: 
# 将 1.13.x-00 中的 x 替换为最新的版本
apt-mark unhold kubectl && \
apt-get update && apt-get install -y kubectl=1.13.x-00 && \
apt-mark hold kubectl

# 将 1.13.x-00 中的 x 替换为最新的版本
yum install -y kubectl-1.13.x-0 --disableexcludes=kubernetes

清空控制平面和工作节点

  1. 在维护前，将每个节点标记为不可调度并移除工作负载来做好准备。 运行：

    kubectl drain $NODE --ignore-daemonsets
在控制平面节点上，必须添加 `--ignore-daemonsets`:

```shell
kubectl drain ip-172-31-85-18
node "ip-172-31-85-18" cordoned
error: unable to drain node "ip-172-31-85-18", aborting command...

There are pending nodes to be drained:
ip-172-31-85-18
error: DaemonSet-managed pods (use --ignore-daemonsets to ignore): calico-node-5798d, kube-proxy-thjp9
```

```
kubectl drain ip-172-31-85-18 --ignore-daemonsets
node "ip-172-31-85-18" already cordoned
WARNING: Ignoring DaemonSet-managed pods: calico-node-5798d, kube-proxy-thjp9
node "ip-172-31-85-18" drained
```

在工作节点上更新 kubelet 配置

  1. 在除控制平面节点的其它节点上，更新 kubelet 配置:

    kubeadm upgrade node config --kubelet-version v1.13.x
将 `x` 替换为您本次将要升级到的版本号。

在每个工作节点上升级 kubelet

  1. 在每个 $NODE 节点上使用您的发行版对应的 Linux 包管理器: 
# # 将 1.13.x-00 中的 x 替换为最新的版本
apt-mark unhold kubelet kubeadm
apt-get update
apt-get install -y kubelet=1.13.x-00 kubeadm=1.13.x-00
apt-mark hold kubelet kubeadm

# # 将 1.13.x-00 中的 x 替换为最新的版本
yum install -y kubelet-1.13.x-0 kubeadm-1.13.x-0 --disableexcludes=kubernetes

重启所有节点上的 kubelet

  1. 重启所有节点上的 kubelet 进程:

    systemctl restart kubelet

  2. 验证节点上新版本的 kubelet 处于运行状态:

    systemctl status kubelet

  3. 将节点状态标记为可调度:

    kubectl uncordon $NODE

  4. 在所有节点上升级 kubelet 之后，通过从任何 kubectl 可以访问集群的位置运行以下命令来验证所有节点是否可用：

    kubectl get nodes

    您所有节点的 STATUS 列应为 Ready，并且版本信息应该是更新后的.

从失败状态恢复

如果 kubeadm upgrade 失败且没有回滚，比如在运行过程中意外被关闭，您可以重新运行kubeadm upgrade 。 此命令是幂等的，并最终确保实际状态是您声明的所需状态。

要从错误状态恢复，您还可以运行 kubeadm upgrade --force 而无需更改群集正在运行的版本。

工作原理

kubeadm upgrade apply 指令做了如下操作:

反馈