Kubernetes集群中怎么利用CRD 实现分批发布

这篇文章给大家介绍Kubernetes集群中怎么利用CRD 实现分批发布，内容非常详细，感兴趣的小伙伴们可以参考借鉴，希望对大家能有所帮助。

使用kubectl进行分批发布

新集群默认安装了alicloud-application-controller，老集群请先手动安装，安装方式，
kubectl create -f alicloud-application-controller.yml

apiVersion: extensions/v1beta1 kind: Deploymentmetadata:   name: alicloud-application-controller   labels:     owner: aliyun     app: alicloud-application-controller   namespace: kube-systemspec:   replicas: 1   selector:     matchLabels:       owner: aliyun       app: alicloud-application-controller   template:     metadata:       labels:         owner: aliyun         app: alicloud-application-controller       annotations:         scheduler.alpha.kubernetes.io/critical-pod: ''     spec:       tolerations:       - effect: NoSchedule         operator: Exists         key: node-role.kubernetes.io/master       - effect: NoSchedule         operator: Exists         key: node.cloudprovider.kubernetes.io/uninitialized       containers:         - name: alicloud-application-controller           image: registry.cn-hangzhou.aliyuncs.com/acs/aliyun-app-lifecycle-manager:0.1-93095ba           imagePullPolicy: IfNotPresent       serviceAccount: admin

我们这里以下面这个StatefulSet为例子演示分批发布的流程，

apiVersion: apps/v1 kind: StatefulSetmetadata:   name: webspec:   selector:     matchLabels:       app: nginx # has to match .spec.template.metadata.labels   serviceName: "nginx"   replicas: 3 # by default is 1   template:     metadata:       labels:         app: nginx # has to match .spec.selector.matchLabels     spec:       terminationGracePeriodSeconds: 10       containers:       - name: nginx         image: registry.cn-hangzhou.aliyuncs.com/xianlu/old-nginx         ports:         - containerPort: 80           name: web

这是一个含有三个实例的nginx 实例，为了暴露此容器，我们使用Service来暴露，Service的Yaml如下

apiVersion: v1 kind: Servicemetadata:   name: nginx   labels:     app: nginxspec:   ports:   - port: 80     name: web   selector:     app: nginx   type: LoadBalancer

这里通过SLB 暴露nginx 服务。
为了保证用户的Yaml安全性，分批发布需要使用Secret来存储最终的yaml，这里需要将yaml执行一下base64操作，再存储。
这个是StatefulSet的Secret Yaml

apiVersion: v1 kind: Secretmetadata:   name: mysts type: Opaquedata:   yaml: YXBpVmVyc2lvbjogYXBwcy92MQpraW5kOiBTdGF0ZWZ1bFNldAptZXRhZGF0YToKICBuYW1lOiB3ZWIKc3BlYzoKICBzZWxlY3RvcjoKICAgIG1hdGNoTGFiZWxzOgogICAgICBhcHA6IG5naW54ICMgaGFzIHRvIG1hdGNoIC5zcGVjLnRlbXBsYXRlLm1ldGFkYXRhLmxhYmVscwogIHNlcnZpY2VOYW1lOiAibmdpbngiCiAgcmVwbGljYXM6IDMgIyBieSBkZWZhdWx0IGlzIDEKICB0ZW1wbGF0ZToKICAgIG1ldGFkYXRhOgogICAgICBsYWJlbHM6CiAgICAgICAgYXBwOiBuZ2lueCAjIGhhcyB0byBtYXRjaCAuc3BlYy5zZWxlY3Rvci5tYXRjaExhYmVscwogICAgc3BlYzoKICAgICAgdGVybWluYXRpb25HcmFjZVBlcmlvZFNlY29uZHM6IDEwCiAgICAgIGNvbnRhaW5lcnM6CiAgICAgIC0gbmFtZTogbmdpbngKICAgICAgICBpbWFnZTogcmVnaXN0cnkuY24taGFuZ3pob3UuYWxpeXVuY3MuY29tL3hpYW5sdS9vbGQtbmdpbngKICAgICAgICBwb3J0czoKICAgICAgICAtIGNvbnRhaW5lclBvcnQ6IDgwCiAgICAgICAgICBuYW1lOiB3ZWI=

下面的为Service的Secret Yaml

apiVersion: v1 kind: Secretmetadata:   name: mysvc type: Opaquedata:   yaml: YXBpVmVyc2lvbjogdjEKa2luZDogU2VydmljZQptZXRhZGF0YToKICBuYW1lOiBuZ2lueAogIGxhYmVsczoKICAgIGFwcDogbmdpbngKc3BlYzoKICBwb3J0czoKICAtIHBvcnQ6IDgwCiAgICBuYW1lOiB3ZWIKICBzZWxlY3RvcjoKICAgIGFwcDogbmdpbngKICB0eXBlOiBMb2FkQmFsYW5jZXI=

分别将上面的两个Secret 创建出来，kubectl create -f xxxxx

cdn.com/a20640c6e505c48a39cec1b05f30d748796d2221.png">

可以看到两个Secret都已经创建完毕，下面就可以来创建分批发布的 CRD。分批发布的 CRD 格式如下，

apiVersion: alicloud.com/v1beta1 kind: BatchReleasemetadata:   name: example-batch-release   annotations:     aliyun.batchnum: "2"spec:   statefulSetSecretName: mysts   serviceSecretName: mysvc

�aliyun.batchnum: 代表分几批发布，目前默认是2批
statefulSetSecretName: 表示存储StatefulSet的Secret 名称
serviceSecretName: 表示存储Service的Secret 名称
使用kubectl create -f xxx 创建此CRD分批发布

可以看到，描述资源都已经创建出来了。

[root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl get stsNAME      DESIRED   CURRENT   AGE web       3         3         6m [root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl get pods -o=wideNAME      READY     STATUS    RESTARTS   AGE       IP             NODE web-0     1/1       Running   0          7m        172.16.2.2     cn-hangzhou.i-bp199b7a244chaux4ozh web-1     1/1       Running   0          6m        172.16.2.131   cn-hangzhou.i-bp199b7a244chaux4ozi web-2     1/1       Running   0          6m        172.16.1.136   cn-hangzhou.i-bp199b7a244chaux4ozg [root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl get svcNAME         TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)        AGE kubernetes   ClusterIP      172.19.0.1      <none>           443/TCP        4h nginx        LoadBalancer   172.19.12.163   120.55.148.238   80:32065/TCP   7m [root@iZbp11x2k7by5gfy6pkrl1Z ~]# curl 120.55.148.238old

可以看到，nginx 的StatefulSet和对应的Service 都已经创建出来，并且curl 是可以直接访问。

下面我们开始分批发布一个新版本的StatefulSet，这里我们新的StatefulSet Yaml 模板如下

apiVersion: apps/v1 kind: StatefulSet metadata:   name: web spec:   selector:     matchLabels:       app: nginx # has to match .spec.template.metadata.labels   serviceName: "nginx"   replicas: 3 # by default is 1   template:     metadata:       labels:         app: nginx # has to match .spec.selector.matchLabels     spec:       terminationGracePeriodSeconds: 10       containers:       - name: nginx         image: registry.cn-hangzhou.aliyuncs.com/xianlu/new-nginx         ports:         - containerPort: 80           name: web

可以看到，新老StatefulSet的区别在于，更换了新版本的镜像。
同样，我们需要创建一个Secret 来存储这个新的StatefulSet

apiVersion: v1 kind: Secretmetadata:   name: my-new-sts type: Opaquedata:   yaml: YXBpVmVyc2lvbjogYXBwcy92MQpraW5kOiBTdGF0ZWZ1bFNldAptZXRhZGF0YToKICBuYW1lOiB3ZWIKc3BlYzoKICBzZWxlY3RvcjoKICAgIG1hdGNoTGFiZWxzOgogICAgICBhcHA6IG5naW54ICMgaGFzIHRvIG1hdGNoIC5zcGVjLnRlbXBsYXRlLm1ldGFkYXRhLmxhYmVscwogIHNlcnZpY2VOYW1lOiAibmdpbngiCiAgcmVwbGljYXM6IDMgIyBieSBkZWZhdWx0IGlzIDEKICB0ZW1wbGF0ZToKICAgIG1ldGFkYXRhOgogICAgICBsYWJlbHM6CiAgICAgICAgYXBwOiBuZ2lueCAjIGhhcyB0byBtYXRjaCAuc3BlYy5zZWxlY3Rvci5tYXRjaExhYmVscwogICAgc3BlYzoKICAgICAgdGVybWluYXRpb25HcmFjZVBlcmlvZFNlY29uZHM6IDEwCiAgICAgIGNvbnRhaW5lcnM6CiAgICAgIC0gbmFtZTogbmdpbngKICAgICAgICBpbWFnZTogcmVnaXN0cnkuY24taGFuZ3pob3UuYWxpeXVuY3MuY29tL3hpYW5sdS9uZXctbmdpbngKICAgICAgICBwb3J0czoKICAgICAgICAtIGNvbnRhaW5lclBvcnQ6IDgwCiAgICAgICAgICBuYW1lOiB3ZWI=

然后我们更新刚才创建的 CRD，将statefulSetSecretName改成新的StatefulSet名称。
修改后的CRD Yaml 如下:

[root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl get BatchRelease example-batch-release -o=yaml apiVersion: alicloud.com/v1beta1 kind: BatchRelease metadata:   annotations:     aliyun.batchnum: "2"   clusterName: ""   creationTimestamp: 2018-07-31T08:17:17Z   generation: 1   name: example-batch-release  namespace: default   resourceVersion: "43484"   selfLink: /apis/alicloud.com/v1beta1/namespaces/default/batchreleases/example-batch-release   uid: 2386ddfd-949a-11e8-a3c2-00163e086528spec:   serviceSecretName: mysvc   statefulSetSecretName: my-new-sts

[root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl describe BatchRelease example-batch-releaseName:         example-batch-release Namespace:    default Labels:       <none> Annotations:  aliyun.batchnum=2API Version:  alicloud.com/v1beta1 Kind:         BatchRelease Metadata:   Cluster Name:   Creation Timestamp:  2018-07-31T08:17:17Z   Generation:          1   Resource Version:    43484   Self Link:           /apis/alicloud.com/v1beta1/namespaces/default/batchreleases/example-batch-release   UID:                 2386ddfd-949a-11e8-a3c2-00163e086528 Spec:   Service Secret Name:       mysvc   Stateful Set Secret Name:  my-new-sts Status:   Control:   Release:     Batch _ Order:  1     Progress:       finished     Status:         WaitingForConfirm   Resources:     Service:       Name:       nginx       Namespace:  default       Status:     Updating     Stateful Set:       Name:       web       Namespace:  default       Status:     Updating Events:   Type    Reason             Age   From               Message   ----    ------             ----  ----               -------   Normal  CreateSubResource  2m    aliyun-controller  Create StatefulSet resource sucessfully   Normal  CreateSubResource  2m    aliyun-controller  Create service resource sucessfully   Normal  Synced             2m    aliyun-controller  Batch CRD synced successfully   Normal  UpdateResource     1m    aliyun-controller  Begin to update StatefulSet web with partition 2   Normal  UpdateResource     1m    aliyun-controller  Waiting StatefulSet default:web ready, readyReplicas 2 replicas 3 times 1   Normal  UpdateResource     1m    aliyun-controller  Waiting StatefulSet default:web ready, readyReplicas 2 replicas 3 times 2   Normal  UpdateResource     1m    aliyun-controller  Begin to update service with new selector map[app:nginx aliyun.version:2]   Normal  UpdateResource     1m    aliyun-controller  Update StatefulSet and Service sucessfully, waiting confirm [root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl get pods -o=wideNAME      READY     STATUS    RESTARTS   AGE       IP             NODE web-0     1/1       Running   0          2m        172.16.2.132   cn-hangzhou.i-bp199b7a244chaux4ozi web-1     1/1       Running   0          2m        172.16.2.3     cn-hangzhou.i-bp199b7a244chaux4ozh web-2     1/1       Running   0          1m        172.16.1.138   cn-hangzhou.i-bp199b7a244chaux4ozg [root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl get svcNAME         TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE kubernetes   ClusterIP      172.19.0.1     <none>           443/TCP        5h nginx        LoadBalancer   172.19.2.184   101.37.107.187   80:32173/TCP   2m[root@iZbp11x2k7by5gfy6pkrl1Z ~]# curl 101.37.107.187new [root@iZbp11x2k7by5gfy6pkrl1Z ~]#

我们可以通过event 看到 CRD 的所有事件。同时可以看到，nginx的3个pod，已经有一个 pod发生了改变，变成了最新镜像，同时 curl对应的service，会发现流量已经指向了新的pod。这样用户就可以快速试错了。

下面我们来演示如何快速回滚，如果流量打到新的 pod后发现有问题，需要回滚，那么我们需要编辑CRD,

将Status 的action 置为rollback 就可以快速回滚。

[root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl edit BatchRelease example-batch-releasebatchrelease.alicloud.com "example-batch-release" edited [root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl get pods -o=wideNAME      READY     STATUS    RESTARTS   AGE       IP             NODE web-0     1/1       Running   0          7m        172.16.2.132   cn-hangzhou.i-bp199b7a244chaux4ozi web-1     1/1       Running   0          7m        172.16.2.3     cn-hangzhou.i-bp199b7a244chaux4ozh web-2     1/1       Running   0          3s        172.16.1.139   cn-hangzhou.i-bp199b7a244chaux4ozg [root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl get svcNAME         TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE kubernetes   ClusterIP      172.19.0.1     <none>           443/TCP        5h nginx        LoadBalancer   172.19.2.184   101.37.107.187   80:32173/TCP   7m [root@iZbp11x2k7by5gfy6pkrl1Z ~]# curl 101.37.107.187old

可以看到原来web-2所在的pod已经被回滚到了老镜像，现在curl service 也都是老的服务了。

发布第二批

发布第二批需要在action 置为continue 就可以

[root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl get pods -o=wide NAME      READY     STATUS    RESTARTS   AGE       IP             NODE web-0     1/1       Running   0          3s        172.16.2.138   cn-hangzhou.i-bp199b7a244chaux4ozi web-1     1/1       Running   0          13s       172.16.2.12    cn-hangzhou.i-bp199b7a244chaux4ozh web-2     1/1       Running   0          2m        172.16.1.153   cn-hangzhou.i-bp199b7a244chaux4ozg [root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl get BatchRelease example-batch-release -o=yaml apiVersion: alicloud.com/v1beta1 kind: BatchRelease metadata:   annotations:     aliyun.batchnum: "2"   clusterName: ""   creationTimestamp: 2018-07-31T09:44:58Z   generation: 1   name: example-batch-release  namespace: default   resourceVersion: "58066"   selfLink: /apis/alicloud.com/v1beta1/namespaces/default/batchreleases/example-batch-release   uid: 6321dcaf-94a6-11e8-a3c2-00163e086528spec:   serviceSecretName: mysvc   statefulSetSecretName: my-new-sts status:   control: {}   release:     batch_order: "2"     progress: finished     status: WaitingForConfirm   resources:     Service:       name: nginx      namespace: default       status: Updating     StatefulSet:       name: web      namespace: default       status: Updating

可以看到StatefulSet的三个 pod都被更新了。

这个时候就可以确认发布了

置为confirm 就确定了这次发布。

[root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl get BatchRelease example-batch-release -o=yaml apiVersion: alicloud.com/v1beta1 kind: BatchRelease metadata:   annotations:     aliyun.batchnum: "2"   clusterName: ""   creationTimestamp: 2018-07-31T09:44:58Z   generation: 1   name: example-batch-release  namespace: default   resourceVersion: "58542"   selfLink: /apis/alicloud.com/v1beta1/namespaces/default/batchreleases/example-batch-release   uid: 6321dcaf-94a6-11e8-a3c2-00163e086528spec:   serviceSecretName: mysvc   statefulSetSecretName: my-new-sts status:   control: {}   release:     progress: finished     status: Success   resources:     Service:       name: nginx      namespace: default       status: Success     StatefulSet:       name: web      namespace: default       status: Success [root@iZbp11x2k7by5gfy6pkrl1Z ~]# kubectl get secret -l owner=aliyun -n=kube-system NAME                       TYPE      DATA      AGE example-batch-release.v1   Opaque    4         18m example-batch-release.v2   Opaque    4         56s

关于Kubernetes集群中怎么利用CRD 实现分批发布就分享到这里了，希望以上内容可以对大家有一定的帮助，可以学到更多知识。如果觉得文章不错，可以把它分享出去让更多的人看到。