本文主要给大家介绍helm如何安装mysql教程，其所涉及的东西，从理论知识来获悉，有很多书籍、文献可供大家参考，从现实意义角度出发，亿速云累计多年的实践经验可分享给大家。
导航：
一、前提 1.版本信息-2.查看方法-3.查看k8s集群节点状态-4.配置docker仓库下载加速 二、安装部署helm 1.下载软件包-2.安装helm-3.初始化 三、使用helm安装mysql1.搜索mysql的helm包-2.准备后端存储-3.准备docker image-4.配置集群内的kube-dns-5.安装mysql的helm包-6.查看mysql的有状态应用、pod、svc、pvc等k8s资源-7.客户端验证 四、故障排除 五、参考资料使用helm安装有状态应用mysql主备集群，首先确保你已经搭建好一个3节点的k8s集群(如何搭建k8s集群、k8s集群网络配置flannel请参考百度)。
k8s: v1.5.2 docker: 1.13.1 flannel:0.7.1 helm: v3.0.0kubectl version docker version flanneld --version helm version[root@master ~]# kubectl get node NAME STATUS AGE master Ready 4d node2 Ready 18h node3 Ready 4m [root@master ~]# 使用163的国内docker仓库，提高docker image的下载速度。
[root@master ~]# cat /etc/docker/daemon.json {"registry-mirrors": ["http://hub-mirror.c.163.com"]} [root@master ~]# systemctl restart docker #重启docker服务生效 [root@master ~]# 使用helm的v3版本，v3版本不用配置服务端组件 Tiller。
下载helm的二进制包。
下载地址：
https://space.dingtalk.com/s/gwHOAF91cwLOABfU0gPaACBkNjc5YmMwMDUzZmQ0YWQwYjg2MmNhZWRiMGU3YmU1ZA 密码: 5kEK
使用xshell或者winscp将安装包传输到linux中。
[root@master ~]# tar -tvf helm-v3.0.0-alpha.1-linux-amd64.tar.gz #查看压缩包中的内容 drwxr-xr-x circleci/circleci 0 2019-05-16 05:44 linux-amd64/ -rwxr-xr-x circleci/circleci 39771936 2019-05-16 05:42 linux-amd64/helm -rw-r--r-- circleci/circleci 11373 2019-05-16 05:44 linux-amd64/LICENSE -rw-r--r-- circleci/circleci 3159 2019-05-16 05:44 linux-amd64/README.md [root@master ~]# 将二进制文件复制到$PATH路径下即可使用。
[root@master ~]# which helm #查看helm二进制文件所在的路径，将helm复制到$PATH中即可直接使用，例如将helm放在/usr/bin/helm的路径 /usr/bin/helm [root@master ~]# helm --help The Kubernetes package manager To begin working with Helm, run the 'helm init' command: $ helm init This will set up any necessary local configuration. Common actions from this point include: - helm search: search for charts - helm fetch: download a chart to your local directory to view - helm install: upload the chart to Kubernetes - helm list: list releases of charts Environment: $HELM_HOME set an alternative location for Helm files. By default, these are stored in ~/.helm $HELM_DRIVER set the backend storage driver. Values are: configmap, secret, memory $HELM_NO_PLUGINS disable plugins. Set HELM_NO_PLUGINS=1 to disable plugins. $KUBECONFIG set an alternative Kubernetes configuration file (default "~/.kube/config") Usage: helm [command] Available Commands: chart push, pull, tag, or remove Helm charts completion Generate autocompletions script for the specified shell (bash or zsh) create create a new chart with the given name dependency manage a chart's dependencies get download a named release help Help about any command history fetch release history home displays the location of HELM_HOME init initialize Helm client install install a chart lint examines a chart for possible issues list list releases package package a chart directory into a chart archive plugin add, list, or remove Helm plugins pull download a chart from a repository and (optionally) unpack it in local directory registry login to or logout from a registry repo add, list, remove, update, and index chart repositories rollback roll back a release to a previous revision search search for a keyword in charts show inspect a chart status displays the status of the named release template locally render templates test test a release or cleanup test artifacts uninstall uninstall a release upgrade upgrade a release verify verify that a chart at the given path has been signed and is valid version print the client version information Flags: --debug enable verbose output -h, --help help for helm --home string location of your Helm config. Overrides $HELM_HOME (default "/root/.helm") --kube-context string name of the kubeconfig context to use --kubeconfig string path to the kubeconfig file -n, --namespace string namespace scope for this request Use "helm [command] --help" for more information about a command. [root@master ~]# 使用helm前，需要初始化。
[root@docker repository]# helm init --stable-repo-url=https://apphub.aliyuncs.com #配置helm使用的仓库为国内阿里云地址，国外的访问不了，而且比较慢 Creating /root/.helm/repository/repositories.yaml Adding stable repo with URL: https://apphub.aliyuncs.com $HELM_HOME has been configured at /root/.helm. Happy Helming! [root@docker repository]# ls cache local repositories.yaml [root@docker repository]# cat repositories.yaml #helm初始化完后会有一个配置文件生成，下面是repositories.yaml的内容 apiVersion: v1 generated: "2019-08-25T07:08:38.917953914+08:00" repositories: - caFile: "" cache: /root/.helm/repository/cache/stable-index.yaml certFile: "" keyFile: "" name: stable password: "" url: https://apphub.aliyuncs.com username: ""使用helm安装有状态应用mysql，使用到的docker镜像有：
docker.io/mysql:latest docker.io/ist0ne/xtrabackup:latest构建mysql的1主2从，主备集群，使用xtrabackup工具进行数据同步。
[root@docker ~]# helm search mysql #搜索mysql的helm包 NAME CHART VERSION APP VERSION DESCRIPTION apphub/mysql 6.3.1 8.0.17 Chart to create a Highly available MySQL cluster apphub/mysqldump 2.6.0 2.4.1 A Helm chart to help backup MySQL databases usi... apphub/mysqlha 1.0.0 5.7.13 MySQL cluster with a single master and zero or ... apphub/prometheus-mysql-exporter 0.5.1 v0.11.0 A Helm chart for prometheus mysql exporter with... stable/mysql 6.3.1 8.0.17 Chart to create a Highly available MySQL cluster stable/mysqldump 2.6.0 2.4.1 A Helm chart to help backup MySQL databases usi... stable/mysqlha 1.0.0 5.7.13 MySQL cluster with a single master and zero or ... stable/prometheus-mysql-exporter 0.5.1 v0.11.0 A Helm chart for prometheus mysql exporter with... apphub/percona 1.1.0 5.7.17 free, fully compatible, enhanced, open source d... apphub/percona-xtradb-cluster 1.0.1 5.7.19 free, fully compatible, enhanced, open source d... apphub/phpmyadmin 3.0.1 4.9.0-1 phpMyAdmin is an mysql administration frontend stable/percona 1.1.0 5.7.17 free, fully compatible, enhanced, open source d... stable/percona-xtradb-cluster 1.0.1 5.7.19 free, fully compatible, enhanced, open source d... stable/phpmyadmin 3.0.1 4.9.0-1 phpMyAdmin is an mysql administration frontend apphub/mariadb 6.8.1 10.3.17 Fast, reliable, scalable, and easy to use open-... apphub/mariadb-galera 0.2.1 10.3.17 MariaDB Galera is a multi-master database clust... stable/mariadb 6.8.1 10.3.17 Fast, reliable, scalable, and easy to use open-... stable/mariadb-galera 0.2.1 10.3.17 MariaDB Galera is a multi-master database clust...由于mysql的helm包需要使用pv，而开源的k8s集群默认没有配置pv后端存储，因此需要手动添加nfs云服务器(如何搭建nfs服务器请参考百度)为后端存储。
nfs配置：
[root@master ~]# cat /etc/exports /nfsdata/pv001 *(rw,no_root_squash) #添加no_root_squash不压缩root权限，因为容器里会使用root权限写入数据，否则后面会有权限不足的报错 /nfsdata/pv002 *(rw,no_root_squash) /nfsdata/pv003 *(rw,no_root_squash) [root@master ~]# ls -ld /nfsdata/ #配置目录的权限为nfsnobody drwxr-xr-x 5 nfsnobody nfsnobody 45 Aug 26 01:18 /nfsdata/ [root@master ~]# ls -ll /nfsdata/ total 0 drwxr-xr-x 2 nfsnobody nfsnobody 6 Aug 26 01:18 pv001 drwxr-xr-x 2 nfsnobody nfsnobody 6 Aug 26 01:18 pv002 drwxr-xr-x 2 nfsnobody nfsnobody 6 Aug 26 01:18 pv003 [root@master ~]# 创建pv的yaml文件：
[root@master hrr]# cat pv.yml apiVersion: v1 kind: PersistentVolume metadata: name: pv003 spec: capacity: storage: 10Gi accessModes: - ReadWriteOnce #accessModes需要和pvc的模式一样 nfs: path: /nfsdata/pv003 server: 192.168.222.129 [root@master hrr]# 创建pv:
[root@docker hrr]# kubectl create -f pv.yml persistentvolume "pv003" created [root@docker hrr]#查看pv：
[root@master hrr]# kubectl get pv #Available表示没有被PVC使用 NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM REASON AGE pv001 10Gi RWO Retain Available 26s pv002 10Gi RWO Retain Available 14s pv003 10Gi RWO Retain Available 3s [root@master hrr]# [root@master hrr]# kubectl get pv #Bound表示被PVC使用 NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM REASON AGE pv001 10Gi RWO Retain Bound default/data-mysql-mysqlha-0 32s pv002 10Gi RWO Retain Bound default/data-mysql-mysqlha-1 17s pv003 10Gi RWO Retain Bound default/data-mysql-mysqlha-2 4s [root@master hrr]#在安装mysql的helm包时需要mysql与xtrabackup的docker镜像。
[root@docker docker]# docker pull docker.io/ist0ne/xtrabackup #下载xtrabackup镜像 Using default tag: latest Trying to pull repository docker.io/ist0ne/xtrabackup ... latest: Pulling from docker.io/ist0ne/xtrabackup 386a066cd84a: Pull complete 40f175e652e1: Pull complete 5eb6b5905d55: Pull complete Digest: sha256:e5e5519b874363f1bf35541cef2caed5803645937e29834817be51b30aabc479 Status: Downloaded newer image for docker.io/ist0ne/xtrabackup:latest [root@docker docker]# [root@docker docker]# docker pull docker.io/mysql #下载mysql镜像查看本地的镜像：
[root@master hrr]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE 192.168.222.129:5000/nginx latest 5a3221f0137b 9 days ago 126 MB docker.io/nginx latest 5a3221f0137b 9 days ago 126 MB docker.io/mysql latest 62a9f311b99c 11 days ago 445 MB mysql 5.7.13 62a9f311b99c 11 days ago 445 MB docker.io/ist0ne/xtrabackup latest c415dbd7af07 2 years ago 265 MB registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup 1.0 c415dbd7af07 2 years ago 265 MB 192.168.222.129:5000/pod-infrastructure latest f9d5de079539 5 years ago 240 kB docker.io/kubernetes/pause latest f9d5de079539 5 years ago 240 kB [root@master hrr]#将镜像打tag，用于后面创建mysql的helm包：
[root@docker hrr]# docker tag docker.io/mysql:latest mysql:5.7.13 [root@docker docker]# docker tag docker.io/ist0ne/xtrabackup:latest registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0将docker镜像导出，用于导入到其它k8s节点中：
[root@master ~]# docker save mysql:5.7.13 -o mysql5713.tgz #将docker镜像导出，名为mysql5713.tgz [root@master ~]# docker save registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0 -o xtrabackup10.tgz [root@master ~]# scp mysql5713.tgz xtrabackup10.tgz root@192.168.222.130: root@192.168.222.130's password: mysql5713.tgz 100% 429MB 35.7MB/s 00:12 xtrabackup10.tgz 100% 260MB 43.4MB/s 00:06 [root@master ~]#在其它k8s节点中导入docker镜像：
[root@node2 ~]# docker load -i mysql5713.tgz #将导出的docker镜像mysql5713.tgz导入 8fa655db5360: Loading layer [==================================================>] 58.48 MB/58.48 MB f5741d086b76: Loading layer [==================================================>] 338.4 kB/338.4 kB 9f77b78f01a7: Loading layer [==================================================>] 10.44 MB/10.44 MB 7f33ce1066af: Loading layer [==================================================>] 4.472 MB/4.472 MB 1dd5f3e365cf: Loading layer [==================================================>] 1.536 kB/1.536 kB 65430c57aee2: Loading layer [==================================================>] 46.15 MB/46.15 MB bb1ef34119b2: Loading layer [==================================================>] 32.77 kB/32.77 kB 995fa4bb8afe: Loading layer [==================================================>] 3.584 kB/3.584 kB bdebd19e878f: Loading layer [==================================================>] 329.8 MB/329.8 MB 79a46f1eb9d0: Loading layer [==================================================>] 5.632 kB/5.632 kB 5b96d5043191: Loading layer [==================================================>] 9.728 kB/9.728 kB 647229410363: Loading layer [==================================================>] 1.536 kB/1.536 kB Loaded image: mysql:5.7.13 [root@node2 ~]# docker load -i xtrabackup10.tgz fe4c16cbf7a4: Loading layer [==================================================>] 128.9 MB/128.9 MB 82d548d175dd: Loading layer [==================================================>] 9.216 kB/9.216 kB f85c58969eb0: Loading layer [==================================================>] 144 MB/144 MB Loaded image: registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0 [root@node2 ~]#node3做同样的操作。
[root@master ~]# helm install mysql apphub/mysqlha NAME: mysql LAST DEPLOYED: 2019-08-25 09:43:33.16471802 +0800 CST m=+0.898633546 NAMESPACE: default STATUS: deployed NOTES: The MySQL cluster is comprised of 3 MySQL pods: 1 master and 2 slaves. Each instance is accessible within the cluster through: <pod-name>.mysql-mysqlha `mysql-mysqlha-0.mysql-mysqlha` is designated as the master and where all writes should be executed against. Read queries can be executed against the `mysql-mysqlha-readonly` service which distributes connections across all MySQL pods. To connect to your database: 1. Obtain the root password: kubectl get secret --namespace default mysql-mysqlha -o jsonpath="{.data.mysql-root-password}" | base64 --decode; echo 2. Run a pod to use as a client: kubectl run mysql-client --image=mysql:5.7.13 -it --rm --restart='Never' --namespace default -- /bin/sh 3. To connect to Master service (read/write): mysql -h mysql-mysqlha-0.mysql-mysqlha -u root -p 4. To connect to slave service (read-only): mysql -h mysql-mysqlha-readonly -u root -p [root@master ~]#安装完后，在本地会有mysql的helm包：
[root@master archive]# pwd /root/.helm/cache/archive [root@master archive]# ls guestbook-0.2.0.tgz mysqlha-1.0.0.tgz [root@master archive]# [root@master archive]# tar -tvf mysqlha-1.0.0.tgz -rwxr-xr-x 0/0 378 2019-08-22 13:20 mysqlha/Chart.yaml -rwxr-xr-x 0/0 2561 2019-08-22 13:20 mysqlha/values.yaml -rwxr-xr-x 0/0 1216 2019-08-22 13:20 mysqlha/templates/NOTES.txt -rwxr-xr-x 0/0 961 2019-08-22 13:20 mysqlha/templates/_helpers.tpl -rwxr-xr-x 0/0 756 2019-08-22 13:20 mysqlha/templates/configmap.yaml -rwxr-xr-x 0/0 967 2019-08-22 13:20 mysqlha/templates/secret.yaml -rwxr-xr-x 0/0 10905 2019-08-22 13:20 mysqlha/templates/statefulset.yaml -rwxr-xr-x 0/0 1203 2019-08-22 13:20 mysqlha/templates/svc.yaml -rwxr-xr-x 0/0 56 2019-08-22 13:20 mysqlha/OWNERS -rwxr-xr-x 0/0 6724 2019-08-22 13:20 mysqlha/README.md [root@master archive]# 将mysqlha-1.0.0.tgz解压，查看文件内容：
[root@master mysqlha]# ls Chart.yaml OWNERS README.md templates values.yaml [root@master mysqlha]# cat README.md # MySQL - Single Master, Multiple Slaves [MySQL](https://MySQL.org) is one of the most popular database servers in the world. Notable users include Wikipedia, Facebook and Google. ## Introduction This chart bootstraps a single master and multiple slave MySQL deployment on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager. Largely inspired by this [tutorial](https://kubernetes.io/docs/tutorials/stateful-application/run-replicated-stateful-application/), further work was made to 'production-ize' the example. ## Prerequisites - Kubernetes 1.6+ - PV provisioner support in the underlying infrastructure ## Installing the Chart To install the chart with the release name `my-release`: $ helm install --name my-release <helm-repo>/mysqlha The command deploys MySQL cluster on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation. ### Uninstall To uninstall/delete the `my-release` deployment: $ helm delete my-release ## Configuration The following table lists the configurable parameters of the MySQL chart and their default values. | Parameter | Description | Default | | ----------------------------------------- | ------------------------------------------------- | -------------------------------------- | | `mysqlImage` | `mysql` image and tag. | `mysql:5.7.13` | | `xtraBackupImage` | `xtrabackup` image and tag. | `gcr.io/google-samples/xtrabackup:1.0` | | `imagePullPolicy` | Image pull policy. | `IfNotPresent` | | `nameOverride` | `String to partially override mysqlha.fullname template with a string (will prepend the release name)` | `nil` | | `fullnameOverride` | `String to fully override mysqlha.fullname template with a string` | `nil` | | `replicaCount` | Number of MySQL replicas | 3 | | `mysqlRootPassword` | Password for the `root` user. | Randomly generated | | `mysqlUser` | Username of new user to create. | `nil` | | `mysqlPassword` | Password for the new user. | Randomly generated | | `mysqlReplicationUser` | Username for replication user | `repl` | | `mysqlReplicationPassword` | Password for replication user. | Randomly generated | | `mysqlDatabase` | Name of the new Database to create | `nil` | | `configFiles.master.cnf` | Master configuration file | See `values.yaml` | | `configFiles.slave.cnf` | Slave configuration file | See `values.yaml` | | `podAnnotations` | Pod annotations | `nil` | | `schedulerName` | Name of the k8s scheduler (other than default) | `nil` | | `persistence.enabled` | Create a volume to store data | true | | `persistence.size` | Size of persistent volume claim | 10Gi | | `persistence.storageClass` | Type of persistent volume claim | `nil` | | `persistence.accessModes` | Persistent volume access modes | `[ReadWriteOnce]` | | `persistence.annotations` | Persistent volume annotations | `{}` | | `resources` | CPU/Memory resource requests/limits | Memory: `128Mi`, CPU: `100m` | | `metrics.enabled` | Start a side-car prometheus exporter | false | | `metrics.image` | Exporter image | `prom/mysqld-exporter` | | `metrics.imageTag` | Exporter image | `v0.10.0` | | `metrics.imagePullPolicy` | Exporter image pull policy | `IfNotPresent` | | `metrics.resources` | Exporter resource requests/limit | See `values.yaml` | | `metrics.livenessProbe.initialDelaySeconds` | Delay before metrics liveness probe is initiated | 15 | | `metrics.livenessProbe.timeoutSeconds` | When the probe times out | 5 | | `metrics.readinessProbe.initialDelaySeconds` | Delay before metrics readiness probe is initiated | 5 | | `metrics.readinessProbe.timeoutSeconds` | When the probe times out | 1 | Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example, ## Persistence The [MySQL](https://hub.docker.com/_/mysql/) image stores the MySQL data and configurations at the `/var/lib/mysql` path of the container. By default persistence is enabled, and a PersistentVolumeClaim is created and mounted in that directory. As a result, a persistent volume will need to be defined: # https://kubernetes.io/docs/user-guide/persistent-volumes/#azure-disk kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: fast provisioner: kubernetes.io/azure-disk parameters: skuName: Premium_LRS location: westus In order to disable this functionality you can change the values.yaml to disable persistence and use an emptyDir instead.查看values.yaml文件：
[root@master mysqlha]# cat values.yaml ## mysql image version ## ref: https://hub.docker.com/r/library/mysql/tags/ ## mysqlImage: mysql:5.7.13 #前面下载的mysql镜像 xtraBackupImage: registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0 #前面下载的xtrabackup镜像 ## Specify an imagePullPolicy (Required) ## It's recommended to change this to 'Always' if the image tag is 'latest' ## ref: http://kubernetes.io/docs/user-guide/images/#updating-images ## imagePullPolicy: IfNotPresent ## String to partially override orangehrm.fullname template (will maintain the release name) ## # nameOverride: "" ## String to fully override orangehrm.fullname template ## # fullnameOverride: "" mysqlha: replicaCount: 3 ## Password for MySQL root user ## # mysqlRootPassword: ## Default: random 10 character string ## Username/password for MySQL replication user ## mysqlReplicationUser: repl # mysqlReplicationPassword: ## Create a database user ## # mysqlUser: # mysqlPassword: ## Default: random 10 character string ## Allow unauthenticated access, uncomment to enable ## # mysqlAllowEmptyPassword: true ## Create database with name and grant all permissions to user on startup, if needed # mysqlDatabase: ## Configuration files for the master and slaves ## configFiles: master.cnf: | # Apply this config only on the master. [mysqld] log-bin skip_name_resolve slave.cnf: | # Apply this config only on slaves. [mysqld] super-read-only skip_name_resolve podAnnotations: {} ## Use an alternate scheduler, e.g. "stork". ## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/ ## # schedulerName: ## Enable persistence using Persistent Volume Claims ## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/ ## persistence: enabled: true ## If defined, storageClassName: <storageClass> ## If set to "-", storageClassName: "", which disables dynamic provisioning ## If undefined (the default) or set to null, no storageClassName spec is ## set, choosing the default provisioner. (gp2 on AWS, azure-disk on ## Azure, standard on GKE, AWS & OpenStack) ## # storageClass: "-" accessModes: - ReadWriteOnce size: 10Gi annotations: {} resources: requests: cpu: 100m memory: 128Mi metrics: enabled: false image: prom/mysqld-exporter imageTag: v0.10.0 annotations: {} livenessProbe: initialDelaySeconds: 15 timeoutSeconds: 5 readinessProbe: initialDelaySeconds: 5 timeoutSeconds: 1 resources: requests: cpu: 100m memory: 100Mi [root@master mysqlha]# [root@master hrr]# kubectl get statefulset NAME DESIRED CURRENT AGE mysql-mysqlha 3 1 18s [root@master hrr]# kubectl get svc NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes 10.254.0.1 <none> 443/TCP 4d mysql-mysqlha None <none> 3306/TCP 27s mysql-mysqlha-readonly 10.254.54.188 <none> 3306/TCP 27s [root@master hrr]# kubectl get pod NAME READY STATUS RESTARTS AGE mysql-mysqlha-0 2/2 Running 0 45s mysql-mysqlha-1 2/2 Running 0 24s mysql-mysqlha-2 1/2 Running 0 4s [root@master hrr]# kubectl get pv NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM REASON AGE pv001 10Gi RWO Retain Bound default/data-mysql-mysqlha-0 2m pv002 10Gi RWO Retain Bound default/data-mysql-mysqlha-2 2m pv003 10Gi RWO Retain Bound default/data-mysql-mysqlha-1 1m [root@master hrr]# kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESSMODES AGE data-mysql-mysqlha-0 Bound pv001 10Gi RWO 51s data-mysql-mysqlha-1 Bound pv003 10Gi RWO 51s data-mysql-mysqlha-2 Bound pv002 10Gi RWO 51s [root@master hrr]# kubectl get configmap NAME DATA AGE mysql-mysqlha 4 54s [root@master hrr]# kubectl get secret NAME TYPE DATA AGE mysql-mysqlha Opaque 2 51m mysql.v1 helm.sh/release 1 51m [root@master hrr]# [root@master hrr]# kubectl get pod -owide NAME READY STATUS RESTARTS AGE IP NODE mysql-mysqlha-0 2/2 Running 0 1m 172.17.93.2 node3 mysql-mysqlha-1 2/2 Running 0 41s 172.17.78.2 node2 mysql-mysqlha-2 2/2 Running 0 21s 172.17.86.2 master查看第一个节点的docker容器
[root@master hrr]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 41ec983706b8 registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0 "/bin/bash -c 'set..." 25 seconds ago Up 24 seconds k8s_xtrabackup.516a4a61_mysql-mysqlha-2_default_387e4cdb-c772-11e9-83c0-000c296be0a4_72aba107 691066dcf870 mysql:5.7.13 "docker-entrypoint..." 25 seconds ago Up 24 seconds k8s_mysql.b9eef278_mysql-mysqlha-2_default_387e4cdb-c772-11e9-83c0-000c296be0a4_894896de d83f031101ef 192.168.222.129:5000/pod-infrastructure:latest "/pause" 26 seconds ago Up 25 seconds k8s_POD.62b22cd9_mysql-mysqlha-2_default_387e4cdb-c772-11e9-83c0-000c296be0a4_c46314b7查看第二个节点的docker容器
[root@node2 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 211a5deed23f registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0 "/bin/bash -c 'set..." 52 seconds ago Up 51 seconds k8s_xtrabackup.516a4a61_mysql-mysqlha-1_default_2c7419c8-c772-11e9-83c0-000c296be0a4_982995a3 51d9e7cd3d1d mysql:5.7.13 "docker-entrypoint..." 52 seconds ago Up 51 seconds k8s_mysql.b9eef278_mysql-mysqlha-1_default_2c7419c8-c772-11e9-83c0-000c296be0a4_3acd8bf2 71bb21654249 192.168.222.129:5000/pod-infrastructure:latest "/pause" 52 seconds ago Up 51 seconds k8s_POD.62b22cd9_mysql-mysqlha-1_default_2c7419c8-c772-11e9-83c0-000c296be0a4_cce640c3 [root@node2 ~]# 查看第三个节点的docker容器
[root@node3 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d159b422bcdf registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0 "/bin/bash -c 'set..." About a minute ago Up About a minute k8s_xtrabackup.516a4a61_mysql-mysqlha-0_default_1fbcfe77-c772-11e9-83c0-000c296be0a4_63f2a879 6aae997c995d mysql:5.7.13 "docker-entrypoint..." About a minute ago Up About a minute k8s_mysql.b9eef278_mysql-mysqlha-0_default_1fbcfe77-c772-11e9-83c0-000c296be0a4_7dee904b 40695f86aaf7 192.168.222.129:5000/pod-infrastructure:latest "/pause" About a minute ago Up About a minute k8s_POD.62b22cd9_mysql-mysqlha-0_default_1fbcfe77-c772-11e9-83c0-000c296be0a4_58c67c1e [root@node3 ~]#查看pvc里的数据
[root@master hrr]# ll /nfsdata/pv001 total 4 drwxr-xr-x 6 polkitd input 4096 Aug 26 03:54 mysql [root@master hrr]# ll /nfsdata/pv001/mysql/ total 177164 -rw-r----- 1 polkitd input 56 Aug 26 03:54 auto.cnf -rw-r----- 1 polkitd input 3091158 Aug 26 03:54 binlog.000001 -rw-r----- 1 polkitd input 155 Aug 26 03:54 binlog.000002 -rw-r----- 1 polkitd input 32 Aug 26 03:54 binlog.index -rw------- 1 polkitd input 1676 Aug 26 03:54 ca-key.pem -rw-r--r-- 1 polkitd input 1112 Aug 26 03:54 ca.pem -rw-r--r-- 1 polkitd input 1112 Aug 26 03:54 client-cert.pem -rw------- 1 polkitd input 1680 Aug 26 03:54 client-key.pem -rw-r----- 1 polkitd input 5419 Aug 26 03:54 ib_buffer_pool -rw-r----- 1 polkitd input 12582912 Aug 26 03:54 ibdata1 -rw-r----- 1 polkitd input 50331648 Aug 26 03:54 ib_logfile0 -rw-r----- 1 polkitd input 50331648 Aug 26 03:54 ib_logfile1 -rw-r----- 1 polkitd input 12582912 Aug 26 03:55 ibtmp1 drwxr-x--- 2 polkitd input 187 Aug 26 03:54 #innodb_temp drwxr-x--- 2 polkitd input 143 Aug 26 03:54 mysql -rw-r----- 1 polkitd input 29360128 Aug 26 03:54 mysql.ibd drwxr-x--- 2 polkitd input 8192 Aug 26 03:54 performance_schema -rw------- 1 polkitd input 1680 Aug 26 03:54 private_key.pem -rw-r--r-- 1 polkitd input 452 Aug 26 03:54 public_key.pem -rw-r--r-- 1 polkitd input 1112 Aug 26 03:54 server-cert.pem -rw------- 1 polkitd input 1680 Aug 26 03:54 server-key.pem drwxr-x--- 2 polkitd input 28 Aug 26 03:54 sys -rw-r----- 1 polkitd input 12582912 Aug 26 03:54 undo_001 -rw-r----- 1 polkitd input 10485760 Aug 26 03:54 undo_002查看svc
[root@master hrr]# kubectl describe svc mysql-mysqlha Name: mysql-mysqlha Namespace: default Labels: app=mysql-mysqlha chart=mysqlha-1.0.0 heritage=Helm release=mysql Selector: app=mysql-mysqlha Type: ClusterIP IP: None Port: mysql-mysqlha 3306/TCP Endpoints: 172.17.78.2:3306,172.17.86.2:3306,172.17.93.2:3306 Session Affinity: None No events. [root@master hrr]# kubectl describe svc mysql-mysqlha-readonly Name: mysql-mysqlha-readonly Namespace: default Labels: app=mysql-mysqlha chart=mysqlha-1.0.0 heritage=Helm release=mysql Selector: app=mysql-mysqlha Type: ClusterIP IP: 10.254.54.188 Port: mysql-mysqlha 3306/TCP Endpoints: 172.17.78.2:3306,172.17.86.2:3306,172.17.93.2:3306 Session Affinity: None No events. [root@master hrr]# 查看有状态应用stateful
[root@master hrr]# kubectl get statefulset -oyaml apiVersion: v1 items: - apiVersion: apps/v1beta1 kind: StatefulSet metadata: creationTimestamp: 2019-08-25T19:54:18Z generation: 1 labels: app: mysql-mysqlha chart: mysqlha-1.0.0 heritage: Helm release: mysql name: mysql-mysqlha namespace: default resourceVersion: "94760" selfLink: /apis/apps/v1beta1/namespaces/default/statefulsets/mysql-mysqlha uid: 1fa6f230-c772-11e9-83c0-000c296be0a4 spec: replicas: 3 selector: matchLabels: app: mysql-mysqlha serviceName: mysql-mysqlha template: metadata: creationTimestamp: null labels: app: mysql-mysqlha spec: containers: - env: - name: MYSQL_DATABASE - name: MYSQL_ROOT_PASSWORD valueFrom: secretKeyRef: key: mysql-root-password name: mysql-mysqlha - name: MYSQL_REPLICATION_USER value: repl - name: MYSQL_REPLICATION_PASSWORD valueFrom: secretKeyRef: key: mysql-replication-password name: mysql-mysqlha image: mysql:5.7.13 imagePullPolicy: IfNotPresent livenessProbe: exec: command: - /bin/sh - -c - mysqladmin ping -h 127.0.0.1 -u root -p${MYSQL_ROOT_PASSWORD} failureThreshold: 3 initialDelaySeconds: 30 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 5 name: mysql ports: - containerPort: 3306 name: mysql protocol: TCP readinessProbe: exec: command: - /bin/sh - -c - MYSQL_PWD="${MYSQL_ROOT_PASSWORD}" - mysql -h 127.0.0.1 -u root -e "SELECT 1" failureThreshold: 3 initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 resources: requests: cpu: 100m memory: 128Mi terminationMessagePath: /dev/termination-log volumeMounts: - mountPath: /var/lib/mysql name: data subPath: mysql - mountPath: /etc/mysql/conf.d name: conf - args: - -c - | set -ex echo "Waiting for mysqld to be ready (accepting connections)" until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 5; done # Create replication user cd /mnt/scripts # file exists and is not empty with -s if [[ -s create-replication-user.sh ]]; then ls -la ./create-replication-user.sh fi cd /var/lib/mysql # Determine binlog position of cloned data, if any. if [[ -f xtrabackup_slave_info ]]; then # XtraBackup already generated a partial "CHANGE MASTER TO" query # because we're cloning from an existing slave. cp xtrabackup_slave_info change_master_to.sql.in elif [[ -f xtrabackup_binlog_info ]]; then # We're cloning directly from master. Parse binlog position. [[ $(cat xtrabackup_binlog_info) =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1 echo "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\ MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.in fi # Check if we need to complete a clone by starting replication. if [[ -f change_master_to.sql.in ]]; then # In case of container restart, attempt this at-most-once. cp change_master_to.sql.in change_master_to.sql.orig mysql -h 127.0.0.1 --verbose<<EOF STOP SLAVE IO_THREAD; $(<change_master_to.sql.orig), MASTER_HOST='mysql-mysqlha-0.mysql-mysqlha', MASTER_USER='${MYSQL_REPLICATION_USER}', MASTER_PASSWORD='${MYSQL_REPLICATION_PASSWORD}', MASTER_CONNECT_RETRY=10; START SLAVE; EOF fi # Start a server to send backups when requested by peers. exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \ "xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=${MYSQL_REPLICATION_USER} --password=${MYSQL_REPLICATION_PASSWORD}" command: - /bin/bash env: - name: MYSQL_PWD valueFrom: secretKeyRef: key: mysql-root-password name: mysql-mysqlha - name: MYSQL_REPLICATION_USER value: repl - name: MYSQL_REPLICATION_PASSWORD valueFrom: secretKeyRef: key: mysql-replication-password name: mysql-mysqlha image: registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0 imagePullPolicy: IfNotPresent name: xtrabackup ports: - containerPort: 3307 name: xtrabackup protocol: TCP resources: requests: cpu: 100m memory: 100Mi terminationMessagePath: /dev/termination-log volumeMounts: - mountPath: /var/lib/mysql name: data subPath: mysql - mountPath: /etc/mysql/conf.d name: conf - mountPath: /mnt/scripts name: scripts dnsPolicy: ClusterFirst restartPolicy: Always securityContext: {} terminationGracePeriodSeconds: 30 volumes: - emptyDir: {} name: conf - configMap: defaultMode: 420 name: mysql-mysqlha name: config-map - emptyDir: {} name: scripts volumeClaimTemplates: - metadata: creationTimestamp: null name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi status: phase: Pending status: replicas: 3 kind: List metadata: {} resourceVersion: "" selfLink: "" [root@master hrr]# 创建mysql-client容器，参考前面“ helm install mysql apphub/mysqlha”中的
[root@master hrr]# kubectl run mysql-client --image=mysql:5.7.13 -it --rm --restart='Never' --namespace default -- /bin/sh Waiting for pod default/mysql-client to be running, status is Pending, pod ready: false If you don't see a command prompt, try pressing enter. Error attaching, falling back to logs: error dialing backend: dial tcp: lookup node3 on 192.168.222.2:53: no such host Error from server: Get https://node3:10250/containerLogs/default/mysql-client/mysql-client: dial tcp: lookup node3 on 192.168.222.2:53: no such host [root@master hrr]# [root@master hrr]# kubectl get pod -owide NAME READY STATUS RESTARTS AGE IP NODE mysql-client 1/1 Running 0 48s 172.17.93.3 node3 mysql-mysqlha-0 2/2 Running 0 54m 172.17.93.2 node3 mysql-mysqlha-1 2/2 Running 0 53m 172.17.78.2 node2 mysql-mysqlha-2 2/2 Running 0 53m 172.17.86.2 master [root@master hrr]# 在node3上查看
[root@node3 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 8261abe363b0 mysql:5.7.13 "docker-entrypoint..." 56 seconds ago Up 55 seconds k8s_mysql-client.d32e2e5_mysql-client_default_90905092-c779-11e9-83c0-000c296be0a4_0d4f202f 1fac37badaa8 192.168.222.129:5000/pod-infrastructure:latest "/pause" 56 seconds ago Up 56 seconds k8s_POD.cace165_mysql-client_default_90905092-c779-11e9-83c0-000c296be0a4_a0f9ab22 d159b422bcdf registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0 "/bin/bash -c 'set..." 54 minutes ago Up 54 minutes k8s_xtrabackup.516a4a61_mysql-mysqlha-0_default_1fbcfe77-c772-11e9-83c0-000c296be0a4_63f2a879 6aae997c995d mysql:5.7.13 "docker-entrypoint..." 54 minutes ago Up 54 minutes k8s_mysql.b9eef278_mysql-mysqlha-0_default_1fbcfe77-c772-11e9-83c0-000c296be0a4_7dee904b 40695f86aaf7 192.168.222.129:5000/pod-infrastructure:latest "/pause" 54 minutes ago Up 54 minutes k8s_POD.62b22cd9_mysql-mysqlha-0_default_1fbcfe77-c772-11e9-83c0-000c296be0a4_58c67c1e [root@node3 ~]# 写入数据
[root@node3 ~]# docker exec -ti 8261abe363b0 sh # mysql -h272.17.93.2 -uroot ERROR 1045 (28000): Access denied for user 'root'@'172.17.93.3' (using password: NO) # mysql -h272.17.93.2 -uroot -pghScMZT1bTJN mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1018 Server version: 8.0.17 MySQL Community Server - GPL Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 4 rows in set (0.01 sec) mysql> CREATE DATABASE test; Query OK, 1 row affected (0.02 sec) mysql> CREATE TABLE test.messages (message VARCHAR(250)); Query OK, 0 rows affected (0.03 sec) mysql> INSERT INTO test.messages VALUES ('hello'); Query OK, 1 row affected (0.03 sec) mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ 5 rows in set (0.00 sec) mysql> use test; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> select * from messages; +---------+ | message | +---------+ | hello | +---------+ 1 row in set (0.01 sec) mysql> exit Bye # 连接svc查看，这个是read-only的，可以看到是轮询的，查询了2次才到有test中的库去
# mysql -h20.254.54.188 -uroot -pghScMZT1bTJN mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1058 Server version: 8.0.17 MySQL Community Server - GPL Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 4 rows in set (0.00 sec) mysql> exit Bye # mysql -h20.254.54.188 -uroot -pghScMZT1bTJN mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1054 Server version: 8.0.17 MySQL Community Server - GPL Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 4 rows in set (0.01 sec) mysql> exit Bye # mysql -h20.254.54.188 -uroot -pghScMZT1bTJN mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1072 Server version: 8.0.17 MySQL Community Server - GPL Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ 5 rows in set (0.00 sec) mysql> exit Bye #故障现象：
[root@docker ~]# helm init Creating /root/.helm/repository/repositories.yaml Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com Error: https://kubernetes-charts.storage.googleapis.com is not a valid chart repository or cannot be reached: Get https://kubernetes-charts.storage.googleapis.com/index.yaml: net/http: TLS handshake timeout [root@docker ~]# 可能原因：
可能国内访问不了https://kubernetes-charts.storage.googleapis.com/index.yaml，并且网速慢。
解决方法：
在初始化时，使用--stable-repo-url参数，将仓库地址改为阿里云的helm仓库地址。helm init --stable-repo-url=https://apphub.aliyuncs.com
故障现象：
[root@docker ~]# kubectl get pod NAME READY STATUS RESTARTS AGE mysql-mysqlha-0 0/2 Pending 0 35s nginx 1/1 Running 0 2m [root@docker ~]# [root@docker ~]# kubectl describe pod mysql-mysqlha-0 …… Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 1m 4s 8 {default-scheduler } Warning FailedScheduling SchedulerPredicates failed due to PersistentVolumeClaim is not bound: "data-mysql-mysqlha-0", which is unexpected.可能原因：
第一次使用helm安装mysql时，没有提前创建pv，导致报pvc没有bound的错。
解决方法：
查看pvc的模式，创建pv，创建过程请参考前面。
故障现象：
[root@docker hrr]# kubectl get pod NAME READY STATUS RESTARTS AGE mysql-mysqlha-0 0/2 RunContainerError 0 19s nginx 1/1 Running 1 1h [root@docker hrr]# [root@docker ~]# kubectl describe pod mysql-mysqlha-0 …… Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 18m 5m 49 {default-scheduler } Warning FailedScheduling SchedulerPredicates failed due to PersistentVolumeClaim is not bound: "data-mysql-mysqlha-0", which is unexpected. 4m 4m 1 {default-scheduler } Normal Scheduled Successfully assigned mysql-mysqlha-0 to master 4m 4m 1 {kubelet master} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy. 4m 4m 1 {kubelet master} spec.containers{mysql} Normal Pulling pulling image "mysql:5.7.13" 10s 10s 1 {kubelet master} spec.containers{mysql} Warning Failed Failed to pull image "mysql:5.7.13": Get https://registry-1.docker.io/v2/library/mysql/manifests/5.7.13: Get https://auth.docker.io/token?scope=repository%3Alibrary%2Fmysql%3Apull&service=registry.docker.io: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 10s 10s 1 {kubelet master} spec.containers{xtrabackup} Normal Pulling pulling image "registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0" 9s 9s 1 {kubelet master} spec.containers{xtrabackup} Warning Failed Failed to pull image "registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0": image pull failed for registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0, this may be because there are no credentials on this request. details: (repository registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup not found: does not exist or no pull access) 9s 9s 1 {kubelet master} Warning FailedSync Error syncing pod, skipping: [failed to "StartContainer" for "mysql" with ErrImagePull: "Get https://registry-1.docker.io/v2/library/mysql/manifests/5.7.13: Get https://auth.docker.io/token?scope=repository%3Alibrary%2Fmysql%3Apull&service=registry.docker.io: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" , failed to "StartContainer" for "xtrabackup" with ErrImagePull: "image pull failed for registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0, this may be because there are no credentials on this request. details: (repository registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup not found: does not exist or no pull access)" ] 8s 8s 1 {kubelet master} spec.containers{mysql} Normal BackOff Back-off pulling image "mysql:5.7.13" 8s 8s 1 {kubelet master} spec.containers{xtrabackup} Normal BackOff Back-off pulling image "registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0" 8s 8s 1 {kubelet master} Warning FailedSync Error syncing pod, skipping: [failed to "StartContainer" for "mysql" with ImagePullBackOff: "Back-off pulling image \"mysql:5.7.13\"" , failed to "StartContainer" for "xtrabackup" with ImagePullBackOff: "Back-off pulling image \"registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0\"" ] [root@docker hrr]# 可能原因：
下载不到docker image，本地没有mysql镜像。
解决方法：
手动下载mysql镜像，将镜像docker tag为配置中的名字，如何手动下载，请参考上面。
故障现象：
[root@docker docker]# kubectl get pod NAME READY STATUS RESTARTS AGE mysql-mysqlha-0 0/2 RunContainerError 0 9s nginx 1/1 Running 2 1h [root@docker docker]# [root@docker docker]# kubectl describe pod mysql-mysqlha-0 …… Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 15s 15s 1 {default-scheduler } Normal Scheduled Successfully assigned mysql-mysqlha-0 to master 14s 14s 1 {kubelet master} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy. 14s 3s 3 {kubelet master} spec.containers{mysql} Normal Pulled Container image "mysql:5.7.13" already present on machine 14s 3s 3 {kubelet master} spec.containers{xtrabackup} Normal Pulled Container image "registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0" already present on machine 14s 3s 3 {kubelet master} Warning FailedSync Error syncing pod, skipping: [failed to "StartContainer" for "mysql" with RunContainerError: "GenerateRunContainerOptions: mkdir /var/lib/kubelet/pods/18f319d5-c6d2-11e9-baf7-000c296be0a4/volumes/kubernetes.io~nfs/pv001/mysql: permission denied" , failed to "StartContainer" for "xtrabackup" with RunContainerError: "GenerateRunContainerOptions: mkdir /var/lib/kubelet/pods/18f319d5-c6d2-11e9-baf7-000c296be0a4/volumes/kubernetes.io~nfs/pv001/mysql: permission denied" ] [root@docker docker]# [root@docker docker]# kubectl get pod NAME READY STATUS RESTARTS AGE mysql-mysqlha-0 1/2 CrashLoopBackOff 2 5m nginx 1/1 Running 2 1h [root@docker docker]# Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 5m 5m 1 {default-scheduler } Normal Scheduled Successfully assigned mysql-mysqlha-0 to master 1m 1m 1 {kubelet master} Warning FailedSync Error syncing pod, skipping: [failed to "StartContainer" for "xtrabackup" with RunContainerError: "GenerateRunContainerOptions: mkdir /var/lib/kubelet/pods/18f319d5-c6d2-11e9-baf7-000c296be0a4/volumes/kubernetes.io~nfs/pv001/mysql: permission denied" , failed to "StartContainer" for "mysql" with RunContainerError: "GenerateRunContainerOptions: mkdir /var/lib/kubelet/pods/18f319d5-c6d2-11e9-baf7-000c296be0a4/volumes/kubernetes.io~nfs/pv001/mysql: permission denied" ] 5m 44s 23 {kubelet master} Warning FailedSync Error syncing pod, skipping: [failed to "StartContainer" for "mysql" with RunContainerError: "GenerateRunContainerOptions: mkdir /var/lib/kubelet/pods/18f319d5-c6d2-11e9-baf7-000c296be0a4/volumes/kubernetes.io~nfs/pv001/mysql: permission denied" , failed to "StartContainer" for "xtrabackup" with RunContainerError: "GenerateRunContainerOptions: mkdir /var/lib/kubelet/pods/18f319d5-c6d2-11e9-baf7-000c296be0a4/volumes/kubernetes.io~nfs/pv001/mysql: permission denied" ] 32s 32s 1 {kubelet master} spec.containers{mysql} Normal Started Started container with docker id d2f5e2c645b8 32s 32s 1 {kubelet master} spec.containers{xtrabackup} Normal Started Started container with docker id 6f2fb901a587 5m 32s 25 {kubelet master} spec.containers{xtrabackup} Normal Pulled Container image "registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0" already present on machine 32s 32s 1 {kubelet master} spec.containers{mysql} Normal Created Created container with docker id d2f5e2c645b8; Security:[seccomp=unconfined] 32s 32s 1 {kubelet master} spec.containers{xtrabackup} Normal Created Created container with docker id 6f2fb901a587; Security:[seccomp=unconfined] 30s 30s 1 {kubelet master} spec.containers{mysql} Normal Created Created container with docker id 104f64db6e8c; Security:[seccomp=unconfined] 30s 30s 1 {kubelet master} spec.containers{mysql} Normal Started Started container with docker id 104f64db6e8c 28s 26s 3 {kubelet master} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "mysql" with CrashLoopBackOff: "Back-off 10s restarting failed container=mysql pod=mysql-mysqlha-0_default(18f319d5-c6d2-11e9-baf7-000c296be0a4)" 5m 11s 27 {kubelet master} spec.containers{mysql} Normal Pulled Container image "mysql:5.7.13" already present on machine 5m 11s 5 {kubelet master} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy. 11s 11s 1 {kubelet master} spec.containers{mysql} Normal Created Created container with docker id ec8f0827294b; Security:[seccomp=unconfined] 11s 11s 1 {kubelet master} spec.containers{mysql} Normal Started Started container with docker id ec8f0827294b 28s 7s 5 {kubelet master} spec.containers{mysql} Warning BackOff Back-off restarting failed docker container 10s 7s 2 {kubelet master} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "mysql" with CrashLoopBackOff: "Back-off 20s restarting failed container=mysql pod=mysql-mysqlha-0_default(18f319d5-c6d2-11e9-baf7-000c296be0a4)" [root@docker docker]# [root@docker docker]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 24a5e40cfbe6 mysql:5.7.13 "docker-entrypoint..." About a minute ago Exited (1) About a minute ago k8s_mysql.b9eef278_mysql-mysqlha-0_default_18f319d5-c6d2-11e9-baf7-000c296be0a4_f37f9154 6f2fb901a587 registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0 "/bin/bash -c 'set..." 5 minutes ago Up 5 minutes k8s_xtrabackup.516a4a61_mysql-mysqlha-0_default_18f319d5-c6d2-11e9-baf7-000c296be0a4_4c633d64 61a82b9ad65a 192.168.222.129:5000/pod-infrastructure:latest "/pause" 10 minutes ago Up 10 minutes k8s_POD.62b22cd9_mysql-mysqlha-0_default_18f319d5-c6d2-11e9-baf7-000c296be0a4_9f7337a5 dc6071e8f613 192.168.222.129:5000/nginx "nginx -g 'daemon ..." 26 minutes ago Up 26 minutes k8s_nginx.3556dff8_nginx_default_30a915ea-c6c5-11e9-bbd7-000c296be0a4_9d257b60 15929311253f 192.168.222.129:5000/pod-infrastructure:latest "/pause" 26 minutes ago Up 26 minutes k8s_POD.cace165_nginx_default_30a915ea-c6c5-11e9-bbd7-000c296be0a4_d9ad1e0f 48f464ad9c9b 192.168.222.129:5000/pod-infrastructure:latest "/pause" 37 minutes ago Exited (2) 26 minutes ago k8s_POD.cace165_nginx_default_30a915ea-c6c5-11e9-bbd7-000c296be0a4_eb3bebdb c01c41763b06 192.168.222.129:5000/nginx "nginx -g 'daemon ..." About an hour ago Exited (0) 37 minutes ago k8s_nginx.3556dff8_nginx_default_30a915ea-c6c5-11e9-bbd7-000c296be0a4_0848bc8b [root@docker docker]# docker logs 24a5e40cfbe6 chown: changing ownership of '/var/lib/mysql/': Operation not permitted [root@docker docker]#可能原因：
docker对pvc没有权限，由于这里使用的是nfs作后端存储，所以可能是nfs目录的权限没放开。
解决方法：
将nfs目录权限设置为chown nfsnobody:nfsnobody -R /nfsdata，并且/etc/exportfs中设置no_root_squash，不压缩root权限，即在docker可以使用任何用户写入数据，具体操作步骤请参考上面。
故障现象：
[root@master ~]# kubectl describe pod mysql-mysqlha-0 …… Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 56m 56m 1 {default-scheduler } Normal Scheduled Successfully assigned mysql-mysqlha-0 to master 56m 56m 3 {kubelet master} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy. 56m 56m 1 {kubelet master} spec.containers{mysql} Normal Pulled Container image "mysql:5.7.13" already present on machine 56m 56m 1 {kubelet master} spec.containers{mysql} Normal Created Created container with docker id 31013a7822ae; Security:[seccomp=unconfined] 56m 56m 1 {kubelet master} spec.containers{mysql} Normal Started Started container with docker id 31013a7822ae 56m 56m 1 {kubelet master} spec.containers{xtrabackup} Normal Pulled Container image "registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0" already present on machine 56m 56m 1 {kubelet master} spec.containers{xtrabackup} Normal Created Created container with docker id a5f2f49c1eef; Security:[seccomp=unconfined] 56m 56m 1 {kubelet master} spec.containers{xtrabackup} Normal Started Started container with docker id a5f2f49c1eef可能原因：
没有配置kube-dns。
解决方法：
请参考上面配置kube-dns。
故障现象：
可能原因：
解决方法：
看了以上helm如何安装mysql教程介绍，希望能给大家在实际运用中带来一定的帮助。本文由于篇幅有限，难免会有不足和需要补充的地方，大家可以继续关注亿速云行业资讯板块，会定期给大家更新行业新闻和知识，如有需要更加专业的解答，可在官网联系我们的24小时售前售后，随时帮您解答问题的。
免责声明：本站发布的内容（图片、视频和文字）以原创、转载和分享为主，文章观点不代表本网站立场，如果涉及侵权请联系站长邮箱：is@yisu.com进行举报，并提供相关证据，一经查实，将立刻删除涉嫌侵权内容。
