Kubernetes Disk Expansion
So your Workload's Persistent Volume is almost full. How can you expand it? With Kubernetes 1.11 and above, this can now easily be done by just updating the Persistent Volume Claim storage specification.
My example in this post is tested in Google's Kubernetes Engine.
Let's get started!
Enabling Volume Expansion in Storage Class
To enable the disk resizing feature, ensure that the Storage Class that your Workload's Persistent Volume Claim uses is configured with allowVolumeExpansion: true.
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: standard parameters: type: pd-standard provisioner: kubernetes.io/gce-pd allowVolumeExpansion: true # ensure that this is true A Side Note for StatefulSet Workloads
If you want to resize a disk for a StatefulSet with a configured volumeClaimTemplates like the following,
volumeClaimTemplates: - metadata: name: postgresql-storage spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi storageClassName: standard you will only have to modify the Persistent Volume Claim that it has provisioned.
Resizing a Persistent Volume Claim
Find the Persistent Volume Claim to resize.
kubectl get pvc -l app=postgresql Let's say you want to update postgresql-storage-postgresql-02-1 from 1Gi to 10Gi
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE postgresql-storage-postgresql-01-0 Bound pvc-613c74c3-66b5-11e9-a602-42010a8400c0 10Gi RWO standard 6m12s postgresql-storage-postgresql-01-1 Bound pvc-88f897b1-60d4-11e9-a602-42010a8400c0 5Gi RWO standard 7d11h postgresql-storage-postgresql-02-0 Bound pvc-727c167c-6089-11e9-a602-42010a8400c0 10Gi RWO standard 7d20h postgresql-storage-postgresql-02-1 Bound pvc-80421dab-6089-11e9-a602-42010a8400c0 1Gi RWO standard 7d20h kubectl edit pvc postgresql-storage-postgresql-02-1 From your text editor, change 1Gi to 10Gi
apiVersion: v1 kind: PersistentVolumeClaim metadata: annotations: pv.kubernetes.io/bind-completed: "yes" pv.kubernetes.io/bound-by-controller: "yes" volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/gce-pd creationTimestamp: "2019-04-16T20:52:09Z" finalizers: - kubernetes.io/pvc-protection labels: app: postgresql name: postgresql-storage-postgresql-02-1 namespace: default resourceVersion: "29892621" selfLink: /api/v1/namespaces/default/persistentvolumeclaims/postgresql-storage-postgresql-02-1 uid: 80421dab-6089-11e9-a602-42010a8400c0 spec: accessModes: - ReadWriteOnce dataSource: null resources: requests: storage: 1Gi # change this to 10Gi storageClassName: standard volumeName: pvc-80421dab-6089-11e9-a602-42010a8400c0 status: accessModes: - ReadWriteOnce capacity: storage: 1Gi # change this to 10Gi phase: Bound After successfully modifying the Persistent Volume Claim, observe its status.
kubectl get pvc postgresql-storage-postgresql-02-1 -o yaml This should be the example output. Observe the status at the very end of the YAML.
apiVersion: v1 kind: PersistentVolumeClaim metadata: annotations: pv.kubernetes.io/bind-completed: "yes" pv.kubernetes.io/bound-by-controller: "yes" volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/gce-pd creationTimestamp: "2019-04-16T20:52:09Z" finalizers: - kubernetes.io/pvc-protection labels: app: postgresql name: postgresql-storage-postgresql-02-1 namespace: default resourceVersion: "33540657" selfLink: /api/v1/namespaces/default/persistentvolumeclaims/postgresql-storage-postgresql-02-1 uid: 80421dab-6089-11e9-a602-42010a8400c0 spec: accessModes: - ReadWriteOnce dataSource: null resources: requests: storage: 10Gi storageClassName: standard volumeName: pvc-80421dab-6089-11e9-a602-42010a8400c0 status: accessModes: - ReadWriteOnce capacity: storage: 1Gi conditions: - lastProbeTime: null lastTransitionTime: "2019-04-24T17:30:51Z" message: Waiting for user to (re-)start a pod to finish file system resize of volume on node. status: "True" type: FileSystemResizePending phase: Bound Restart the Pod
The next step is to restart the Pod as recommended by Kubernetes.
kubectl delete pod postgresql-02-1 Then wait until the Pod starts running again.
Observe the Changes
After a successful Pod restart, check the status of the Persistent Volume Claim again and you should see that the storage has been updated.
kubectl get pvc postgresql-storage-postgresql-02-1 -o yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: annotations: pv.kubernetes.io/bind-completed: "yes" pv.kubernetes.io/bound-by-controller: "yes" volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/gce-pd creationTimestamp: "2019-04-16T20:52:09Z" finalizers: - kubernetes.io/pvc-protection labels: app: postgresql name: postgresql-storage-postgresql-02-1 namespace: default resourceVersion: "33541444" selfLink: /api/v1/namespaces/default/persistentvolumeclaims/postgresql-storage-postgresql-02-1 uid: 80421dab-6089-11e9-a602-42010a8400c0 spec: accessModes: - ReadWriteOnce dataSource: null resources: requests: storage: 10Gi storageClassName: standard volumeName: pvc-80421dab-6089-11e9-a602-42010a8400c0 status: accessModes: - ReadWriteOnce capacity: storage: 10Gi phase: Bound Do kubectl exec to the Pod and verify that the disk has been resized!
Run df -h.
# df -h Filesystem Size Used Avail Use% Mounted on overlay 5.7G 4.7G 1.1G 83% / tmpfs 1.9G 0 1.9G 0% /dev tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup /dev/sda1 5.7G 4.7G 1.1G 83% /etc/hosts shm 64M 0 64M 0% /dev/shm /dev/sdc 9.9G 1.3G 8.6G 13% /var/lib/postgresql tmpfs 1.9G 12K 1.9G 1% /run/secrets/kubernetes.io/serviceaccount tmpfs 1.9G 0 1.9G 0% /sys/firmware Magic!
Top comments (0)