Posted on Oct 4, 2023

Installing Openshift Cluster on vSphere7

Pre-requisites
Generate Pull secret from Redhat
Creating openshift cluster
Fixing Internal Image registry
Deploy a sample nginx application

1. Pre-requisites

a) Connectivity to vCenter on port 443 from openshift network.

b) Connectivity to ESXi hosts on port 443 from openshift network.

c) Generate ssh keys (we can use the existing), this needs to be passed during cluster creation.

d) Working DHCP for openshift cluster nodes.

e) Two static ip's for API and Apps, for Step d.

f) DNS entry for "api.." and "*.apps.." .

In our case we mapped as below in our DNS,
api.openshift-test01.tanzu.local => 192.168.144.22
*.apps.openshift-test01.tanzu.local => 192.168.144.23

2. Generate Pull secret from Redhat

Lets get the Pull secret and also download the installer and client tools.

a) Register with https://console.redhat.com/openshift/ using your personal email or official email.

b) Once logged in, Click on Create Cluster.

c) Choose "Datacenter" and scroll down

d) Click on vSphere

e) Click on Automated installation

f) Download the Installer, Pull secret, and Command line tools.

3. Creating openshift cluster

We use a linux jumpserver which is in same network as openshift network to create the cluster so the installer can connect to API server to verify the installation without any dependencies on Firewall.

root@linux-vm-automation:~/openshift# ls -ltr total 414864 -rw-r--r-- 1 root root 2783 May 30 17:30 pull-secret.txt -rw-r--r-- 1 root root 59819571 May 30 17:31 openshift-client-linux.tar.gz -rw-r--r-- 1 root root 364993703 May 30 17:31 openshift-install-linux.tar.gz root@linux-vm-automation:~/openshift# root@linux-vm-automation:~/openshift# root@linux-vm-automation:~/openshift# tar -xvf openshift-install-linux.tar.gz README.md openshift-install root@linux-vm-automation:~/openshift# ll total 975252 drwxr-xr-x 2 root root 146 May 30 18:04 ./ drwx------ 22 root root 4096 May 30 18:02 ../ -rw-r--r-- 1 root root 59819571 May 30 17:31 openshift-client-linux.tar.gz -rwxr-xr-x 1 root root 573825024 May 9 18:10 openshift-install* -rw-r--r-- 1 root root 364993703 May 30 17:31 openshift-install-linux.tar.gz -rw-r--r-- 1 root root 2783 May 30 17:30 pull-secret.txt -rw-r--r-- 1 root root 706 May 9 18:10 README.md root@linux-vm-automation:~/openshift#

Parameters we passed to the installer are below, so ensure you have the details ready.

a) ssh public key.
b) select vsphere as platform.
c) vcenter ip address.
d) vcenter username and password with required previleges.
e) datacenter.
f) datastore.
g) network.
h) VIP for API and Ingress.
i) Domain Name.
j) cluster name.
k) enter the pull secret that we copied from redhat console.

root@linux-vm-automation:~/openshift# ./openshift-install create cluster ? SSH Public Key /root/.ssh/id_rsa.pub ? Platform vsphere ? vCenter 172.17.22.118 ? Username administrator@vsphere.local ? Password [? for help] ************* INFO Connecting to vCenter 172.17.22.118 INFO Defaulting to only available datacenter: vcenter-datacenter ? Cluster tenant-cluster ? Default Datastore SSD_Storage ? Network tenant43-ntw-72a59d1a-398e-4018-8dbd-5afa8ca60d40 ? Virtual IP Address for API 192.168.144.22 ? Virtual IP Address for Ingress 192.168.144.23 ? Base Domain tanzu.local ? Cluster Name openshift-test01 ? Pull Secret [? for help] ******************************************************************************************************************************************************************************************************************INFO Obtaining RHCOS image file from 'https://rhcos.mirror.openshift.com/art/storage/prod/streams/4.13-9.2/builds/413.92.202305021736-0/x86_64/rhcos-413.92.202305021736-0-vmware.x86_64.ova?sha256=' INFO The file was found in cache: /root/.cache/openshift-installer/image_cache/rhcos-413.92.202305021736-0-vmware.x86_64.ova. Reusing... INFO Creating infrastructure resources... INFO Waiting up to 20m0s (until 8:22AM) for the Kubernetes API at https://api.openshift-test01.tanzu.local:6443... INFO API v1.26.3+b404935 up INFO Waiting up to 30m0s (until 8:35AM) for bootstrapping to complete... INFO Destroying the bootstrap resources... INFO Waiting up to 40m0s (until 9:05AM) for the cluster at https://api.openshift-test01.tanzu.local:6443 to initialize... INFO Checking to see if there is a route at openshift-console/console... INFO Install complete! INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/root/openshift/auth/kubeconfig' INFO Access the OpenShift web-console here: https://console-openshift-console.apps.openshift-test01.tanzu.local INFO Login to the console with user: "kubeadmin", and password: "c9T8a-ALwe9-ZU7D2-ENTDh" INFO Time elapsed: 44m32s root@linux-vm-automation:~/openshift#

So Cluster is created, lets login and verify.

Installer above provided the url and credentials to login

INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/root/openshift/auth/kubeconfig' INFO Access the OpenShift web-console here: https://console-openshift-console.apps.openshift-test01.tanzu.local INFO Login to the console with user: "kubeadmin", and password: "c9T8a-ALwe9-ZU7D2-ENTDh" INFO Time elapsed: 44m32s

Access the cluster using oc or kubectl,

We already download oc tool "openshift-client-linux.tar.gz" from the redhat console, extract it and place it in /usr/local/bin/ or the location that you prefer.

export KUBECONFIG=/root/openshift/auth/kubeconfig root@linux-vm-automation:~/openshift# oc get nodes NAME STATUS ROLES AGE VERSION openshift-test01-pg8s9-master-0 Ready control-plane,master 35m v1.26.3+b404935 openshift-test01-pg8s9-master-1 Ready control-plane,master 35m v1.26.3+b404935 openshift-test01-pg8s9-master-2 Ready control-plane,master 34m v1.26.3+b404935 openshift-test01-pg8s9-worker-0-5c42f Ready worker 14m v1.26.3+b404935 openshift-test01-pg8s9-worker-0-djzl5 Ready worker 15m v1.26.3+b404935 openshift-test01-pg8s9-worker-0-mtgzh Ready worker 14m v1.26.3+b404935 root@linux-vm-automation:~/openshift#

4. Fixing Internal Image registry

In vSphere environment, Openshift Internal Image registry won't be available since shareable stroage ReadWriteMany can't be created on vSphere storage.

If you try to create a pod with image pointing to internal image registry,

It will fail like below,

To Fix it, first create a PVC

root@linux-vm-automation:~/openshift# cat openshift-image-registry-pvc.yaml kind: PersistentVolumeClaim apiVersion: v1 metadata: name: image-registry-storage namespace: openshift-image-registry spec: accessModes: - ReadWriteOnce resources: requests: storage: 100Gi root@linux-vm-automation:~/openshift# root@linux-vm-automation:~/openshift# oc create -f openshift-image-registry-pvc.yaml -n openshift-image-registry persistentvolumeclaim/image-registry-storage created root@linux-vm-automation:~/openshift#

Update the Registry CR spec with the following command.

oc edit configs.imageregistry.operator.openshift.io -n openshift-image-registry Change spec.managementState from Removed to Managed. Change spec.storage from {} to: claim: image-registry-storage spec: managementState: Managed storage: pvc: claim: image-registry-storage

After updating it should look like below,

spec: managementState: Managed storage: pvc: claim: image-registry-storage

Once image registry pod is running fine, images from the internal image registry should be available.

The example which was not running earlier is running now,

5. Deploy a sample nginx application.

You should already be familiar on how to deploy a pod. Below we created a deployment using nginx image and created a service.

Lets create a route in openshift,

Note : This is not a Kubernetes object like Service.

Top comments (1)

aseques a • Dec 9 '24

Hi @balajivedagiri since your previous guide was on rancher and now you are testing openshift. Does this mean that you are leaning to the later or you are just creating guides on all the popular technologies?

DEV Community