Task:
Install Origin Community Distribution of Kubernetes (OKD) on CentOS Stream CoreOS.
Starting 2023 two OKD "flavors":
- OKD based on Fedora CoreOS (FCOS)
- OKD-SCOS based on CentOS Stream CoreOS (SCOS)
Parts:
A. Download FCOS or SCOS and OKD
B. Install FCOS/SCOS on Boostrap VM (and Control Plane Nodes and Worker/Compute Nodes if Manual Setup)
C. Install required packages on the Boostrap VM
D1. Run openshift-install to Create the install-config.yaml file
D2. Run openshift-install to Create the Ignition Config Files: bootstrap.ign, master.ign, and worker.ign
E. Install with ISO and Kernel Command Parameters
F. Run openshift-install to complete bootstrap
G. Post Install Tasks Short List:
Notes:
- This deployment mode is the User-Provisioned Infrastructure (UPI) one
- The later 4.x and 5.0.0 pre-release run on CentOS Stream CoreOS instead of FCOS.
- OKD is upstream open-source of Red Hat OpenShift
- OKD verisons match the Openshift versions major and minor releases
- Network-based (external) DNS and DHCP already exist with reverse pointers for the hosts/nodes. It is possible to work around this for a "pure" encapsulated environment with host files, but its not scalable and would have to be manually updated when DHCP records change.
- LVMs are NOT supported, ensure using hardware RAID 10, etc.
Prerequisites:
- Requires 1 temporary bootstrap VM, 3 control plane hosts, and at least 2 worker/compute hosts. You can also run 0 workers and configure the control plan hosts to also run work loads.
- If installing on RHV/oVirt or vSphere, create VMs. If doing this, think about switching to the more automated Installer-Provisioned Infrastructure (IPI) install method.
- VM/Host Requirements: 4+cores, 16GB+ memory, 100 GB+ storage SSD or 15k drives min.
- DNS and DHCP
- CentOS Stream CoreOS
- Before installing OKD, API and the Ingress LBs must be provisioned
Part A: Download FCOS/SCOS and ODK:
1. Download FCOS or SCOS:
- FCOS:
getfedora.org/coreos
- SCOS
(Download the iso for a manual install, and download the PXE images if using network boot install.)
cloud.centos.org/centos/scos/10/prod/streams/latest/
2. Download ODK:
(At the time of this document's last update, the 5.0.0 pre-release and 4.22 are the current versions.)
github.com/okd-project/okd/releases
or
For OCP:
mirror.openshift.com/pub/openshift-v4/clients/ocp/latest/openshift-install-linux.tar.gz
Ensure you have:
- installer binary: openshift-install
- "CLI" tool, aka oc.
Part B: Install FCOS/SCOS on VMs or Hosts/Nodes:
Note:
- Delay install of the 3 control plane nodes, and 2 worker nodes until the Bootstrap VM has created the ignition files.
- Not using the ignition files will require manual set-up of the host and networking.
1. Insert the ISO to the bootstrap VM, and run the install:
a. Run the install
- Manually set the hostname and domain
- Confirm the network is set to DHCP so the bootstrap VM picks up its proper IP.
b. Skip the following step unless manually setting up control plane and worker nodes. Proceed to Part C.
2. Manual set-up of the control plane and worker nodes:
5 or 3 Host VMs or Hosts nodes, and the bootstrap VM, and run the install on each.
(3 control plane VMs/nodes, 2 compute/worker VMs/nodes, 1 bootstrap/installer VM)
a. During set-up, set the hostname and domain of each.
b. Verify that the network is set to DHCP, and it should retrieve its IP from the DHCP VM which has the prerequisite DNS and reverse DNS records required.
c. Repeat for rest of the control planes, worker/compute nodes, and the bootstrap VM.
d. Install the HAProxy VM(s) as required.
e. Create and install a SSH key for the bootstrap VM to have passwordless login to the 3 or 5 control plane and worker VMs/nodes.
(Refer to a technote in this repository containing steps.)
Part C: Install packages on the Bootstrap VM required to perform the cluster initialization:
a. Install Apache to serve the ignition files.
- Install and configure:
$ sudo dnf install -y httpd
$ sudo sed -i 's/Listen 80/Listen 8080/' /etc/httpd/conf/httpd.conf
$ sudo sed -i 's/Listen 443/Listen 8443/' /etc/httpd/conf.d/ssl.conf
$ sudo setsebool -P httpd_read_user_content 1
$ sudo systemctl enable --now httpd.service
- Configure the firewall and open the ports to the local network:
$ sudo firewall-cmd --permanent --add-port=8080/tcp
$ sudo firewall-cmd --permanent --add-port=8443/tcp
$ sudo firewall-cmd --reload
- Verify Apache is available:
$ curl localhost:8080
b. Install HAProxy for the API service since it must exist.
Note:
- If you have the HAProxy installed on two other VMs for both the API and Apps already, skip this step.
- Install HAProxy:
$ sudo dnf install haproxy -y
$ sudo setsebool -P haproxy_connect_any 1
$ sudo systemctl enable --now haproxy.service
- configure HAProxy using the sample guides.
(Refer to the technote in this repository containing 2 examples.)
$ sudo vi /etc/haproxy/happroxy.cfg
...
- Configure the firewall and open the ports to the local network:
$ sudo firewall-cmd --permanent --add-port=6443/tcp
$ sudo firewall-cmd --permanent --add-port=22623/tcp
$ sudo firewall-cmd --permanent --add-service=http
$ sudo firewall-cmd --permanent --add-service=https
$ sudo firewall-cmd --reload
Part D: Run openshift-install to create the install-config.yaml file:
Note:
- Before your start, ensure you have:
- - Base domain: mindwatering.net
- - Cluster name: okd
- - Pull secret: <use OKD-specific or fake secret for community>
- - Platform: <none> (for bare metal/UPI)
- - SSH Key: <paste from previous
1. On the bootstrap VM, run the downloaded openshift-intstall binary:
$ ./openshift-install create install-config --dir=okd-cluster
<enter the base domain, cluster name, pull secret, platform, and SSH key>
Review the generated install-config.yaml.
2. Create the ignition Files:
$ ./openshift-install create ignition-configs --dir=okd-cluster
Review the generated files:
- bootstrap.ign (for the bootstrap VM)
- master.ign (for the control plane nodes)
- worker.ign (for the worker/compute nodes)
Note:
- Use ignition .ign files in the coreos boot arguments as below:
coreos.inst.install_dev=/dev/sda
coreos.inst.image_url=<SCOS-Live-ISO-URL>
coreos.inst.ignition_url=http://<http_server>/<node-type>.ign
Copy the generated files to the Apache html root folder:
$ cp bootstrap.ign /var/www/html/
$ cp master.ign /var/www/html/
$ cp worker.ign /var/www/html/
Part E: Install with ISO and Kernel Command Parameters
1. Insert ISO and boot control plane VM or node.
2. Interrupt the boot at the boot menu and override with the ignition file:
- At the boot menu, press either <tab> (if isolinux) or e (if grub) to abort the boot and edit the kernel boot commands.
- Use the ignition lines in Part D2 above, and update:
- - For a control plane node:
coreos.inst.install_dev=/dev/sda
coreos.inst.ignition_url=http(s)://bootstrap.mindwatering.net/master.ign
- - For a worker/compute node:
coreos.inst.install_dev=/dev/sda
coreos.inst.ignition_url=http(s)://bootstrap.mindwatering.net/worker.ign
- Press <enter> (for isolinux) or <cntl+x> (for grub) to continue the install.
3. Repeat for rest of control plane VM(s) or Nodes and worker/compute VM(s) or Nodes.
Part F: Run openshift-install to complete bootstrap
1. Complete the bootstrap back in the bootstrap VM:
$ ./openshift-install wait-for bootstrap-complete --dir=okd-cluster --log-level=info
<wait>
2. Assuming completed successfully, shut down the bootstrap VM.
$ shutdown -h now
3. On the primary control-plane node (e.g. control plane 0 node), complete the install:
$ ./openshift-install wait-for install-complete --dir=okd-cluster
<wait>
If successful, installation will print:
- the web console UI URL
- the store location of the Admin credential (auth/kubeadmin-password)
4. Validate the cluster using the cli:
$ oc login -u kubeadmin -p <kubeadmin-password> <web-console-ui-url>
<confirm all nodes are healthy, and the cluster status is Ready>
Part G: Post Install Tasks Short List:
1. Configure OAuth ID (no HTPasswd for multi-node clusters or production OKD)
2. Install Operators (OperatorHub)
3. Set up Storage
4. Create projects/namespaces
5. Set-up monitoring connections to built-in Prometheus and Grafana
previous page
|