Archy's Blog

Roughly three years ago I've started my RHCA Journey  for the first time. Since then, quite a lot has happened and my knowledge gained in the first round has still served me well so far but since the Certs are expiring soon, it's time to get 5 new Certs in order to maintain my status. This time, I'have gone for the following certifications: EX280 - OpenShift Administration (242/300) EX380 - OpenShift Automation and Integration (300/300) EX316 - OpenShift Virtualization (257/300) EX188 - Containers (274/300) EX480 - Multicluster Management with RHACM and RHACS (252/300) Unfortunately, I don't have detailed stats like last time but I've started to learn for EX280 roughly mid May 2024 and achieved my results for EX480 on 18th September 2024, so roughly 4 months. I had some experience with OpenShift before starting this journey but I had no exposure at all to RHACM and RHACS and only briefly touched OpenShift Virtualization. Here are my thoughts on each Exam: EX280 (v4.

After upgrading to Foreman 3.11 and Katello 4.13 I've noticed I wasn't able to configure Storage and Network Settings for compute profiles using Libvirt Compute Resources and VMs were created with the default values: Disk Size: '10G' Storage Pool: 'default' Network: 'default' Using 'foreman-tail', I found these log lines: 2024-09-02T17:34:02 [I|app|3b6bf363] Deface: [WARNING] No :original defined for 'remove_networks_and_volumes_partial', you should change its definition to include: 3b6bf363 | :original => 'eb4c199bc19e113181e82700bff46538b01f8c44' 2024-09-02T17:34:02 [I|app|3b6bf363] Deface: [WARNING] No :original defined for 'remove_networks_and_volumes_partial', you should change its definition to include:

    NOTE: this guide exists for Upgrading from v3.9 to v3.10 as well -->  here With the current version, the  official documentation  is quite good and can be referenced. I would recommend executing all of these commands in a tmux session so that your session will remain on the server in case anything happens to your workstation. Start by checking for running tasks that would prohibit an update: [root@katello01 ~]# foreman-rake katello:upgrade_check Next, update the katello host and reboot if yum tells you to: [root@katello01 ~]# dnf -4y --refresh update [root@katello01 ~]# dnf needs-restarting -r If there were any updates to foreman-related packages, make sure foreman is in a consistent state: [root@katello01 ~]# foreman-maintain service stop [root@katello01 ~]# foreman-installer --scenario katello When the katello services have started again, upgrade the release-rpms: [root@katello01 ~]# dnf -4y update https://yum.theforeman.org/releases/3.11/el8/x86_64/foreman-rel

I've just upgraded my (local-only) zimbra installation from 9.0.0 to 10.0.0 and just after the upgrade mailboxd and a few other services wouldn't start: [root@zimbra01 ~]# su -ls /usr/bin/bash -c 'zmcontrol status' zimbra load average: 0.91 1.81 1.88 Host zimbra01.archyslife.lan amavis Running antispam Running antivirus Running dnscache Running ldap Running logger Running mailbox Stopped zmmailboxdctl is not running. memcached Running mta Running opendkim Running proxy Running service webapp Stopped

The easiest step to download the OpenShift installation artifacts is using the openshift-installer to get the urls for the rootfs, pxe and iso files. First, make sure to install the openshift-installer binary which can be downloaded from the RedHat Customer Portal. Once the installer is downloaded and extracted from the archive, run the following command: [archy@helper01 ~]$ openshift-install coreos print-stream-json | jq -C .architectures.x86_64.artifacts.metal.formats This will give you a json list with all the urls for the required files. Here's an example using the okd version of the openshift-installer: [archy@helper01 ~]$ openshift-install coreos print-stream-json | jq -C .architectures.x86_64.artifacts.metal.formats { "4k.raw.xz": { "disk": { "location": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/39.20231101.3.0/x86_64/fedora-coreos-39.20231101.3.0-metal4k.x86_64.raw.xz", "signa

We'll be working on the Servers that are surrounded by the continous lines in this drawing: The Cluster is in the process of building if you've arrived here. However, there's still some steps left to do like: remove the bootstrap node from haproxy approve the signing requests for the worker nodes monitor the cluster state and cluster operators create an initial htpasswd auth provider for a fallback admin account create a storage class for the openshift internal image registry Let's start with removing the bootstrap Node after the bootstrap monitoring command tells us it's safe to remove: [archy@helper01 ~]# sudo vim /etc/haproxy/haproxy.cfg [archy@helper02 ~]# sudo vim /etc/haproxy/haproxy.cfg global log 127.0.0.1 local2 pidfile /var/run/haproxy.pid maxconn 4000 daemon defaults mode http log global option dontlognull option http-server-close option redispatch retries 3 timeout http-request 10s tim

We'll be working on the Servers that are surrounded by the continous lines in this drawing: Most of the Setup is already done, from here on out the heavylifting will be done from the installer. But first, there's still a few small things left to do: getting the installation artifacts. Specifically, I'm talking about these artifacts that still need to be downloaded: fedora coreos image openshift-installer openshift-client helm butane Since we've set up a shared-storage for the webservers that will host these files, they will only need to be downloaded once and can be served from the interal share. I'll download all the artifacts on one of the helper nodes: [archy@helper01 ~]$ sudo -Hiu root [root@helper01 ~]# curl -4kLo '/var/www/html/okd/fcos-39.iso' -X 'GET' -H 'Accept: application/octet-stream' -H 'User-Agent: curl/7.76.1' 'https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/39.20240210.3.0/x86_64/fedora-coreo

We'll be working on the Servers that are surrounded by the continous lines in this drawing: Keepalived is a useful tool to share a single Virtual IP (VIP) between multiple nodes without dealing with pacemaker, corosync and fencing. Keepalived is also fairly lightweight and easy to configure, so a good fit four this setup. Start by installing keepalived: [archy@helper01 ~]$ sudo dnf -4y --refresh install keepalived [archy@helper02 ~]$ sudo dnf -4y --refresh install keepalived After the packages are done installing, configure keepalived on the first helper node: [archy@helper01 ~]$ sudo vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { admins@archyslife.lab } notification_email_from root@helper01.okd.archyslife.lab smtp_server 127.0.0.1 smtp_connect_timeout 30 script_user root root } vrrp_script chk_haproxy { script '/usr/bin/pidof haproxy' interval 2

We'll be working on the Servers that are surrounded by the continous lines in this drawing: The HTTPD Setup for this setup is only for sharing installer required files, meaning I'll barely use any features and leave the default config in place for most things. First, install httpd: [archy@helper01 ~]$ sudo dnf -4y --refresh install httpd Now, let's edit the httpd config and change the default ports since they're already in use by HAProxy for the 'http-ingress' and 'https-ingress' services: [archy@helper01 ~]$ sudo sed -i 's/Listen 80/Listen 8080/g' /etc/httpd/conf/httpd.conf [archy@helper01 ~]$ sudo sed -i 's/Listen 443/Listen 8443/g' /etc/httpd/conf/httpd.conf Also, edit the dummy ssl config to listen on Port 10443 to ensure there are no conflics with port bindings on reboots: [archy@helper01 ~]$ sudo sed -i 's/Listen 443/Listen 10443/g' /etc/httpd/conf.d/ssl.conf Using Port 10443/tcp will also require adding the '