Skip to main content

Posts

Latest Post

Quay - Build a On-Premise Registry using Podman

I usually prefer to host Quay using the Operator on OpenShift. It's clean, declarative, and integrates perfectly. But sometimes, that OpenShift Cluster is just not available yet, and you need a place to put your images right now . In those situations, you have to improvise and host the initial registry on a standalone virtual machine. Here is how you can spin up a quick, containerized Quay Proof of Concept (PoC) environment using Podman, PostgreSQL, and Redis. First, we need to create some directories that we're going to use for the different backing services that Quay requires. We'll set these up under '/var/podman': [archy@quay ~]$ mkdir -p -m 755 /var/podman/{postgresql,redis,quay} [archy@quay ~]$ mkdir -p -m 755 /var/podman/redis/{config,data} [archy@quay ~]$ mkdir -p -m 755 /var/podman/quay/{config,storage} Next, we need to set some ACLs so that the PostgreSQL and Quay containers can write to their respective storage locations. Without this, you will ru...
Recent posts

Foreman - Upgrade to Foreman 3.19 and Katello 4.21

           NOTE: this guide exists for Upgrading from v3.17 to v3.18 as well --> here With the current version, the official documentation is quite good and can be referenced. I would recommend executing all of these commands in a tmux session so that your session will remain on the server in case anything happens to your workstation. Start by checking for running tasks that would prohibit an update: [root@katello01 ~]# foreman-rake katello:upgrade_check Next, update the katello host and reboot if yum tells you to: [root@katello01 ~]# dnf -y --refresh upgrade [root@katello01 ~]# dnf needs-restarting -r If there were any updates to foreman-related packages, make sure foreman is in a consistent state: [root@katello01 ~]# foreman-maintain service stop [root@katello01 ~]# foreman-installer --scenario katello When the katello services have started again, upgrade the release-rpms: [root@katello01 ~]# dnf -y --refresh upgrade https://yum.thefor...

NMState - Migrating my Hypervisor from imperative to declarative Network Configuration

I've been working with NMState for quite a bit of time, mostly in OpenShift environments, and I've really grown to like the declarative way of handling network configuration . Since I still mainly use EL9 and NMState is packaged directly in the official repositories, I figured it was time to finally switch my traditional, imperative NetworkManager configuration over . First, let's iron out some basics. NMState doesn't completely replace NetworkManager; it relies on it and generates standard .nmconnection files under the hood. Think of it as a clean, declarative wrapper to configure NetworkManager and manage connections seamlessly. There are two distinct ways you can run NMState : Imperatively with a Failsafe: Best for initial testing and manual updates As a Service on Boot: Automatically enforces your declared configuration during startup (here be dragons). Let's look at the imperative mode first, which features a brilliant safeguard known as a "dead man'...

Foreman - Upgrade to Foreman 3.18 and Katello 4.20

          NOTE: this guide exists for Upgrading from v3.16 to v3.17 as well -->  here With the current version, the  official documentation  is quite good and can be referenced. I would recommend executing all of these commands in a tmux session so that your session will remain on the server in case anything happens to your workstation. Start by checking for running tasks that would prohibit an update: [root@katello01 ~]# foreman-rake katello:upgrade_check Next, update the katello host and reboot if yum tells you to: [root@katello01 ~]# dnf -y --refresh upgrade [root@katello01 ~]# dnf needs-restarting -r If there were any updates to foreman-related packages, make sure foreman is in a consistent state: [root@katello01 ~]# foreman-maintain service stop [root@katello01 ~]# foreman-installer --scenario katello When the katello services have started again, upgrade the release-rpms: [root@katello01 ~]# dnf -y --refresh upgrade https:/...

FreeIPA - Un-bricking DNSSEC after SoftHSM Token Loss

I've been using DNSSEC to sign my internal DNS zones with FreeIPA for quite some time now and have never had any problems. But suddenly, I noticed ipa-dnskeysyncd.service was continuously failing to start, throwing this traceback: dnssec-keyfromlabel: fatal: failed to get key RSASHA256: not found ipa-dnskeysyncd.service: Main process exited, code=exited, status=1/FAILURE Looking deeper into the logs, I could see that LDAP was happily reporting it was adding key metadata to my zones, but the local token was instantly rejecting it: The public key was not found at: pkcs11:object=... The root cause turned out to be a cryptographic "split-brain" situation. The metadata stored in the LDAP directory was correct, but the actual cryptographic private/public key material stored by the local SoftHSM token was missing. You can verify if your saved keys are actually in the token by pointing pkcs11-tool at the FreeIPA SoftHSM database: [root@ipa03 ~]# export SOFTHSM2_CONF=/e...

Mikrotik - Split Tunnel VPN to circumvent ISP Peering Issues

For the past week, my ISP has been struggling with connectivity to resources hosted on the Cloudflare network. The issues manifested as: Packet loss High latency Low download speeds (despite upload speeds remaining unaffected) To mitigate this, I set up a split tunnel VPN to route all Cloudflare traffic through a WireGuard connection on my MikroTik router. Here is a quick guide on the setup. First, obtain a WireGuard configuration file from your VPN provider. Ensure this file contains all necessary connection parameters, including the private key, endpoint address, and public key. Below is an example of the typical configuration data you will need: [Interface] PrivateKey = <private-key-data> Address = 10.2.0.2/32 DNS = 10.2.0.1 [Peer] PublicKey = <public-key-data> AllowedIPs = 0.0.0.0/0, ::/0 Endpoint = <vpn_provider_endpoint_address>:<vpn_provider_endpoint_port> PersistentKeepalive = 25 Create the WireGuard interface on the router. ...

Foreman - Upgrade to Foreman 3.17 and Katello 4.19

         NOTE: this guide exists for Upgrading from v3.15 to v3.16 as well -->  here With the current version, the  official documentation  is quite good and can be referenced. I would recommend executing all of these commands in a tmux session so that your session will remain on the server in case anything happens to your workstation. Start by checking for running tasks that would prohibit an update: [root@katello01 ~]# foreman-rake katello:upgrade_check Next, update the katello host and reboot if yum tells you to: [root@katello01 ~]# dnf -y --refresh upgrade [root@katello01 ~]# dnf needs-restarting -r If there were any updates to foreman-related packages, make sure foreman is in a consistent state: [root@katello01 ~]# foreman-maintain service stop [root@katello01 ~]# foreman-installer --scenario katello When the katello services have started again, upgrade the release-rpms: [root@katello01 ~]# dnf -y --refresh upgrade https://...

Talos - Enable ETCD Metrics Scraping for the Kube-Prometheus-Stack

When deploying the kube-prometheus-stack on Talos Linux, you might notice that ETCD metrics are missing by default. This occurs because Talos secures ETCD using mTLS, and the default Prometheus configuration does not have the necessary certificates to authenticate against the ETCD endpoints. Here is a quick guide on how to extract the necessary certificates and configure the monitoring stack to scrape ETCD metrics successfully. First, we need to export the client certificates from a Talos control-plane node. These certificates are required for Prometheus to authenticate with ETCD. Run the following commands to copy the certificate authority, server certificate, and key to your local machine: [archy@admin42 ~]$ mkdir -p -m 700 ~/etcd [archy@admin42 ~]$ MASTER_NODE=master01.talos.archyslife.lan [archy@admin42 ~]$ talosctl -e ${MASTER_NODE} -n ${MASTER_NODE} copy /system/secrets/etcd/ca.crt ~/etcd [archy@admin42 ~]$ talosctl -e ${MASTER_NODE} -n ${MASTER_NODE} copy /system/secre...

Foreman - Upgrade to Foreman 3.16 and Katello 4.18

        NOTE: this guide exists for Upgrading from v3.14 to v3.15 as well -->  here With the current version, the  official documentation  is quite good and can be referenced. I would recommend executing all of these commands in a tmux session so that your session will remain on the server in case anything happens to your workstation. Start by checking for running tasks that would prohibit an update: [root@katello01 ~]# foreman-rake katello:upgrade_check Next, update the katello host and reboot if yum tells you to: [root@katello01 ~]# dnf -y --refresh upgrade [root@katello01 ~]# dnf needs-restarting -r If there were any updates to foreman-related packages, make sure foreman is in a consistent state: [root@katello01 ~]# foreman-maintain service stop [root@katello01 ~]# foreman-installer --scenario katello When the katello services have started again, upgrade the release-rpms: [root@katello01 ~]# dnf -y --refresh upgrade https://yum.th...