Archy's Blog

When deploying the kube-prometheus-stack on Talos Linux, you might notice that ETCD metrics are missing by default. This occurs because Talos secures ETCD using mTLS, and the default Prometheus configuration does not have the necessary certificates to authenticate against the ETCD endpoints. Here is a quick guide on how to extract the necessary certificates and configure the monitoring stack to scrape ETCD metrics successfully. First, we need to export the client certificates from a Talos control-plane node. These certificates are required for Prometheus to authenticate with ETCD. Run the following commands to copy the certificate authority, server certificate, and key to your local machine: [archy@admin42 ~]$ mkdir -p -m 700 ~/etcd [archy@admin42 ~]$ MASTER_NODE=master01.talos.archyslife.lan [archy@admin42 ~]$ talosctl -e ${MASTER_NODE} -n ${MASTER_NODE} copy /system/secrets/etcd/ca.crt ~/etcd [archy@admin42 ~]$ talosctl -e ${MASTER_NODE} -n ${MASTER_NODE} copy /system/secre...

Since I'm mostly working with OpenShift I'm used to the Monitoring Stack being already deployed. However, if you're rolling your own Kubernetes Stack, you'll have to take care of monitoring yourself and I'd like to stick to the Prometheus-Grafana Stack since I'm fairly familiar with it. This is not intended to be a production-ready Deployment but more in the category of 'Proof-of-Concept'. This setup will require a working Kubernetes Cluster with the following Features: default Storage Class ('managed-nfs' in my case) working Ingress Class (I'll be using 'nginx-ingress') Additionally, access to the 'helm' binary on the workstation. Since this is a demo, I'll also provide a self-signed Cert for the Ingress. This is also what we're starting with: [archy@workstation ~]$ DEPLOYMENT=grafana [archy@workstation ~]$ KEY="${DEPLOYMENT}.key" [archy@workstation ~]$ CRT="${DEPLOYMENT}.crt" [archy@work...

My original installation of Zabbix (v4.0) has been installed on centos7 and it's time to migrate it to a new el8 (v5.0) installation. I will use almalinux8 here but these steps should be identical for rockylinux or any other rhel8 clone. I'll use postgresql and nginx for my setup, so the steps might vary when using mysql or httpd. First, enable the postgresql module stream. Version 10 is enabled by default but I'll go for the latest available which is version 13 as of writing this. [root@zabbix-new ~]# yum -d 2 -y module enable postgresql:13 Now that the module stream is enabled, install the required packages: [root@zabbix-new ~]# yum -d 2 -y install postgresql postgresql-server postgresql-contrib Initialize the database and start it: [root@zabbix-new ~]# su - postgres -c 'initdb' [root@zabbix-new ~]# systemctl enable --now postgresql.service Create the zabbix user and database in postgresql (creating the user will prompt for a password): [root@zabbix-n...

By default, zabbix runs on http and sends all login data in cleartext which is not ideal in terms of security. This is a short writeup on how to configure the zabbix-frontend to run on https. First up, you'll need a certificate. This can be obtained from your internal CA or you can generate a self-signed certificate using this handy command: [archy@zabbix ~]$ sudo openssl req -x509 -nodes -days 3650 -newkey rsa:4096 -keyout /etc/pki/tls/private/zabbix.archyslife.lan.key -out /etc/pki/tls/certs/zabbix.archyslife.lan.cert Now that you've got the certificate, reconfigure httpd to also listen on port 443. Add this line to /etc/httpd/conf/httpd.conf: [archy@zabbix ~]$ sudo vim /etc/httpd/conf/httpd.conf Listen 443 Now, let's configure the zabbix virtualhost. The virtualhost config is a slightly altered version of the default version provided by zabbix. [archy@zabbix ~]$ sudo vim /etc/httpd/conf.d/zabbix.conf Alias /zabbix /usr/share/zabbix <VirtualHost *:443>...

I recently had to set up LDAP-Authentication for Icinga2   Even though the Documentation on the user-setup is quite good, the Documentation about groups and ldap is missing some informations. So here is what I had to do, to get this working. In this environment I already had two IPA-Servers set up with replication. These will take care of LDAP and DNS for me. My Client is joined to the IPA-Domain. First, let's create a resouce for our ldap-connection, the parameters are pretty self explanatory. [archy@icinga2 ~]$ sudo vim /etc/icingaweb2/resources.ini ... [auth-ipa] type = "ldap" hostname = "ipasrv02.archyslife.lan" port = "636" root_dn = "dc=archyslife,dc=lan" bind_dn = "uid=icinga-bind,cn=users,cn=accounts,dc=archyslife,dc=lan" bind_pw = "some_secret_password" encryption = "ldaps" ... Next, let's take care of the user-authentication using our LDAP-Connection...

I recently decided to change my internal monitoring at home from nagios to Zabbix. The main reason was that I won't have to write config files with zabbix. I used a raspberry pi 3 running raspbian 9 which I had idling around and for homeuse this should be enough since I will only monitor around 4-5 devices using the agent. The shown steps should be the same on a updated debian9 system. Anyway, I'd suggest to push the database (in my case postgresql) to a separate HDD or anything else not on the sd-card since the db will have a lot of write-cycles which will tear the flash down very quickly. I checked the internet for some good HowTo's but even the documentation on the zabbix page was missing some information. So I decided to write down the steps it took me to get up and running with zabbix. First you'll have to install the zabbix-server-pgsql package and if you want the webfrontend, also install the zabbix-webfrontend-php package. [archy@zabbix ~]$ sudo apt-...