Fedora Upgrade with LVM Snapshots

Fedora 31 has been released at the end of October 2019 so it's time for me to update. Fedora Updates have proven to be reliable but Updates can go wrong and for that case, I want to have a fallback that I can utilize to restore my system without much of a hassle. I use LVM for all my Storage so the solution for me was fairly easy: LVM Snapshots.

I'm using one volume group for the system named 'vg_base'. If you are copy-pasting the commands, this is the one parameter you'll have to adjust. Without further ado, let's get through the upgrade scenario. While these commands can be executed as user with sudo privileges, I find it easier to do these as root.

Install the system-upgrade plugin

 [root@castle-bravo ~]# dnf -y install dnf-plugin-system-upgrade  

Now create the snapshots of all the system lvs. Note that you would only need to snapshot the rpmdb as well as the logical volumes containing /etc/, /bin, /sbin, /usr/bin and /usr/sbin but to be on the safe side I usually snapshot all the volumes required to run the system.

 [root@castle-bravo ~]# for lv in $(lvs vg_base | awk '{print $1}' | grep -vE '^LV$|^lv_swap$'); do echo "lvcreate -s -n "$lv"_snapshot -L 20G /dev/vg_base/"$lv | bash -x; done  

Let's break this command down a bit. This is a simple for-loop in bash that creates a snapshot with a Copy-On-Write capacity of 20G for every item in the output of 'lvs vg_base'. If you do not want to run it instantly but want to see what this would do, sort of a dry-run, remove the 'bash -x' part.
Here's an example:

 [root@castle-bravo ~]# # for lv in $(lvs vg_base | awk '{print $1}' | grep -vE '^LV$|^lv_swap$'); do echo "lvcreate -s -n "$lv"_snapshot -L 20G /dev/vg_base/"$lv; done  

 lvcreate -s -n lv_home_snapshot -L 20G /dev/vg_base/lv_home  
 lvcreate -s -n lv_root_snapshot -L 20G /dev/vg_base/lv_root  
 lvcreate -s -n lv_tmp_snapshot -L 20G /dev/vg_base/lv_tmp  
 lvcreate -s -n lv_usr_snapshot -L 20G /dev/vg_base/lv_usr  
 lvcreate -s -n lv_var_snapshot -L 20G /dev/vg_base/lv_var  
 lvcreate -s -n lv_var_log_snapshot -L 20G /dev/vg_base/lv_var_log  

The creation of the snapshots will take only a few seconds. Once this loop is done, you can download the new release version of fedora using dnf. This might take a while depending on your connection speed.

 [root@castle-bravo ~]# dnf system-upgrade download --releasever=31  

Once everything is downloaded and the transaction checks have passed successfully, you can start the upgrade by rebooting.

 [root@castle-bravo ~]# dnf system-upgrade reboot  

The update can go two ways, either successfully or it can fail. In case of success, you can delete the snapshots by running this command

 [root@castle-bravo ~]# for snapshot in $(lvs vg_base | awk '{print $1}' | grep -iE 'snapshot'); do lvremove -f /dev/vg_base/$snapshot; done  

If the update fails and you want to revert to the snapshot you've taken, you'll have to boot your machine using a live environment, such as the installer, or any other live distribution with lvm support (or availability in the repos) and run this command

 [root@castle-bravo ~]# for snapshot in $(lvs vg_base | awk '{print $1}' | grep -iE 'snapshot'); do lvconvert --mergesnapshot /dev/vg_base/$snapshot; done  

This will merge the snapshots you've taken back to the original volumes and restore your system to the point where you've taken the snapshot.

Feel free to comment and / or suggest a topic.