Table of Contents

Fix initramfs from broken update

Overview

Update: Did this with Almalinux 8 which is RHEL8 / Rocky Linux 8 equivalent and the below instructions apply to those as well. This was done with hardware server through IPMI.

You can ran into this problem with failed updates and also by editing your mdadm.conf to rule out your root md device, which i managed to do in my last endevour. Be sure to update your mdadm.conf to exclude any DEVICE or other statements that would exclude your root device.

I updated my CentOS 7 virtual guest and after boot it could not locate my partitions. I had actually 2 instances, an LVM guest and a non-LVM guest. LVM guest waited for a few minutes until it told me some errors about dracut. Non-LVM guest waited indefinitely for vda partition mount.

In practice you may see this kind of error message:

dracut-initqueue timeout

And in detail i just got this on my screen :)

dracut-initqueue[247]: Warning: dracut-initqueue timeout - starting timeout scripts

and get thrown into dracut shell.

Solution

I booted the virtual instance with CentOS 7 / Almalinux 8 ISO, selected rescue mode, let it mount existing system as read-write to /mnt/sysimage and chrooted to it.

While chrooted I rebuilt my initramfs with following command:

dracut -H -f /boot/initramfs-3.10.0-1127.8.2.el7.x86_64.img 3.10.0-1127.8.2.el7.x86_64 || dracut -fv --regenerate-all

Where 3.10.0-1127.8.2.el7.x86_64 is the kernel version, which you must check before issuing the command. Using uname is not recommended as you are doing this with a rescue kernel.

After that just sync and reboot. You may need to reboot twice.

Mounting root partition

If your rescue disk fails to mount your OS partitions here is some advice on how to do it manually.

Firstly locate the devices the OS devices are in. Try command “lsblk”, “lvs” and “lvdisplay” to locate the devices. You can locate lvs volumes under /dev/volumegroupname/volumename.

Then mount the hardware devices under /mnt/sysimage. Some examples below:

mount /dev/xxx /mnt/sysimage
mount /dev/xxx /mnt/sysimage/var
mount /dev/xxx /mnt/sysimage/boot

Then mount the proc, sys, dev filesystems from parent OS:

mount -t proc none /mnt/sysimage/proc
mount --rbind /sys /mnt/sysimage/sys
mount --rbind /dev /mnt/sysimage/dev
mount --rbind /run /mnt/sysimage/run
mount --rbind /sys /mnt/sysimage/sys

Comments

All comments and corrections are welcome.