Diagnosing and Fixing GRUB Boot Failures in Linux

📌

Nothing is more frustrating than rebooting your Linux server only to be greeted with a GRUB rescue prompt. Bootloader failures are among the most common startup issues, but they are also some of the most recoverable—if you know what to do.

If you see errors like:
🔴 error: no such partition
🔴 grub rescue> prompt
🔴 kernel panic - not syncing: VFS: Unable to mount root fs

📌 Don't panic! In this guide, you will learn:
✅ How GRUB works and why it fails
✅ The difference between grub> and grub rescue>
✅ Step-by-step fixes for GRUB misconfigurations, missing partitions, and bootloader corruption
✅ Enterprise-grade troubleshooting strategies for fast recovery
✅ Best practices to prevent GRUB failures in production

🔜 Next in the series: Recovering from Kernel Panics & Initramfs Issues

🔍 1. What is GRUB and Why Does It Fail?

GRUB (GRand Unified Bootloader) is the default bootloader for most Linux distributions. It loads the Linux kernel into memory and hands over control to the OS.

💡 Common Reasons for GRUB Failures

🔍 2. Diagnosing GRUB Boot Failures

💡 If your Linux machine fails to boot, check the type of GRUB prompt you see:

✅ grub> prompt → Partial GRUB working, just missing config
✅ grub rescue> prompt → GRUB cannot find critical boot files
✅ Black screen, stuck at BIOS → Possible bootloader corruption

📌 Step 1: List Available Partitions

From the grub> or grub rescue> prompt, run:

ls

📌 Output Example:

(hd0) (hd0,msdos1) (hd0,msdos2) (hd0,msdos3)

💡 This command lists all available disks and partitions. Identify where Linux is installed.

📌 Step 2: Locate the GRUB Configuration

Try to find the /boot/grub2/ directory:

ls (hd0,msdos2)/boot/grub2/

📌 Expected Output:

grub.cfg  grubenv  themes  fonts

💡 If you find grub.cfg, GRUB exists but isn’t loading correctly.

🔍 3. Fixing GRUB Boot Failures

🛠️ Fix 1: Temporary Boot into Linux

If you reach the grub> prompt but GRUB isn't loading properly, you can manually boot into Linux:

set root=(hd0,msdos2)
linux /vmlinuz root=/dev/sda2 ro
initrd /initramfs.img
boot

📌 Command Breakdown:

• set root=(hd0,msdos2) → Set the partition where Linux is installed
• linux /vmlinuz root=/dev/sda2 ro → Load the kernel
• initrd /initramfs.img → Load the initramfs
• boot → Start the OS

✅ If this works, login and run the following to permanently fix GRUB:

grub2-mkconfig -o /boot/grub2/grub.cfg
grub2-install /dev/sda
reboot

🛠️ Fix 2: Restore GRUB from a Live CD

If grub rescue> appears, you may need to boot from a Linux Live USB and restore GRUB.

1️⃣ Boot from a Linux Live USB
2️⃣ Identify the root partition:

fdisk -l

3️⃣ Mount the partition and chroot into the system:

mount /dev/sda2 /mnt
mount --bind /dev /mnt/dev
mount --bind /proc /mnt/proc
mount --bind /sys /mnt/sys
chroot /mnt

4️⃣ Reinstall GRUB:

grub2-install /dev/sda
grub2-mkconfig -o /boot/grub2/grub.cfg
exit
reboot

📌 Expected Outcome: GRUB is restored, and the system boots normally.

🛠️ Fix 3: Check for Missing Kernel or Initramfs

If you get kernel panic - not syncing, check if the kernel and initramfs exist:

ls /boot

📌 Expected Output:

vmlinuz-5.10.0 initramfs-5.10.0.img

💡 If files are missing, regenerate initramfs and reinstall the kernel:

dracut -f
yum reinstall kernel
reboot

🔍 4. Enterprise Case Study: GRUB Failure in Production

📌 Scenario:
A data center running RHEL 8 experienced sudden downtime on a critical server cluster. After a scheduled update, systems rebooted into GRUB rescue mode.

📌 Investigation:

• Engineers connected via iLO (HP) & DRAC (Dell)
• Running ls in GRUB rescue showed missing partitions
• Bootloader corruption detected after RAID firmware updates

📌 Solution:
🔹 Engineers booted from a Live Rescue USB
🔹 Chrooted into the system and reinstalled GRUB
🔹 Fixed boot order in BIOS settings

📌 Lesson Learned:
⚠️ Always test kernel and bootloader updates in staging environments first
⚠️ Keep offline recovery tools available for remote repair

🔍 5. Best Practices to Prevent GRUB Failures

📌 To minimize downtime, follow these enterprise best practices:

✅ Test bootloader updates in staging before deployment
✅ Keep multiple kernel versions installed in case a rollback is needed
✅ Regularly back up GRUB configuration (grub.cfg)
✅ Use hardware-based remote management (IPMI/iLO/DRAC) for emergencies
✅ Enable automatic boot monitoring via logs (journalctl -b -p err)

📌 Summary

💡 Want to learn more? Check out the next article: "Recovering from Kernel Panics & Initramfs Issues" 🚀

💬 Join the Discussion!

💬 Have you encountered GRUB boot failures in production?
💡 What strategies do you use to quickly recover from boot issues?
🚀 Share your experience in the comments!