Server is not booting on Centos Kernel : OVH server

[santosh]

Just sharing one issue with the OVH server:

We were facing a booting issue with the one of our OVH server running on Centos 7.6, the server was booting on OVH kernel without any issue. We were getting the following error while booting on Centos Kernel.

A start job is running for dev-md1.device

Screenshot :



I tried with compiling multiple kernels on the server but the booting issue still persists, for e.g, I used the following commands to install and boot the server on CentOS kernel.

# yum install kernel

[root@server ~]# yum reinstall kernel*
Loaded plugins: fastestmirror, universal-hooks
Loading mirror speeds from cached hostfile
* EA4: 149.56.181.210
* cpanel-addons-production-feed: 149.56.181.210
* cpanel-plugins: 149.56.181.210
Resolving Dependencies
--> Running transaction check
---> Package kernel.x86_64 0:3.10.0-957.5.1.el7 will be installed
---> Package kernel-abi-whitelists.noarch 0:3.10.0-957.5.1.el7 will be reinstalled
---> Package kernel-debug.x86_64 0:3.10.0-957.5.1.el7 will be installed
---> Package kernel-debug-devel.x86_64 0:3.10.0-957.5.1.el7 will be reinstalled
---> Package kernel-devel.x86_64 0:3.10.0-957.5.1.el7 will be installed
---> Package kernel-doc.noarch 0:3.10.0-957.5.1.el7 will be reinstalled
---> Package kernel-headers.x86_64 0:3.10.0-957.5.1.el7 will be reinstalled
---> Package kernel-tools.x86_64 0:3.10.0-957.5.1.el7 will be reinstalled
---> Package kernel-tools-libs.x86_64 0:3.10.0-957.5.1.el7 will be reinstalled
---> Package kernel-tools-libs-devel.x86_64 0:3.10.0-957.5.1.el7 will be reinstalled
--> Finished Dependency Resolution

Dependencies Resolved

================================================== ================================================== ================================================== ========
Package Arch Version Repository Size
================================================== ================================================== ================================================== ========
Reinstalling:
kernel x86_64 3.10.0-957.5.1.el7 updates 48 M
kernel-abi-whitelists noarch 3.10.0-957.5.1.el7 updates 7.0 M
kernel-debug x86_64 3.10.0-957.5.1.el7 updates 50 M
kernel-debug-devel x86_64 3.10.0-957.5.1.el7 updates 17 M
kernel-devel x86_64 3.10.0-957.5.1.el7 updates 17 M
kernel-doc noarch 3.10.0-957.5.1.el7 updates 18 M
kernel-headers x86_64 3.10.0-957.5.1.el7 updates 8.0 M
kernel-tools x86_64 3.10.0-957.5.1.el7 updates 7.1 M
kernel-tools-libs x86_64 3.10.0-957.5.1.el7 updates 7.0 M
kernel-tools-libs-devel x86_64 3.10.0-957.5.1.el7 updates 7.0 M

Transaction Summary
================================================== ================================================== ================================================== ========
Reinstall 10 Packages

Total download size: 186 M
Installed size: 256 M
Is this ok [y/d/N]: y
Downloading packages:
(1/10): kernel-abi-whitelists-3.10.0-957.5.1.el7.noarch.rpm | 7.0 MB 00:00:01
(2/10): kernel-3.10.0-957.5.1.el7.x86_64.rpm | 48 MB 00:00:02
(3/10): kernel-debug-3.10.0-957.5.1.el7.x86_64.rpm | 50 MB 00:00:02
(4/10): kernel-debug-devel-3.10.0-957.5.1.el7.x86_64.rpm | 17 MB 00:00:01
(5/10): kernel-devel-3.10.0-957.5.1.el7.x86_64.rpm | 17 MB 00:00:00
(6/10): kernel-doc-3.10.0-957.5.1.el7.noarch.rpm | 18 MB 00:00:00
(7/10): kernel-tools-3.10.0-957.5.1.el7.x86_64.rpm | 7.1 MB 00:00:00
(8/10): kernel-tools-libs-3.10.0-957.5.1.el7.x86_64.rpm | 7.0 MB 00:00:00
(9/10): kernel-tools-libs-devel-3.10.0-957.5.1.el7.x86_64.rpm | 7.0 MB 00:00:00
(10/10): kernel-headers-3.10.0-957.5.1.el7.x86_64.rpm | 8.0 MB 00:00:00
--------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 4.6 MB/s | 186 MB 00:00:40
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : kernel-tools-libs-3.10.0-957.5.1.el7.x86_64 1/10
Installing : kernel-tools-3.10.0-957.5.1.el7.x86_64 2/10
Installing : kernel-tools-libs-devel-3.10.0-957.5.1.el7.x86_64 3/10
Installing : kernel-debug-3.10.0-957.5.1.el7.x86_64 4/10
Installing : kernel-debug-devel-3.10.0-957.5.1.el7.x86_64 5/10
Installing : kernel-abi-whitelists-3.10.0-957.5.1.el7.noarch 6/10
Installing : kernel-3.10.0-957.5.1.el7.x86_64 7/10
Installing : kernel-devel-3.10.0-957.5.1.el7.x86_64 8/10
Installing : kernel-headers-3.10.0-957.5.1.el7.x86_64 9/10
Installing : kernel-doc-3.10.0-957.5.1.el7.noarch 10/10
Verifying : kernel-tools-libs-3.10.0-957.5.1.el7.x86_64 1/10
Verifying : kernel-doc-3.10.0-957.5.1.el7.noarch 2/10
Verifying : kernel-tools-3.10.0-957.5.1.el7.x86_64 3/10
Verifying : kernel-headers-3.10.0-957.5.1.el7.x86_64 4/10
Verifying : kernel-devel-3.10.0-957.5.1.el7.x86_64 5/10
Verifying : kernel-3.10.0-957.5.1.el7.x86_64 6/10
Verifying : kernel-abi-whitelists-3.10.0-957.5.1.el7.noarch 7/10
Verifying : kernel-debug-devel-3.10.0-957.5.1.el7.x86_64 8/10
Verifying : kernel-debug-3.10.0-957.5.1.el7.x86_64 9/10
Verifying : kernel-tools-libs-devel-3.10.0-957.5.1.el7.x86_64 10/10

Installed:
kernel.x86_64 0:3.10.0-957.5.1.el7 kernel-abi-whitelists.noarch 0:3.10.0-957.5.1.el7 kernel-debug.x86_64 0:3.10.0-957.5.1.el7
kernel-debug-devel.x86_64 0:3.10.0-957.5.1.el7 kernel-devel.x86_64 0:3.10.0-957.5.1.el7 kernel-doc.noarch 0:3.10.0-957.5.1.el7
kernel-headers.x86_64 0:3.10.0-957.5.1.el7 kernel-tools.x86_64 0:3.10.0-957.5.1.el7 kernel-tools-libs.x86_64 0:3.10.0-957.5.1.el7
kernel-tools-libs-devel.x86_64 0:3.10.0-957.5.1.el7

Complete!
[root@server ~]#

# grub2-mkconfig -o /etc/grub2.cfg

[root@server boot]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-957.5.1.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-957.5.1.el7.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-957.5.1.el7.x86_64.debug
Found initrd image: /boot/initramfs-3.10.0-957.5.1.el7.x86_64.debug.img
done

[root@server boot]# awk -F\' '$1=="menuentry " {print i++ " : " $2}' /etc/grub2.cfg
0 : CentOS Linux (3.10.0-957.5.1.el7.x86_64) 7 (Core)
1 : CentOS Linux (3.10.0-957.5.1.el7.x86_64.debug) 7 (Core)

But, the issue remains the same.

Lastly, I tried with rebuilding the initramfs on the server using the following command and rebooted the server.

dracut --mdadmconf --fstab --add="mdraid" --add-drivers="raid1" -f /boot/initramfs-3.10.0-957.5.1.el7.x86_64.img 3.10.0-957.5.1.el7.x86_64

and the server came up on Centos Kernel.

[root@server ~]# uname -ir
3.10.0-957.5.1.el7.x86_64 x86_64

That's it!!!

[Medha]

Thank you, Great information!