I recently encountered a weird problem when updating my Debian stretch Docker server at OVH. During the upgrade process I noticed that both service Containerd and the Docker daemon (which depends on
containerd) failed to start, so I started investigating and was looking for suspicious things in my syslog logfile and found the following:
Nov 8 16:25:03 ns3085344 modprobe: modprobe: FATAL: Module overlay not found in directory /lib/modules/4.9.33-mod-std-ipv6-64
I got confused, because the file was right there.. nothing out of the ordinary.
[email protected]:~# stat /lib/modules/4.9.33-mod-std-ipv6-64/modules.builtin.bin
So I started looking through some forums, but couldn’t find anything useful. I contacted a good friend of mine, Shyim. After digging around he found some interesting resources regard
the OVH kernel. It seems like OVHs custom linux kernel is the cause of this issue.
So we started investigating and tried to remove the kernel from the server and replace it with the official one from Debian. Installing the new kernel was fairly easy:
After installing, I connected to my server using OVHs IPMI web console and chosed the new installed kernel during the boot process. Everything was working again, yay!
But I wanted to get rid of the nasty kernel and remove the boot option. I simply moved the configuration file and updated grub:
[email protected]:~# mv /etc/grub.d/06_OVHKernel ~/06_OVHKernel.backup
One more reboot aaaand YES. The old kernel is gone, all my services are working again and my bootloader config has been updated.
I still don’t know what caused this issue, but as far as I can tell I’m not the only one having this issue. We also noticed they probably just copied their kernel onto my server, because we
couldn’t find it as installed package
I hope this post helps you in case you got the same problem.