How to prevent NVIDIA from automatically upgrading the driver on Ubuntu?
I was training models last night on my Ubuntu workstation, and then woke up this morning and saw this message:
Failed to initialize NVML: Driver/library version mismatch

Apparently the NVIDIA system driver automatically updated itself, and now I need to reboot the machine to use my GPUs... How do I prevent automatic updates from NVIDIA?

I think I have had the same issue. It is because of so-called unattended upgrades on Ubuntu.
Solution 1: check the changed packages and revert the updates
Check the apt history logs
less /var/log/apt/history.log

Then you can see what packages have changed. Use apt or aptitude to revert the changes.
Solution 2: disable unattended upgrades
Use this guide to disable unattended upgrades. Please consider if this solution works for you as you have to install security updates manually after this change.
Solution 3: hold specific packages
Use this guide on how to hold certain packages. Read the apt history as mentioned above to determine which packages you have to put on hold. Probably CUDA related packages such as nvidia-cuda-toolkit. Hard to say since some information is missing from your post. You can see all nvidia related packages like this
dpkg -l *nvidia*

I hope at least one of my solutions works for you :)
P.S. you have to change the title. NVIDIA isn't upgrading anything on your system by itself. Ubuntu is the one causing your trouble ;)
