As an example of not a problem, but a little complication, when I got my new PC last November I had to install the unstable branch of Debian to get rudimentary vulkan ray tracing support with the open source driver (mesa/radv) for the graphics card. Otherwise I would have had to revert to the AMD drivers (amdgpu or amdgpu pro). The latter would have been a manual installation, which means I break the updates from the repositories. I try to avoid such things, as one day there may be incompatibilities between kernel and driver.
Sure, you say I could read the 10 pages of small print of the release notes before updating, and you're right, but ... errr ... well ... :-)
Meanwhile, the mesa/radv driver is reported to be better and more performant than the amdgpu driver, and it has arrived in the testing branch of my OS which means it is available for everyone without having to climb any fences.
Suffice to say that with nvidia there's no choice really which driver to use (the proprietary is it) for graphics programming and gaming. The open source nouveau driver has a lot of problems, only because nvidia refuses to work with the open source community.