Yeah. Part of why we moved out of Unity and into Godot was it was taking 5 minutes for me and my artist to open a project with our own assets, a bunch of plugins, and other messy stuff. Changing code took 30 seconds to compile even though I deliberately set up C# assemblies to break up the process. We hadn't even started putting in full world maps or anything. It was just a handful of characters. Another aspect is Unity caches everything in its library with metadata so you can reference assets without relying on their direct path. This is a nice QoL feature and I do miss it going into Godot, since everything's based on asset paths and adjusting folders and names is a pain, but for Unity it was disproportionately messy when things added up.
Godot is lean in that it seems to just grabs assets as told to for the current situation and doesn't compile (unless building a pak? I assume GDScript has a bytecode-ized variant for export) So it doesn't have to worry. Your worst bottleneck is probably if you're on an HDD or your computer is powered by potatoes and lemons. To be fair, I've not reached the point our project is too big yet, but if Godot isn't really doing any asset library maintenance on boot and just dragging assets into exported paks, then it's probably not going scale into a big problem.
Load times, well, it's probably just "here's asset path, unzip and decode, dump raw into RAM" and probably not very bad unless the asset itself is the problem or you're using encrypted files on a very weak CPU.
I'd actually like to see if there's any good source of info for Godot's management of assets and experience on how it gets when your asset count is high. The freedom is dangerous in it seems easy to lose grasp of good organization practices and start dumping things where they probably don't belong.