Is there any way to change this? I despise the brown fading to white. I want to change it to a solid colour or my own gradient of choice.

Project Settings - Environment - Default Clear Color

Specially for 3D, you can create a world environment, set property Background - Sky - Sky Material = PanoramaSkyMaterial

Then set property PanoramaSkyMaterial - panorama = GradientTexture2D, set your custom gradient, finished.