Oh my gosh. I deleted the environment, and increased the light size, then turned on the sun. Now it works.
Now I just have to get my skybox to work. It seems so weird that I can't just make a CG box and add different textures to the sides. If I go back to an environment, the light stops working again.