Unsolicited opinion:
Games have the potential and at times can be an art form and some of the best completely throw realism out the window. What's the point of learning how to build a world if its something you've already seen before? What Remains of Edith Finch i think is a great example of a game that has a healthy splash of realism but not so much where it becomes boring.
I think realism is good in a game if it serves a purpose towards the gameplay. I can definitely see where you come from about it being wasteful in some/lots of cases. I've played plenty of "pretty" or realistic games that were quite hollow when it came to substance. That being said...
Example of "good" realism:
The physics of impact when a sword hits your shield. A recoil feels good and keeps you engaged with haptic feedback, like a controller vibration, to communicate the hit with more than sound.
Example of completely pointless realism:
When a fighting game with female characters wastes time on their "jiggling" physics of anatomy that are not important to combat.