Tomcat That worked! Guess I'll use Krita to convert my exr back to exr from now on... What a weird issue.
I checker my normal view again using my PNG and the colors shifts as I rotate (which I thought was normal at the time), which is not the case for your EXR:
MY PNG:
YOUR EXR: