Just had a very interesting eureka moment, without the animations, the FPS jumps pretty significantly, it almost double, so it looks like that's what I'm going to have to look at optimising the most is how the animations are being played through the code, will need to have a look at these methods when I'm in a better mood for it. The animations seem to be confirmed as hitting the performance the most.
I wonder if there's something I can do about the way the navmesh is dealing with the code? I'll have to have a look at that video again myself.