Xanz I actually managed to fix this issue but posting the solution in case anyone else is attempting to use the implementation.
So there is two issues, In Godot 4 the default capsule orientation is upright rather than aligned along the Z axis. Since the implementation uses the code from Godot 3.5 the alignment is still aligned to the Z axis instead of upright. You can fix this in the bullet code so that instead of creating and returning a BtCapsuleShapeZ you return a BtCapsuleShape, This will fix the alignment issues and the strange invisible force.
Alongside this another issue encountered is that the capsule height is no longer aligned with the debug widgets heights and just the general capsule height as it seems like Godot 4 expects that to be the exact height. This causes an issue as bullet expects a half height(This is just knowledge from implementing bullet in a custom engine so could be slightly incorrect but seems to resolve this issue).
This will cause issues with capsules appearing like they are floating or just general strange height related issues. To solve this simply divide the height received in the CapsuleBulletShape::setup method by 2(You could do this earlier in the callstack but it seems like a sensible place to do it as it keeps it within the bullet module). This will give you the half height which is what bullet expects.
Good luck to anyone else using this!