Hi All,
I thought I'd start a thread on this to let you guys know what I'm working on, get some input and who knows, see if this is useful to anyone.
I'm still new to Godot but been building my own things for awhile, after discovering Godot I thought it would be fun to learn it and see if I can contribute to the project instead of reinventing the wheel in my own little thing in areas where I am not very good at (my experience lies mostly in 3D rendering, not in things like sound or GUI).
Also one of the friends I've made at regular developer meeting I attend here in Sydney has done a lot of work modding existing games to add stereoscopic support to them and I was impressed by how well this can work when done properly. Its always felt like a gimmick to me until I saw his version of Abe's Oddysee in hi-def stereoscopic 3D and it is something I've added to my own project as well. Now with VR it's even more of an interesting thing. I know that the Godot devs are looking into adding VR to 3.0, maybe I can help here :)
Earlier in the week Lamberto Tedaldi made a post on the Godot facebook page introducing his solution for split screen stereoscopic on an Android phone. It gave me a bit of a boost in understand a few things I hadn't grasped about Godot and a nice starting point to work from.
One of the issues I believe with his solution that can't be solved without diving into the C++ source code is that his solution results in a 'Toe-in' of the two cameras as both 'lookat' a single convergence point set some distance away to the front. Now I haven't mastered the theory behind why this is actually a problem and whether my info is based on what is written about 3D displays and whether it thus applies to HMDs as well but so far I've been lead to believe the cameras should look in a parallel direction but that there is a skewing of the frustum based on the convergence distance. A good bit of information on the subject can be found in the PDF from NVidia: https://www.nvidia.com/content/GTC-2010/pdfs/2010_GTC2010.pdf
Anyways, that is the approach I had implemented in my own engine some time ago and it has indeed give much better results on my 3D monitor (I'm using a 3D TV at home that can take splitscreen input but also build in support for cards that have hardware support for 3D with a separate left and right buffer). Now I want to replicate that in Godot :)
As part of learning more of the internals of Godot I've started modifying the camera code to support configuring a camera as a proper left eye or right eye camera implementing an asymmetric frustum. I got a fair way last night and eventually got to a point where i think the correct projection matrix is applied to each eye but where I think it is not using that frustum for clipping but using the normal projection frustum. It was late and I decided it was time for bed :)
I'll spend some more time on it tonight time allowing and am planning on checking in a first version into my fork on my github page in the weekend. I just wanted to introduce what I'm doing and see if there are any people interested in having a look over my shoulder and seeing if what I'm doing makes sense and the approach I'm taking fits in the overal way Godot is designed to work. I also want to make sure I'm not re-inventing the wheel here if others are already going down this path.
My first milestone will simply be getting the left and right camera projection matrices to work in combination with the split screen approach that Lamberto created and then adjusting the approach to be able to switch between the side by side normal aspect ratio approach for HMD and for 3DTV where the two images get stretched and I need to double the aspect ratio as a result.
I then want to start looking into what it'll take to start tying into hardware support with left/right buffer and see what it'll take to support things like NVidias 3D Vision. One thing here will be looking into ensuring we're not doubling up work as it would be possible to render to both left and right buffers simultaneously or at the very least more parallel and remove a lot of overhead in rendering a scene twice.
Cheers,
Bas