Sorry that it took so long, I did simple not find the time to reply.
I don't want to modify the vertices, I want to modify the fragments actual 3D position or displace the pixels like @award wrote.
Let me explain once again what I'm basically doing:
Import a 3D texture where each data point is the color of a voxel (or empty, when alpha is zero) like this one
Render a box in the size of the voxel model (like 20x40x10) with flipped faces and the raymarching shader applied to it
The vertex shader just transforms the vertices of the box
The fragment shader will trace a ray from the camera position in the direction of the 3D pixel coordinate
If it hits a voxel along the rays path, that is the actual position, depth, color and normal of the pixel.
If no voxel is hit, discard that fragment.Use a depth prepass for depth testing
Use the voxel position instead of the vertex position from the shader to calculate shadows, lights etc.
So, I've managed to get everything working except the shadows because I'm not able to tell the final shader to use my displaced position from the raymarching algorithm.
I've extracted the final shader via RenderDoc and it looks like it should in theory be possible:
void fragment_shader(SceneData scene_data)
{
uint instance_index = instance_index_interp;
vec3 vertex = vertex_interp;
vec3 eye_offset = vec3(0.0);
vec3 view = -normalize(vertex_interp);
vec3 albedo = vec3(1.0);
// ... more variables
// ... source from gdshader file
albedo = m_albedo;
normal = m_normal;
gl_FragDepth = m_depth;
// and if here I could do:
vertex = worldPos;
view = -normalize(vertex);
But without this, the shadow will be applied to the box just like this:
So I hope this makes is more clear now
Also, here is a very interesting presentation of how the game Teardown does it: