I have so many questions about this method that I'm actually going to have to do a write up but I think I'm getting something of a breakthrough with it thanks to the video I found on the 2D version of it. At the moment though, I've got some errors on it and I don't know why. I also have no idea where I'm supposed to be placing the quad mesh with the viewport texture on it in order for the multiply blend to work on the shader.

I'll be coming back to this thread when I've had more time to think on the specifics.

E 0:00:01.245 get_path: Cannot get path of node as it is not in a scene tree.
<C++ Error> Condition "!is_inside_tree()" is true. Returned: NodePath()
<C++ Source> scene/main/node.cpp:1587 @ get_path()

E 0:00:01.247 get_node: (Node not found: "ViewportContainer/Viewport" (relative to "").)
<C++ Error> Condition "!node" is true. Returned: nullptr
<C++ Source> scene/main/node.cpp:1325 @ get_node()

E 0:00:01.248 setup_local_to_scene: ViewportTexture: Path to node is invalid.
<C++ Error> Condition "!vpn" is true.
<C++ Source> scene/main/viewport.cpp:69 @ setup_local_to_scene()

Right, I've at least made something happen now with this confusing viewport texture method, however because I'm still learning how it all works I'm struggling to have the rings even do anything with this effect. I'm 99% sure it's probably not the shader because even in the 2D example I managed to find it was all viewport black magic trickery.

I tried to keep things as simple as possible for now with both cameras pointed down on top of each other and in the exact same position. I've managed to get a viewport texture onto the quad mesh and it's set to multiply. I do understand how these viewports seem to work, that concept I'm fine with it's just the way the multiply blend mode is working in this case with preventing the overlap that I'm totally struggling with. I don't even really know as you can probably tell where the quad is supposed to be and what it's supposed to be doing to deal with the circle overlap problem.

You can see in the screenshot that there's nothing really happening with the circles on the quad mesh, I've shown how I've got my hierarchy setup as well a where the viewport texture actually is because it's a really awesome effect when you can pull it off and can be used for all kind of pretty borders among other things. I need to do something about these duplicates as well that seem to keep popping up when it should be purely happening on the quad?

Holy crap I really am having something of a breakthrough with this technique, I'm going to do a full write up tutorial on this thread just to detail the steps for anyone trying to look at this effect because it's ridiculous now even on how older engines people don't really seem to have looked at the topic so much. Probably because it's a pain in the arse to implement if you don't know anything about viewports, blend modes and how they work.

There are problems I'm running into with this effect though at the moment, there is currently some z fighting I think happening with the outlines. I think maybe Z Write off will fix that, I will test it, the resolution for the quad is too small so I'm going to have to do some more fiddling. I hadn't realised when you posted in your example @cybereality that when you meant make the cylinders white as in a solid white, I thought I needed to keep them transparent and use multiply mode on those as well. Is this the effect that's supposed to happen when you blend the two solid shapes in on the quad?

I remember people on the Unity forums describing the same sort of method and back then I had no idea how to really use it effectively and this time it was for a fog of war. I had no idea this was what they were on about but I did keep a note of. This will take some tweaking but I'm definitely going to do a proper written tutorial on this.

This is what I've managed so far with some resolution tweaking, but I need to get rid of the black and the quad is making the borders look fairly 2D and flat, I don't know if that's maybe the camera setup though.

The clear color of the viewport also needs to be white. But you are pretty close.

Got it! So I couldn't seem to set the clear color in the UI for the viewport but I changed the default colour to white through the code, I re-enabled the background and now the quad is completely transparent thanks to the multiply mode. Definitely doing a step by step write up on this when I'm in the mood to help anyone who likes this sort of effect.

My main complaint with the overlapping and I'm convinced that this is z-fighting is there are some tiny bits of pixels still left reading the overlap of the outline. I'll also do a write up with how to do the vertex positioning which should be more straightforward ironically because it's just grabbing the right ones and raycasting them to the floor each time there's a change in scale.

I've noticed though there are some input problems, I think the quad might be interfering with the raycasting and camera movement somehow despite collision being off but I'll double check that. It may be I just need to take another look at the code and check my second viewport camera is working correctly.

So hilariously this works great in the viewport but I need to know how to set the viewport's blend mode to multiply, there must be a blend mode option tucked away somewhere? Because of the way multiply works if I set the background to transparent that's why it showed up black initially. I found out that in the GUI if you change the custom colour rather than the clear colour you don't need code for that bit.

Any ideas? I did have a glance through the documentation but this is very unfamiliar to me so I'm experimenting now.

You don't need to use transparent. You want it to be opaque with a white background. The viewport is just basically a render texture. If you want a specific clear color, this would be set on the camera that is tied to that viewport.

    cybereality Yeah that's what I hadn't realised about how multiply mode works, my problem now is though when I start the project my camera is solid white so that's the next problem to deal with.

      Yep, stuck for now, not sure why it's doing things correctly in the viewport but not the build.

      Weird, okay, now I know the steps better I did the classic thing of wiping and starting over so I must have clicked something somewhere that screwed up but I now don't have the white background anymore and it's all showing correctly.

      Now there's something that's bothering me quite a bit and I just think it might be the nature of the technique being used because it's projecting the circles on a flat surface. You can see there's a line still visible very thinly even though the intersection is now mostly gone, there's also a bit of an issue I'd like to tweak with the circles now looking quite flat since bearing in mind this is a fully 3D cylinder I have going. Does anyone have any ideas how I could fix this and made it work across the terrain I have? I know how I'm going to do the positioning itself of the vertices but the outline is doing this billboard effect which I don't really want.

      Oh for crying out loud I fixed the main angle and positioning problems right after I made that post, I put the quad over my main player camera and then the quad a child of the camera so it would follow no matter where the camera went and the circles are looking a lot better and almost in the position they should be within the game world and are also looking more 3D.

      The thin lines are still an issue, not sure how to deal with that and the quad behaviour still needs more tweaking, there are an awful lot of steps to getting this right but I'm almost at the end now. It's going to be really interesting for me seeing how this looks with the vertices placed properly compared to the other mesh generated version I had before.

      Very close now that I've adjusted the viewport resolution to match with the build resolution but I'll leave it here for now until I've had more of a think on how to fix things. The quad isn't exactly following the movement as it should, I think I need to make it a child of my pivot like I have done with the camera and yes it did work. Just a quick write up on that and there was a bit of colour in the quad that was still there and I had forgotten I needed to flag the quad as unshaded, now it's all looking like it should in the main view.

      Still not satisfied yet though, there are bits I need to tweak like the Z-Fighting.

      You probably want to use a torus, not a cylinder. Or find a way to give the cylinder a border thickness. Tori are round, but perhaps there is a way to model it with a flat top.

        cybereality I could experiment with some blender trickery again to see what I can do now I know more about how the effect works. One of the reason I used Blender with making this current cylinder is because it enabled me to place the outline on just the sites which is how I got the effect on the outline in the first place. Alternatively, instead of a torus, I could bevel the cylinder to give it more rounded edges.

        Any ideas on how to deal with the zfighting in the overlapping parts of the circle?

          16 days later

          I thought I'd post a video now to show where exactly I'm at with the results as a video in a lot of cases can be better than a bunch of screenshots in showing some behaviour that I'm not all that happy with. The intersection is working, I think in order to fix the pixelation problem in the centre which I swear looks like z-fighting to me I need to turn z-write off for the shader itself. I just don't know what the syntax is for that.

          The biggest problem at the moment though that I have with the viewport's behaviour is the rings are not staying put. Now, I think I know why this is and it's because they're a child of the viewport which is following the camera. As a result this is causing the influence rings to move around in an almost billboard like effect.

          I suppose the question I have is how do I keep the rings static? Once I do that I'll have them working exactly as I want and can then work on dealing with the vertices. I suppose I could override the child behaviour somehow? That might work, should I set as top level in the ready function? Or would that be too much of a hacky work around? In other games like Stellaris and Civilization VI they can keep the borders completely still without any movement so it must be possible.

          Oh for crying out loud, I think setting as top level partly did the job, however because of the way the viewport is seeing the influence rings it's still not quite how I want it yet and the viewport is rotating with the camera in a bit of an odd way. I think the main culprit of this behaviour is definitely the quad I'm using. Depending on where the camera is rotating I don't think the quad is quite covering the camera as it should as well and moving it closer screws up the positioning of the rings.

          I'm now experimenting with vertices code to see if I can finish off the functional side of this effect that I'm working on while I think about how to neaten up the way the shader and viewport is behaving. Do I need to do something extra to grab the vertices of the mesh I'm editing for this to work? I'm getting an error.

          extends MeshInstance

func  _ready():
	SetVerticesPositions()
	

func SetVerticesPositions():
	var mesh = ArrayMesh.new()
	var meshDataTool = MeshDataTool.new()
	
	meshDataTool.set_vertex(0, Vector3(0, 100, 0))

          E 0:00:00.579 set_vertex: Index p_idx = 0 is out of bounds (vertices.size() = 0).
          <C++ Source> scene/resources/mesh_data_tool.cpp:355 @ set_vertex()
          <Stack Trace> CylinderMeshVerticesPosition.gd:11 @ SetVerticesPositions()
          CylinderMeshVerticesPosition.gd:4 @ _ready()

          I'm just trying to get a vertices to change position at the moment, nothing too complex as of yet, I'm wanting to grab individual vertices on the mesh and edit them.

          Oh I've already got the effect just about now if you check the video I uploaded, it's just the way the quad is behaving that I need to tweak as well as a bit of zfighting I think. Now I'm looking at vertices positions and I'm going to use raycasts to position them on the ground. FIrst though I just need to experiment with how to even move the vertices in code using the MeshDataTool. I had a look at that solution awhile ago but it's 2D only and I wanted my outline to be properly 3D which I've achieved.