How to smoothly render too many unique meshes?

woopdeedoo

kuligs2
I did that, but it didn't make any difference, unfortunately. I mentioned it in the OP. It's still something I'm considering using later on, as it may still help with larger amounts of brushes, but for now it's not doing anything, because the brushes are still forcing a ton of draw calls for being unique meshes.

I think I must figure out a way to merge their meshes in groups to reduce draw calls.

LODs are also not useful in this, because brushes are already as low detail as they can get.

xyz What's a "brush"?

To put it as simply as I can, fundamentally a brush is a 3D shape you use to block out the maps, which can have any number of faces, but must obey a few rules:

it must be convex
every face must be perfectly flat
every vertex must have 3 or more edges connected to it

Don't ask me why they named them "brushes", though... 🤷‍♂️

xyz

woopdeedoo Why are you using this concept of "brush" to describe things while not using any of mentioned optimizations? I also find it hard to believe that some other editor would run 18000 such brushes while each being an unique mesh. What would even be the purpose of so many unique objects?

Since you're not using any optimizations that those brushes are using, you're not really using "brushes". You're using plain mesh instances. Why call them "brushes" in Godot context?

To optimize in general, start by clumping brushes/meshes that have the same geometry into multimesh instance nodes. You may additionally use some heuristic to group by some proximity threshold into multiple multimeshes to benefit from frustum culling.

You can use visibility range as described in that video to cull off far away objects and/or swap them with sprites.

I still don't fully understand what a brush really is, but from the description @kuligs2 linked there is a distinction between editor map and runtime (compiled) map. It appears that brushes end up in (or constitute) a BPS tree at runtime, maybe even it the editor. This could significantly improve the performance via tree culling. So if you want comparable optimizations, you'll have to implement a BSP tree.

There's also not much use in benchmarking some bruteforce scenario. Make something that's comparable to an actual use case.

woopdeedoo

xyz

The picture below is a real map I made for The Dark Mod. Nearly everything you see in this picture is brushwork. In practice, the chances of any one brush being an exact copy of another are very slim. There are possible exceptions, like the cornerstones of the tower building. But even those are not guaranteed to be exactly the same, as the textures can have been aligned differently, and thus each of them may still have different UVs in at least one face.

Each brush is created either individually by dragging the mouse, or by duplicating an existing brush. When you duplicate a brush, that's the only situation where you can easily know that this brush is an exact copy of another brush, without having to iterate through the entire list of brushes and comparing every vertex and UV coordinate. But then what usually happens is that the duplicate gets immediately modified in order to make up some different part of the geometry. Cases where you do use exact copies are very rare.

Even if you duplicate a brush to makes bars for a prison cell, for example, it's very likely that you'll still align each of their textures differently, just so they don't all look the exact same in game.

This is why brushes have to be unique. It's very rare that they are the same, and I don't suppose there's an efficient way to compare brushes to check that. So if this can be optimized, it has to be in different ways.

I tried looking at the source code from DarkRadiant and TrenchBroom, but so far I haven't been able to find how they handle these things. (I know TrenchBroom uses a "RenderBatch", but I'm still in the process of wrapping my head around how the whole thing works, and if it could be replicated in Godot.)

xyz You can use visibility range as described in that video to cull off far away objects and/or swap them with sprites.

The visibility range is supposed to be an option that users will be able to change to their own needs in the map editor, so I can't use that as an optimization. But then also, there's no telling how many brushes there might be in close proximity. In an editor like this, you can create brushes of any sizes in any place you want. You can easily have more than 1000 brushes within the visibility range. The picture above does have more than that (the tower alone is nearly 500).

xyz I still don't fully understand what a brush really is, but from the description @kuligs2 linked there is a distinction between editor map and runtime (compiled) map. It appears that brushes end up in (or constitute) a BPS tree at runtime, maybe even it the editor. This could significantly improve the performance via tree culling. So if you want comparable optimizations, you'll have to implement a BSP tree.

BSP trees are a map compiler thing that is unrelated to the editors. It's an algorithm in map compilers for quake/doom derived engines (including Source), that creates a tree representation of the map, so it can cull parts of it in game. It's basically a different way of doing occlusion culling in-game, and has nothing to do with the level editor. The editor doesn't care about any of that at all. The map files you save from the editors contain all the information about every single individual brush.

Toxe

woopdeedoo There is too much "brush" in that post. 😆

When you export that level, do these individual objects get baked together into one (or a couple of) big meshes, so that the engine can render them efficiently? Or would it be one draw call for every single little mesh? Can't really imagine that though, because that would be a disaster.

xyz

woopdeedoo How do you know the editor is not using BSP culling when rendering the scene?

How do you create a "brush" in Godot?

If draw calls are your bottleneck you can partition the space into a grid and merge everything that is in a cell into a single mesh. If something in the cell is selected, dissolve it back into individual instances for editing. When deselected, assemble back.

DarkRadiant probably does some kind of merging. You can try to capture one of its frames with something like RenderDoc and check how is vertex data grouped into draw calls.

Toxe

So, generally speaking, this is my understanding of what "brush" usually means and its origins.

To no one's surprise the term comes from "paint brush" and goes back to (very) old paint programs, like Deluxe Paint on the Amiga. I actually used that one myself back in the day. In DPaint you could copy small regions of the picture as a "brush" (like a sprite) and then "paint" with it, either by single clicking or holding the mouse button. Then you would paint something that has the look and shape of your brush. Shocking, I know.

This BTW is also why Guybrush Threepwood from Monkey Island is called Guybrush. Because his sprite was saved to a file called guy.brush in DPaint and after a while the name stuck.

In modern 3D tools a brush is usually a predefined piece of raw geometry that you could drop into a level to create 3D objects directly inside the engine editor. At least that's how it works in Unreal. You have a couple of presets like stairs, spheres or simply a cube and then you can scale and rotate it, modify its vertices or apply some boolean operation to cut holes into the mesh. And it's called a brush because you can copy the mesh and then paste it all around the map, similar to a paint program.

And while you could create simple objects this way and even texture them etc. it's usually used for blocking out areas and defining your level geometry. Add some colliders and you can already run around the level while the 3D artists fill those areas with proper models, piece by piece.

I never cared much about 3D in Godot so I have no idea if Godot has a similar concept of in-editor editable raw meshes for quickly adding some WIP geometry (don't think so, though) but, yeah, this is IMO usually what people mean when they talk about brushes: Single, independent ad hoc meshes, that won't make it into the final product.

woopdeedoo

Toxe
Yes, I don't know the specifics of what the Doom3 engine does, but presumably it merges the geometry wherever it can. It probably doesn't make the whole map into one big mesh, because it has to be able to cull parts of it during gameplay, but it probably merges whatever it can. It also discards any faces that are never visible to the player. The final playable map is highly optimized.

When we use TrenchBroom or NetRadiant Custom with Godot, the FuncGodot plugin does the same thing, but in this case it's advisable for the mappers to manually separate their maps into chunks of geometry to improve occlusion culling in-game. It's not an automatic process when working with Godot. If you keep all the geometry in one single entity, the plugin will make the whole map a single mesh.

Toxe

woopdeedoo Alrighty, now I understand what you are trying to do here and what the issue is. As I said, I don't know much about 3D in Godot but did you take a look at CSGs? Not saying that this solves all your problems but maybe the links on the doc page can give you further ideas.

https://docs.godotengine.org/en/stable/tutorials/3d/csg_tools.html

Toxe

xyz If draw calls are your bottleneck you can partition the space into a grid and merge everything that is in a cell into a single mesh. If something in the cell is selected, dissolve it back into individual instances for editing. When deselected, assemble back.

Something like that would be my suggestion as well, but you phrased it a lot better than I would have.

woopdeedoo

I may be onto something. I tried building one mesh per material for the entire brushwork, and I think this might go a long way. It reduces it to 1 draw call per material in total. I ran some tests, and it takes a few seconds to rebuild the mesh for 8000 brushes, but that's because I only have 1 material for now, so the whole map is just a single big mesh. The end result runs as smooth as butter, of course, since it's just one draw call.

In a real map there's always many materials (the one in the screenshot above has about 300), so editing one or a few brushes might trigger the rebuild of only a few meshes. If each of them runs on a thread, it may be fast enough. I lose frustum culling this way, but it probably won't matter.

Gonna go with this for now and see how it goes.

xyz If draw calls are your bottleneck you can partition the space into a grid and merge everything that is in a cell into a single mesh. If something in the cell is selected, dissolve it back into individual instances for editing. When deselected, assemble back.

This is also something I'm keeping in mind for sure.

xyz How do you know the editor is not using BSP culling when rendering the scene?

Because the tree representation requires compiling the map, and it would slow down the editor, as it would have to rebuild the visibility tree every time you made changes to the brushwork. The BSP algorithm also requires the map to be completely sealed off from the void (https://developer.valvesoftware.com/wiki/Leak), which is something that you don't always have in the editor (because you're building the map).

Moreover, the BSP algorithm also has to cut many brushes in half or however it makes sense for culling. It's a destructive process. The game needs it because it has to render a whole lot more than the editor (lights, shadows, higher res textures with normal maps and all that, animations, etc) on top of having to calculate physics and such.

xyz

woopdeedoo I'd definitely look into the rendered frame structure of one of those Quake editors via RenderDoc. By looking at vertex buffer content at each draw call, you can get a good clue on how exactly they are merging vertex data (per material, per proximity...).

For performant realtime merging you'd probably need to go to native code via gdextension, but you can still first make a prototype in script. If the overall vertex count is relatively low and some sort of partitioning is used, the script version may even end up being sufficiently fast.

woopdeedoo

xyz yea, I'm gonna have to test it out and see. Chunking will probably be required. I like to avoid GDExtension as much as possible, as it slows down development, so I'll probably leave it for a later stage.

Editing brushes is the part I'm trying to figure out now. If one brush is selected, it's easy to just isolate its mesh and make changes to it individually, and update the main mesh(es) only once to exclude that one in the meanwhile. In fact, as it is now, each brush is already keeping an individual Mesh in memory, because I'm using that to easily generate the convex collision meshes for selecting brushes. So I could jsut use that.

(The mesh is just for visual representation, btw. The actual (simplified) geometry is being stored in arrays of Face, Edge and Vertex objects -- the mesh gets updated from those.)

But if too many brushes are selected, changing many such individual meshes as you move gizmos around, may become slow (unless maybe they can be threaded). If I merge the meshes of the edited brushes (maybe also with chunking), it may also become slow, as it will need to be constantly merging the brushes.