How to smoothly render too many unique meshes?

woopdeedoo

I'm trying to render thousands of meshes (as many as possible), but I'm having the problem that I can't optimize them in the usual ways.

I'm making a brush-based 3D level editor, akin to Valve Hammer, Trenchbroom, DarkRadiant and others. And one problem I'd like to get solved from the beginning is how to render many brushes with good performance.

As of now, if I have 1000 brushes on the screen, my frame rate drops to 40. That's not a lot of brushes to be capped at, so I need to improve this. My system is old and slow, but that shouldn't be a concern. DarkRadiant can render upwards of 18,000 brushes at 60fps, no sweat, on my system (I got sick of adding more brushes at some point).

The main problem I have is that brushes can't be optimized in the obvious ways. For example, most brushes in a map are going to be different from all other brushes, so they must be unique meshes, and that means I can't use MultiMeshes. On top of that, each face of a brush can have an arbitrary material, so there's a high likelihood of there being multiple surfaces per brush mesh. But then each surface in a mesh is a separate draw call.

Right now my tests involve simple cuboid brushes, with 6 surfaces each. And so as it stands, rendering 1000 of those brushes, results in 6000 draw calls. I'm testing with the worst case scenario, where each face of every brush has a different material.

So I'm trying to figure out how to optimize this. I tried using the RenderingServer, but it didn't improve performance at all (at least not with just 1000 brushes; it may be useful down the line, but not now, it seems).

I tried occlusion culling too. The results seemed promising, but it has a problem when you fly into a brush it culls brushes you should be able to see from there. I tried turning off the occluder of the brush the camera is in, but with too many brushes in the scene the occlusion algorithm takes too long to update and you see the brushes disappearing momentarily.

An idea I had, was to merge brush meshes, or face meshes that share the same material. The problem is doing this blindly might result in meshes being merged despite their brushes being too far away, and then many of them might lose frustum culling. I presume this method might need to be chunked with a quadtree or octree or something. But doing some kind of chunking for brushes doesn't sound as intuitive as for voxels, because brushes are at arbitrary positions and sizes, and any leaf in a tree might be densely packed or nearly empty.

I thought about using an AtlasTexture, which would allow all brushes to have a single surface, but in 3D with big textures you'll hit the atlas texture size limits quickly (4096x4096). I'm also concerned that with an AtlasTexture I can't do texture tiling on the brush faces. Another option that I stumbled on was the Texture2DArray, but it seems to require that all textures have the same size, so that's no good.

I'd be interested in hearing any other thoughts on this.

xyz

woopdeedoo What's a "brush"?

kuligs2

xyz i think https://developer.valvesoftware.com/wiki/Brush

woopdeedoo

How are you adding brushes?

Maybe insted of adding them via nodetree, you use render server?

woopdeedoo

kuligs2
I did that, but it didn't make any difference, unfortunately. I mentioned it in the OP. It's still something I'm considering using later on, as it may still help with larger amounts of brushes, but for now it's not doing anything, because the brushes are still forcing a ton of draw calls for being unique meshes.

I think I must figure out a way to merge their meshes in groups to reduce draw calls.

LODs are also not useful in this, because brushes are already as low detail as they can get.

xyz What's a "brush"?

To put it as simply as I can, fundamentally a brush is a 3D shape you use to block out the maps, which can have any number of faces, but must obey a few rules:

it must be convex
every face must be perfectly flat
every vertex must have 3 or more edges connected to it

Don't ask me why they named them "brushes", though... 🤷‍♂️

xyz

woopdeedoo Why are you using this concept of "brush" to describe things while not using any of mentioned optimizations? I also find it hard to believe that some other editor would run 18000 such brushes while each being an unique mesh. What would even be the purpose of so many unique objects?

Since you're not using any optimizations that those brushes are using, you're not really using "brushes". You're using plain mesh instances. Why call them "brushes" in Godot context?

To optimize in general, start by clumping brushes/meshes that have the same geometry into multimesh instance nodes. You may additionally use some heuristic to group by some proximity threshold into multiple multimeshes to benefit from frustum culling.

You can use visibility range as described in that video to cull off far away objects and/or swap them with sprites.

I still don't fully understand what a brush really is, but from the description @kuligs2 linked there is a distinction between editor map and runtime (compiled) map. It appears that brushes end up in (or constitute) a BPS tree at runtime, maybe even it the editor. This could significantly improve the performance via tree culling. So if you want comparable optimizations, you'll have to implement a BSP tree.

There's also not much use in benchmarking some bruteforce scenario. Make something that's comparable to an actual use case.

woopdeedoo

xyz

The picture below is a real map I made for The Dark Mod. Nearly everything you see in this picture is brushwork. In practice, the chances of any one brush being an exact copy of another are very slim. There are possible exceptions, like the cornerstones of the tower building. But even those are not guaranteed to be exactly the same, as the textures can have been aligned differently, and thus each of them may still have different UVs in at least one face.

Each brush is created either individually by dragging the mouse, or by duplicating an existing brush. When you duplicate a brush, that's the only situation where you can easily know that this brush is an exact copy of another brush, without having to iterate through the entire list of brushes and comparing every vertex and UV coordinate. But then what usually happens is that the duplicate gets immediately modified in order to make up some different part of the geometry. Cases where you do use exact copies are very rare.

Even if you duplicate a brush to makes bars for a prison cell, for example, it's very likely that you'll still align each of their textures differently, just so they don't all look the exact same in game.

This is why brushes have to be unique. It's very rare that they are the same, and I don't suppose there's an efficient way to compare brushes to check that. So if this can be optimized, it has to be in different ways.

I tried looking at the source code from DarkRadiant and TrenchBroom, but so far I haven't been able to find how they handle these things. (I know TrenchBroom uses a "RenderBatch", but I'm still in the process of wrapping my head around how the whole thing works, and if it could be replicated in Godot.)

xyz You can use visibility range as described in that video to cull off far away objects and/or swap them with sprites.

The visibility range is supposed to be an option that users will be able to change to their own needs in the map editor, so I can't use that as an optimization. But then also, there's no telling how many brushes there might be in close proximity. In an editor like this, you can create brushes of any sizes in any place you want. You can easily have more than 1000 brushes within the visibility range. The picture above does have more than that (the tower alone is nearly 500).

xyz I still don't fully understand what a brush really is, but from the description @kuligs2 linked there is a distinction between editor map and runtime (compiled) map. It appears that brushes end up in (or constitute) a BPS tree at runtime, maybe even it the editor. This could significantly improve the performance via tree culling. So if you want comparable optimizations, you'll have to implement a BSP tree.

BSP trees are a map compiler thing that is unrelated to the editors. It's an algorithm in map compilers for quake/doom derived engines (including Source), that creates a tree representation of the map, so it can cull parts of it in game. It's basically a different way of doing occlusion culling in-game, and has nothing to do with the level editor. The editor doesn't care about any of that at all. The map files you save from the editors contain all the information about every single individual brush.

Toxe

woopdeedoo There is too much "brush" in that post. 😆

When you export that level, do these individual objects get baked together into one (or a couple of) big meshes, so that the engine can render them efficiently? Or would it be one draw call for every single little mesh? Can't really imagine that though, because that would be a disaster.

xyz

woopdeedoo How do you know the editor is not using BSP culling when rendering the scene?

How do you create a "brush" in Godot?

If draw calls are your bottleneck you can partition the space into a grid and merge everything that is in a cell into a single mesh. If something in the cell is selected, dissolve it back into individual instances for editing. When deselected, assemble back.

DarkRadiant probably does some kind of merging. You can try to capture one of its frames with something like RenderDoc and check how is vertex data grouped into draw calls.

Toxe

So, generally speaking, this is my understanding of what "brush" usually means and its origins.

To no one's surprise the term comes from "paint brush" and goes back to (very) old paint programs, like Deluxe Paint on the Amiga. I actually used that one myself back in the day. In DPaint you could copy small regions of the picture as a "brush" (like a sprite) and then "paint" with it, either by single clicking or holding the mouse button. Then you would paint something that has the look and shape of your brush. Shocking, I know.

This BTW is also why Guybrush Threepwood from Monkey Island is called Guybrush. Because his sprite was saved to a file called guy.brush in DPaint and after a while the name stuck.

In modern 3D tools a brush is usually a predefined piece of raw geometry that you could drop into a level to create 3D objects directly inside the engine editor. At least that's how it works in Unreal. You have a couple of presets like stairs, spheres or simply a cube and then you can scale and rotate it, modify its vertices or apply some boolean operation to cut holes into the mesh. And it's called a brush because you can copy the mesh and then paste it all around the map, similar to a paint program.

And while you could create simple objects this way and even texture them etc. it's usually used for blocking out areas and defining your level geometry. Add some colliders and you can already run around the level while the 3D artists fill those areas with proper models, piece by piece.

I never cared much about 3D in Godot so I have no idea if Godot has a similar concept of in-editor editable raw meshes for quickly adding some WIP geometry (don't think so, though) but, yeah, this is IMO usually what people mean when they talk about brushes: Single, independent ad hoc meshes, that won't make it into the final product.

woopdeedoo

Toxe
Yes, I don't know the specifics of what the Doom3 engine does, but presumably it merges the geometry wherever it can. It probably doesn't make the whole map into one big mesh, because it has to be able to cull parts of it during gameplay, but it probably merges whatever it can. It also discards any faces that are never visible to the player. The final playable map is highly optimized.

When we use TrenchBroom or NetRadiant Custom with Godot, the FuncGodot plugin does the same thing, but in this case it's advisable for the mappers to manually separate their maps into chunks of geometry to improve occlusion culling in-game. It's not an automatic process when working with Godot. If you keep all the geometry in one single entity, the plugin will make the whole map a single mesh.

Toxe

woopdeedoo Alrighty, now I understand what you are trying to do here and what the issue is. As I said, I don't know much about 3D in Godot but did you take a look at CSGs? Not saying that this solves all your problems but maybe the links on the doc page can give you further ideas.

https://docs.godotengine.org/en/stable/tutorials/3d/csg_tools.html

Toxe

xyz If draw calls are your bottleneck you can partition the space into a grid and merge everything that is in a cell into a single mesh. If something in the cell is selected, dissolve it back into individual instances for editing. When deselected, assemble back.

Something like that would be my suggestion as well, but you phrased it a lot better than I would have.

woopdeedoo

I may be onto something. I tried building one mesh per material for the entire brushwork, and I think this might go a long way. It reduces it to 1 draw call per material in total. I ran some tests, and it takes a few seconds to rebuild the mesh for 8000 brushes, but that's because I only have 1 material for now, so the whole map is just a single big mesh. The end result runs as smooth as butter, of course, since it's just one draw call.

In a real map there's always many materials (the one in the screenshot above has about 300), so editing one or a few brushes might trigger the rebuild of only a few meshes. If each of them runs on a thread, it may be fast enough. I lose frustum culling this way, but it probably won't matter.

Gonna go with this for now and see how it goes.

xyz If draw calls are your bottleneck you can partition the space into a grid and merge everything that is in a cell into a single mesh. If something in the cell is selected, dissolve it back into individual instances for editing. When deselected, assemble back.

This is also something I'm keeping in mind for sure.

xyz How do you know the editor is not using BSP culling when rendering the scene?

Because the tree representation requires compiling the map, and it would slow down the editor, as it would have to rebuild the visibility tree every time you made changes to the brushwork. The BSP algorithm also requires the map to be completely sealed off from the void (https://developer.valvesoftware.com/wiki/Leak), which is something that you don't always have in the editor (because you're building the map).

Moreover, the BSP algorithm also has to cut many brushes in half or however it makes sense for culling. It's a destructive process. The game needs it because it has to render a whole lot more than the editor (lights, shadows, higher res textures with normal maps and all that, animations, etc) on top of having to calculate physics and such.

xyz

woopdeedoo I'd definitely look into the rendered frame structure of one of those Quake editors via RenderDoc. By looking at vertex buffer content at each draw call, you can get a good clue on how exactly they are merging vertex data (per material, per proximity...).

For performant realtime merging you'd probably need to go to native code via gdextension, but you can still first make a prototype in script. If the overall vertex count is relatively low and some sort of partitioning is used, the script version may even end up being sufficiently fast.

woopdeedoo

xyz yea, I'm gonna have to test it out and see. Chunking will probably be required. I like to avoid GDExtension as much as possible, as it slows down development, so I'll probably leave it for a later stage.

Editing brushes is the part I'm trying to figure out now. If one brush is selected, it's easy to just isolate its mesh and make changes to it individually, and update the main mesh(es) only once to exclude that one in the meanwhile. In fact, as it is now, each brush is already keeping an individual Mesh in memory, because I'm using that to easily generate the convex collision meshes for selecting brushes. So I could jsut use that.

(The mesh is just for visual representation, btw. The actual (simplified) geometry is being stored in arrays of Face, Edge and Vertex objects -- the mesh gets updated from those.)

But if too many brushes are selected, changing many such individual meshes as you move gizmos around, may become slow (unless maybe they can be threaded). If I merge the meshes of the edited brushes (maybe also with chunking), it may also become slow, as it will need to be constantly merging the brushes.