• Godot Help3D
  • How exactly does `Skin` work for a skeletal mesh?

Can someone knowledgeable explain in simple words here what information exactly does a Skin store?
I am trying to figure it out without much success.

Here's what I know:

  1. The ArrayMesh contains data inside arrays[Mesh.ARRAY_BONES] that is supposed to be the indexes of the bones that influence the mesh vertex, for each vertex (it stores 4 consecutive indexes per vertex, or 8, depending on the mesh flag value).
  2. Somehow, a Skin remaps these indexes to different indexes? It's not clear from where to where this mapping goes.
  3. When I import a blend / gltf file with an armature, the imported mesh has a skin. Removing that skin from the mesh breaks the deformation (it appears to ignore the skeleton). When I look at the data inside arrays[Mesh.ARRAY_BONES] I see nonsensical bone indexes, the only explanation for which that I can concoct is: these are not bone indexes, but skin bind indexes that are somehow remapped to bone indexes?

I would really like to understand the mathematics behind how mesh vertex positions are deformed by the bones in the presence of the Skin object.

To make the question more concrete: I am trying to implement a class similar to BoneAttachment, except it should follow the exact position of a vertex of the mesh.
Here's my attempt to replicate the transformation of a single vertex:

@tool
class_name MeshVertexAttachment
extends Node3D

@export var surface_index: int = 0
@export var vertex_index: int = 0

func _process(_delta):
	transform = compute_transform()

func get_mesh_instance() -> MeshInstance3D:
	var mesh_instance := get_parent_node_3d()
	if not mesh_instance: return null
	assert(
		mesh_instance is MeshInstance3D,
		"MeshVertexAttachmentmust be a child of MeshInstance3D."
	)
	return mesh_instance as MeshInstance3D

func compute_transform() -> Transform3D:
	var mesh_inst := get_mesh_instance()
	var arrays := mesh_inst.mesh.surface_get_arrays(surface_index)
	var skeleton = mesh_inst.get_node(mesh_inst.skeleton)
	var pos: Vector3 = arrays[Mesh.ARRAY_VERTEX][vertex_index]
	var norm: Vector3 = arrays[Mesh.ARRAY_NORMAL][vertex_index].normalized()
	if skeleton:
		assert(skeleton is Skeleton3D)
		var pos_modified := Vector3.ZERO
		var norm_rotated := Vector3.ZERO
		var skel := skeleton as Skeleton3D
		var bone_group_count = 4  # TODO: conditionally set this to 8 depending on a flag value
		for i in bone_group_count:
			var bone_ind = int(arrays[Mesh.ARRAY_BONES][bone_group_count * vertex_index + i])
			if bone_ind < 0: continue
			var bone_weight = float(arrays[Mesh.ARRAY_WEIGHTS][bone_group_count * vertex_index + i])
			var rest_pose := skel.get_bone_global_rest(bone_ind)
			var bone_trans: Transform3D = (
				skel.get_bone_global_pose(bone_ind)
				* rest_pose.inverse()
			)
			pos_modified += bone_weight * (bone_trans * pos)
			norm_rotated += bone_weight * (bone_trans * (pos + norm) - bone_trans * pos)
		if pos_modified != Vector3.ZERO: pos = pos_modified
		if norm_rotated != Vector3.ZERO: norm = norm_rotated
	var up := mesh_inst.to_local(Vector3.UP).normalized()
	# Orthonormalize.
	up -= norm * norm.dot(up)
	up = up.normalized()
	var right := up.cross(norm)
	return Transform3D(Basis(right, up, norm), pos)

It seems like I got the math right, except in the presence of a Skin object it doesn't work at all. Looking at the contents of the array[Mesh.ARRAY_BONES] array, it has nonsensical bone indexes. I suspect those are somehow remapped by the Skin, but with the lack of documentation, I can't figure out the details of that mapping.

  • xyz replied to this.

    @xyz

    Attaching the node to a random vertex on the character's foot, and printing the bones (in the bone_index bone_weight bone_name format):

    1 0.75571829080582 mixamorig2_Spine
    2 0.2175936549902 mixamorig2_Spine1
    0 0.02667277120054 mixamorig2_Hips
    0 0 mixamorig2_Hips

    There are 3 non-zero-weight bones which are somehow hips, spine and spine1, which makes no sense at all to me.
    The vertex that the attachment is attached to in the rest pose moves correctly with the bones, while the attachment itself does not.

    Attaching it to a vertex somewhere on the character's hand actually works exactly as expected. Here's an example output:

    8 0.95664912462234 mixamorig2_LeftArm
    7 0.03854428976774 mixamorig2_LeftShoulder
    25 0.00479133287445 mixamorig2_LeftHandRing3
    0 0 mixamorig2_Hips

    This looks correct.

    I suppose the other explanation would be that the character mesh is botched somehow...
    But it transforms correctly with bone rotations, all parts of it.

    • xyz replied to this.

      silly_billy What values are returned by Skin::get_bind_bone() if you pass it indexes from the first printout?

        xyz It returns -1 for each of those indexes 0, 1, 2.
        Strangely, there are 14 bindings I see in the inspector in the Skin object.
        That's part of my question: I also expected it to return something meaningful. I don't understand the Skin object.

        • xyz replied to this.

          xyz sure!

          Open character.tscn, click on the skeleton and try rotating some of the bones.
          The red sphere (hand attachment) works as expected.
          The blue sphere (foot attachment) doesn't...

          meshvertexattachmentdemo.zip
          8MB