I think this is because of how the HBoxContainer and VBoxContainer work. From what I remember, both try to size and arrange the objects so they fit into rows going horizontal/vertical, with the largest Min Size property of all of the children being the size used. Once you have gotten the size you like in the Godot editor, I would try setting the Min Size property and see if that fixes the issue.
Also, welcome to the forums!