Ok, you might try start with one or a combination of very few standard collision shapes (remember to not scale the shapes but edit their extents instead) instead of the trimesh and see if it makes any difference. Did you also set the Rigidbodies weight correctly?
By the way: The form of the visible mesh itself is unimportant for physics and collisions.
