There has to be some type of torque tube ball housing support for 1953 and earlier manual transmission trucks with a torque tube (edited). The torque tube ball housing needs to fixed in place with the truck frame. Inside the ball housing, the inner ball swivels with the motion of the rear axle and the splined shaft coupling moves in-out direction also with the motion of the rear axle.
I think you said it all with that statement. The torque tube moves allot and creates allot of additional forces, all at the tail of the transmission. Without that support you would multiply those forces by the length of the tranny to the rear motor mounts. You only find the rear transmission crossmember on torque tubes so I think you hit the nail on the head!