You are looking for a captive nut, like this maybe

https://www.mcmaster.com/products/captive-nuts/aligning-weld-nuts-7/

Chuck


Hank: 46 Chev 1/2ton shortbed
2023 Miata RF Club