Any explanation on why they need to be closed? Probably easier to find a later 1956-62 235 unvented valve cover and swap it out, using a vented filler cap as you mentioned. Don't get rid of the '54 valve cover. If you don't intend on keeping it, put an ad in the
Truck Parts for Sale forum as I'm sure someone else will want it.

Side note - your signature line says your truck is a 1954 GMC. Guessing the original engine has been transplanted by the 235???