You shouldn't have to pound very hard to install the pilot bushing, but the freezer suggestion is a good idea. There is a bushing driver for installing the bushing, but nearly any bushing driver of the proper outside diameter will work better than a hammer and punch.
Another possibility for your trouble is misalignment between the transmission case and the bell housing. Before you install the engine, make sure the clutch releases properly, then attach the transmission and see if the clutch releases with the transmission installed. If it doesn't, there is a misalignment with the case. There is a procedure in the shop manuals to align the case with shims, but you have to disassemble the transmission and use a dial indicator to take measuements. I have the procedure, I think, and can get it for you if it comes to that.
Harvester