There were a number of miner differences but the main difference is the mounting. Your 54 is mounted in the front center. The 61 engine will not be drilled for the front mount but it is easy to drill for the mount.
There is also a problem with the throttle linkage pivot.
1960-62 engines dropped the threaded boss where the 54 pivot mounted.


See the USA in your vintage Chevrolet!
My Blog