John of Champlin,
Yes, the things the pushrods rest in are called valve lifters. They need to be removed or at the very least pulled up so the cam lobes can slither by as you pull the camshaft out. The fuel pump actuation arm is also resting on a cam lobe and it (the fuel pump) needs to be taken completely off the engine. The cam will not just pull straight out, it must be turned, supported and gently coaxed as it comes through its bearings.
Stuart