There's probably a lifter hanging down in the way. The lifter body just above the cam lobe never goes completely into the lifter bore; in time, this area collects a varnish coating so the lifter doesn't want to move up farther than it ran. Use a wood dowel rod to knock the lifters up enough to get the cam out, then drive the lifters down and out.
