It's possible the lifter is worn/mushroomed enough on the bottom to make it difficult to remove. If so, both cam and lifters should be replaced. A good lifter will have a slight convex shape on the bottom where it rides the cam, taller in the middle than the edges. This, plus the lifter riding slightly off-center on the cam lobe causes it to spin as it lifts, equalizing the wear pattern. Flat-bottomed or cupped-out lifters are worn out. Check the bottoms with a straightedge like a machinist's 6-inch rule.
Jerry