Of course the wheel is off...I can move the drum outward just a little bit, Maybe the brake shoes are catching it and kind of holding it in place?