OK, that might work- - - -use wet/dry sandpaper, maybe 600 or 1000 grit, and keep it wet with lightweight oil or maybe kerosene, charcoal lighter fluid, etc. Sneak up on the clearance a little at a time, and measure frequently by flushing the piston with brake cleaner and try a slip fit with a lightly lubricated pin. You can always make the hole a little bigger, but it you oversize it there's no way to shrink it! Instead of using a drill motor in a vise, how about using a drill press?
Jerry