i would braze the riveted center collar onto the fan before making the hole larger, there might not be enough wall remaining to hold it tight in the long run if you successfully expand that hole
also, use a reamer not a drill bit if you can/have one.
also have you considered just turning down the motor shaft?
clamp the motor in a vise, power it on, and use a file on spinning the shaft to make it smaller until it fits the collar hole,
i've turned down many small objects that way with a power drill
it works like a charm.
-s
