After you let the clutch out that first time, goose the engine and then pop the clutch in. It should shift in a little quicker (and quieter).
The way the old pros double-clutch is: they jump on the gas when they're letting the clutch out that first time (like when you shift up) and then they shove it back in again and slide the shifter into gear.
I haven't managed to do it quite as well as them, but I can get it to downshift all the same.
