yes, there's always some minor gaps where outside air can infiltrate, but most likely your smell problem is the outlet position of your exhaust - I'll guess either the pipes don't go all the way back or they exit barely beyond the back of the box in the center third
Bill