There should not be any power to the directional switch with the key turned off. Find the power wire ( usually red, but could be any color ) with a test light or meter. Trace it from that switch and see where it goes. It must be hooked to something that is powered all the time. Change it so that it is only powered when the key is in the on position..
Some directional switches have the brake light switch wire running through them. If that is the case, are your brake lights sticking on also ??
A few minutes spent investigating with a test light will solve the trouble..