When I put the float in. What way should the rear tab be facing? Up or down?
In most carbs the tab pushes on the needle valve, so up. Just remember though, up is down when the carb top is upside down on the work bench!

Also you might want to let everyone know what type of carb it is, it will help with the diagnosis. It does sound like the needle valve is not closing though. Is there pooled gasoline on the floor of the intake?