Sounds like a vacuum leak. In the majority of cases, holding your hand over a carbs intake and the speed picks up suggests it's sucking air between the carb and the intake valve(s). Adjusting the fuel air generally doesn't eliminate it because theres generally not enough fuel/air mixture available to offset a big volume of air entering via vacuum leak "after" the idle fuel/air mixture port.
The carb could be have a vacuum leak itself, the carb to manifold junction, the manifold and/or manifold to head junction.
Theres several ways to test/discount this, but it definitely sounds like a vacuum leak.
Good luck
Dave