Originally, there was a big bypass hose, something like 3/4", which connected the thermostat housing to the water pump. With the no longer available stat installed, that hose got blocked off as the stat opened, routing the bypass coolant to the radiator. Your current hose routing allows flow through the heater core all the time, engine hot or cold. Without the proper t-stat, one of the ports at the water pump, and the one in the lower stat housing should be plugged.
Jerry