The fact that you are going about the same speed for the same amount of discomfor with the larger tires makes me feel fairly confident that the issue is road and/or fan noise. Soundproofing it relatively inexpensive and will be beneficial whether it is a final solution or not.
Sounds like you are off and running on this one.
Good luck and please come back and show us your solution amd how well it worked (or didn't work).
