Thanks everyone! It didn't seem right that it came right off. I will have to have it drilled and tapped.

The key that was mentioned above is starting to wear. I don't even know that this is the correct pulley for the truck, but if I actually fasten it to the engine, it will probably work.