It is not recommended to use a neoprene seal on a shimmed engine.
The shims will hold the ends apart and leave a gap near the crank.
If you trim the shim around the seal the oil will most likely go through the gap between the seal and shim. If some sealer is applied in that gap it might work.
Your bearing is not looking very good either.
