I've found that getting that stuff off of where you don't want it is a booger. On raw concrete I'll bet muriatic acid is your only hope - etch off the top layer. Have you considered just epoxy painting the whole floor? With a nice POR base, it won't rust wink .