You could have bent the hub. It's easy enough to check with a dial indicator. Remove the wheel and caliper and use the lugnuts to hold the rotor to the hub, Indicate the rotor. Then remove the rotor and reinstall
180 degrees from where it was and reindicate. From this you should be able to determine if the hub is straight or not. If the hub isn't straight, you'll never get the car smooth at speed but can still be smooth braking.