It's not a shim, it's a thrust washer (tanged), and there's only one.
If the shaft, as you say, is able to raise a whole quarter of an inch out of
the camshaft gear, then you have some excessive wear either at the cam gear
or the distributor gear, or both. Typically, there is a small amount of
run-out, maybe a sixteenth of an inch to the eye.
If your shaft is able to raise a quarter of an inch the rotor could come too
close to the electrodes on the cap. This could cause a variance of the air
gap between the rotor electrode and the cap electrodes. It can also cause
excessive wear on rotor contact, and will wreak havoc with the points
follower and alignment in relationship to the distributor shaft cam.
Yes this will alter your dwell as well as your timing. Sound like the
distributor is worn out.