Most likely the fan motor is on the way out. Very common. To test this...
put fan on high when it is not working... and then hit the passenger side
lower dash hard... if fan comes on, then it is definitely the motor.
You should also change the fan fuse regardless of condition.
If above two doesn't work, then turn the ignition key back a bit more
without shutting off the engine, if this work, then you need to change the
Above three are the most common solution to this problem.