Brad,
I've removed dozens, rebuilt hundreds, and installed dozens of these motors. Can you tell me what year model your car is? The early motors use three wires. red, yellow, and black. The black is the return path for all motor current.
12vdc goes to red to raise the top and 12vdc goes to yellow to lower the top. The black ground wire must have a clean tight connection or performance will suffer. The relays supply the 12vdc, so if they are bad, performance will suffer. You say it happens in both directions, but I doubt both relays would go bad at once. If your pump assembly has only the red and yellow wire it is a permanent magnet type motor. One wire will have 12vdc and the other will be the return. They then switch via the relays to move the top in the opposite direction. Your symptoms lead to three possible faults depending on the type motor you have. The switch, the ground wire or the motor brushes are worn out. Feel free to email me if you have questions about troubleshooting methods.