I have permanently removed the power steering pump on my '92 SC during its transformation into a track-only car.
I would STRONGLY recommend against doing this on a street-driven vehicle. Some Saturns came with manual steering from the factory and had a different steering rack (plain SL and SW models, I believe). But if you bypass the pulley on a car that came with a power steering rack, your stationary pump then becomes a restriction. As you turn the steering wheel, I'd imagine the rack is trying to push the fluid which will not want to easily go through a pump that is not running. When I removed my pump, I put a resorvoir tank in its place which allows the fluid to flow freely when turning the wheel.
If using the steering rack in this way that was meant to be assisted by the pump, parallel parking becomes extremely difficult. Unless the car is rolling at least 10 mph, it takes a huge amount of effort to turn the wheel. How much? Turn your key on but leave the car off. While your car is standing still, attempt to turn the wheel lock-to-lock. That's what it will feel like all of the time below 10 mph or whenever you try to do any low-speed parking.
In addition, when I removed my pump I had to swap the idler pulley from a smooth model to one that had grooves as the belt was now routed differently and the opposite side now traveled over the idler. But I removed my AC compressor at the same time. I'm not sure if any Saturns over the years came without a power steering pump but with air conditioning. If they did, you'd need to find a belt routing diagram from one, get that size belt, and determine if your current idler pulley will do the job.
You can find a replacement pump on eBay very cheaply, probably less than $30 or so. You would save a lot of trouble by just replacing the pump. Good luck.
Lane [ lane (at) evilplastic.com ]