98 Lexus ES300 RPM Problem

I have 98 Lexus ES300 with 77,000 miles. Here is my problem. The car starts and cranks fine and runs fine usually. But sometime right after I start the car and it is still in park the RPM gauge drops to the last line before 0, but the engine is still running and you can feel a very slight vibration or rough noise, very hard to feel or hear Must turn off A/c and radio. When you put it into gear say D the car doesn't roll forward until you press the gas. Usually with automatics the car rolls forward a bit even with out pressing the gas. If I step on the throttle a few times or just ingonore it and drive say for 30 seconds like leaving a parking lot. Then the idle goes back to normal around 1 or the line right below 1 and the car is fine. It so far has only happend right after I drive and try to restart the engine, like quick stops for

5 min breaks. It has never happend yet when the car has sat for a while right before starting it like in the morning.

Does anyone know what is causing this problem? Is it expensive to fix? Is the engion oil sludge?

Thanks