Well, YEAH!
There is a club in town, the Inland Jeep Freeks, that makes at least one trip out each month. Some trips are day trips, and some are over-nighters.
I am near Cal Oaks and Clinton Keith. The hills above Clinton Keith have some cool trails -- take Nutmeg and go up the hill.
The Cleveland National Forest is close by, you can get there through La Cresta.
Los Coyotes Indian Reservation is the really good spot to go wheeling, but it is about an hour south of Temecula.