There have been a couple of waves of enthusiasm for turbocharged gasoline engines for more or less the reasons you mention. The one in the late 70s and early 80s was a bit ahead of its time in the engine-controls department, and also it took awhile for manufacturers to consumer-proof the way they cooled the bearings.
In the late 80s, turbos came back, but went away again, mostly supplanted this time with bigger as well as better naturally aspirated engines. Note also that instead of a third wave of the energy crisis, oil was getting cheaper and cheaper, in inflation-corrected terms, in the US market, so the regulatory and economic forces that had been pushing increased fuel economy lost a lot of momentum.
In both instances, the turbos seemed mainly oriented toward a performance variant of any given car rather than toward getting away with a smaller engine for economy.
Toyota made a version of their 4WD pickup that had a turbocharged Four. It was not a huge success in the marketplace. Frankly, neither "turbo lag" nor the need to maintain high revs are what most people are after off-road, or even on-highway in a truck.
Which brings us to one of the other troubles with getting huge power out of a small engine: real-world driveability. Under most circumstances it's more pleasant to drive a lower-powered vehicle with broader torque and power curves than one that makes monster numbers but only with constant attention to gear and rpm to stay in a narrow sweet-spot.
This applies mostly to gassers, of course. For reasons of both thermodynamics and physical robustness, turbos go with diesels like basil goes with tomatoes. They have a good bit to do with diesels' success in both super-duty pickup trucks and economy cars in recent years.
In either case, you also get plumbing/packaging complications, which can range from trivial to damnable depending on the engine and vehicle.
Cheers,
--Joe