A few "hard data" questions in the survey doesn't turn the overall survey into a hard data gathering exercise. The main questions in the survey are clearly eliciting an opinion. Questions like - "If you had any problems with your car in the past 12 months that you considered SERIOUS because of cost, failure, safety or downtime, select the appropriate box(es)...." or "How satisfied are you with this vehicle with respect to each of the following factors?" The answers to these questions are opinions, not facts. The chief information gathered is all based on the opinions of the reader. This makes it "clearly an opinion survey." When the pollsters come around and ask me how GWB is doing, they also ask me my age and race. Just because they ask me some hard data questions, does that make my answers to the other questions facts?
You mean the data gathered where they asked the respondents to decide what is "SERIOUS?"
No, I am saying that CR readers tend to spit back the opinions of the magazine. They have been told repeatedly that car A is very reliable, so when they are asked to report SERIOUS problems with car A, they tend to think problems with such a reliable car had can't be serious. I am not saying it is a massive case of deliberate misreporting. Because of the limited nature of the survey, very small shifts can appears very significant when the data is reduced to the little circles. In my opinion, the data gathering method is unscientific and the reporting methods can make it appear that there are great differences when in fact, there may be no statistically valid differences at all between component systems of two different cars. This means that very subtle biases can skew the results and make it appear there are significant differences when in fact they don't actually exist.
It is not just that the samples are too small, they are both too small and from a non-random group.
For years Buicks have shown up better in reliability ratings (like JD Powers) than other GM products (at times even better than Cadillac). Why? In most cases the basic parts of the car are exactly the same as Pontiacs and Oldsmobiles and they are built on the same assembly lines by the same workers. There is no reason to believe that Buicks are more reliable than Oldsmobiles, but if you look at JD Powers surveys, Buicks always come out much better. This also used to be the case with CR surveys, but in recent years it appears that CR started blending the data for similar cars sold by different divisions to avoid having to explain this phenomenon.
You have to be kidding. When a new Toyota comes out the CR editors will say they expect it to be reliable. When a new Buick comes out they will say it is a new model and the reliability is unknown.
The reliability reports are based on the respondents deciding what is "SERIOUS." This is an opinion based question. Cars from all manufacturers have very few problems of any type (according to JD Powers the average new car has 1.24 problems, and the average 3 year old car has 2.27 problems). It only takes a subtle shift in determining what is "SERIOUS" to make large differences in the survey. For all I know, Toyota owners might be overly picky and over report problems compared to Buick owners. The CR reporting methods (the little circles) can make tiny differences appear to be very significant.
The reviews are full of opinions. For instance:
"This large, front-wheel-drive four-door sedan replaced the LeSabre and the Park Avenue. The standard power plant is a rough-sounding 3.8-liter V6. A potent 4.6-liter, 275-hp V8 powers the CXS. The four-speed automatic transmission shifts smoothly enough. The Lucerne has a quiet, comfortable ride, especially at low speeds. Handling is not agile and the steering lacks feedback. It has a tendency to fishtail easily at its limits. Stability control is only available on the CXS V8. Braking is unimpressive. The back seat is roomier than the one in the LaCrosse. First-year reliability has been above average."
Opinions: "rough-sounding 3.8-liter V6" "the four-speed automatic transmission shifts smoothly enough" "has a quiet, comfortable ride" "Braking is unimpressive"
"The Mercury Milan and similar Ford Fusion are new midsized sedans that are very agile and satisfying to drive. These sedans rate even higher in our testing than the Mazda6 sedan on which they're based. They have a sporty feel; the V6 and automatic transmission make a smooth and responsive powertrain. The four-cylinder is a bit coarse. The interior is well made and space is generous."
Opinions: "are new midsized sedans that are very agile and satisfying to drive" "They have a sporty feel" "The four-cylinder is a bit coarse" "The interior is well made and space is generous"
"The redesigned Toyota Camry is roomy, quiet, has a comfortable ride, and is refined. The addition of a telescoping steering column is a plus. Power comes from a strong 3.5-liter V6 mated to a six-speed automatic transmission. It returns 23 mpg overall, just one mpg less than the four-cylinder. The base 2.4-liter four-cylinder is also responsive and relatively refined. A four-cylinder hybrid version returned an impressive 34 mpg overall. Handling is responsive and secure but not sporty. The interior is spacious, with reclining rear seats in the high-end XLE. Curtain air bags are standard, but stability control remains optional."
Opinions: "is roomy, quiet, has a comfortable ride, and is refined" "......a strong 3.5-liter V6" "The base 2.4-liter four-cylinder is also responsive and relatively refined" "Handling is responsive and secure but not sporty" "The interior is spacious"
etc., etc., etc.
The reviews are full of opinions.
You don't think the whole continual retesting of the Suzuki Samurai wasn't an attempt to skew the ratings? Every newspaper / magazine / TV news program / etc that I am aware of skews their content to please / attract the readers / viewers. You are naive if you don't think CR does as well.
People keep buying Motor Trend, and I am sure most of the people who buy it understand that "Car / Truck / SUV of the Year" is up for sale to the highest bidder. People keep watching NBC's Dateline, yet most people know they faked the whole Chevrolet exploding gas tank report. 60 Minutes has told so many lies it is hard to imagine they have any credibility, yet it remains a very popular program. I think most people know WWE is scripted, but it still remains very popular. I think CR is biased towards certain makes, but I still enjoy reading it. Heck I often even agree with their opinions. I thought the Samurai was a death trap, but I also thought CR when over the line in trying to prove it.