Consumer Reports 2007 Auto Survey

I'm not ignoring that at all. It isn't relavent to the fact that this isn't simply an opinion poll.

It is statistically significant because the results are consistant enough to prove it. You failed to show any "clear bias" on the part of the poll-takers.

Brent didn't say that. He said "It [sic] clearly an opinion survey." Even if he had said that, the results show the statement to be untrue.

I didn't say much of anything in this thread and did not even mention the freaking survey, both of you are confused.

What I did post on was the religious following of CR where the faithful just ignore the obvious flaws in method and buy into the nonsense that people at CR have some god-like lack of bias.

CR at best is like a testing department run amuck that doesn't have to correlate its tests to actual use. At worst it's about as good as usenet in gathering reliablity information.

I never said it was a hard data gathering exercise, just that it's not simply an opinion poll.

How do you determine which questions are the "main" ones?

I said so in my post.

The only thing that makes it "chief information" is your opinion, which is clearly biased. I mentioned that I was referring specifically to the reliability data--which means the only relavent questions in the poll are factual ones.

There's a problem with your analogy. If the poll results included the average age and racial data of the persons taking the poll, and some people were only interested in the average age and race of people who were polled rather than what they thought of GWB, then absolutely those are hard data questions pertaining to the information at which those folks are looking.

Yes. I trust the law of averages because as I mentioned before, the results have enough internal consistency to be believed. When two different makes and models of a car get rated, the results are usually pretty similar if not always exactly the same. If they were at different ends of the spectrum, that would be another thing altogether. I expect some variation because of the small sample sizes, however if you look at general trends you get a picture based on more data.

Your basic argument is that the owners of certain companies' cars will use a significantly more restrictive criteria for determining whether or not a repair problem is "serious." You haven't provided any evidence or reasoning that support your claim, IMO.

Does CR express the opinion that certain cars are very reliable? From what I've seen, they don't. They might mention that a car is expected to be reliable based on the poll results, maybe that's what you mean. But then we get to a case of the chicken or the egg. Did CR first tell its readers that certain cars are more reliable and then start sending out polls? Or did they send out the polls and then mention the results? I don't have any issues from before they started collecting repair data from readers, so I don't know.

That's your opinion, but it doesn't make sense to me. It would make more sense to me that if people shell out money for a car that they are EXPECTING to be ultra-reliable and then end up having to repair it, they'll report even the most *minor* incidents as serious.

I would agree with you there and to some extent what you are saying based on that point. Expecting people being polled to use their own judgement on "serious" repairs is a bad way to collect data, because when you ask judgement-based questions, the smaller samples are bound to be less accurate. However I disagree with your other conclusion, which is that the results would be skewed in favor of certain brands and/or against others. I think they would just have a wider range of accuracy (meaning less reliable.)

Yep, I agree again--I want to see:

1. A much larger survey response.
2. No relying on opinion or judgement questions! They should be asking specific questions like "What was repaired" and "How much did the repair cost."
3. The results should reflect the actual numbers. For example, what percentage of owners of 1997 Corollas with 100,000 to 130,000 miles on the vehicle had to repair their vehicle's transmission last year, and on average how much was spent per repair job?

I'm saying the polling and reporting procedures are flawed, not that they don't provide a reasonably reliable (if somewhat vague) indication of a car's dependability.

Can you give me an example of a Buick and its twin Olds vehicle that this happened with? No offense, but I would like to look it up myself--I am pretty sure I can find past issues of CR frombefore they started combining the data for similar models.

By the way, are you sure they did start blending differently-branded instances of the same basic vehicle? I notice that the Vibe is separate from the Matrix in the reliability records, and from what I understand they are basically the same vehicle.

I haven't seen that, but then I don't usually pay attention to statements like that, so they very well could. However, I believe they used to say that about Mercedes-Benz autos as well, because they usually topped the CR reliability records... yet now, you find Mercedes cars at the bottom of the reliability charts, while Toyotas have sonsistently remained at the top. Unless CR started telling Mercedes owners that their cars were very unreliable, this would indicate that people don't tend to minimize their repair problems based on CR reliability reports, otherwise Mercedes owners would still be considering most or all their repairs to be "not serious" and not reporting them.

Good point about the goofy circles. However, they also list percentages--at least on their web site.

Yep. I never said they weren't. However, they aren't "opinion pieces" because they also contain factual data, like "the rear seats fold all the way down to make a completely flat cargo area" etc. An opinion piece would not have any use for statements like that. They are reviews, which contain a mix of factual data and descriptions of their experience, which have to be opinions. For example, you can measure how loud engine noise is, but you can't measure how an engine *sounds* or how a shifter *feels* in any way that would make sense to people. You have to use your best judgement, which is of course opinion. As I've said, I have found that their descriptions of such things tends to agree with what I have found. If they say one car has an engine sound that's "rough" and don't say that about another car, it's obviously a judgement but again I have in the past agreed with most of their judgements so I

That's yet another opinion. I'd say you're naive if you think they do, because CR isn't any newspaper/magazing/TV program etc--their existence is based on the trust that they are truthful and objective. If they break that trust, no one has any reason to buy their magazine, subscribe to their web site, etc--and this isn't the case with most newspapers, magazines, TV programs, etc--it's the opposite for them in fact. People pick which news station, newspaper, fashion magazine, etc based on it telling what they want to hear--but with CR it's different. The purpose of CR is to give the reader information which will help them spend their money on purchases they won't later regret making. And that's pretty much all it's for.

Some do, some don't, most don't care--they just want to see their own car praised so they can feel smart for buying it. If their car isn't one of the favored ones, then they don't get Motor Trend. I suppose some people might base their buying decisions on what MT says, but I feel sorry for them.

Right, because it's entertainment. It's supposed to be true, but even if it isn't, it's still interesting. CR is about the most boring, driest read you can buy. I have trouble forcing myself to even read the reviews of things I'm interested in. Without impeccable credibility, I'd pay quite a bit if I had to just to *avoid* reading CR.

You actually enjoy reading CR. I am astounded, flabbergasted, amazed and befuddled. I guess we all have our own tastes. I won't even consider that there enough people who enjoy reading CR that they could make it without credibility. I mean good grief, it would be like reading weather reports for places that don't exist.

I look forward to your response.

Except those vehicles built on the same assembly line sometimes ARE different. GM's Prizm, for example, didn't include suspension components that the Corolla did have. Same car but different. Does the Vibe come with AC-Delco parts or Nippon Denso parts?

And, once you've purchased it, your GM car is serviced by a GM dealer and your Toyota is serviced by Toyota. That could have an impact on satisfaction and it might have an impact on frequency of service, too.

There's also no reason to believe your contention, that the outlook of CR does anything besides select whoever reads the magazine. If a CR subscriber buys a Ford, he does it for a reason and he's just as invested in having a positive outcome to the transaction as a Toyota buyer. In other words, he's as likely to overlook problems as the Toyota purchaser.

The percentage of CR readers who own domestic cars might be lower than the percentage of domestic owners across the US as a whole but as long as they get a decent sample size, they can still evaluate satisfaction.

And, for those that were whining about statistical sampling, determining sample size is just plain math. If sampling wasn't a valid way of assessing quality, pretty much every manufacturer's QC operation would turn out the lights and go home.

The Detroit Fan Club can piss and moan about CR and other surveys all they like but the fact of the matter is, Toyota and Honda are doing SOMETHING to make their customers believe in them and it's paying off in $$$.

The selection is very important, not just the sample size. The CR survey is asking a bunch of more or less like-minded people (afterall, those who don't like CR for whatever reason are not subscribing to it generally speaking) what they think if they want to share their thoughts. It is like sending out a questionaire to neo-cons asking how Bush Jr is doing as president. They can fill it out or just toss it in the trash. Odds are the results are going to be skewed.

I apologize for attributing to you a quote that you didn't make.

I'm sure there are people like that. Obviously they are wrong--everyone is biased. CR reviews are like any other. It doesn't matter whether or not the reviewer will like the product being reviewed. What matters is whether or not the person reading the review gets an accurate picture of how they'll feel about the product.

CR can rave about a product all they like, but if it doesn't excel at the things you want it to excel at and in the correct way, you won't be happy with it. I'm sure you know this, but I'm hoping that if any of the CR faithful are reading this it might switch on a light bulb so to speak.

I like CR because it gives me information which I find useful in helping me decide which product to buy. For example, I bought a certain camera not because they recommended it but because it had the features I wanted, such as low shutter lag (which they measured to the tenth of a second at .3 seconds, and as far as I can tell that's accurate.) It saved me the trouble of having to go to several different stores and asking them to unchain each of the demo models, put batteries in, supply it with a memory card if necessary, and let me take some photos. That would have taken hours at least, and I would have had to deal with salespeople, which I often find frustrating and sometimes infuriating. Some stores probably have a "you-cram-it-up-someone's-ass, you-bought-it" policy, and I don't want a camera that's been up anyone's ass, even if I'm the one who put it there.

Well, every time I've used their info, I've found it to be accurate. However, I've only used their info about probably 30 times or so, and they have done thousands and thousands of reviews--so maybe I've been lucky with the thirty I used.

I don't trust CR implicitly, though. If they start providing information that I find to be incorrect, I won't hesitate to let as many people know about it as I can. I will give *specific* examples that they can look up for themselves, though, rather than making vague statements about "the religious following of CR" because I don't want to look like a fanatic member of the anti-CR religion.

CR comes up with on the fly test criteria that fail products they think should fail and passes products they think should pass.

Considering CR's testing, there is no way of making such a determination.

There's this thing called the web now where one can find exacting specifications of devices like cameras. CR does list out anywhere near the detail that can often be found online.

I've been in product design a fair number of years of now. Lab tests are well, lab tests. Currently I am dealing with issues on a couple products that do not occur in the field, only in lab tests. No relationship between these lab tests and field failures has ever been established, it's just 'gut-feel' arbitary tests. And in the end that's what CR does as well.

You can have a product that internally a manufacturer put through all sorts of arbitary abusive testing and it passed only to have CR come up with something different and break it immediately. They are just arbitary tests, neither better than the other.

Did you not read the actual questions? The reliability data is partially based on the respondent deciding what is SERIOUS. This is the respondents opinon.

But if you are trying to decide what vehicle is reliable, you want reliable reliability data. CR is giving you the CR reader's opinion of the reliability of thier cars. These are not the same thing. One is based on hard facts (dollars spent, hours out of service, etc.), the other is based on whether people felt they were inconvenienced.

Garbage in, garbage out. Too small a sample size; non-random samples, etc. They depend on people remembering problems, and deciding they were serious. If you average the opinions of all the RNC members, you would conclude GWB is doing a great job.

I guess I missed your evidence that they don't. No doubt this is speculation, but I don't think you can make the opposite claim (i.e., that people are totally objective when filling out the survey - for one thing, they have no clear guide what should be considered serious). .....

I think the opposite is true in many cases. I think people that buy cars because they are said to be reliable tend to not want to admit they made a mistake if the car turns out to be unreliable. Humans have a hard time admitting mistakes (even me).

And I am saying that CR is reporting a very limited group's opinion of the reliability of a car, not the actual releiability.

Try around 1992. Or just go look at the JD Powers Survey results on line to see how results are slewed by expectations (JDP definitely does not average results across brands). According to the JDP 2005 Vehicle Dependability Study, the average 3 year old Buick has 1.63 problems, the average 3 year old Oldsmobile has 2.42. That is a retty signficant difference (over 30%).

CR rates the 2006 models exactly the same (suspicious?). JD Power gave them radically different ratings (Vibe got 3 balls for manufacturing quality, Matrix got 4.5; Vibe got 2 balls for design quality, Matrix got 3).

Mercedes is so far down the list CR would have no credibility if they continued to claim they were reliable. Besides, these days which audience is it more important for CR to please, Toyota buyers or Mercedes buyers?

Percetage of what? The only percentages I see are in the graphs where they rate cars comapred to the CR average "score." How is the score calculated? They don't actually tell (at least as far as I can see). They take a lot of pride in mentioning that they are basing the ratings on 1.3 million vehicles spanning three year models. But since they are rating over 300 models, over

3 model years, the ":average" model/year only has 1,300 data points - and that is the averge. Many of the models must have only a few hundred. CR could just publish the raw results and then we could decide for ourselves. They could put it online for a minimal cost. I wonder why they don't do it.

Even your example could be an example of bias. Choosing which facts to present involves making an editorial decision - which is another way of saying, expressing an opinion. CR decides what is important to report on. They love ESC. I think it is over hyped and not worth the cost. If I was writing the article it wouldn't be a factor. One thing that has always bugged me is the entry and exit from cars. I hate Crown Victorias because they are too low, which makes them hard for me (tall/large) to get into. My

2001 Mustang was easier to get in than my Mothers' Grand Marquis. I rarely see this sort of thing mentioned, and it is often a problem for me with Japanese cars. I have to practically fall into my Sister's Civic because it is so low to the ground. Selective reporting is as much a bias factor as stating a clearly identifiable opinion.

.........

Yes I expressed andopinion. Here is another - I think you are naive if you believe what you just wrote. Do you think people who subscribe to CR need CR's opinions on a monthly basis to make a purchase? If they only read CR to gather information for an occasional purchase, why wouldn't they just buy the yearly buying guide, or pick-up the occasional issue that addresses their next big purchase, or read it at the library. People subscribe to CR for the same sort of reasons they subscribe to Car and Driver, or Popular Science, or People. People like to have information. People like to read things. I enjoy CR, I like to read their opinions, even when I don't agree with them, and they do include interesting features.

So only Motor Trend buyers want to see their car praised? I think you jsut swung over to my side of the discussion. --> Toyota owners like to see their cars praised, so they subscribe to CR. CR needs to keep those Toyota owners happy, so they continue to praise Toyotas.

The Suzuki Samurai episode was hardly an example of "impeccable credibility." Or the recent baby seat test fiasco. I've seen them do some really stupid tests. I find some articles boring, but still find enough interesting to keep subscribing. I'd just as soon read CR's car comparisons as Car and Drivers. I just wish CR would test a few more interesting cars (I am still waiting for the Ferrari / Aston Martin comparison).

If the only people who subscribed to CR were people who were making a major purchase, CR would already be out of business. CR needs the faithful subscriber base to stay in business. In fact, they are far more beholden to their subscriber than publications that accept advertising. C&D can afford to tick off a few subscribers as long as they keep the big advertisers happy. I wonder what would happen next month if CR had a road test that trashed the new Camry.

So how do you explain the very poor reliability ratings that CR subscribers have given to the new Nissan Quest and Nissan Titan, which scored highly in tests, or the above-average reliability rating for the low-scoring Chevy Impala?

But aren't repairs and service so much more expensive for Mercedes that it can cost more to maintain a reliable Mercedes than an unreliable Chevy?

Then why did CR score the Ford Futura/Mercury Milan so highly and its readers give it very good reliability ratings?

I wanted to make a few more comment about the reliability "percentages" shown on the CR website.

There inconsistencies in these "percentages" that should make you wonder about the value of the CR reliability data. For instance:

A Honda Accord Hybrid is shown as being around 75% better than average. A 4 cylinder Accord is about 60% better than average. A V-6 Accord is only about 40% better than average. Do you really think there is that much difference in the reliability between the three "types" of Accords? Of course if the difference was only very small then this would make sense, but then the difference probably would not be statistically significant, which of course makes the whole comparison an exercise of making almost non-existent difference look important.

Another Case -

A V-6 Camry is a little over 40% better than average. A 4 cylinder Camry is only average or a little worse. Do you think a V-6 Camry is significantly more reliable than a 4 cylinder Camry? I don't. Yet CR's percentages are structured in such a way as to make it appear that the V-6 Camrys are far more reliable. And why would a Lexus ES350 be close to 100% more reliable than average, if the V-6 Camry is only 40% more reliable than average and the Avalon is only 35% more reliable than average? Despite the much higher cost, the basic underpinnings of the ES350 are still standard Toyota components. I cannot believe there is a significant difference in ACTUAL reliability (as opposed to the Customer's opinion of reliability). And while you are looking at Toyotas, look at the Solara - 4 cylinder Solaras are actually rated as being more reliable than V-6 Solaras (50% to 40%). This wouldn't be particularly significant, except it is completely at odds with the comparison of 4 cylinder and V-6 Camrys. They share drivetrains. You would assume if 4 cylinder Camrys were much less reliable than V-6 Camrys, 4 cylinder Solaras would be much less reliable than V-6 Solaras - but that is not what CR's data shows.

This fits into my contention that CR's survey is poorly constructed. It is not random and there are too few data points to make the results meaningful. They take this bad data, and then over emphasize the differences. What are probably very small and possibly statistically insignificant differences are presented in such a way as to make it appear they represent important differences.

So you ask - why would CR do this? Easy! If they printed results where the reliability of most cars was "average" or very close to average with no meaningful differences, who would bother to read the magazine? They are doing exactly the same thing other media outlets do - hyping the story to attract Customers (in this case subscribers). It is little different than Dateline using rockets to make sure the Chevy truck would burn in their infamous story.

Ed

I said they "tend" spit back opinions. Since I think the survey is poorly constructed and not statistically valid, I expect inconsistent results to be the norm. And I was not referring to driving quality, I was talking about reliability. I am confident that BMWs score really well in terms of driving qualities, but they aren't particularly reliable. Besides CR has consistently mentioned that the Titan and Quest are not reliable. And interestingly, the V-8 Impala has poor reliability. The V-6 is average. I don't see how you can call the Impala "low scoring" since CR "recommends" the V-6 Impala.

Ed

"larry moe 'n curly" wrote in message news: snipped-for-privacy@y5g2000hsa.googlegroups.com...

A bias does not imply that they are outright making things up. If the Ford Fusion / Mercury Milan / Lincoln Zephyr really did well in the survey, they can't just say the results were bad. But when they write the reviews they can either not emphasize that or emphasize other areas (either good or bad) to create a biased impression.

The devil advocate in me is suspicious of the great rating for the Fusion. I think it is possible that CR has trashed US vehicles to the point that the only CR readers that still buy domestic brands are hard core Ford/Chevy/etc buyers. When these hard core buyers get the survey they shade their answers to the point that the results are too good. Of course I have thought that hard core foreign brand owners have been doing that for years. Think about it. Compared to domestic competitors Toyotas are generally over priced by hundreds, even thousands of dollars. For cars in similar categories, they also are often smaller. So why would someone buy less car for more money? Gas mileage and reliability are two rational reasons for doing so. So if you go out and spend more for less car because it is supposedly more reliable, isn't it likely you'd want to validate your reasons for buying the more expensive car and maybe shade your answers to the survey? Wouldn't you feel stupid if the Camry you bought because it was so reliable was actually less reliable than a Fusion which costs thousands less - oh wait, it is. I guess I am wrong, all vehicle owners are completely honest and Fusions are more reliable than Camrys and are a far better value. Toyota sales will soon crash and Ford will rule the world. Ford's problems are solved. Rational Consumer Reports reading car buyers will quit buying over priced average reliability Camrys and start buying lower priced more reliable Fusions...sure that is going to happen.

Ed

Here is a bit of irony - I just got a survey request from Consumer Reports asking me to rate the annual Car Issue.....

Ed

If CR "recommends" the V-6 Impala, does the CR "groupthink" now include Chevy purchasers?

Get a grip. Among other things, CR probably tests various iterations of their survey and found that asking the customers for "serious" problems by the customers' own lights didn't deviate significantly from CR supplying some criteria for what was "serious" and what was not. So, they made the form simpler and probably increased response by making the questions more approachable.

Further, CR is, to some extent, an opinion-maker. But around here, you're going to find people who have taken CRs advice and find that it has helped put money in their pocket. This is not our opinion, this is our experience.

And if you want to whine about bias and groupthink and so forth, I'd suggest you try some other web auto forums (not even UseNet) because there's plenty of them out there that have, for instance, an apparent pathological hatred of domestic cars that leaves any possible anti-domestic bias on CRs part in the dust.

It probably will. If you consider the Camry and the Fusion as comparable, the Fusion offers a price advantage and if the reliability is there, people will carefully consider the Fusion.

However, it won't happen overnight. I was impressed enough with the ratings of Ford on some things to consider one this summer. My wife, however, remembers very well how Ford treated us before and said, "NO!"

The reason that GM and Ford still sell any cars at all is because it take time to lose market share. The flip side is that it also takes time to gain it. They're also lucky that 40% of US auto buyers will absolutely not consider an Asian car, no matter what. And their market share is getting down towards this hard core.

I don't have the figures in front of me but let's say, for the sake of argument, that CR says Ford is 20% more reliable than a Camry. Personal Ford history and spousal input aside, what do I do next year if I want a sedan? There's a good chance I'd buy a Camry. Why? Because my current Toyotas have given satisfaction and, on a personal basis, the Ford is unkown. I can lean towards it over other makes but if I'm happy with what I have, why switch? If a Camry is significantly more expensive, I might reconsider. If the Fusion has features I really like, I'll consider that.

If you look at CR, you can tell which pages are Honda and Toyota pages by the dominant colors. Those two automakers, while not quite perfect, have been v-e-r-y consistent and consistent across all their products. This is important. Ford can have a great frist year with the Fusion but they're going to need several years of consistent performance to build trust. And some of us are going to look at the other Ford models when deciding what to buy, so Ford's got some work to do there, too. Why? Well, suppose they switch production at one plant from to Fusions? Or changes plant managers or the engineering team or the QC team... or any kind of changes. What happens to quality? If Ford has a consistent "quality ethic" that shows across their entire product line, then you know that when they make changes they're going to do them right.

They report on depreciation but it is not a factor in their ratings.

By the way, talking about the Fusion, my brother in law rents one regularly when visiting the area and although they appear to be decent cars the weatherstripping looks incredibly cheap.

I am having a hard time understanding this comment. I thought Ford went overboard on the weather stipping. They not only have the sandard door frame to door stripping, they also have an additional section that fills in the gap between the edge of the door and the roof / A / B / C pillars.

Ed