Re: Consumer Reports 2007 Auto Survey

So how do you explain the very poor reliability ratings that CR subscribers have given to the new Nissan Quest and Nissan Titan, which scored highly in tests, or the above-average reliability rating for the low-scoring Chevy Impala?

But aren't repairs and service so much more expensive for Mercedes that it can cost more to maintain a reliable Mercedes than an unreliable Chevy?

Then why did CR score the Ford Futura/Mercury Milan so highly and its readers give it very good reliability ratings?

I wanted to make a few more comment about the reliability "percentages" shown on the CR website.

There inconsistencies in these "percentages" that should make you wonder about the value of the CR reliability data. For instance:

A Honda Accord Hybrid is shown as being around 75% better than average. A 4 cylinder Accord is about 60% better than average. A V-6 Accord is only about 40% better than average. Do you really think there is that much difference in the reliability between the three "types" of Accords? Of course if the difference was only very small then this would make sense, but then the difference probably would not be statistically significant, which of course makes the whole comparison an exercise of making almost non-existent difference look important.

Another Case -

A V-6 Camry is a little over 40% better than average. A 4 cylinder Camry is only average or a little worse. Do you think a V-6 Camry is significantly more reliable than a 4 cylinder Camry? I don't. Yet CR's percentages are structured in such a way as to make it appear that the V-6 Camrys are far more reliable. And why would a Lexus ES350 be close to 100% more reliable than average, if the V-6 Camry is only 40% more reliable than average and the Avalon is only 35% more reliable than average? Despite the much higher cost, the basic underpinnings of the ES350 are still standard Toyota components. I cannot believe there is a significant difference in ACTUAL reliability (as opposed to the Customer's opinion of reliability). And while you are looking at Toyotas, look at the Solara - 4 cylinder Solaras are actually rated as being more reliable than V-6 Solaras (50% to 40%). This wouldn't be particularly significant, except it is completely at odds with the comparison of 4 cylinder and V-6 Camrys. They share drivetrains. You would assume if 4 cylinder Camrys were much less reliable than V-6 Camrys, 4 cylinder Solaras would be much less reliable than V-6 Solaras - but that is not what CR's data shows.

This fits into my contention that CR's survey is poorly constructed. It is not random and there are too few data points to make the results meaningful. They take this bad data, and then over emphasize the differences. What are probably very small and possibly statistically insignificant differences are presented in such a way as to make it appear they represent important differences.

So you ask - why would CR do this? Easy! If they printed results where the reliability of most cars was "average" or very close to average with no meaningful differences, who would bother to read the magazine? They are doing exactly the same thing other media outlets do - hyping the story to attract Customers (in this case subscribers). It is little different than Dateline using rockets to make sure the Chevy truck would burn in their infamous story.

Ed

I said they "tend" spit back opinions. Since I think the survey is poorly constructed and not statistically valid, I expect inconsistent results to be the norm. And I was not referring to driving quality, I was talking about reliability. I am confident that BMWs score really well in terms of driving qualities, but they aren't particularly reliable. Besides CR has consistently mentioned that the Titan and Quest are not reliable. And interestingly, the V-8 Impala has poor reliability. The V-6 is average. I don't see how you can call the Impala "low scoring" since CR "recommends" the V-6 Impala.

Ed

"larry moe 'n curly" wrote in message news: snipped-for-privacy@y5g2000hsa.googlegroups.com...

A bias does not imply that they are outright making things up. If the Ford Fusion / Mercury Milan / Lincoln Zephyr really did well in the survey, they can't just say the results were bad. But when they write the reviews they can either not emphasize that or emphasize other areas (either good or bad) to create a biased impression.

The devil advocate in me is suspicious of the great rating for the Fusion. I think it is possible that CR has trashed US vehicles to the point that the only CR readers that still buy domestic brands are hard core Ford/Chevy/etc buyers. When these hard core buyers get the survey they shade their answers to the point that the results are too good. Of course I have thought that hard core foreign brand owners have been doing that for years. Think about it. Compared to domestic competitors Toyotas are generally over priced by hundreds, even thousands of dollars. For cars in similar categories, they also are often smaller. So why would someone buy less car for more money? Gas mileage and reliability are two rational reasons for doing so. So if you go out and spend more for less car because it is supposedly more reliable, isn't it likely you'd want to validate your reasons for buying the more expensive car and maybe shade your answers to the survey? Wouldn't you feel stupid if the Camry you bought because it was so reliable was actually less reliable than a Fusion which costs thousands less - oh wait, it is. I guess I am wrong, all vehicle owners are completely honest and Fusions are more reliable than Camrys and are a far better value. Toyota sales will soon crash and Ford will rule the world. Ford's problems are solved. Rational Consumer Reports reading car buyers will quit buying over priced average reliability Camrys and start buying lower priced more reliable Fusions...sure that is going to happen.

Ed

If CR "recommends" the V-6 Impala, does the CR "groupthink" now include Chevy purchasers?

Get a grip. Among other things, CR probably tests various iterations of their survey and found that asking the customers for "serious" problems by the customers' own lights didn't deviate significantly from CR supplying some criteria for what was "serious" and what was not. So, they made the form simpler and probably increased response by making the questions more approachable.

Further, CR is, to some extent, an opinion-maker. But around here, you're going to find people who have taken CRs advice and find that it has helped put money in their pocket. This is not our opinion, this is our experience.

And if you want to whine about bias and groupthink and so forth, I'd suggest you try some other web auto forums (not even UseNet) because there's plenty of them out there that have, for instance, an apparent pathological hatred of domestic cars that leaves any possible anti-domestic bias on CRs part in the dust.

It probably will. If you consider the Camry and the Fusion as comparable, the Fusion offers a price advantage and if the reliability is there, people will carefully consider the Fusion.

However, it won't happen overnight. I was impressed enough with the ratings of Ford on some things to consider one this summer. My wife, however, remembers very well how Ford treated us before and said, "NO!"

The reason that GM and Ford still sell any cars at all is because it take time to lose market share. The flip side is that it also takes time to gain it. They're also lucky that 40% of US auto buyers will absolutely not consider an Asian car, no matter what. And their market share is getting down towards this hard core.

I don't have the figures in front of me but let's say, for the sake of argument, that CR says Ford is 20% more reliable than a Camry. Personal Ford history and spousal input aside, what do I do next year if I want a sedan? There's a good chance I'd buy a Camry. Why? Because my current Toyotas have given satisfaction and, on a personal basis, the Ford is unkown. I can lean towards it over other makes but if I'm happy with what I have, why switch? If a Camry is significantly more expensive, I might reconsider. If the Fusion has features I really like, I'll consider that.

If you look at CR, you can tell which pages are Honda and Toyota pages by the dominant colors. Those two automakers, while not quite perfect, have been v-e-r-y consistent and consistent across all their products. This is important. Ford can have a great frist year with the Fusion but they're going to need several years of consistent performance to build trust. And some of us are going to look at the other Ford models when deciding what to buy, so Ford's got some work to do there, too. Why? Well, suppose they switch production at one plant from to Fusions? Or changes plant managers or the engineering team or the QC team... or any kind of changes. What happens to quality? If Ford has a consistent "quality ethic" that shows across their entire product line, then you know that when they make changes they're going to do them right.

They report on depreciation but it is not a factor in their ratings.

By the way, talking about the Fusion, my brother in law rents one regularly when visiting the area and although they appear to be decent cars the weatherstripping looks incredibly cheap.

I am having a hard time understanding this comment. I thought Ford went overboard on the weather stipping. They not only have the sandard door frame to door stripping, they also have an additional section that fills in the gap between the edge of the door and the roof / A / B / C pillars.

Ed

I realize that, but there are many other examples, and if respondents to the reliabilitly survey are biased in favor of cars that score well in CR's tests, why do they often say they're highly satisfied with cars that they say are very unreliable or dissatisfied with cars that they say are reliable?

Different assembly factories? It's been said that Nissan's least- reliable vehicles all come from one factory in Mississippi, and in the case of my old Ford Escort, CR said the sedan, which were made at their Hermosillo, Mexico factory, were more reliable than the hatchbacks, which were assembled in a US factory.

It got 63 points, putting into the lower range of the "Very Good" catagory, and ranked 18th out of 23 "Family Sedans Over $25,000". In this catagory only the bottom-ranked Pontiac Grand Prix wasn't recommended among cars with average or better reliabilitly records. The GP was more reliable than average but scored only 38 points, putting it in the upper range of the "Fair" range.

Ashton Crusher wrote to Art:

My father has been subscribing for a long time, despite complaining about their "stupid socialism".

That didn't seem to be the case with the Toyota Matrix/Pontiac Vibe, either in test scores or reliability ratings, or with the Toyota Corolla/Chevy Nova/Geo Prizm/Chevy Prizm, which were built simultaneously on the same assembly line in California, although with cosmetic and mechanical differences (like the ABS).

It's more the sample size. The be statistically meaningful it has to be a proper representation of the population. For example, if most the people who buy Toyotas are women and most of the people who buy fords are men there is a very high probability that what the women consider a "significant problem" will be different then what men consider a significant problem. So if both fords and Toyotas each have the same number of alternators go bad, they could well get different ratings because the men may shrug it off but the women may come unglued when the "battery light" comes on and stop on the side of the road and wind up with a HUGE incident over a minor alternator problem.

Yes, my point exactly. CR briefly reported that info, which made their favorite cars look bad, so they stopped reporting it. Some people would like to know BOTH bits of info. It's nice to have a 'reliable' car but if you have to choose between a Chevy that will break down once every two years and cost you $300 to fix, versus a Mercedes that will breakdown every 3 years but cost you $3000 to fix, you might well decide the "unreliable" Chevy is a MUCH better deal. Keep in mind, that the actual difference in the amount of problems on most of these cars is quite trivial despite how CR makes them look with all the black dots.

Because you can only bend the truth just so far.

Please provide specific examples, otherwise you'll start to look like one of the anti-CR religious nuts.

Sarcasm? That doesn't lend weight to your argument--it just makes me think you're emotionally invested in it, which is a good sign that someone has abandoned objectivity and is just looking for reasons to justify their belief. You can counteract this easily by providing evidence for all of the claims you are making against CR. It shouldn't be too difficult, libraries often have issues going back for years.

If you mean specs provided by manufacturers, they often fudge by using different definitions for things, like the CRT monitor manufacturers used to claim up to two inches bigger than the actual screen size, and hard drive manufacturers use a different definition of kilobyte, megabyte, gigabyte and terabyte than everyone else in the computer industry to make their hard drives seem bigger than they really are.

If you mean other (non-manufacturer) product testing sites, I've seen a few. I noticed that most of them were supported by ads. The ones without ads I trusted even less, because who knows where their money is coming from.

And you're saying CR does that? Do you read CR? Because they don't do reliability testing for the most part... and that's ALL you've mentioned here. Oh, and can you provide specific examples so you can back up your claim?

I haven't picked up a CR in over a decade. The religion is CR, as your rabid response serves to indicate. Believe what you want, however CR does not correlate their tests to actual field failures or even performance, thusly they are simply arbitary on their face.

Nice projection. I could care less about CR. I dismissed their methods as worthless long before I ever saw a thread on usenet about them.

My aren't you testy... Guess you have to lash out at a CR heretic like me.

reliability, performance whatever you want to call it. Same difference. They come up with a test and a standard of 'pass' for said test. If the device broke in the process it fails.

Apparently you feel I've attacked your belief system, however the tests that CR does are just the tests they feel judge reliability (or as some would call it, performance) under certain arbitary conditions. These may or may not have any bearing to the real world. Manufacturers, or at any worth a damn have their own internal testing which may be much better or just as flawed as CR. However the engineers must design for the internal tests, not CRs.

Yes, that's what it says on their web site. Thirty-five percent seems like a huge range of difference for the same car with different drivetrains! Also, their site does say "Note that the average stretches 20 points on either side of the zero line, so it's possible for a car to have an average Predicted Reliability Rating even if its bar is in the negative zone." To me, this means that the cars are fairly close--in other words, if a difference of forty points separates the high and low ends of the average field, I would expect that a difference between any two cars of less than forty points would be meaningless. I believe you said something to this effect brfore. Again, your point.

Not if they are all the same basic car, made in the same factories, and in the same class for comparison. If these criteria are met (which they could be--I don't know) then I'd agree the wide variance would need to be explained--otherwise I would agree that there would be some doubt as to whether or not the numbers are meaningful. Good point.

Well, forst I'd like to thank you for providing examples. I understand much better what you're getting at and agree that the percentages aren't nearly as helpful as I had thought. From this point on, I doubt if I'd let a difference of sixty or eighty points or less influence my buying decision. More than eighty points would make a difference to me, though.

Once again, I disagree that the commonality of the poll participants being CR subscribers would skew the result in favor of, say, Toyota consistently enough to show much better than average reliability for at least 25 years--I think that over time the results would even out more. I believe the companies that get consistently high ratings in the CR polls do make better cars--though how much better is anyone's guess. It might not even be meaningful.

For example, if 73 of 1000 Accord owners have to make major repairs within the first four years after the warranty expires and 86 of 1000 Impala owbers have to do the same (given equal definitions for the word "major"), should it even be a factor in my buying decision that I have (if my math is correct) a 1.3 percent greater liklihood of having to make a major repair within the first four years after the warranty expires with the Impala than with the Accord? Yes, but only a *slight* one. If I like the cars equally otherwise and they are the same price, I would go with the Accord, but if the Impala is $150 cheaper, I might just get that. That 1.3% increased risk seems like a fair tradeoff for $150. If that translates to an 80-point difference on CR's scale, then the scale doesn't do me much good. (This is an example, all the numbers are made up.)

Of course, CR doesn't post the raw data, so we can't make such a determination for ourselves, even if the sample is statistically significant.

I don't agree that the data has been proven to be bad, but the small sample combined with questionable and mysterious methods for displaying it do make me re-think my whole attitude toward their reliability reports.

That's a valid line of reasoning for why they would do this, and the fact that their percent range is odd and potentially misleading combined with the fact that they don't release their raw data (do they?) makes their chart highly suspect *at best*.

One thing I've noticed is that most US cab companies which own their vehicles and rent them to the drivers buy American cars. Since the cab companies are paying to repair these vehicles, I would guess they would be strongly inclined to buy Japanese cars instead--if Japanese cars lasted longer and needed fewer repairs than American cars. It would save them a bundle, cabs get driven a lot of miles and need lots of fixing. For other fleets I would suspect image concerns, for example I suspect Geek Squad chose VW Bugs for image alone, but when it comes to cabs most people just want one that's available or will be soon.

That's where I think you're wrong, I haven't seen any evidence of CR readers tending to spit back opinions nor any reasoning to suggest they would.

That's where I'd agree with you. So it doesn't really matter if CR readers are sheep to the CR shephard or whether they tend to be independant thinkers; the samples are too small to be reliable for individual models.

I wouldn't say they are the norm. For example, the Matrix and Vibe are very close, probably within five points. Other different makes of the same vehicle seem to be very close as well. That's actually unexpected given the sample sizes. Plus, if you were correct and the survey results were usually inconsistent, then certain auto makers (like Toyota) would never have stayed consistently near the top for the last 25 years at least, while others have gone from near the top to near the bottom (Like Mercedes.) I believe this shows that the results do *in general* show which makes are producing more reliable models over the years and which ones less so, but that doesn't matter if the differences aren't significant.

My "rabid response"--ROFL! If you can't beat them with reason, attack them personally, is that it? Who's the rabid one here? I ask for some evidence backing up your claim and that's a "rabid response"--wow. I'd hate to see what happened if I actually got emotional with you about this.

I believe what the evidence shows, not what someone on the internet says is true, especially when they start hiding behind insults when politely asked to provide evidence supporting their claims. You expect someone to just accept your claims without any evidence? If so, maybe you should examine that expectation, because I've found that most people won't accept any claim that goes against what they believe even with a mountain of evidence. You're in for an enormous amount of frustration.

Personally, I'm not emotionally invested in CR, so I can look at your claims objectively. C. E. White is making similar claims, and through evidence and solid reasoning, he's convinced me that CR's reliability data is misleading and not particularly helpful. I'm not convinced it's completely worthless, but I would no longer change by car-buying decision (all other things being equal) for a percent difference of at least sixty to eighty percentage points, and I wouldn't expect the frequency of repairs on two compared models to be substantially different unless the spread was more than a hundred twenty to a hundred forty points. Before, I had considered anything over five to ten points significant--because I didn't know as much about the testing methods and display methods. I think that speaks to whether or not I'm a CR fanatic. I'm not one to go making wild claims and then when politely requested to supply evidence supporting those claims respond with senseless insults. That's one mark of a fanatic.

The statement "I could care less" means you care. I'm going to assume you meant "I couldn't care less" which means you don't care. If that's what you're trying to say, it couldn't be more obviously untrue. You've spent a great deal of time criticizing them online. If you didn't care, you wouldn't be trying so hard to shoot them down.

And what does that prove?

"Testy"? "lash out"? ROFL!!! That's enough, you're obviously delusional. There's nothing in the preceeding two paragraphs that would remotely be considered "testy" or "lashing out" by any stretch of the imagination. I hope you get better soon. Bye.