Re: Consumer Reports 2007 Auto Survey

CR comes up with on the fly test criteria that fail products they think should fail and passes products they think should pass.

Considering CR's testing, there is no way of making such a determination.

There's this thing called the web now where one can find exacting specifications of devices like cameras. CR does list out anywhere near the detail that can often be found online.

I've been in product design a fair number of years of now. Lab tests are well, lab tests. Currently I am dealing with issues on a couple products that do not occur in the field, only in lab tests. No relationship between these lab tests and field failures has ever been established, it's just 'gut-feel' arbitary tests. And in the end that's what CR does as well.

You can have a product that internally a manufacturer put through all sorts of arbitary abusive testing and it passed only to have CR come up with something different and break it immediately. They are just arbitary tests, neither better than the other.

Reply to
Brent P
Loading thread data ...

Did you not read the actual questions? The reliability data is partially based on the respondent deciding what is SERIOUS. This is the respondents opinon.

But if you are trying to decide what vehicle is reliable, you want reliable reliability data. CR is giving you the CR reader's opinion of the reliability of thier cars. These are not the same thing. One is based on hard facts (dollars spent, hours out of service, etc.), the other is based on whether people felt they were inconvenienced.

Garbage in, garbage out. Too small a sample size; non-random samples, etc. They depend on people remembering problems, and deciding they were serious. If you average the opinions of all the RNC members, you would conclude GWB is doing a great job.

I guess I missed your evidence that they don't. No doubt this is speculation, but I don't think you can make the opposite claim (i.e., that people are totally objective when filling out the survey - for one thing, they have no clear guide what should be considered serious). .....

I think the opposite is true in many cases. I think people that buy cars because they are said to be reliable tend to not want to admit they made a mistake if the car turns out to be unreliable. Humans have a hard time admitting mistakes (even me).

And I am saying that CR is reporting a very limited group's opinion of the reliability of a car, not the actual releiability.

Try around 1992. Or just go look at the JD Powers Survey results on line to see how results are slewed by expectations (JDP definitely does not average results across brands). According to the JDP 2005 Vehicle Dependability Study, the average 3 year old Buick has 1.63 problems, the average 3 year old Oldsmobile has 2.42. That is a retty signficant difference (over 30%).

CR rates the 2006 models exactly the same (suspicious?). JD Power gave them radically different ratings (Vibe got 3 balls for manufacturing quality, Matrix got 4.5; Vibe got 2 balls for design quality, Matrix got 3).

Mercedes is so far down the list CR would have no credibility if they continued to claim they were reliable. Besides, these days which audience is it more important for CR to please, Toyota buyers or Mercedes buyers?

Percetage of what? The only percentages I see are in the graphs where they rate cars comapred to the CR average "score." How is the score calculated? They don't actually tell (at least as far as I can see). They take a lot of pride in mentioning that they are basing the ratings on 1.3 million vehicles spanning three year models. But since they are rating over 300 models, over

3 model years, the ":average" model/year only has 1,300 data points - and that is the averge. Many of the models must have only a few hundred. CR could just publish the raw results and then we could decide for ourselves. They could put it online for a minimal cost. I wonder why they don't do it.

Even your example could be an example of bias. Choosing which facts to present involves making an editorial decision - which is another way of saying, expressing an opinion. CR decides what is important to report on. They love ESC. I think it is over hyped and not worth the cost. If I was writing the article it wouldn't be a factor. One thing that has always bugged me is the entry and exit from cars. I hate Crown Victorias because they are too low, which makes them hard for me (tall/large) to get into. My

2001 Mustang was easier to get in than my Mothers' Grand Marquis. I rarely see this sort of thing mentioned, and it is often a problem for me with Japanese cars. I have to practically fall into my Sister's Civic because it is so low to the ground. Selective reporting is as much a bias factor as stating a clearly identifiable opinion.

.........

Yes I expressed andopinion. Here is another - I think you are naive if you believe what you just wrote. Do you think people who subscribe to CR need CR's opinions on a monthly basis to make a purchase? If they only read CR to gather information for an occasional purchase, why wouldn't they just buy the yearly buying guide, or pick-up the occasional issue that addresses their next big purchase, or read it at the library. People subscribe to CR for the same sort of reasons they subscribe to Car and Driver, or Popular Science, or People. People like to have information. People like to read things. I enjoy CR, I like to read their opinions, even when I don't agree with them, and they do include interesting features.

So only Motor Trend buyers want to see their car praised? I think you jsut swung over to my side of the discussion. --> Toyota owners like to see their cars praised, so they subscribe to CR. CR needs to keep those Toyota owners happy, so they continue to praise Toyotas.

The Suzuki Samurai episode was hardly an example of "impeccable credibility." Or the recent baby seat test fiasco. I've seen them do some really stupid tests. I find some articles boring, but still find enough interesting to keep subscribing. I'd just as soon read CR's car comparisons as Car and Drivers. I just wish CR would test a few more interesting cars (I am still waiting for the Ferrari / Aston Martin comparison).

If the only people who subscribed to CR were people who were making a major purchase, CR would already be out of business. CR needs the faithful subscriber base to stay in business. In fact, they are far more beholden to their subscriber than publications that accept advertising. C&D can afford to tick off a few subscribers as long as they keep the big advertisers happy. I wonder what would happen next month if CR had a road test that trashed the new Camry.

Reply to
C. E. White

So how do you explain the very poor reliability ratings that CR subscribers have given to the new Nissan Quest and Nissan Titan, which scored highly in tests, or the above-average reliability rating for the low-scoring Chevy Impala?

Reply to
larry moe 'n curly

But aren't repairs and service so much more expensive for Mercedes that it can cost more to maintain a reliable Mercedes than an unreliable Chevy?

Then why did CR score the Ford Futura/Mercury Milan so highly and its readers give it very good reliability ratings?

Reply to
larry moe 'n curly

I wanted to make a few more comment about the reliability "percentages" shown on the CR website.

There inconsistencies in these "percentages" that should make you wonder about the value of the CR reliability data. For instance:

A Honda Accord Hybrid is shown as being around 75% better than average. A 4 cylinder Accord is about 60% better than average. A V-6 Accord is only about 40% better than average. Do you really think there is that much difference in the reliability between the three "types" of Accords? Of course if the difference was only very small then this would make sense, but then the difference probably would not be statistically significant, which of course makes the whole comparison an exercise of making almost non-existent difference look important.

Another Case -

A V-6 Camry is a little over 40% better than average. A 4 cylinder Camry is only average or a little worse. Do you think a V-6 Camry is significantly more reliable than a 4 cylinder Camry? I don't. Yet CR's percentages are structured in such a way as to make it appear that the V-6 Camrys are far more reliable. And why would a Lexus ES350 be close to 100% more reliable than average, if the V-6 Camry is only 40% more reliable than average and the Avalon is only 35% more reliable than average? Despite the much higher cost, the basic underpinnings of the ES350 are still standard Toyota components. I cannot believe there is a significant difference in ACTUAL reliability (as opposed to the Customer's opinion of reliability). And while you are looking at Toyotas, look at the Solara - 4 cylinder Solaras are actually rated as being more reliable than V-6 Solaras (50% to 40%). This wouldn't be particularly significant, except it is completely at odds with the comparison of 4 cylinder and V-6 Camrys. They share drivetrains. You would assume if 4 cylinder Camrys were much less reliable than V-6 Camrys, 4 cylinder Solaras would be much less reliable than V-6 Solaras - but that is not what CR's data shows.

This fits into my contention that CR's survey is poorly constructed. It is not random and there are too few data points to make the results meaningful. They take this bad data, and then over emphasize the differences. What are probably very small and possibly statistically insignificant differences are presented in such a way as to make it appear they represent important differences.

So you ask - why would CR do this? Easy! If they printed results where the reliability of most cars was "average" or very close to average with no meaningful differences, who would bother to read the magazine? They are doing exactly the same thing other media outlets do - hyping the story to attract Customers (in this case subscribers). It is little different than Dateline using rockets to make sure the Chevy truck would burn in their infamous story.

Ed

Reply to
C. E. White

I said they "tend" spit back opinions. Since I think the survey is poorly constructed and not statistically valid, I expect inconsistent results to be the norm. And I was not referring to driving quality, I was talking about reliability. I am confident that BMWs score really well in terms of driving qualities, but they aren't particularly reliable. Besides CR has consistently mentioned that the Titan and Quest are not reliable. And interestingly, the V-8 Impala has poor reliability. The V-6 is average. I don't see how you can call the Impala "low scoring" since CR "recommends" the V-6 Impala.

Ed

Reply to
C. E. White

"larry moe 'n curly" wrote in message news: snipped-for-privacy@y5g2000hsa.googlegroups.com...

A bias does not imply that they are outright making things up. If the Ford Fusion / Mercury Milan / Lincoln Zephyr really did well in the survey, they can't just say the results were bad. But when they write the reviews they can either not emphasize that or emphasize other areas (either good or bad) to create a biased impression.

The devil advocate in me is suspicious of the great rating for the Fusion. I think it is possible that CR has trashed US vehicles to the point that the only CR readers that still buy domestic brands are hard core Ford/Chevy/etc buyers. When these hard core buyers get the survey they shade their answers to the point that the results are too good. Of course I have thought that hard core foreign brand owners have been doing that for years. Think about it. Compared to domestic competitors Toyotas are generally over priced by hundreds, even thousands of dollars. For cars in similar categories, they also are often smaller. So why would someone buy less car for more money? Gas mileage and reliability are two rational reasons for doing so. So if you go out and spend more for less car because it is supposedly more reliable, isn't it likely you'd want to validate your reasons for buying the more expensive car and maybe shade your answers to the survey? Wouldn't you feel stupid if the Camry you bought because it was so reliable was actually less reliable than a Fusion which costs thousands less - oh wait, it is. I guess I am wrong, all vehicle owners are completely honest and Fusions are more reliable than Camrys and are a far better value. Toyota sales will soon crash and Ford will rule the world. Ford's problems are solved. Rational Consumer Reports reading car buyers will quit buying over priced average reliability Camrys and start buying lower priced more reliable Fusions...sure that is going to happen.

Ed

Reply to
C. E. White

If CR "recommends" the V-6 Impala, does the CR "groupthink" now include Chevy purchasers?

Get a grip. Among other things, CR probably tests various iterations of their survey and found that asking the customers for "serious" problems by the customers' own lights didn't deviate significantly from CR supplying some criteria for what was "serious" and what was not. So, they made the form simpler and probably increased response by making the questions more approachable.

Further, CR is, to some extent, an opinion-maker. But around here, you're going to find people who have taken CRs advice and find that it has helped put money in their pocket. This is not our opinion, this is our experience.

And if you want to whine about bias and groupthink and so forth, I'd suggest you try some other web auto forums (not even UseNet) because there's plenty of them out there that have, for instance, an apparent pathological hatred of domestic cars that leaves any possible anti-domestic bias on CRs part in the dust.

Reply to
DH

It probably will. If you consider the Camry and the Fusion as comparable, the Fusion offers a price advantage and if the reliability is there, people will carefully consider the Fusion.

However, it won't happen overnight. I was impressed enough with the ratings of Ford on some things to consider one this summer. My wife, however, remembers very well how Ford treated us before and said, "NO!"

The reason that GM and Ford still sell any cars at all is because it take time to lose market share. The flip side is that it also takes time to gain it. They're also lucky that 40% of US auto buyers will absolutely not consider an Asian car, no matter what. And their market share is getting down towards this hard core.

I don't have the figures in front of me but let's say, for the sake of argument, that CR says Ford is 20% more reliable than a Camry. Personal Ford history and spousal input aside, what do I do next year if I want a sedan? There's a good chance I'd buy a Camry. Why? Because my current Toyotas have given satisfaction and, on a personal basis, the Ford is unkown. I can lean towards it over other makes but if I'm happy with what I have, why switch? If a Camry is significantly more expensive, I might reconsider. If the Fusion has features I really like, I'll consider that.

If you look at CR, you can tell which pages are Honda and Toyota pages by the dominant colors. Those two automakers, while not quite perfect, have been v-e-r-y consistent and consistent across all their products. This is important. Ford can have a great frist year with the Fusion but they're going to need several years of consistent performance to build trust. And some of us are going to look at the other Ford models when deciding what to buy, so Ford's got some work to do there, too. Why? Well, suppose they switch production at one plant from to Fusions? Or changes plant managers or the engineering team or the QC team... or any kind of changes. What happens to quality? If Ford has a consistent "quality ethic" that shows across their entire product line, then you know that when they make changes they're going to do them right.

Reply to
DH

They report on depreciation but it is not a factor in their ratings.

Reply to
Art

By the way, talking about the Fusion, my brother in law rents one regularly when visiting the area and although they appear to be decent cars the weatherstripping looks incredibly cheap.

Reply to
Art

I am having a hard time understanding this comment. I thought Ford went overboard on the weather stipping. They not only have the sandard door frame to door stripping, they also have an additional section that fills in the gap between the edge of the door and the roof / A / B / C pillars.

Ed

Reply to
Ed White

I realize that, but there are many other examples, and if respondents to the reliabilitly survey are biased in favor of cars that score well in CR's tests, why do they often say they're highly satisfied with cars that they say are very unreliable or dissatisfied with cars that they say are reliable?

Different assembly factories? It's been said that Nissan's least- reliable vehicles all come from one factory in Mississippi, and in the case of my old Ford Escort, CR said the sedan, which were made at their Hermosillo, Mexico factory, were more reliable than the hatchbacks, which were assembled in a US factory.

It got 63 points, putting into the lower range of the "Very Good" catagory, and ranked 18th out of 23 "Family Sedans Over $25,000". In this catagory only the bottom-ranked Pontiac Grand Prix wasn't recommended among cars with average or better reliabilitly records. The GP was more reliable than average but scored only 38 points, putting it in the upper range of the "Fair" range.

Reply to
larry moe 'n curly

Ashton Crusher wrote to Art:

My father has been subscribing for a long time, despite complaining about their "stupid socialism".

That didn't seem to be the case with the Toyota Matrix/Pontiac Vibe, either in test scores or reliability ratings, or with the Toyota Corolla/Chevy Nova/Geo Prizm/Chevy Prizm, which were built simultaneously on the same assembly line in California, although with cosmetic and mechanical differences (like the ABS).

Reply to
larry moe 'n curly

It's more the sample size. The be statistically meaningful it has to be a proper representation of the population. For example, if most the people who buy Toyotas are women and most of the people who buy fords are men there is a very high probability that what the women consider a "significant problem" will be different then what men consider a significant problem. So if both fords and Toyotas each have the same number of alternators go bad, they could well get different ratings because the men may shrug it off but the women may come unglued when the "battery light" comes on and stop on the side of the road and wind up with a HUGE incident over a minor alternator problem.

Reply to
Ashton Crusher

Yes, my point exactly. CR briefly reported that info, which made their favorite cars look bad, so they stopped reporting it. Some people would like to know BOTH bits of info. It's nice to have a 'reliable' car but if you have to choose between a Chevy that will break down once every two years and cost you $300 to fix, versus a Mercedes that will breakdown every 3 years but cost you $3000 to fix, you might well decide the "unreliable" Chevy is a MUCH better deal. Keep in mind, that the actual difference in the amount of problems on most of these cars is quite trivial despite how CR makes them look with all the black dots.

Because you can only bend the truth just so far.

Reply to
Ashton Crusher

Please provide specific examples, otherwise you'll start to look like one of the anti-CR religious nuts.

Sarcasm? That doesn't lend weight to your argument--it just makes me think you're emotionally invested in it, which is a good sign that someone has abandoned objectivity and is just looking for reasons to justify their belief. You can counteract this easily by providing evidence for all of the claims you are making against CR. It shouldn't be too difficult, libraries often have issues going back for years.

If you mean specs provided by manufacturers, they often fudge by using different definitions for things, like the CRT monitor manufacturers used to claim up to two inches bigger than the actual screen size, and hard drive manufacturers use a different definition of kilobyte, megabyte, gigabyte and terabyte than everyone else in the computer industry to make their hard drives seem bigger than they really are.

If you mean other (non-manufacturer) product testing sites, I've seen a few. I noticed that most of them were supported by ads. The ones without ads I trusted even less, because who knows where their money is coming from.

And you're saying CR does that? Do you read CR? Because they don't do reliability testing for the most part... and that's ALL you've mentioned here. Oh, and can you provide specific examples so you can back up your claim?

Reply to
That Guy

I haven't picked up a CR in over a decade. The religion is CR, as your rabid response serves to indicate. Believe what you want, however CR does not correlate their tests to actual field failures or even performance, thusly they are simply arbitary on their face.

Nice projection. I could care less about CR. I dismissed their methods as worthless long before I ever saw a thread on usenet about them.

My aren't you testy... Guess you have to lash out at a CR heretic like me.

reliability, performance whatever you want to call it. Same difference. They come up with a test and a standard of 'pass' for said test. If the device broke in the process it fails.

Apparently you feel I've attacked your belief system, however the tests that CR does are just the tests they feel judge reliability (or as some would call it, performance) under certain arbitary conditions. These may or may not have any bearing to the real world. Manufacturers, or at any worth a damn have their own internal testing which may be much better or just as flawed as CR. However the engineers must design for the internal tests, not CRs.

Reply to
Brent P

Yes, that's what it says on their web site. Thirty-five percent seems like a huge range of difference for the same car with different drivetrains! Also, their site does say "Note that the average stretches 20 points on either side of the zero line, so it's possible for a car to have an average Predicted Reliability Rating even if its bar is in the negative zone." To me, this means that the cars are fairly close--in other words, if a difference of forty points separates the high and low ends of the average field, I would expect that a difference between any two cars of less than forty points would be meaningless. I believe you said something to this effect brfore. Again, your point.

Not if they are all the same basic car, made in the same factories, and in the same class for comparison. If these criteria are met (which they could be--I don't know) then I'd agree the wide variance would need to be explained--otherwise I would agree that there would be some doubt as to whether or not the numbers are meaningful. Good point.

Well, forst I'd like to thank you for providing examples. I understand much better what you're getting at and agree that the percentages aren't nearly as helpful as I had thought. From this point on, I doubt if I'd let a difference of sixty or eighty points or less influence my buying decision. More than eighty points would make a difference to me, though.

Once again, I disagree that the commonality of the poll participants being CR subscribers would skew the result in favor of, say, Toyota consistently enough to show much better than average reliability for at least 25 years--I think that over time the results would even out more. I believe the companies that get consistently high ratings in the CR polls do make better cars--though how much better is anyone's guess. It might not even be meaningful.

For example, if 73 of 1000 Accord owners have to make major repairs within the first four years after the warranty expires and 86 of 1000 Impala owbers have to do the same (given equal definitions for the word "major"), should it even be a factor in my buying decision that I have (if my math is correct) a 1.3 percent greater liklihood of having to make a major repair within the first four years after the warranty expires with the Impala than with the Accord? Yes, but only a *slight* one. If I like the cars equally otherwise and they are the same price, I would go with the Accord, but if the Impala is $150 cheaper, I might just get that. That 1.3% increased risk seems like a fair tradeoff for $150. If that translates to an 80-point difference on CR's scale, then the scale doesn't do me much good. (This is an example, all the numbers are made up.)

Of course, CR doesn't post the raw data, so we can't make such a determination for ourselves, even if the sample is statistically significant.

I don't agree that the data has been proven to be bad, but the small sample combined with questionable and mysterious methods for displaying it do make me re-think my whole attitude toward their reliability reports.

That's a valid line of reasoning for why they would do this, and the fact that their percent range is odd and potentially misleading combined with the fact that they don't release their raw data (do they?) makes their chart highly suspect *at best*.

One thing I've noticed is that most US cab companies which own their vehicles and rent them to the drivers buy American cars. Since the cab companies are paying to repair these vehicles, I would guess they would be strongly inclined to buy Japanese cars instead--if Japanese cars lasted longer and needed fewer repairs than American cars. It would save them a bundle, cabs get driven a lot of miles and need lots of fixing. For other fleets I would suspect image concerns, for example I suspect Geek Squad chose VW Bugs for image alone, but when it comes to cabs most people just want one that's available or will be soon.

Reply to
That Guy

That's where I think you're wrong, I haven't seen any evidence of CR readers tending to spit back opinions nor any reasoning to suggest they would.

That's where I'd agree with you. So it doesn't really matter if CR readers are sheep to the CR shephard or whether they tend to be independant thinkers; the samples are too small to be reliable for individual models.

I wouldn't say they are the norm. For example, the Matrix and Vibe are very close, probably within five points. Other different makes of the same vehicle seem to be very close as well. That's actually unexpected given the sample sizes. Plus, if you were correct and the survey results were usually inconsistent, then certain auto makers (like Toyota) would never have stayed consistently near the top for the last 25 years at least, while others have gone from near the top to near the bottom (Like Mercedes.) I believe this shows that the results do *in general* show which makes are producing more reliable models over the years and which ones less so, but that doesn't matter if the differences aren't significant.

Reply to
That Guy

MotorsForum website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.