Re: Consumer Reports 2007 Auto Survey

Page 1 of 3  


All I did was post the questions asked in CR's annual auto survey and comment that "from this limited questionnaire, that is only sent to Consumer Reports subscribers, CR generates "data" that million use to make buying decisions." Anything false in that statement? Is it even a complaint?
Do you believe that the few questions they asked of a very specific group (CR subscribers) provides hard data? It clearly an opinion survey of a specific group who's opinions are influenced by the group doing the surveying. If you want a real survey, with better data, you need to survey a random group of car owners and collect hard cost data. Depending on people to determine what problems are serious leaves a lot up in the air. I consider a problem "serious" if I have to have it fixed no matter how little it costs and I want almost everything wrong with a car "fixed." I consider a rock strike in the windshield "serious." My Sister has been riding around with a cracked windshield for three years. She doesn't think it is serious. The paint has fallen off the bumpers of her car - I'd be freaking out if mine did that - she doesn't care. One of her sun visors fell off - again she didn't care. A guy I work with constantly complained that the rear end of his pick-up truck hummed. I rode with him and never heard a thing. Eventually he got rid of the truck because it was so annoying - to him.
I guess the point I am trying to make is this - People who read CR and follow CR's advice have a tendency to share the opinions of the CR editorial staff. Surveying that group is likely to confirm the opinions of CRs editors.
I enjoy reading Consumer Reports. The OPINIONS of the editorial staff are interesting. Sometimes I agree with them, sometimes I don't. I think they do a good job of collecting hard data (0-60 times, braking distances, interior volume, etc.), but I have a lot less faith in the subscriber opinion survey results.
Even though CR "is not funded in any way by the corporations which produce the items," they still need to generate revenue. To do this they must create a magazine that attracts subscribers and contributors. They have a constituent they need to please just as surely as if they did accept funds from corporations.
Why do you think expressing an opinion is complaining? And why would I be a lot happier if I did not express my opinions? You do not have to agree with me, but I think it is a good thing to listen to and considered the opinions of others. I actually consider the results of the CR opinion survey, but I don't consider it to be hard data. To me, the survey results are just another opinion to be considered.
Ed
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload


<...>
The CR true believers cannot be swayed by actual science and proper methods. It's a faith. What really gets me is that they actually think that a lack of advertising makes CR unbiased. As if bias cannot form without advertising money. They have their opinions and biases, it was pretty clear from the articles I read many years ago.
Anyway it's a question of faith.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
wrote in message

I think Brent has the right idea. I read CR (at the library, sometimes) with some faith and trust in their opinions, but sometimes I sense a feeling of "we'll let this one skate" but "I didn't like the feeling of the seat in that" in their articles. They seem to ignore rather obvious flaws in auto design, (like a plugugly car) and focus on the fact that the cupholders are mounted at an unfortunate angle for left-handed albino dwarfs. Their opinions are a bit too subjective for my liking. I stopped thinking of CR as gospel about 53 years ago, when they compared a Chrysler Windsor with a Hudson, saying the Chrysler's body was a little torquey, and the Hudson was more rigid. They recommended the Hudson over the Chrysler, totally ignoring the fact that at the time, if you bought a Hudson, you must realize that at trade-in time, you'd better buy another Hudson, because your trade-in value at any other car dealer would be nickels and dimes. CR overall is a good guide to avoid junk, but in a lot of cases, by the time their ratings are published, the products are a year or more old, and some are discontinued at retail outlets, and the models that supersede the products may or may not be similar in features and quality.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
"mack"

C. E. White wrote:

Anyone who just uses one source for information on a high-ticket product *deserves* to get ripped off. I think of CR as a good general guide, but it's by no means infallible.

All I can say is I've never bought a product that was recommended by CR, and turned out to be a rip-off. Since it's compiled and published by humans, there is no way it would be perfect, but I still think it's about the best you're going to get for an unbiased opinion, though certainly not *completely* objective.

FIFTY-THREE YEARS ago?! Don't you think they may have honed their methods since then?

With ya on the age of the product, but since they only do the survey once a year, you can't expect that in a more timely fashion. As for their own ratings, I'm sure it takes a lot of time to test so bloody many products. Give them a break, already! LOL
Natalie
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Not really a whole lot. There's still a lot of subjectivism that creeps in, and often a well-known product and highly advertised is not even included in their testing. E.G. - The vacuum cleaner articles usually don't even mention the Oreck models, yet they're a big selling machine with an extensive dealer network. And they only get around to carpet cleaning steam machines about every four years if then .....we bought one recently and I found I had to dig back into the files at the library about that length of time to find their last article on such machines. Yet they'll often review some bullsh*t item like the roomba robotic floor vacuum, which I've never seen anyone buy in a store, in fact most stores don't even carry them. One thing I DO appreciate about CR is the way they alert customers to recalls and faulty merchandise.

That's true, Nat, they test an awful lot of stuff, and a lot of the time I agree with their findings.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
CR does not consider trade in value when rating cars. Nor do they consider ugliness.
wrote in message

Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
wrote in message

Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
They report on depreciation but it is not a factor in their ratings.
wrote in message

Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

You've said that before and you're wrong. It is NOT "clearly an opinion survey." It is a survey which asks SOME questions which elicit answers based on opinions, some questions which require answers based on factual numbers, and some which require answers based on a combination of facts and judgments. To say it is "clearly an opinion survey" shows your own strong bias against their surveys: you're focusing ONLY on the part of the survey that is about opinion and completely ignoring the part which is about facts.
For example, the questions "Was any work done on this vehicle in the past 12 months (excluding tire replacement or accident damage) covered by a warranty or by a recall notice?" and "Did this vehicle get any MAINTENANCE (e.g., oil change/lube/tire rotation) or REPAIRS in the past 12 months?" are not asking for any opinion at all.
Personally, I don't find any use for the opinion portions "are you satisfied/would you buy again" etc. I look at the reliability data based on repair reports. That's the part of the poll to which I refer from this point on.
o> f a specific group who's opinions are influenced by the group

I agree that would be better because of the hard cost data. Since you seem to be implying that CR readers would answer poll questions favorably towards some cars and unfavorable towards others in a consistent enough way to skew the results, it would be better to have a random set of car owners, not because it makes sense that CR readers as a whole would be biased toward or against any particular nation's cars but so that people like you can't say the test is skewed because the owners who were polled all read the same magazine.

`Your opinion means nothing. My opinion means nothing. When you take hundreds of opinions and average them out, you have something. For example, you take a 1.5 pound weight, and ask one person how heavy it is, he might get it right, but he could be way off. However if you ask hundreds of people how much it weighs and average out the answers, you'll have an answer that is remarkably close to 1.5 pounds. Unfortunately I can't remember what book I read about this, so I can't provide any hard evidence. If it makes sense you can accept it, if not then disregard. Naturally, a larger sample would provide more reliable data. That's the biggest caveat I have about the CR surveys--the samples are far too small.

I have three questions.
1. On what evidence is this belief based? 2. How would the survey takers know the opinions about the reliability of a given auto that the CR editors have? As far as I've seen, the CR editors don't state their opinions about the reliability of autos--they simply publish the survey results and let them to the talking. 3. Even if the people answering the surveys knew the opinions of the CR editors and tended to share them, how would that affect their poll answers? Is it like they'd think, "Well, my engine in mt new Toyota fell out, but the CR editors think Toyotas are reliable, therefore it must not have been a serious problem, so I won't report it on the poll." Is that what you're suggesting? Again, I'm referring to the reliability reports, NOT the owner satisfaction part of the poll.

I never read the opinion pieces, unless you consider the reviews opinion pieces. I don't necessarily like what they like, but the information I've gotten from them has always proven helpful. If they say orange juice A is sweeter than orange juice B and therefore tastes better, I ignore the part OJ A being better and buy OJ B if I like my OJ less sweet.
Every time I've read one of their reviews on a product which I already owned or later purchased, they had the facts straight even if I disagreed with their conclusions--and the facts are what I pay attention to.

If it were ever revealed that anyone at CR was skewing their ratings towards what they thought would please their readers and in the process skewing the ratings away from the truth, how long do you think it would take for them to go out of business? There'd be no reason to buy their magazine or send them money for anything after that.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
The opinion questions have some value. For example Consumer Reports discovered that people loved their mini's and would buy them again even though they are unreliable cars.

Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Yep. I'm not saying they don't have value to anyone, only that they don't have value to me.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

And you're ignoring the facts that a) the sample group is entirely self-selected, and b) even if the sample of CR subscribers were done randomly, it's still not statistically significant because of the limited (and clearly biased) sample population.
Brent is right - the results are no more meaningful than an opinion survey.
--
Drive right. Pass left.

Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
said in rec.autos.driving:

I'm not ignoring that at all. It isn't relavent to the fact that this isn't simply an opinion poll.
> and b) even if the sample of CR subscribers were done

It is statistically significant because the results are consistant enough to prove it. You failed to show any "clear bias" on the part of the poll-takers.

Brent didn't say that. He said "It [sic] clearly an opinion survey." Even if he had said that, the results show the statement to be untrue.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

I didn't say much of anything in this thread and did not even mention the freaking survey, both of you are confused.
What I did post on was the religious following of CR where the faithful just ignore the obvious flaws in method and buy into the nonsense that people at CR have some god-like lack of bias.
CR at best is like a testing department run amuck that doesn't have to correlate its tests to actual use. At worst it's about as good as usenet in gathering reliablity information.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
said in rec.autos.driving:

I apologize for attributing to you a quote that you didn't make.

I'm sure there are people like that. Obviously they are wrong--everyone is biased. CR reviews are like any other. It doesn't matter whether or not the reviewer will like the product being reviewed. What matters is whether or not the person reading the review gets an accurate picture of how they'll feel about the product.
CR can rave about a product all they like, but if it doesn't excel at the things you want it to excel at and in the correct way, you won't be happy with it. I'm sure you know this, but I'm hoping that if any of the CR faithful are reading this it might switch on a light bulb so to speak.
I like CR because it gives me information which I find useful in helping me decide which product to buy. For example, I bought a certain camera not because they recommended it but because it had the features I wanted, such as low shutter lag (which they measured to the tenth of a second at .3 seconds, and as far as I can tell that's accurate.) It saved me the trouble of having to go to several different stores and asking them to unchain each of the demo models, put batteries in, supply it with a memory card if necessary, and let me take some photos. That would have taken hours at least, and I would have had to deal with salespeople, which I often find frustrating and sometimes infuriating. Some stores probably have a "you-cram-it-up-someone's-ass, you-bought-it" policy, and I don't want a camera that's been up anyone's ass, even if I'm the one who put it there.

Well, every time I've used their info, I've found it to be accurate. However, I've only used their info about probably 30 times or so, and they have done thousands and thousands of reviews--so maybe I've been lucky with the thirty I used.
I don't trust CR implicitly, though. If they start providing information that I find to be incorrect, I won't hesitate to let as many people know about it as I can. I will give *specific* examples that they can look up for themselves, though, rather than making vague statements about "the religious following of CR" because I don't want to look like a fanatic member of the anti-CR religion.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

CR comes up with on the fly test criteria that fail products they think should fail and passes products they think should pass.

Considering CR's testing, there is no way of making such a determination.

There's this thing called the web now where one can find exacting specifications of devices like cameras. CR does list out anywhere near the detail that can often be found online.

I've been in product design a fair number of years of now. Lab tests are well, lab tests. Currently I am dealing with issues on a couple products that do not occur in the field, only in lab tests. No relationship between these lab tests and field failures has ever been established, it's just 'gut-feel' arbitary tests. And in the end that's what CR does as well.
You can have a product that internally a manufacturer put through all sorts of arbitary abusive testing and it passed only to have CR come up with something different and break it immediately. They are just arbitary tests, neither better than the other.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
That Guy

Please provide specific examples, otherwise you'll start to look like one of the anti-CR religious nuts.

Sarcasm? That doesn't lend weight to your argument--it just makes me think you're emotionally invested in it, which is a good sign that someone has abandoned objectivity and is just looking for reasons to justify their belief. You can counteract this easily by providing evidence for all of the claims you are making against CR. It shouldn't be too difficult, libraries often have issues going back for years.

If you mean specs provided by manufacturers, they often fudge by using different definitions for things, like the CRT monitor manufacturers used to claim up to two inches bigger than the actual screen size, and hard drive manufacturers use a different definition of kilobyte, megabyte, gigabyte and terabyte than everyone else in the computer industry to make their hard drives seem bigger than they really are.
If you mean other (non-manufacturer) product testing sites, I've seen a few. I noticed that most of them were supported by ads. The ones without ads I trusted even less, because who knows where their money is coming from.

And you're saying CR does that? Do you read CR? Because they don't do reliability testing for the most part... and that's ALL you've mentioned here. Oh, and can you provide specific examples so you can back up your claim?
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

I haven't picked up a CR in over a decade. The religion is CR, as your rabid response serves to indicate. Believe what you want, however CR does not correlate their tests to actual field failures or even performance, thusly they are simply arbitary on their face.

Nice projection. I could care less about CR. I dismissed their methods as worthless long before I ever saw a thread on usenet about them.

My aren't you testy... Guess you have to lash out at a CR heretic like me.

reliability, performance whatever you want to call it. Same difference. They come up with a test and a standard of 'pass' for said test. If the device broke in the process it fails.
Apparently you feel I've attacked your belief system, however the tests that CR does are just the tests they feel judge reliability (or as some would call it, performance) under certain arbitary conditions. These may or may not have any bearing to the real world. Manufacturers, or at any worth a damn have their own internal testing which may be much better or just as flawed as CR. However the engineers must design for the internal tests, not CRs.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
That Guy

My "rabid response"--ROFL! If you can't beat them with reason, attack them personally, is that it? Who's the rabid one here? I ask for some evidence backing up your claim and that's a "rabid response"--wow. I'd hate to see what happened if I actually got emotional with you about this.

I believe what the evidence shows, not what someone on the internet says is true, especially when they start hiding behind insults when politely asked to provide evidence supporting their claims. You expect someone to just accept your claims without any evidence? If so, maybe you should examine that expectation, because I've found that most people won't accept any claim that goes against what they believe even with a mountain of evidence. You're in for an enormous amount of frustration.
Personally, I'm not emotionally invested in CR, so I can look at your claims objectively. C. E. White is making similar claims, and through evidence and solid reasoning, he's convinced me that CR's reliability data is misleading and not particularly helpful. I'm not convinced it's completely worthless, but I would no longer change by car-buying decision (all other things being equal) for a percent difference of at least sixty to eighty percentage points, and I wouldn't expect the frequency of repairs on two compared models to be substantially different unless the spread was more than a hundred twenty to a hundred forty points. Before, I had considered anything over five to ten points significant--because I didn't know as much about the testing methods and display methods. I think that speaks to whether or not I'm a CR fanatic. I'm not one to go making wild claims and then when politely requested to supply evidence supporting those claims respond with senseless insults. That's one mark of a fanatic.

The statement "I could care less" means you care. I'm going to assume you meant "I couldn't care less" which means you don't care. If that's what you're trying to say, it couldn't be more obviously untrue. You've spent a great deal of time criticizing them online. If you didn't care, you wouldn't be trying so hard to shoot them down.

And what does that prove?

"Testy"? "lash out"? ROFL!!! That's enough, you're obviously delusional. There's nothing in the preceeding two paragraphs that would remotely be considered "testy" or "lashing out" by any stretch of the imagination. I hope you get better soon. Bye.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload


Let's see you call me an anti-CR zealot and I say your response is rabid. Think about that for a moment. Now who's making the personal attack? That would be you.

Your post consisted of quite a bit more, but you trimmed it from the quoted material. I am in no mood for these childish usenet games.


Let's see, you opened up with insulting me, calling me an anti-CR zealot, etc and so forth. Pot-kettle-black.

I don't expect anything. I could go through an issue of CR and say how their tests aren't correlated to the real world,etc and so forth and it would be a pointless excerise because you'd just go to the next level of denial... I've done this dance before and don't care enough about CR to spend the time *AGAIN* on it. Yes, this is an old topic, and in years past I went through the detail as did others.

At least you admit you're a religous CR zealot. So, that means you haven't been insulted in any way.

The way you lashed out at me indicates otherwise.
I am sure the lloyd parker CR threads of the mid-late 1990s are in the google archive if you want to do the research and get up to speed. With each passing day, usenet re-runs become less and less interesting.
Buh-bye.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Related Threads

    Motorsforum.com is a website by car enthusiasts for car enthusiasts. It is not affiliated with any of the car or spare part manufacturers or car dealers discussed here. All logos and trade names are the property of their respective owners.