LumberJocks

A convention for the "stars" on the review

  • Advertise with us

« back to Power Tools, Hardware and Accessories forum

Forum topic by b2rtch posted 07-28-2012 03:37 PM 1011 views 0 times favorited 13 replies Add to Favorites Watch
View b2rtch's profile

b2rtch

4351 posts in 1803 days


07-28-2012 03:37 PM

Topic tags/keywords: question

To make the reviews more useful for everyone, we need to speak the same language.
Could we “officially” establish a convention on how to use the “stars” when writing a review?
I used to put five starts for any product that meet my expectation but I read a while ago on this website that this is not the right way.

The way I understand the start system now is:

#1 star= lousy product, do not meet expectation at all.

#2 stars=no very good but somewhat usable.

#3 stars= meet expectation but not more

#4 stars= better than expected.

#5 stars= much better than expected. exceptional product.

Is this OK with everyone?

-- Bert


13 replies so far

View ShaneA's profile (online now)

ShaneA

5453 posts in 1353 days


#1 posted 07-28-2012 04:08 PM

Not to be a downer, but expectations really shouldnt factor into a rating system, should they? My logic being, that if I expect the “perfecto tool” to be perfect, and it is….that would be 3 stars. Seems if it was perfect, no matter what I expect, it should be 5 stars. I guess, I would just like as much info included in the review as possible. Give the reader as much detail as possible. I dont put too much into the stars any how. Just my opinion. I agree that it can be improved upon, but it will be tough.

View poopiekat's profile

poopiekat

3748 posts in 2489 days


#2 posted 07-28-2012 04:12 PM

What if somebody gives a right-outta-the-box 5-star rating to a tool they’ve never operated in their life? Like a HF lathe, for example?

-- Einstein: "The intuitive mind is a sacred gift, and the rational mind is a faithful servant. We have created a society that honors the servant and has forgotten the gift." I'm Poopiekat!!

View lumberjoe's profile

lumberjoe

2847 posts in 1003 days


#3 posted 07-28-2012 05:48 PM

I put way more stock in the meat of the review, not the star rating. You can tell if someone is just in love with it because they simply own it, or if they have actually used it and basing their review on actual use and past experience with other products. When I review something I like to be as detailed as possible and “show my work”. I appreciate it when others do also

-- www.etsy.com/shop/KandJWoodCrafts

View WhoMe's profile

WhoMe

1127 posts in 1998 days


#4 posted 07-28-2012 06:34 PM

I think having a general convention/guideline is a good idea. Like Lumberjoe, I put more stock into the meat of the review but I do look at the number of stars as a “interest point” to see what a reviewer rated something. And, I do feel expectations do have a bearing on the review but it should be minor as that is something that is more emotional. Obviously, if you are buying a top quality product, expectations should be high for a good reason so it would have a bearing on the review but it should be backed up by FACTS and usage of the product.

Like when I go onto Amazon. I almost never look at 1 star or 5 star ratings. To me, the 1 star ratings are based on someone who bought something with no clue in what they were buying or it was a factory defect and the reviewer has not bothered to return it for another and give it a meaningful rating. In that light, to me a 5 star rating means that someone is a ringer (paid to give it 5 stars to feed the purchases) or they bought it and did a “out of the box” review without actually using it.
In my opinion, the most meaningful reviews are the 2 to 4 star reviews. These are almost always more thoughtfully written by someone that has used the product for a while and actually thought about writing the review with pros, cons and any issues they have had.

So here are my guidelines on how I rate things after I have used them several times or more: #1 star= Absolutely no redeeming qualities, pure junk even if I knew it was a cheap tool – has too many factory flaws to be a one off item. —I have never given a item this as I do my research and buy a better product anyway #2 stars= Known lower/bottom quality item. Many flaws in workmanship/ design/ usability. Way more cons than pros. Does not perform as expected even though it was known to be a lesser quality item. Also if item was a better quality item but had all of the above qualities. #3 stars= meet expectation but not more – This would again be based on the price paid/quality and usability of the item reviewed. The item would have as many pros and cons and it basically worked for it’s intended purpose. #4 stars= This was a known better quality product with a good reputation. Has mostly positive usability, quality and price point. Maybe 1 or 2 cons but minor ones at that. This would be a tool that I bought to be used on a frequent basis and it was a better quality manufacturer. #5 stars= I have not given one of these yet. This would be a high quality product with maybe 1 con and a extremely minor one at that. After a month or 2 of use, the product has performed flawlessly and there are no issues with usability.

Anyway, that is how I do it.

-- I'm not clumsy.. It's just the floor hates me, the tables and chairs are bullies and the wall gets in the way.. - Mike -

View bondogaposis's profile

bondogaposis

2765 posts in 1105 days


#5 posted 07-28-2012 07:54 PM

I agree w/ Shane, expectations should not be part of it. If I rate a HF sander and expect poor performance, but does better, then I give it 4 stars. If I then rate a Festool sander and expect to perform excellently and it does, I only give 3 stars? That really muddies things don’t you think?

-- Bondo Gaposis

View b2rtch's profile

b2rtch

4351 posts in 1803 days


#6 posted 07-28-2012 08:27 PM

bondogaposis, not really . 3 stars= as expected. 4stars: better than expected.

-- Bert

View Tedstor's profile

Tedstor

1507 posts in 1387 days


#7 posted 07-28-2012 09:21 PM

Better yet, do away with the star rating all together. That would compel people to actually read the review rather than judge a product based on an arbitrary rating system.

View MonteCristo's profile

MonteCristo

2098 posts in 943 days


#8 posted 07-29-2012 03:44 AM

I think Tedstor is on to something !

-- Dwight - "Free legal advice available - contact Dewey, Cheetam & Howe""

View justahobby's profile

justahobby

19 posts in 1332 days


#9 posted 07-29-2012 09:57 AM

A consistent point of reference has always been the problem in reviews. I fall more or less with WhoMe:

1 – this product should not even have been manufatured
2 – you can make it work correctly if you want to put the time in, not comfortable to use out of box
3 – decent quality / product value (for price?)
4 – this is worth buying (functionally/ task performance)
5 – stands out among field of peers, exceptional quality, all the details thought of

Although price is often a bracketing (not really defining, but that is a semantic argument for another time) factor, I find it difficult to bring price into the mix, as I know my values do not necessarily match those of others. I feel reviews should always include price paid so the reivew and any rating can focus on how the product functions and affects the user’s work flow (or just plain enjoyment of the task, for us hobbiests). In this framework, a product is rated and reviewed on function and quality, and each individual reader can then assign ‘value’ to the product based on price and translating the reviewers experience to what is likely to be their own with the product. I find when I look at something in terms of ‘expectations’, I bring price into the mix before I have even used the product, and this is too personal for doing a review.

Maybe reviewers could only write the copy, other readers assign a ‘star’ rating based on what they interpret from the reivew: sort of a ‘meta’ rating or average based on multiple readings of each reivew. I’ve never seen this done before, and it wouldn’t be without ways to game the system, so it would require coding. Or maybe we could just use multiple rating categories: quality, comfort/ fit and finish, price/ value, ease of use, et cetera.

View Craftsman on the lake's profile

Craftsman on the lake

2420 posts in 2192 days


#10 posted 07-29-2012 10:43 AM

Like in Education, the only way to have a rating system is to have standards. So, one needs to define what one-five stars is. The only problem with this is that the stuff we review is diverse. If we only reviewed routers then we could develop standards for routers and use those to give them ratings. i.e. amount of play in the router shaft, ease of adjustment, fit and finish. And then again even with standards developed for these things the standards need to be fairly detailed to be used.

In lieu of that, the narrative is the only way to successfully review a diverse bunch of items. Although not perfect at least one can get a feel for how the person writes. You can feel if the person may be happy with the crappy router because it’s his first one and doesn’t know the difference. You can tell if he/she is writing a review with useful data gathered while using it or just giving an off the cuff opinion. You can sometimes see pictures of issues or successes or video showing the unit in use and how it performs. If 20 people review the same item, then you can choose which ones seem to be doing it right and go from there.

That said, we’ve got a pretty good narrative system that lets us be the judge of the judger. If we have the stars at the top, well, some like it some don’t but it doesn’t hurt anyone. If that was all there was then it would be pretty much worthless.

-- The smell of wood, coffee in the cup, the wife let's me do my thing, the lake is peaceful.

View DamnYankee's profile

DamnYankee

3240 posts in 1316 days


#11 posted 07-29-2012 10:56 AM

I think b2rtch and whome are essentially saying the same thing.

For me there are a couple of factors at work, some of which will NEVER be satisfactory to every LJ due to the wide range of experience and abilities. – price is a factor in ratings, or at least in terms of price-value. That said what I think is reasonable price range may not be for others, due to budgets, amount of woodworking they do, type of woodworking, hobby, pro, etc. I buy a $1000 tablesaw, that is doable but on the limits of my budget as I am not much more than an exagerated hobbiest. To my fellow LJs this might be in the “junk” price range, to others far outside their means. Additionally, let say I did end up with the “mac daddy undeniable best in the land” table saw (say I won it) and the retail price was $10,000, given my capabilities or needs as a hobbiest I might not really notice (or care) a $9,000 increase in perfection. I’ll take this to the other extreme. You can buy a 6’ tape measure for 99cents or you can spend $20. Is there really a price-value differenct of $19? Yes, no, maybe, depends. All of those. Depends on purpose for which it was purchased. If I am measuring for rough dimension, no there is nor value added. If I need the absolute precision the $20 provides then maybe. I would suspect though that for most of us $20 is overkill and is not worth the extra $19. That said for most of we probably want say the $10 tape for its addd durability, accuracy, etc and a more reasonable price.
-Capabilities and experiences. This was mentioned above but it goes beyond price. I got into woodworking from DIY to carpentry to woodworking. So most of my background was in circular saws, bench saws and hand drills. So for me to rate my firstever experienced/used planer probably would not mean much. Now, that said, I am not stupid, so I can probably recognize poor junk soon after I put it to work, I can probably tell you if the set up manual was useful, and hopefully I can tell if the tool appears to do what it was intended to do (this assumes I knew what a planer does before I purchased it). However that limited (non-existent) experience probably wouldn’t mean much to the owner of multiple planers.

This kind of reminds me when a while ago someone asked me what the difference between carpentry and woodworking was, and I answered the difference between a 1/16” and .0001”. A matter of precision. Doing carpentry (such as when I framed out my screened in porch) 1/16” was nothing and so a miter saw that was not very accurate was not a major issue. Whereas when I make boxes with mitered corners its EXTREMELY important and so is the tool. If all you have known is carpentry then your expetations of a miter saw is different than if all you know is woodworking. If you do both you can see value at both ends. Doing carpentry on a job site I want relative accuracy but almost more import in durability of the tool. Doing woodworking in my shop durability is not likely as much of an issue where as repeatability is.

So I can see where an HF tool bought for $20 could end up with the same rating as a Festool for $300 provided the reviewer had a reasonable set of expectations between the two. And really that to me is the problem when I read reviews, a lack of reasonable expectation. A lack of reasonable expectations is often tied to a lack of experience/knowledge about the tool they bought.

-- Shameless - Winner of two Stumpy Nubs Awards

View b2rtch's profile

b2rtch

4351 posts in 1803 days


#12 posted 07-29-2012 11:33 AM

Thank you all for your replies

-- Bert

View knotscott's profile

knotscott

5610 posts in 2130 days


#13 posted 07-29-2012 12:04 PM

There’s never going to be a universally accepted rating system because it’s subjective at best, and we’re all coming from different directions on what’s good, so we define that term differently ….. kind of like asking what’s the best color? I remember being pretty darn happy with my first Delta 36-600 compact saw with a universal motor. With 11 years experience behind me and exposure to a lot more saws, I have far more to compare it to and am harder to please. Just one of the many reasons I read past the “star” rating, and don’t put too much temphasis on anyone’s reviews (including magazines). Some reviewers are more objective and credible that others, so I also evaluate the source while I’m processing their evaluation of the tool.

I don’t know how you could not be influenced somewhat by expectations, which are heavily influenced by price. I’d be a lot happier with an $18 Delta Industrial 40T saw blade that performed well than I would be with a $110 Forrest that performed comparably….high performance it’s simply expected with a premium product, it’s a plus with a bargain product, and I would have settled for “acceptable”....not so with the Forrest.

To me, price being factored in is a given. Unavoidable. I have near zero tolerance for a mediocre version of a premium tool with a premium price tag (just think of all the posts from really ticked off Laguna customers who paid top dollar only to receive common quality Asian imports, then got lousy customer service to boot….similar deal… their expectations were simply higher and they thought they were buying better because of how Laguna markets themselves). I’m much more forgiving on something that was a bargain, but I make a point of including that information in my comments during a review. It’s very frustrating to pay more and not get more.

-- Happiness is like wetting your pants...everyone can see it, but only you can feel the warmth....

Have your say...

You must be signed in to reply.

DISCLAIMER: Any posts on LJ are posted by individuals acting in their own right and do not necessarily reflect the views of LJ. LJ will not be held liable for the actions of any user.

Latest Projects | Latest Blog Entries | Latest Forum Topics

HomeRefurbers.com

Latest Projects | Latest Blog Entries | Latest Forum Topics

GardenTenders.com :: gardening showcase