//this is the mailchimp popup form

Be Careful of Online Software Reviews

Here are some review strategies that are slightly more novel and specific to software reviews...

Phil Strazzulla
HR Tech Expert, Harvard MBA, Finance and Software Nerd
May 12, 2021
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

A year ago we wrote about how an unknown application tracking system from South Africa was the #2 rated ATS on Capterra.  The TL/DR of that article is that a very small company was able to manipulate their reviews in order to seem like a global player that could compete with the best ATS’s.

Of course, you’ve probably seen the many scams that companies have perpetrated on sites like Amazon, Yelp, Glassdoor, etc to enhance their online reputations and get more customers (or hires) via reviews.

While many of these various strategies are well documented, we ran into one that is slightly more novel and specific to software reviews that we wanted to make our audience aware of.

The $1 million rule

On SSR, we generally don’t review solutions that have less than $1 million of annualized recurring revenues.

After a company has $1 million in revenues, they are much more likely to have a fully developed product, at least decent customer service, a sales team that can handle different types of customers, and they are much more likely to be around in 5 years vs a company with $200k in ARR.

We chose the $1 million cutoff as a VC friend had done an extensive analysis of their portfolio spanning a few decades and hundreds of companies. They found that once a company hit $1 million, they have a MUCH higher chance of making it.

“Fake” reviews on software review sites

A few weeks ago a vendor emailed us asking to be reviewed by our team.  Their basic pitch was that they had over 250 reviews, almost all of which were 5 stars.

I asked how many customers they had, and what their revenues were and was a bit perplexed at the answer.  They had <150 customers, and around $600k in revenues.  Since they weren't at the $1 million marker, we started digging into their reviews to better understand if we were missing something and should feature them on the site.

The first question we had was: how could you have 250 reviews if you have <150 customers?

Well, this company had asked their customers for reviews, but mostly asked the employees who use the software to review it as buyers.  Definitely a smart marketing tactic, but in my opinion a bit devious.

First, the users of the software have a very different perspective vs buyers.  Their "love it!" is relevant to the buyer, but doesn't give the full story.  Secondly and more importantly, if your review data is what you lead with in your pitch, you are marketing yourself in a misleading way.

To give you a sense of how much, Greenhouse Software is rumored to be around $80 million of revenues (133x the revenues of this company!) and has only 2x the number of reviews.  They’ve also been around for 10 years, vs 2 years for this company.

Workday is a $60 billion market cap company and has 730 reviews on Capterra….only 2.9x this $600k revenue company!

If you were to read the 250 reviews, you would start to realize many of them are irrelevant to the HR team buying the software (except for understanding the employee experience - important but just part of the puzzle).

Of course, who has time to read 250 reviews?  What we’re most likely doing as buyers is internalizing that number as a point of social proof to justify a purchase.

And, if your brain has internalized that hundreds of reviews = vendor at scale, you’ve been duped by this inflated review count from a tiny company!

So, just like we should dig a few levels deeper on many vendor claims (and generally buy software like a toddler), having a deeper understanding of what the true signal of a company’s online reviews are is essential to the buying process.

Share this article

Subscribe to weekly updates

Join 20,000 HR Tech Nerds who get our weekly insights
Thanks for signing up, we send our newsletter every Wednesday at 10 AM ET!
Oops! Something went wrong while submitting the form.
Phil Strazzulla

Phil is the founder of SelectSoftware Reviews, a website dedicated to helping HR and Recruiting teams to find and buy the right software through in-depth, expert advice. Phil started his career working in venture capital before getting his MBA from Harvard Business School.

Related posts

Employee Perks & Benefits: 20 Ideas, Types, & Examples

Find out which employee perks and benefits matter most to your current and potential employees.

Buying HR Tech for Startups and SMBs

Looking to improve your startup hiring and retention processes? Read here for top HR tech-buying...

How to motivate employees - 11 protips from Steve Jobs

Learn how to motivate employees using eleven tips from Steve Jobs