Best Developer Assessment Tools - November 2019
The best tech assessment tools, pricing guidelines, benefits, pitfalls, and more
If you’re looking to hire more quality software engineers faster, then you are probably researching the various coding interview platforms.
These tools allow recruiting and engineering teams to effectively screen more applicants in an unbiased way that allows for the right candidates with the right skills to get the job.
Below is our research into this space - everything you need to know before deciding on whether or not to move forward with buying a coding interview solution, along with suggested partners that we’ve personally vetted. Good luck and let us know if we’re missing anything!
We are a reader-supported site and may earn a fee when you click on vendor links on our site. Learn More
Qualified allows your team to effectively assess technical talent while saving your recruiting and engineering team time. The platform was developed by the creators of CodeWars, a community where over 1 million engineers sharpen their skills. They’re focused on unit-testing and test driven development to design assessments that mirror actual programming work as much as possible. Their customers include Apple, GE and UC Berkeley.
Worth Checking Out: This is a great platform if you want to accurately screen engineers with real world problems. Learn More About Qualified
CodinGame make the evaluation and shortlisting of programmers simple and fast. By challenging developers on real-life problems, they help recruiters and CTOs reduce hiring costs with reliable results on candidates' skills. Clients include Nasdaq, Thales and Ubisoft.
Worth checking out: You’ll soon be able to boost candidate experience and gamify your CodinGame Assessment campaigns with innovative game-based programming exercises. Learn More About CodinGame
Codility is laser focused on helping the right people get the right jobs based on skills, not background. One of the things we like the most about this platform is the positive feedback around the accuracy of assessment from developers who’ve taken the tests.
Worth Checking Out: Codility has put together a comprehensive developer demographic report full of data to help you recruit engineers.
CodeSignal’s developer vetting process aspires to be as close to the real world job as possible to get to the heart of a person’s skills and aptitude. The reality is that many qualified applicants have unhelpful resumes, and aren’t good at interviewing. But, these people may be great for the job. If you want to get nerdy, CodeSignal has published several in depth papers on how their vetting algorithms work.
Worth Checking Out: CodeSignal has an automated proctoring feature that allows for scaled supervision during interviews.
HackerRank is definitely a leader in this market with customers like AirBnB, Dropbox, and Cisco. They have a free trial so you can get a feel for how their platform works with your hiring workflows.
Byteboard was incubated inside of Google's Area 120. Their solution enables companies to assess engineering candidates through real world engineering exercises vs more academic constructs like algorithms and data structures.
Worth Checking out: Interviews are blind and paired with structured rubrics to limit bias in evaluation.
TripleByte gives companies access to the top 3% of developers who take their assessment via a marketplace of vetted tech talent. This solution can help SMBs through enterprise with their tech recruiting.
Worth Checking Out: TripleByte also licenses their assessment tool to companies who want to run their own process vs using their talent marketplace.
Benefits of Tech Recruiting Software
There are several key advantages that talent acquisition teams can reap from using software to vet engineering candidates. Like any pre-employment test, online coding tests are meant to test an applicant’s likelihood of succeeding in a potential role. However, there are specific benefits to this segment of online assessment platforms.
Engineer Time: First off, your engineering teams don’t have to spend nearly as much time in the earliest parts of the recruiting cycle. This means software engineers can spend more time coding. In many cases, this increase in productivity is similar to a company hiring 10% more engineers.
Better Hires: Your engineering team is no doubt brilliant, but are they the best interviewers? How many job applicants do they unfairly disqualify in the early days simply because their specialty isn’t hiring. Unbiased coding tests allow you to better asses your top job applicants through standardized interview questions that test skills in the programming languages you care the most about.
Candidate Experience: Because your software engineer screens are now automated, you can evaluate a larger number of applicants. This allows you to give more people the opportunity to go through the screen a show their coding skills beyond what is on their resume.
Diversity Hiring: An engineering assessment tool evaluates skills objectively and isn’t biased by gender, ethnicity, age, nor educational pedigree. This means the people who have the skills to do the job will make it to the next phase.
More Signal: Generally speaking a resume is a poor proxy for how strong a developer is. Coding tests allow employers to expand their pipeline beyond people who simply look good on paper.
Easy to Get Started: Most of these vendors offer free trials to get started. This means you can quickly understand the workflow for a given tool, and how it can impact your overall hiring process. Our advice is to have someone from recruiting and engineering involved during any pilot or trial phase as both organizations need to buy into this initiative.
Companies Using Tech Assessment Platforms: Andela
Andela is a fast growing company faced with a massive challenge - how do we efficiently find and hire the top 1% of developers across Africa at scale (they get over 4,000 applicants/mo!).
Maya Neria, Senior Product Manager, was in charge of figuring out how to replace a home grown tech assessment platform that wasn’t cutting it with a solution that would mimic the real world while also providing feedback to candidates. After demo’ing over 100 solutions, Andela selected a tech assessment platform, Qualified, and moved forward with overhauling their developer screening process.
Common Pitfalls and Misconceptions about Online Coding Interview Platforms
We strongly believe that online coding tests are a great way to streamline your engineering hiring while producing a meaningful return on investment. That said, there are several misconceptions and pitfalls that we see from the talent acquisition community that we want you to be aware of.
Screening, not interviewing: Engineer interview tools are great at screening candidates. However, they are not a replacement for the actual interview. You still need to understand how someone deals with conflict, compromise, communication, etc before hiring them. Coding tools are merely the first step.
Diversity: Make sure you understand how diversity candidates are scoring on the tests you are administering. One vendor in this space found their tests were biased against female engineers simply because no females were originally involved in writing the tests. Getting women involved in building the question set fixed this problem.
Engineer buy in: Your engineering team needs to trust in these assessments. If they don’t, this solution provides very little value as they’ll still need to spend time doing their own screens. Educating the engineering team typically happens from a leader on the recruiting side. Of course, there are hiring managers spearheading this change as well. To ensure a successful buying and implementation of the product, get engineering involved when you are demo’ing/trialing the product.
Fraud: Understand how the screening tool you’re using ensures that candidates are taking the tests themselves and not posting fraudulent scores.
What’s tested matters: You have the ability to customize your coding tests in a way that reflects the specific programming languages and skills relevant to the job. However, what idiosyncrasies are still being left out? Maybe you only pair program, and need to test this. Maybe there should be a psychometric part of this test, or specific aptitudes that need to be measured in addition to skills.
Tool, not marketplace: These tools enable your recruiting to team to vet candidates more effectively. But, they are not sources of candidates like a job board would be (although some of the best tech assessment vendors can help with sourcing).
Don’t leave humans out: Just because you have a new way to vet software engineers doesn’t mean you can simply let your coding assessment handle all candidate interactions. Like any pre-employment screening, you need to realize candidate experience is paramount. Set expectations, sell the person on your culture, and overall just treat them as you’d want to be as an applicant.
Coding Interview Software Pricing
Tech recruiting platform pricing is typically based on the number of interviews you’ll be conducting on a monthly basis. We like this because the more interviews you conduct, the more value you’re going to get.
Pricing starts in the $50/mo range for companies who are just getting started with one req, and can grow into the 6 figure range pretty easily. Here is a chart to give you a general sense for coding test platform pricing:
|Company Size||Annual Cost|
|1-50||$1,000 - $4,500|
|200 - 1,000||$8,000 - $30,000||1,000 - 5,000||$30,000 - $100,000|
Please note that these are ranges and are meant to give you a general idea of how much you might pay for your organization. However, prices will vary depending on a number of factors including: the features you need, support levels, vendor you partner with, HRTech integrations, etc.
Also note that many of the best online coding assessment platforms have free trials.
Are you looking at multiple tech assessment platforms? We thought you might be! Our recommendation is to use a structured rubric to evaluate these tools so that you can make the right decision for your business after demo’ing a few of the platforms. We put together the below spreadsheet so that you can keep track of your key criteria when looking at these engineer hiring solutions. Our hope is that it allows you to make the best decision for your business!
Download the Tech Interviewing Software Demo Organizer Here
You’ll notice there are a few suggested questions in this spreadsheet already. In addition to talking to a sales person to vet these vendors, we recommend opening up a free account to play around with the code editor. Setup a pre-canned screen for a data scientist or programmer, and see the process they’d go through. Then, take a look at how screen applicants are presented to your hiring team.
How does this stack up to other online assessment platforms you’ve looked at? Does it give you confidence you’ll be able to judge an applicant’s programming skills? What other interview questions can you embed into these skill tests? Make sure you think about this both from the recruiting team’s perspective, and the candidate’s.
Final Advice on Tech Assessment
Any company hiring more than a few engineers a year should take a look at using a tech assessment platform. They save your engineering team time. They lead to better hires. And, most are very simple to do a pilot with. You can get a 30 minute demo of one of our pre-vetted vendors to understand how this will fit into the specifics of your recruiting process:
Qualified’s tech assessment platform is designed to help you efficiently vet engineering talent. In this case, efficiently means less engineer and recruiter time on screening candidates, while getting the insights you need to move the right candidates forward.