Best Developer Assessment Tools - September 2020
The best tech assessment tools, pricing guidelines, benefits, pitfalls, and more
If you’re looking to hire more quality software engineers faster, then you are probably researching the various online coding interview platforms.
These tools allow recruiting and engineering teams to effectively screen more applicants in an unbiased way that allows for the right candidates with the right skills to get the job.
Additionally, online coding assessments (along with video interviews) have become more popular in the current COVID-19 pandemic where in person interviews are becoming necessary with most offices closed.
Below is our research into this space - everything you need to know before deciding on whether or not to move forward with buying a coding interview solution, along with suggested partners that we’ve personally vetted. Good luck and let us know if we’re missing anything!
We spend hundreds of hours researching the best HR and recruiting software so you don’t have to. We never take money from vendors during our research phase and rely heavily on practitioners and experts to help us recommend the right software. Most of the vendors featured on our site have no commercial relationship with us, we want to feature the best software for our readers regardless. Learn more
We put in the effort to make this guide awesome
28
Hours Researched
.svg)
7
Experts Consulted
.svg)
16
Vendors Reviewed
Featured in
Top Tech Assessment Platforms
We track thousands of HRTech solutions, these are the best online coding interview tool per our research and expert council as of September 2020
CodinGame make the evaluation and shortlisting of programmers simple and fast. By challenging developers on real-life problems, they help recruiters and CTOs reduce hiring costs with reliable results on candidates' skills. Clients include Nasdaq, Thales and Ubisoft.
Worth Checking Out:
You’ll soon be able to boost candidate experience and gamify your CodinGame Assessment campaigns with innovative game-based programming exercises.
Filtered.ai allows companies to implement a skills based hiring process for their technical hires through in depth virtual assessments. Their solution has more signal than simply screening resumes, sophisticated anti-fraud detection, and specializes in testing more experienced candidates focused on architecting systems vs simply writing code. Marquee customers include Lyft, Cigna, and Pinterest.
Worth Checking Out:
Filtered thinks about the candidate experience, not just the efficiency of the recruiting process, and has an average candidate rating of 4.75/5.0.
Qualified allows your team to effectively assess technical talent while saving your recruiting and engineering team time. The platform was developed by the creators of CodeWars, a community where over 1 million engineers sharpen their skills. They’re focused on unit-testing and test driven development to design assessments that mirror actual programming work as much as possible. Their customers include Apple, GE and UC Berkeley.
Worth Checking Out:
This is a great platform if you want to accurately screen engineers with real world problems.
HackerRank allows companies to deeply understand the skill sets of potential engineer hires through both automated code screens and live programming exercises. Their 2,000 customers include small engineering teams all the way up through Stripe, Goldman Sachs and LinkedIn. From data scientists of full stack engineers, HackerRank can help you identify the best talent, set a consistent hiring bar, and generally improve your recruiting process.
Worth Checking Out:
If you're a developer, you can join a community to 11 million software engineers to sharpen your skills.
TripleByte gives companies access to the top 3% of developers who take their assessment via a marketplace of vetted tech talent. This solution can help SMBs through enterprise with their tech recruiting.
Worth Checking Out:
TripleByte also licenses their assessment tool to companies who want to run their own process vs using their talent marketplace.
CodeSignal’s developer vetting process aspires to be as close to the real world job as possible to get to the heart of a person’s skills and aptitude. The reality is that many qualified applicants have unhelpful resumes, and aren’t good at interviewing. But, these people may be great for the job. If you want to get nerdy, CodeSignal has published several in depth papers on how their vetting algorithms work.
Worth Checking Out:
CodeSignal has an automated proctoring feature that allows for scaled supervision during interviews.
Codility is laser focused on helping the right people get the right jobs based on skills, not background. One of the things we like the most about this platform is the positive feedback around the accuracy of assessment from developers who’ve taken the tests.
Worth Checking Out:
Codility has put together a comprehensive developer demographic report full of data to help you recruit engineers.
Byteboard was incubated inside of Google's Area 120. Their solution enables companies to assess engineering candidates through real world engineering exercises vs more academic constructs like algorithms and data structures.
Worth Checking Out:
Interviews are blind and paired with structured rubrics to limit bias in evaluation.
CoderPad allows technical teams to replace whiteboard interviews (or even worse, over the phone interviews where candidates are expected to program in a text editor that won't even run their code) with a live programming environment that can facilitate both in person and virtual interviews. Their solution lets applicants work in real time on a problem in an IDE where they can test their solutions and collaborate with your team - see their skills, style and smarts as they solve real problems. Customers include Lyft, AirBnB, Dropbox and many more.
Worth Checking Out:
This solution is great for high-signal technical interviews, and interviews with more senior engineers. In addition to the IDE, you can interact over audio and video.
Buying HR Tools? Study Up First
Buying HR Tech is really hard, here's some tactical advice to help you save money and buy the right solution:
- Use these best practices to save time and money
- Make your purchase a success by getting internal buy-in
- Calculate ROI for your new purchase and get the budget you need
- Checkout the best vendors in other HR software categories
Want to get wicked smart and stay up to date on the latest in HR Tech? Sign up for our weekly newsletter!
Benefits of Tech Recruiting Software
There are several key advantages that talent acquisition teams can reap from using software to vet engineering candidates. Some people ask, what is a tech assessment? A technical assessment is a coding test that mirrors the real world work of a computer engineer at a company and asks the job applicant to complete a series of coding exercises in a developer environment that tests their skills as a programmer. Like any pre-employment evaluation tool, it is meant to test an applicant’s likelihood of succeeding in a potential role. However, there are specific benefits to this segment of online assessment platforms.
- Remote Interviews: If you’re looking to hire remote engineers, or cannot meet in person due to the restraints of COVID-19, virtual tech assessments are a great way to continue moving the interview process forward, along with video interviews.
- Engineer Time: First off, your engineering teams don’t have to spend nearly as much time in the earliest parts of the recruiting cycle. This means software engineers can spend more time coding. In many cases, this increase in productivity is similar to a company hiring 10% more engineers.
- Better Hires: Your engineering team is no doubt brilliant, but are they the best interviewers? How many job applicants do they unfairly disqualify in the early days simply because their specialty isn’t hiring. Unbiased coding tests allow you to better asses your top job applicants through standardized interview questions that test skills in the programming languages you care the most about.
- Candidate Experience: Because your software engineer screens are now automated, you can evaluate a larger number of applicants. This allows you to give more people the opportunity to go through the screen a show their coding skills beyond what is on their resume.
- Diversity Hiring: An engineering assessment tool evaluates skills objectively and isn’t biased by gender, ethnicity, age, nor educational pedigree. This means the people who have the skills to do the job will make it to the next phase.
- More Signal: Generally speaking a resume is a poor proxy for how strong a developer is. Coding tests allow employers to expand their pipeline beyond people who simply look good on paper.
- Easy to Get Started: Most of these vendors offer free trials to get started. This means you can quickly understand the workflow for a given tool, and how it can impact your overall hiring process. Our advice is to have someone from recruiting and engineering involved during any pilot or trial phase as both organizations need to buy into this initiative.
Common Pitfalls and Misconceptions about Online Coding Interview Platforms
We strongly believe that online coding tests are a great way to streamline your engineering hiring while producing a meaningful return on investment. That said, there are several misconceptions and pitfalls that we see from the talent acquisition community that we want you to be aware of.
- Screening, not interviewing: Engineer interview tools are great at screening candidates. However, they are not a replacement for the actual interview. You still need to understand how someone deals with conflict, compromise, communication, etc before hiring them. Coding tools are merely the first step.
- Diversity: Make sure you understand how diversity candidates are scoring on the tests you are administering. One vendor in this space found their tests were biased against female engineers simply because no females were originally involved in writing the tests. Getting women involved in building the question set fixed this problem.
- Engineer buy in: Your engineering team needs to trust in these assessments. If they don’t, this solution provides very little value as they’ll still need to spend time doing their own screens. Educating the engineering team typically happens from a leader on the recruiting side. Of course, there are hiring managers spearheading this change as well. To ensure a successful buying and implementation of the product, get engineering involved when you are demo’ing/trialing the product.
- Fraud: Understand how the screening tool you’re using ensures that candidates are taking the tests themselves and not posting fraudulent scores.
- What’s tested matters: You have the ability to customize your coding tests in a way that reflects the specific programming languages and skills relevant to the job. However, what idiosyncrasies are still being left out? Maybe you only pair program, and need to test this. Maybe there should be a psychometric part of this test, or specific aptitudes that need to be measured in addition to skills.
- Tool, not marketplace: These tools enable your recruiting to team to vet candidates more effectively. But, they are not sources of candidates like a job board would be (although some of the best tech assessment vendors can help with sourcing).
- Don’t leave humans out: Just because you have a new way to vet software engineers doesn’t mean you can simply let your coding assessment handle all candidate interactions. Like any pre-employment screening, you need to realize candidate experience is paramount. Set expectations, sell the person on your culture, and overall just treat them as you’d want to be as an applicant.
Coding Interview Software Pricing
Tech recruiting platform pricing is typically based on the number of interviews you’ll be conducting on a monthly basis. We like this because the more interviews you conduct, the more value you’re going to get.
Pricing starts in the $50/mo range for companies who are just getting started with one req, and can grow into the 6 figure range pretty easily.
Please note that these are ranges and are meant to give you a general idea of how much you might pay for your organization. However, prices will vary depending on a number of factors including: the features you need, support levels, vendor you partner with, HRTech integrations, etc.
Also note that many of the best online coding assessment platforms have free trials.
Staying Organized
Are you looking at multiple tech assessment platforms? We thought you might be! Our recommendation is to use a structured rubric to evaluate these tools so that you can make the right decision for your business after demo’ing a few of the platforms. We put together the below spreadsheet so that you can keep track of your key criteria when looking at these engineer hiring solutions. Our hope is that it allows you to make the best decision for your business!
Questions to Ask on Demos
You’ll notice there are a few suggested questions in this spreadsheet already. In addition to talking to a sales person to vet these vendors, we recommend opening up a free account to play around with the code editor. Setup a pre-canned screen for a data scientist or programmer, and see the process they’d go through. Then, take a look at how screen applicants are presented to your hiring team.
How does this stack up to other online assessment platforms you’ve looked at? Does it give you confidence you’ll be able to judge an applicant’s programming skills? What other interview questions can you embed into these skill tests? Make sure you think about this both from the recruiting team’s perspective, and the candidate’s.
Companies Using Tech Assessment Platforms: Andela
Andela is a fast growing company faced with a massive challenge - how do we efficiently find and hire the top 1% of developers across Africa at scale (they get over 4,000 applicants/mo!).
Maya Neria, Senior Product Manager, was in charge of figuring out how to replace a home grown tech assessment platform that wasn’t cutting it with a solution that would mimic the real world while also providing feedback to candidates. After demo’ing over 100 solutions, Andela selected a tech assessment platform, Qualified, and moved forward with overhauling their developer screening process.
Andela now starts their interview process by sending applicants a study packet on Python/Javascript basics, tests knowledge of this packet via a simple quiz through Google Forms, and then moves candidates into the tech assessment platform. Currently 15% of people who pass the tech assessment are hired. Moving to a standardized platform has allowed Andela to save time, improve the candidate experience, and create a more objective scoring rubric for future candidates.
Final Advice on Tech Assessment
Any company hiring more than a few engineers a year should take a look at using a tech assessment platform. They save your engineering team time. They lead to better hires. And, most are very simple to do a pilot with. You can get a 30 minute demo of one of our pre-vetted vendors to understand how this will fit into the specifics of your recruiting process:
Best Tech Assessment Platforms
Want to get wicked smart and stay up to date on the latest in HR Tech? Sign up for our weekly newsletter.
The latest from our Blog
Is HR Getting Back To Normal?
Here's some data the says yes, we may be getting back to normal!
HR Tech in COVID's New Normal, Week of Sept 23rd
Here's what HR tech news we're tuned into this week!
The 5 Must Read HRTech Stories from This Week
Here are the stories we're reading this week!