How to Create a Great Take-Home Coding Test
Programming skills tests are an essential component of hiring developers. Take-home coding challenges provide a convenient way to assess tech skills and test developers because they are asynchronous and don't put candidates under pressure.
A failure to consider some fundamental aspects could lead to incomplete tests, wasted time for both parties, and incorrect assessment of candidates' results.
In this post, we'll cover the basic rules for ensuring that your programming skills test achieves its primary goal, i.e., how to test developers’ skills in order to identify if they have the necessary technical skills and problem-solving required for the job.
With take-home tests, the main challenge is that people have to do them outside of work hours. Therefore, you have to be mindful of the following constraints:
- Candidates are often short on time, so they might not finish the test.
- Take-home tests require a longer feedback loop. Since your candidates are going through several interview processes, you might lose them to someone else.
Is There a Solution?
Sadly, there is no guarantee that all your candidates will complete the coding skills assessment, or that you will have enough time to select the best candidates.
Nevertheless, if you follow these tips, you'll maximise your chances of getting the best results on your developer hiring tests.
How to Test Coding Skills
First, define the main assessment objective and make your exercise revolve around it. The following questions should help you:
- What experience levels do we target?
- What tech skills do we want to assess?
- Do we only want to screen candidates or do a full tech assessment?
- What competencies can we assess automatically using auto-tests, and when would we need a manual code review?
The second step is to create a clear task description. Until now, the following structure has proved successful at DevSkills:
- Introduction - provides a general description of the task.
- Getting started instructions - helps the candidate get started quickly (e.g., describes the setup, how to run tests, describes external dependencies, etc.).
- Expectations - describes what the candidate should focus on.
- Way to get help - describes how to proceed in case the candidate gets stuck.
- Time estimate - helps the candidate plan their time before starting working on the task.
Here is an example of how it could look:
Finally, you can provide candidates with automated tests they can run locally to test if they built exactly what you wanted.
Once your candidate volumes have increased, you can also use the auto-test results to vet your candidates with no developer effort on your side.
Stick to the process developers use every day:
- Instead of sharing your take-home challenge as an email/Word doc/ zip file/, use a Git repository (ideally, use GitHub as most developers already have an account there).
- Instead of asking them to send you an email when they are ready, tell them to create a Pull Request.
- To avoid losing emails, ask them to create a ticket (e.g., GitHub issue) instead on their repository whenever they run into issues.
- Pre-upload a project template (or templates if you expect submissions in multiple tech stacks) so that your candidates don't waste time setting everything up from scratch. You can use the DevSkills upload script that supports a number of tech stacks.
Define a Standardised Code Review Scorecard
As humans, it's easy to get caught up in biases when looking at someone's code.
To ensure no bias creeps in, create a code review scorecard that includes the following:
- The scoring rules. Here's how we define it at DevSkills:
- Define scoring criteria per evaluated area. Here's an example of how it could look:
Embrace Continuous Improvement
In the beginning, you'll likely have some teething problems with your take-home challenges. What's worse is that not all your candidates will be vocal about it.
To ensure that your your programming skills tests are up to standard, use this 2-step rule:
- Add the following to the challenge criteria: "Describe one thing you'd improve about this coding assessment".
- Once there is a good suggestion, implement it right away.
Following this rule will save your tech interviewing team hours of future process revamping and result in hiring great candidates.
Developer hiring is hard. But it doesn't have to be. If you follow the coding test for hiring principles above, you will already be way ahead of most tech recruitment processes.