How to Create a Great Take-Home Coding Test

Anton Fenske / Twitter · LinkedIn

September 20, 2021

Programming skills tests are an essential component of hiring developers. Take-home coding challenges provide a convenient way to assess tech skills and test developers because they are asynchronous and don't put candidates under pressure.

A failure to consider some fundamental aspects could lead to incomplete tests, wasted time for both parties, and incorrect assessment of candidates' results.

In this post, we'll cover the basic rules for ensuring that your programming skills test achieves its primary goal, i.e., how to test developers’ skills in order to identify if they have the necessary technical skills and problem-solving required for the job.

"First, solve the problem. Then, write the code." — John Johnson

— Programming Wisdom (@CodeWisdom) August 16, 2021

What Are Take-Home Coding Challenges?

Take-home coding challenges are small projects you are assigned by your potential employer as a means of assessing your technical skills and knowledge. Generally, the task is moderately complex and might require a few hours to complete.

The employer sees what you will be able to accomplish in a less time-pressured setting than in a whiteboarding interview. By doing this exercise, you are able to research and document, as well as demonstrate other skills, that you wouldn't be able to show on a typical whiteboard.

In theory, the take-home challenge will allow you to work in a relaxed environment in which you are comfortable. This means that you are likely to be able to do your best work.

Problem

With take-home tests, the main challenge is that people have to do them outside of work hours. Therefore, you have to be mindful of the following constraints:

• Candidates are often short on time, so they might not finish the test.
• Take-home tests require a longer feedback loop. Since your candidates are going through several interview processes, you might lose them to someone else.
• Take-home challenges may also be viewed as "free work".
• Among the mistakes hiring managers make are tests that are too long and interviews that do not offer feedback.

Here are a few examples of what developers think about live challenges versus take-home tests.

Live coding challenges are the toughest part of interviewing for most people

Here's a summary from my latest blog post on @frontendeval about how to ace them:

— Mike Chen🌱 (@genericmikechen) February 17, 2022

Just had the initial HR phone screen for a job. When asked about the interview process & coding challenge she said “the coding challenge will be take-home. We don’t believe in live-coding challenges because that’s not what you’ll be doing on the job.” And I’m crying tears of joy.

— hannahrosecodes (@hannahrosecodes) November 18, 2019

Is There a Solution?

Sadly, there is no guarantee that all your candidates will complete the coding skills assessment, or that you will have enough time to select the best candidates.

Nevertheless, if you follow these tips, you'll maximise your chances of getting the best results on your developer hiring tests.

How to Test Coding Skills

First, define the main assessment objective and make your exercise revolve around it. The following questions should help you:

• What experience levels do we target?
• What tech skills do we want to assess?
• Do we only want to screen candidates or do a full tech assessment?
• What competencies can we assess automatically using auto-tests, and when would we need a manual code review?

The second step is to create a clear task description. Until now, the following structure has proved successful at Alva Coding Tests:

• Introduction - provides a general description of the task at hand.
• Getting started instructions - helps the candidate get started quickly (e.g., describes the setup, how to run tests, describes external dependencies, and so forth).
• Expectations - describes what the candidate should focus on, in priority order, and what the assessor’s expectations are in terms of what they’re required to do and what skills they will need to showcase.
• Ways to get help - describes how a candidate should proceed should they get stuck during the assessment.
• Time estimate - helps the candidate effectively plan their time before starting working on the task.

Here is an example of how it could look, based on one of Alva Coding Tests' real-world coding tests:

Finally, your coding test can include automated tests that candidates can run by themselves on their local machine to confirm whether their implementation satisfies the acceptance criteria.

Once your candidate volumes have increased, you can also use the auto-test results to vet your candidates with no developer effort on your side.

Reduce Friction

Stick to the process developers use every day:

• Instead of sharing your take-home challenge as an email/Word doc/ zip file/, use a Git repository (ideally, use GitHub as most developers already have an account there).
• Instead of asking them to send you an email when they are ready, tell them to create a Pull Request.
• To avoid losing emails, ask them to create a ticket (e.g., GitHub issue) instead on their repository whenever they run into issues.
• Pre-upload a project template (or templates if you expect submissions in multiple tech stacks) so that your candidates don't waste time setting everything up from scratch. You can use the Alva Coding Tests import script that supports a number of tech stacks.

Define a Standardised Code Review Scorecard

As humans, it's easy to get caught up in biases when looking at someone's code.

To ensure no bias creeps in, create a code review scorecard that includes the following:

• The scoring rules. Here's how we define it at Alva Coding Tests:

• Define scoring criteria per evaluated area. Here's an example of how it could look:

Embrace Continuous Improvement

In the beginning, you'll likely have some teething problems with your take-home challenges. What's worse is that not all your candidates will be vocal about it.

To ensure that your programming skills tests are up to standard, use this 2-step rule:

1. Add the following to the challenge criteria: "Describe one thing you'd improve about this coding assessment".
2. Once there is a good suggestion, implement it right away.

Following this rule will save your tech interviewing team hours of future process revamping and result in hiring great candidates.

Taking action afterward not only helps you learn valuable things about your candidates (their problem-solving abilities, communication skills, teamwork, etc.), but it helps you perfect the candidate experience as well.

If you give candidates feedback, you ensure that they understand your decision and learn something for their next application. A candidate who has spent hours on a task is incredibly disappointed to get rejected without explanation. Even worse, they might not hear from anyone at all.

In Closing

Developer hiring is hard. But it doesn't have to be. If you follow the coding test for hiring principles above, you will already be way ahead of most tech recruitment processes.

A skills-based approach is the most effective way to identify unqualified candidates early on. It is now an effective way for hiring teams to assess their candidates.

By doing this, you will not only save time, but you will also gain a deeper understanding of how your candidates think, solve real-life problems, and apply this to their roles.