Take-Home Exercises

Take-home exercises are one of the most valuable tools in a hiring manager’s toolkit. However, you need to do them right; you need to be true to your reasons for why you do them, be clear about what you are trying to assess, and be open with your expectations. I want to share how I think about them.

The key thing I always conveyed to the candidates is that the goal is to assess both their ability and interest. The ability part is easy (or easier); we need to know if they can solve basic tech problems. The interest part is more subtle but equally important.

So, here’s how I built take-home exercises at my various jobs. There are 3 important things in a good take-home exercise:

1. Make sure they are aligned with your company’s values.
2. Make sure the scope is limited
3. Follow the process

1. Value Alignment

Here, you are primarily asssesing ability. The goal is not to see if the candidate can code, but see what the output looks and feels like. If fit and finish is important to you, you should let the candidate know in advance and use that as a criterion. Same for performance, or architecture. The key is having a clear and specific set of criteria.

This is also a part great candidates often shine. One of our take-home challenges was primarily a data munging exercise. One candidate took the time not just to work with the data but also build a visualization. We didn’t ask them to do it, but they wanted to. It showed not only initiative, but also the visualization was so tastefully done that we knew we could rely on their taste.

Creating Role-Specific Exercises

Goes without saying, but you want to create very specific exercises for each role you are hiring. It’s likely you won’t get it right the first time and will do some edits, but here are some other guidelines:

1. Specific is better than vague. You should be able to define the exercise in one sentence.
2. The exercise should be reasonably fun—good candidates like interesting problems.
3. The goal is to see if they can code, not solve a hard CS problem (for most SaaS companies).
4. Make sure you have solved the problem yourself, but also be aware that there are multiple paths to god.

2. Limited Scope

This is where you are getting to the assessing the interest part. My goal for all the challenges is that they can be solved “in an afternoon”. For some candidates, this means a couple of hours, and for some others a few days. Either is fine, though the former is slightly better.

The temptation here is to make the exercise large. Resist it. A good exercise is limited in scope but has some incidental complexity. You want to see if they can get to the finish line while doing some of the less exciting parts of coding.

Should You Pay People?

Limited scope also helps answer the “I want to be paid for this” question. I was surprised how rarely this came up; out of hundreds of candidates, only 2 people demanded payment. I rejected both, and one of them did the exercise anyway.

It’s useful to give context on why I don’t offer payment. I openly told candidates the work would be representative of the work but it wouldn’t be something we’d use ever (and it was clear from the exercise). We weren’t trying to get anyone to do free labor.

I consider this part of our due diligence before moving to the next stage, which was either the on-site interview with the team or the work-trial. Both are very expensive in terms of time and resources, especially for a small company. Most people who are serious about a job search can find a few hours in their week to help me assess them and gather resources on my part.

Again, you need to find your own wording here, but I was pleasantly surprised how many people immediately grokked it when I openly shared why we had take-home exercises in the first place. Honesty is always the best policy.

3. Follow the Process

You are now squarely asessing the interest. You want to see if they get to the finish line. It’s still important to be as helpful and clear as possible.

I always had a handout with the instructions and judging criteria. This was helpful for candidates who were new to the process and for me to quickly reference. It also forced me to be concise and clear.

Example

I created a copy of this Notion page for each candidate. I did this more for my own bookkeeping than some operational security concern about the questions leaking. One disgruntled candidate did share the post on a forum after we rejected them, and it hasn’t been a huge problem. We retired the page and kept the same exercise.

I also operated on an honor code and shared it with the candidates — we gave everyone 1 week to finish the exercise and to not spend more than an afternoon. I told them I’d check in after a week if I didn’t hear back. This worked really well; interested candidates solved it in a few days and many people either never returned the exercise or kindly withdrew after I checked in. Every single person who asked for an extension got one but none of those people turned in anything later. Make of that what you will.

Assessment

For me, great candidates didn’t just solve the problem but shared our values. Largely, I wanted folks who had good taste and wrote good code, where good was defined as “easy to change”. Good candidates got the job done, and great candidates often surprised me.

For front-end focused roles, this meant a working prototype that handles all the edge cases such as people actively messing with the UI, double clicks and such. For backend folks, the most valuable signal was always the simplicity of the code; fewer layers of indirection and followed common patterns in the language of their choice.

Here’s a selection of some other criteria that seemed to work:

1. Simpler code almost always meant better outcomes.
2. Great candidated showed their personality and creativity, especially for front-end exercises.
3. Great candidates often wrote tests, especially for backend take-homes.
4. Fewer dependencies meant better code for both front-end and backend.
5. Not testing instructions is a big no-no — we officially didn’t hold anyone back but it sent a message.
6. Good candidates knew where to stop — see #1
7. When in doubt, see Zen of Python

Vibe-Coded or AI-Assisted Work

It’s clear that people will use AI to help write their code. I don’t find it useful to guard against it. After all, it is a key part of everyone’s toolkit these days and my goal is to be honest in my assessment of how people will get stuff done at work.

That being said, I think in the future hiring managers will have to pair more in person (or over Zoom) to go over these take-home projects. Until then, however, I’ve found a few things that work well.

Start by telling candidates that they should be very clear on how they programmed the exercise. I used to tell people that they can use third-party imports and now tell them that they can use AI. However, I do ask them to include what models they’ve used. This alone shows we care about this stuff.

Second, we try to find unique exercises that don’t lend themselves to being “vibe coded”. This is easier said than done. It’s a good idea to give a few frontier models a try with your exercise to see what to expect.

Third, I tell them that our “in-person” section of the interview, should they advance to that stage, will involve further work on this project. This often “scares” people enough to make sure they don’t blindly submit AI-coded projects.

Future Ideas for Assessing AI-Assisted Work

I want to experiment with giving people access to a model and to share their coding threads. I love what Amp is doing here. I’ve seen this in action with the Hashicorp founder Mitchell Hashimoto sharing his entire workflow and it striked me as a great way to see how people work. I expect tools like Claude Code and Codex to follow suit here.

Another idea I have is to actually share a couple vibe-coded examples as part of the instruction. This would help candidates understand what we are looking for, but also make it clear that we are aware these problems can be solved in many ways and we have looked at some AI-coded solutions.

I have considered asking for a commit history but based on how I use commits, I couldn’t bring myself to use that as a criterion. I’ve always used commits as a way to “save my progress” versus a way to organize it and it’d be disiengous to expect others do something I wouldn’t do. Moreover, given how fast frontier methods are evolving, I expect this to be easy to game anyway.

Final Thoughts

In Turkish we have a saying; you want people who are willing to “put their hands under the rock” to lift it up, versus talking about doing the work. Those are the people you want to hire. People interview for tons of reasons; sometimes they are depressed and want to talk to someone. Sometimes they want to kick the tires to see if they can get a raise. And often they are looking for a new type of challenge, feel like they can do more than they are currently doing at their job. I wanted to hire those kinds of people and take-home exercises help with that.

More on Hiring

If you’re curious about the broader pipeline—how many candidates you need to reach out to in order to make one hire—I built an interactive hiring pipeline calculator that might help you think through the numbers.

I’ve also written before about how chaotic and random hiring decisions are at the individual level.