How should we hire less-experienced developers in the age of LLMs?
Over the medium to long term, we’ll at least be able to reflect and think about what’s worked and what hasn’t. But right now, we’re in fresh snow: no tracks, no scents.
When evaluating for technical competency, should we…
- Tell candidates not to use AI? This seems backwards, like asking accounts not to use calculators.
- Push more leetcode? This was already ineffective, high-effort, and filtered out otherwise great candidates.
- Describe a feature & ask candidates to write the tests? I personally like this, but that’s probably because I’m An Old. It’s totally conceivable that effective development flows are going to look wildly different in a couple years, and the testing story when we’re increasingly using engines that are non-deterministic might get weird. Short-term, this definitely still has value.
For those candidates we’re talking with that have learned to sling code with LLM assistance, though, we’ve got an option that previously was much more problematic: the take-home assignment, but accelerated.
Previously, asking someone to write a feature from scratch and bring it back to you, while probably a decent indicator of technical competency, was a really big ask. That could take a day or more of someone’s usable time, and often, firms making this ask would do so without offering compensation. Yuck. Expensive, borderline insulting, and you filter out people that already have jobs!
Now, you can slim down your request to something that someone could reasonably build with an LLM in an hour or so. It’ll be brittle and full of code quirks and flaws, but you’ll have a chance to evaluate how someone breaks down a problem and composes a solution—and that’s the most important technical competency in our industry.
We’ve got the chance to be better at hiring and make the process less painful for everyone, especially job-seekers. Let’s not miss out on it.
