I disagree. I give live coding tests. I very much don’t want the candidate to be stressed. I provide a written and verbal description of the (simple) problem, and provide unit tests. And I talk them through it if they run into problems, but try to give them space to work it out.
I’m not sadistic. I want to see if they can write code.
The few times I skipped the live test because of practical reasons or they were “too senior” I absolutely regretted it.
Is this a serious question? What interview, a meeting specifically to scrutinize someone and compare them to their peers, isn’t stressful for candidates?
Interesting. What do you think happened with those you didn’t test? You think they were making stuff up or senior at their job is a far cry from senior at your job?
Not sure. One seemed either incredibly timid or just way in above his head on simple tasks. I assigned him a bug and had already narrowed it down to a particular return code, in a particular call tree. He could have set 20 breakpoints and found the bug in five minutes. Or put unique error codes and found the bug in ten minutes.
But weeks later he was still asking questions and eventually just moved on without solving the bug or even finding the cause.
Maaaybe he would have aced the live coding test, but I doubt it. He just never seemed to “get it” and I think the live test would have reflected it.
But by “senior” i mean decades of experience. No quibbling about job titles.
fully agree. we’re actually reintroducing live coding interviews into our process because so many candidates made it onsite who then showed that they didn’t really know how to code
I don’t think anyone disputes that, it’s just that nobody has come up with anything better.
Take home exercises were a potentially better option (though they definitely have other big downsides) but they aren’t a sensible choice in the age of AI.
Just taking people’s word for it is clearly worse.
Asking to see people’s open source code is unfair to people who don’t have any.
The only other option I’ve heard - which I quite like the sound of but haven’t had a chance to try - is to get candidates to do “live debugging” on a real world bug. But I expect that would draw exactly the same criticisms as live coding interviews do.
I’m not sure that offline or alone coding tests are any better. A good coding interview should be about a lot more than just seeing if they produce well structured and optimal code. It’s about seeing what kinds of questions they’ll ask, what kind of alternatives and trade offs they’ll consider, probing some of the decisions they make. All the stuff that goes into being a good SWE, which you can demonstrate even if you’re having trouble coming up with the optimal solution to this particular problem.
… that’s why you do a follow up interview and review their code, and maybe leave some things a little ambiguous to see if they ask you questions (telling them it’s okay to email questions and mostly expected)
Why did you decide to do ABC this way? What do you think about having done it XYZ way instead?
I know you didn’t have time to write a full test suite, but what areas of what you wrote would be best to focus on tests and why?
You can ask them so many things about what they wrote.
That’s like… how it works in the real world. They ask questions to product as they come up, they get questioned on their work in code reviews
Unless you work somewhere where you pair code 100% of the time anyway…
If you just look at it as a pass or fail and are not doing a detailed review with them after, you’re doing it wrong.
Sure you can move some parts of the conversation to a review session, though I think the answers will be heavily influenced by hindsight at that point. For example, hearing about dead end paths they considered can be very informative in a way that I think candidates assume is negative. Nobody expects you to get it right the first time and telling the interviewer about your binary tree solution (that actually doesn’t work) can be a good thing.
But the biggest problem I think with not being in the room as an interviewer is that you lose the opportunity to hint and direct the candidate away from unproductive solutions or use of time. There are people who won’t ask questions about things that are ambiguous or they’ll misinterpret the program and that shouldn’t be a deal breaker.
Usually it only takes a very subtle nudge to get things back on track, otherwise you wind up getting a solution that’s not at all what you’re looking for (and more importantly, doesn’t demonstrate the knowledge you’re looking for). Or maybe you wind up with barely a solution because the candidate spent most of their time spinning their wheels. A good portion of the questions I ask during an interview serve this purpose of keeping the focus of the candidate on the right things.
hey, I love this idea, but we tried it and we kept getting candidates who managed to BS their way onsite and then waste our time ultimately.
it just didn’t work. I really want it to work because I hated live coding too but it just didn’t.
you can make live coding interviews that aren’t actually difficult questions and are more about showing that you can think and write the most basic of code. that’s what we do now.
I disagree. I give live coding tests. I very much don’t want the candidate to be stressed. I provide a written and verbal description of the (simple) problem, and provide unit tests. And I talk them through it if they run into problems, but try to give them space to work it out.
I’m not sadistic. I want to see if they can write code.
The few times I skipped the live test because of practical reasons or they were “too senior” I absolutely regretted it.
You seem to be disagreeing with something that isn’t the main point of the article.
That you take those steps doesn’t mean candidates aren’t stressed, despite your intentions.
Sorry an interview is stressful to candidates?
Is this a serious question? What interview, a meeting specifically to scrutinize someone and compare them to their peers, isn’t stressful for candidates?
Fuck yes, and that’s just a regular one.
Interesting. What do you think happened with those you didn’t test? You think they were making stuff up or senior at their job is a far cry from senior at your job?
Not sure. One seemed either incredibly timid or just way in above his head on simple tasks. I assigned him a bug and had already narrowed it down to a particular return code, in a particular call tree. He could have set 20 breakpoints and found the bug in five minutes. Or put unique error codes and found the bug in ten minutes.
But weeks later he was still asking questions and eventually just moved on without solving the bug or even finding the cause.
Maaaybe he would have aced the live coding test, but I doubt it. He just never seemed to “get it” and I think the live test would have reflected it.
But by “senior” i mean decades of experience. No quibbling about job titles.
fully agree. we’re actually reintroducing live coding interviews into our process because so many candidates made it onsite who then showed that they didn’t really know how to code
The article isn’t saying don’t check, it’s saying that live coding interviews are a bad measure.
I don’t think anyone disputes that, it’s just that nobody has come up with anything better.
Take home exercises were a potentially better option (though they definitely have other big downsides) but they aren’t a sensible choice in the age of AI.
Just taking people’s word for it is clearly worse.
Asking to see people’s open source code is unfair to people who don’t have any.
The only other option I’ve heard - which I quite like the sound of but haven’t had a chance to try - is to get candidates to do “live debugging” on a real world bug. But I expect that would draw exactly the same criticisms as live coding interviews do.
What would you do?
I’m not sure that offline or alone coding tests are any better. A good coding interview should be about a lot more than just seeing if they produce well structured and optimal code. It’s about seeing what kinds of questions they’ll ask, what kind of alternatives and trade offs they’ll consider, probing some of the decisions they make. All the stuff that goes into being a good SWE, which you can demonstrate even if you’re having trouble coming up with the optimal solution to this particular problem.
… that’s why you do a follow up interview and review their code, and maybe leave some things a little ambiguous to see if they ask you questions (telling them it’s okay to email questions and mostly expected)
Why did you decide to do ABC this way? What do you think about having done it XYZ way instead?
I know you didn’t have time to write a full test suite, but what areas of what you wrote would be best to focus on tests and why?
You can ask them so many things about what they wrote.
That’s like… how it works in the real world. They ask questions to product as they come up, they get questioned on their work in code reviews
Unless you work somewhere where you pair code 100% of the time anyway…
If you just look at it as a pass or fail and are not doing a detailed review with them after, you’re doing it wrong.
Sure you can move some parts of the conversation to a review session, though I think the answers will be heavily influenced by hindsight at that point. For example, hearing about dead end paths they considered can be very informative in a way that I think candidates assume is negative. Nobody expects you to get it right the first time and telling the interviewer about your binary tree solution (that actually doesn’t work) can be a good thing.
But the biggest problem I think with not being in the room as an interviewer is that you lose the opportunity to hint and direct the candidate away from unproductive solutions or use of time. There are people who won’t ask questions about things that are ambiguous or they’ll misinterpret the program and that shouldn’t be a deal breaker.
Usually it only takes a very subtle nudge to get things back on track, otherwise you wind up getting a solution that’s not at all what you’re looking for (and more importantly, doesn’t demonstrate the knowledge you’re looking for). Or maybe you wind up with barely a solution because the candidate spent most of their time spinning their wheels. A good portion of the questions I ask during an interview serve this purpose of keeping the focus of the candidate on the right things.
hey, I love this idea, but we tried it and we kept getting candidates who managed to BS their way onsite and then waste our time ultimately.
it just didn’t work. I really want it to work because I hated live coding too but it just didn’t.
you can make live coding interviews that aren’t actually difficult questions and are more about showing that you can think and write the most basic of code. that’s what we do now.