- cross-posted to:
- hackernews
- cross-posted to:
- hackernews
You must log in or register to comment.
In my first interview they put me in a room with a PC with Borland C and a copy of K&R and a sheet with a simple problem to solve and some extra enhancements if I had time. They said they would be back in half an hour and left me to it. That I passed fine.
Some twenty-ish years later I was asked to write a C function to reverse a string on a white board and I failed because I’d misformatted the for loop. I don’t think it was because I’ve become a worse C coder in the intervening years.
When I’m actually coding I’m sat with my editor configured Just So with completion, compilation and unit tests at my finger tips. My favourite coding music blasting my speakers and a handy browser window for looking up anything in unsure of. This is my most productive setting and expecting the same performance in a stressful interview setting is foolish in my opinion.
Working through problems on a white board can work well but you are looking for the problem solving approach, not an encyclopedic knowledge of regex syntax. Those same problems get immeasurably harder when explained over a phone call.
My personal preference when evaluating candidates ability to code is reading their actual production code, the break down of commits, the commit messages and the sort of unit tests they add with a feature. The interview is more focused on their soft skills, what about the work excites them and what they are looking to get out of the role.
And yet it seems that with the increased scrutiny of candidates, and somehow the expectation that code should be able to be written in a vacuum on notepad, shit just seems more progressively broken and unusable.
I failed because I’d misformatted the for loop
Unlikely that you failed the interview because of a basic syntax mistake.
My personal preference when evaluating candidates ability to code is reading their actual production code
This would be a great interview method! But 99% of people are not working on open source code professionally so it doesn’t really work in general.
Our industry has no idea how to hire people. Our interview processes are almost designed to filter out obviously bad candidates while accepting that some good candidates will fail, too. Getting a specifically good candidate is almost luck.
Remember this if you’re bummed about a string of rejections.
I mean, this is essentially all hiring processes.
The way to get actually good employees is to be the sort of place that actual good employees want to work for. Good pay, good work-life balance, good managers and company culture, work that is enjoyable and meaningful. Then, you hire through social networks. The founders start off as people who meet through informal social networks. They hire their friends. And then they ask their friends for further recommendations. The best way to know if someone is a good hire is if you have actually worked with them before. And at this point, the interview is really just hanging out, shaking hands, and having lunch before you sign some paperwork.
The problem with only hiring people you have met personally is that you miss out on a whole world of people who would be great to work with but had no chance of ever meeting you or your network. I agree that network recruiting is the safest route, but having diversity in your employees is great. If you only hire through your networks you’ll see quickly quickly how you only get one kind of person.
I have seem this happen a lot in smaller companies. It’s also the story of how I’m typically the sole woman in the department. I by happenstance happen to seed my professional network from college with a lot of men (because I accidentally picked a college that like 80% men). I’m a unicorn because many men’s networks include so few women since in IT they tend to be non-traditional and/or generally excluded from younger men’s social groups.
I get tapped via my network all the time. But if the company basically only does referral based hiring me and perhaps one other woman is there for the whole engineering department. It’s way more balanced at 20%-30% of the department at companies that don’t do this. There is some value in shotgun hiring even if it has a higher fail rate than referral hiring. Different kinds of people can bring fresh perspectives and considerations.
Great points. Thanks for sharing your perspective
deleted by creator
So in other words, focus on hiring extroverts who are great at forming social networks.
I mean, kind of. I wouldnt say extroverts, so much as “people with good/decent social skills”. Introversion/extroversion is a sliding scale, not a dichotomy, and it refers more to your propensity to gain or lose energy from social interactions - not your ability to socialize.
While many more introverted people find socializing more difficult in general, there is no reason why they can’t develop the skill.
Sure, but if someone is more introverted, then, even if they have amazing social skills, they will have a much harder time forming social networks with a lot of reach because it takes more energy for them to do so, whereas it is a lot easier for a more extroverted person to do this.
You don’t know how good you’ve got it. The hiring process in other industries is much worse.
Story time?
I don’t have a particular story. A lot of industries use “competency based questions”, you know “tell us about a time when…”.
They’re awful. If you don’t know what I’m talking about count yourself lucky.
Which would you rather, “write some code to filter off numbers from a list”, or “tell us about a time when you worked with someone difficult. How did you win them over and subsequently become best friends?”.
It is true that the last time I answered that question with, “I put something in their drink every morning to make them more chill,” I did not end up landing that job…
for those who this affects, this lands badly
I think asking one simple coding question during a live interview is a great way to eliminate candidates that have obviously lied on their CVs. Nothing fancy, just a simple problem that everyone should know. But asking leet-code medium or hard problems make no sense.
Depends on what your requirements are. Some times it’s the best way to let a candidate shine, which is usually the goal – give people as much of an opportunity to impress as you can
If a requirement is that a developer writes code then I want to see them write code.
I can see how this could be unfair, but working as a dev sometimes does require you to be on top of things in a high stress atmosphere. For example, what if you’re proposing an excellent technical solution in a meeting but some jaded older engineer is hard to convince? If you can’t outline your thinking in that scenario, your solution could be discarded just because someone was louder than you. As someone who used to have performance anxiety, I believe it’s generally something you can and should practice for. On the other hand, if there really isn’t a need for this type of skill, it totally makes sense to avoid creating interview environments where you are filtering candidates based on it.
I did stress test interviews for DevOps positions. I explicitly told them that and gave them a task and a time limit. I would watch what they did and there was nothing out of bounds as long as they were solving problems. For example, I would give them an account in cloud provider and then task them with spinning up a k8s cluster with a few basic services and make it scalable, then watch and heckle as they googled around and brought up services. The objective wasn’t to complete the task though, it was too see how they approached problem solving. Good times.
Great example. We do the same but to spin up a single ec2 via terraform. Checking the real familiarity with tools. This immediately filter those who lied about their experience.
Good enough example, although I would’ve picked dealing with a live incident
Yeah, that too! When you have some non technical manager breathing down your neck, you might have a hard time not fumbling around even if you normally could resolve the issue in no time.
That sounds fair. I hate “tests” that involve things you’d never do on the job.
If you give live coding tests you’re a moron. Here’s why. Not all but many of us coders are autistic or highly functional autistic and our brain shuts down in high stress social situations with someone watching over you. Plus whiteboarding when I never fucking whiteboard anything. But get us alone in a room with a task and we’ll whip your ass.
My last boss pulled this on me. I almost didn’t get the job. Then I told him to assign me any project as homework. Overnight I produced a program that blew them all away. Got hired.
I disagree. I give live coding tests. I very much don’t want the candidate to be stressed. I provide a written and verbal description of the (simple) problem, and provide unit tests. And I talk them through it if they run into problems, but try to give them space to work it out.
I’m not sadistic. I want to see if they can write code.
The few times I skipped the live test because of practical reasons or they were “too senior” I absolutely regretted it.
You seem to be disagreeing with something that isn’t the main point of the article.
That you take those steps doesn’t mean candidates aren’t stressed, despite your intentions.
Sorry an interview is stressful to candidates?
Fuck yes, and that’s just a regular one.
Is this a serious question? What interview, a meeting specifically to scrutinize someone and compare them to their peers, isn’t stressful for candidates?
fully agree. we’re actually reintroducing live coding interviews into our process because so many candidates made it onsite who then showed that they didn’t really know how to code
The article isn’t saying don’t check, it’s saying that live coding interviews are a bad measure.
I don’t think anyone disputes that, it’s just that nobody has come up with anything better.
Take home exercises were a potentially better option (though they definitely have other big downsides) but they aren’t a sensible choice in the age of AI.
Just taking people’s word for it is clearly worse.
Asking to see people’s open source code is unfair to people who don’t have any.
The only other option I’ve heard - which I quite like the sound of but haven’t had a chance to try - is to get candidates to do “live debugging” on a real world bug. But I expect that would draw exactly the same criticisms as live coding interviews do.
What would you do?
You mention lots of options. Given people are varied, and you want that in a company, how about letting the candidate decide how to prove themselves? It’s pretty established that it’s not “fair” to stick to a single style, so why hang on to that?
I’m not sure that offline or alone coding tests are any better. A good coding interview should be about a lot more than just seeing if they produce well structured and optimal code. It’s about seeing what kinds of questions they’ll ask, what kind of alternatives and trade offs they’ll consider, probing some of the decisions they make. All the stuff that goes into being a good SWE, which you can demonstrate even if you’re having trouble coming up with the optimal solution to this particular problem.
… that’s why you do a follow up interview and review their code, and maybe leave some things a little ambiguous to see if they ask you questions (telling them it’s okay to email questions and mostly expected)
Why did you decide to do ABC this way? What do you think about having done it XYZ way instead?
I know you didn’t have time to write a full test suite, but what areas of what you wrote would be best to focus on tests and why?
You can ask them so many things about what they wrote.
That’s like… how it works in the real world. They ask questions to product as they come up, they get questioned on their work in code reviews
Unless you work somewhere where you pair code 100% of the time anyway…
If you just look at it as a pass or fail and are not doing a detailed review with them after, you’re doing it wrong.
hey, I love this idea, but we tried it and we kept getting candidates who managed to BS their way onsite and then waste our time ultimately.
it just didn’t work. I really want it to work because I hated live coding too but it just didn’t.
you can make live coding interviews that aren’t actually difficult questions and are more about showing that you can think and write the most basic of code. that’s what we do now.
Sure you can move some parts of the conversation to a review session, though I think the answers will be heavily influenced by hindsight at that point. For example, hearing about dead end paths they considered can be very informative in a way that I think candidates assume is negative. Nobody expects you to get it right the first time and telling the interviewer about your binary tree solution (that actually doesn’t work) can be a good thing.
But the biggest problem I think with not being in the room as an interviewer is that you lose the opportunity to hint and direct the candidate away from unproductive solutions or use of time. There are people who won’t ask questions about things that are ambiguous or they’ll misinterpret the program and that shouldn’t be a deal breaker.
Usually it only takes a very subtle nudge to get things back on track, otherwise you wind up getting a solution that’s not at all what you’re looking for (and more importantly, doesn’t demonstrate the knowledge you’re looking for). Or maybe you wind up with barely a solution because the candidate spent most of their time spinning their wheels. A good portion of the questions I ask during an interview serve this purpose of keeping the focus of the candidate on the right things.
Interesting. What do you think happened with those you didn’t test? You think they were making stuff up or senior at their job is a far cry from senior at your job?
Not sure. One seemed either incredibly timid or just way in above his head on simple tasks. I assigned him a bug and had already narrowed it down to a particular return code, in a particular call tree. He could have set 20 breakpoints and found the bug in five minutes. Or put unique error codes and found the bug in ten minutes.
But weeks later he was still asking questions and eventually just moved on without solving the bug or even finding the cause.
Maaaybe he would have aced the live coding test, but I doubt it. He just never seemed to “get it” and I think the live test would have reflected it.
But by “senior” i mean decades of experience. No quibbling about job titles.
IMO this is not a helpful way to put it. They measure skill under stress. Stress may have a large effect on skill level for some people but highly unlikely that it’s so large that performance is completely random.
Nah, they measure memorization under stress. Can you recall that tidbit of information to solve the problem the interviewer has given you? If you never have needed to solve a problem like that then you’re shit out of luck, even though solving that problem for the first time (by whomever) definitely didn’t do it under stress in a job interview.
Some bad interview questions are like that, sure. But they’re supposed to be things you are very unlikely to have done before and can reasonably figure out. It’s not too hard to come up with simple questions like that. (Though I will grant many people don’t seem to bother.)
I’m sorry, but as an interviewer I’ll never not have some form of live coding. Some people just don’t know how to code. I don’t mean that in some elitist, gate keepy way. I mean some people lie on resumes. If I used Excel every day (like I was an accountant) I would not take someone’s word that they know how to use Excel, I’d want to see them do it.
I am fully aware that problems are harder under stress. I advocate for genuinely easy problems in coding challenges. I don’t like brain teasers. I don’t really even care if you finish the problem. So long as I can tell that you know how to make a computer do things you tell it in code, know how to ask questions about a problem, and make progress towards solving it – I’m happy.
At an old job I made the mistake of not conducting a coding session of some form and we hired someone who I genuinely believe didn’t know how to program. They’d ask for help on tasks very often and I’d try to guide them in the right way, but once we paired up, they just wouldn’t ever type anything. After me becoming more and more clued into something being wrong and seeing no progress I finally mentioned it to my manager. I don’t think he ever got fired, just shuffled around.
my current company does live code design challenges instead of straigt codong exercises. seems to work well
I give live coding tests generally based on the level they claim to be at in the language. It doesn’t have to be perfect as I’m more concerned with why they’re doing a thing. I usually pick something fairly basic with some edge cases just to see if they mention it.
As opposed to homework, it also proves that you can at least basically work in the language in question (I’ve had a couple of people who got to my round but seemed to know almost nothing about a language they claimed a lot of experience in (like declaring variables and struct members wrong… seriously). We also caught someone that didn’t seem to have done the homework themselves.
If the candidate makes mistakes or gives an imperfect solution, I try to gently guide them to where we need to be. I ask them to explain why they made decisions they did, any edge cases, and how to improve performance or scale it. I expect them to ask questions when something is vague (and usually something in my problem can be interpreted one or two ways for this reason) because these are things they will encounter working with stakeholders and other engineers. If they can’t do that live and on-the-fly, they’re probably not for us. I fully expect nerves to be a factor and account for that; we’re all nervous in interviews.
People have different levels of “nerves” as others, and it kind of sounds like you may filtering out applicants on an arbitrary metric (how nervous a person may be in an interview). Don’t have enough information about your process to say for sure (obviously), but it may be something to think about. Interviews can be very high-stakes for some people (such as “I may become homeless”), and not for others (“my parents are rich”). After hired, it’s not necessarily as high-staked, and toy problems aren’t what SEs work on day-to-day.
I’m mostly making sure they didn’t completely lie about being able to work in the language and can explain what and they would do, why, and how they respond to feedback. I expect people to be varying levels of nervous and that’s fine. I work with people to get them focused and take the edge off as much as possible.
What I ask for usually is related to what we need to implement, but a more basic chunk of it to, for example, show that the candidate understands concurrency and can use basics in the language to do something with that (which we do frequently).
For many positions, we do not have homework and this is the only coding we get (kinda depends on role and project).
As a newer company and still technically a start-up, the boss paying the bills can decide we need to chase something else and he isn’t being talked out of it. This can lead to very fast collaborative design and coding of PoCs which can be more intense than the interview. I don’t like it but it is what it is. Not everything we do is nice, stable, and long-term.
I can relate to needing that job; I’ve been homeless, so I definitely kno the hat that pressure feels like and why nerves alone are never a deciding factor for me.
deleted by creator
is the point of a question like that not to measure how you perform under stress? the guy who posted it in the screenshot doesn’t seem to realise that either though…
If you’re expecting to be stressed all the time at work then that is a red flag. Some professions may involve a degree of stress, which should be mitigated, from time to time but software engineering shouldn’t.
I guess my question why should anyone feel stressed from live coding? There are some jobs where this is legitimately a common occurrence at your job. Some jobs are big on pair programming. And I don’t think I’ve ever had a single job that at least a couple times a year didn’t have me living coding through a problem. It happened way more often when I was a junior and needed a lot of assistance. If you are stressed by being watched while you code, that’s not great because you are going to have to do it regularly or semi-regularly at your job. That’s whether someone is sitting right next to you or they are screensharing. It’s why I personally am comfortable with live coding. It’s literally a thing I do at work, albeit not with toy problems.
I think there is a difference in setting. Pair coding is a useful exercise but demands a degree of trust that the two of you are working together on a solution rather than one of the pair judging the other.
I don’t agree at all. I’ve definitely been in lair sessions where the other person has been assigned to babysit me to the correct answer. It’s just an experience that mostly happens with juniors. I’ve babysat juniors to the solution myself.
There can also be zero trust between colleagues forced to pair, especially in debug sessions. I have worked a lot of jobs, so maybe it’s just my experience, but I would not say that if categorized every single pair session I’ve had in my entire career anywhere near half involved two colleagues who trusted each other and didn’t judge.
I’ve definitely been judged as a senior for dumb dumb moments and that’s okay. If you care about people’s opinions too personally as a software engineer, I’m not sure this is the career for you. It’s a career that involves a lot of negative feedback even as an experienced professional.
There is a difference between reviewing code and the feedback when you have the job and during an interview when trying to get a job. I’m not saying you should never expect to be pulled up on mistakes just that an interview experience is very different to the work experience.
Maybe there are ways to ameliorate the stress during the interview to get a better view of how a candidate will perform once hired but I think it’s a tricky balance to strike.
That is a difference people make in their mind. I don’t see a difference. The criticism is the critism. If you receive enough negative feedback on PRs after being hired, you will be fired for not be good enough.
The only way to take away the stress of an interview is to not care about the outcome. I don’t and interviews are stressful for me. I present myself as I am and if they don’t like it oh well, but that is a confidence I think most people don’t have. I have been like this since I was junior so it’s not the arrogance of experience. It’s not even the confidence of easily being able to get jobs because in my early career it would sometimes take months to get a new job. I’d ask for feedback if I could get it and accept or disregard it. Some feedback amounted to, “this wasn’t the job for you” and that’s okay.
I just don’t think it’s worth worrying about any particular job when hiring is like dating. You can be perfect for one job and an obvious no for another, so it’s not worth worrying about the outcome. They like you or they don’t.
deleted by creator
Stress is not something you can reason about in the way you imply. It’s an emotional response, and people vary wildly in how they will react. It’s great to hear you don’t get stressed, but judging people who do is, well, concerning.
Live coding in interviews is a completely different experience to pair programming, night and day. I don’t ever recall being asked to code in front of a group.
The literal point if interviews it to judge. The point is to find people who will work in the environment you have. I have done work on codebases where bad code means people die, by indirect or direct results. This probably biases me. For example, I have coded in front of a group several times. This year in fact. Sometimes a problem involves multiple people thinking through it. That’s probably why I don’t care about panel interviews as well. I have had to explain myself in front of a group several times.
These are things that people find stressful, but they are part of my job and have been at nearly every one of the little over half a dozen jobs I have held. My current job isn’t even doing anything important. No one dies if I make a mistake and I’ve still experienced explaining myself in front of a group and coding with several people onlooking. I just assumed that’s how the job is as my friends in the same field have similar kinds of stories
People can be stressed I guess, but is normal and common events in your job are highly stressful, then I still say that’s a sign that it’s not the career path for you. For all we know, these jobs have these things because it’s common on the job and a candidate should really feel at ease doing it. That’s my opinion anyway. We can only form opinions based on experience and apparently, mine differs from yours.
I meant judging people for feeling anxious, as opposed to judging aptitude.
Your post was about stress not anxiety and theu are different things. My point is that these these kind interviews approximate what you will actually do at work. If someone finds them stressful then they should think about if this is the career for them. Feeling anxious is another thing, but you can feel anxious while being confident because anxiety is about fearing and unknown outcome.
My point is that people should fine these interview styles stressful and that has always been my point and what I have been replying to since you never brought up anxiety until now.
I feel we disagree on a number of fundamental things. We’re not going to get anywhere on a site like this
deleted by creator
Pair programming and collaboration do not at all carry the stress of an interview situation with a person you hardly know and have never worked with.
