Is the Engineer Interview Process Broken?


Over on his personal blog, developer Zachary David Forrest (who currently works at Sonos) has a posting on interviewing software engineers that’s attracted some buzz on Hacker News and other sites, and for good reason: He calls into question many common aspects of the interviewing process.

“How you interview a candidate speaks to the quality of the organization,” he wrote. Many companies head into the hiring process with only a vague idea of the candidate they actually want for the job; and rather than start with broad topics or engage in small talk, they fire off intensely complicated coding puzzles that seem more designed to boost the interviewer’s ego than assess the candidate’s skills.

When interviewing, Forrest tries to take a different approach, one that engages the candidate as a human being: discussing past projects and specific ideas, and giving a test project with a somewhat relaxed deadline:

“If you feel good about the candidate at this point, let them know that you’ll be sending them a project to code on their own the next business day. Try to tailor it to your specific needs and if you can manage it, a simplified problem that satisfies an actual business need. Give them a few days to code it on their own. When they submit the code, you’ll quickly discover if they copied and pasted some solution vs. working through it themselves.”

That suggestion could prove controversial for those interviewers dedicated to having candidates whiteboard out problems on extremely tight deadlines. In their quest to root out engineers who somehow might not know their stuff, these interviewers may end up alienating the best and brightest potentials. In addition, not all engineers and developers work the same way; there are brilliant workers out there who prefer to pick through code without someone breathing down their neck—and they probably aren’t working at companies that interview via a whiteboarding gauntlet.

Or as a commentator on TechCrunch once memorably put it: “Despite the worst talent crunch that Silicon Valley has ever experienced, we still regularly throw away huge groups of talent for not perfectly answering the latest hip algorithm question.”

The rest of Forrest’s tips can apply to anyone searching for engineering talent. Remember, you’re not just looking for an excellent engineer—you also want a person who will mesh well with your company and team.

15 Responses to “Is the Engineer Interview Process Broken?”

  1. Fred Bosick

    So, what’s the problem if they cut and pasted from a programmer’s forum? If it solves the problem they were given, it’s alright.

    Do you think a working programmer hand codes everything s/he submits? What do you think all those algorithm books are for? Not to mention the net bean/struts/.NET frameworks that put in more code than you do.

    The true question is whether those esoteric whiteboard problems weren’t submitted to trip up honest US applicants in favor of an H-1B applicant carefully fed the answer beforehand. I have a 3rd hand story of this occurrence at a major online retailer.

  2. Joe Michalides

    I hear you about the H1B candidates. I had that feeling after being asked a trick question by an Indian engineer during an interview. It was nothing you would ever see in a real world program. I have two engineering degrees and 39 years experience as an engineer. I find the white board experience to be demeaning.

  3. Lex Mercatoria

    @Fred – I’d bet money the esoteric whiteboard question experience is designed to do just what you stated: exclude Americans. This would be totally in line with corporate America posting “non-job” ads with absurd skills requirements that *only* Americans are required to have. The foreigners’ resumes are patently bogus, the “C” suite mafia knows it but doesn’t care, and will split such a job–if it’s a real job to begin with–into three separate positions just to accommodate the cheap foreign laborers they really wanted in the first place.

    As a client’s VP explained to a previous employer’s VP why we lost a contract, even though they were quite satisfied with our performance (quoting verbatim): “the ONLY thing we give a damn about is price!”

  4. Sunny Garg

    I love what this GUY has to say. I have been through so many interviews and the ‘high almighty’ interviewer is pissed off because you missed a semi-colon on the board, or an array length check and NOW you are an IDIOT. That’s what compilers and fancy GUI’s are for. In real life you don’t write a function with a 10 minute deadline. Approach to problem and beauty of thought process is what matters. Rest can be fixed in 1-2 iterations.

  5. I don’t think whiteboard or coding test interviews prove anything except that a candidate can think under pressure. If the job is that high pressure, I don’t want it anyway. I was once given a coding test with 2 foreign-born developers watching me code, 1 wearing a long sleeve shirt with a tie and turban. I didn’t get the job and I didn’t care. To me, fit with the existing team is the most important criteria. The wrong fit can totally disrupt the team. As a professional developer, we can learn anything and learn it quickly so it’s not so much what you know but how well you can get the answers you need to solve a business problem in a timely manner.

  6. Scott Mudry

    I am a software engineer with 22 years spanned across five positions. We have handled everything we have been tasked with in the past, everything; anything it takes to hit deadlines, make customers happy, and be the best at what we do. I was laid off in March and this is EXACTLY the type of interview I have had to fight my way through onlyvto hear that im ‘not a fit’ or receive an entry level offer. I learned PHP on the fly, because I had to. I dont understand why I have to re-interview for everything ive done in the past, and for that new hip algorithm. I have referrences to attest to everything. -linked in; Scott J Mudry

  7. Patrick

    I have tools that handle many of the low-level coding for me. In a recent interview I was asked to code something that my tools usually do, at a level that I usually don’t work with. With my tools, which I’ve invested a lot of time streamlining and customizing, it takes about 60 seconds, but I spent 30 minutes on it in the interview, and they thought I was a dumbass. It’s like asking a C# person to write assembler code. If you want me to code, let me bring my tools, and let’s work at a level of abstraction appropriate for the job.

    By the way, I don’t code on a whiteboard. If you can compile code from a whiteboard, then OK, your hiring process is finding the right folks for you. We’re in the computer industry. The level of automation that you bring to the problem is a factor in coding speed.

    Also, my hourly rate is in the triple digits. So no, I will not take time to write your stupid code sample.

    Also, I will not write your stupid quicksort puzzle or any other puzzle. If it’s not related to the job, then why am I being screened on it? I haven’t written quicksort in over 25 years, and if you ask me to write one (which I’ve never had to do in any of my jobs), I’ll say, “Beats the hell out of me.”

    I have a side business that I’ve made a comfortable living on for the last 18 years, and I’m going back to that, because the interview process has gotten more and more ridiculous. The interview process is definitely broken, and I’d rather not have to deal with it.

  8. Timothy Hammons

    It has been a few years since I’ve seriously interviewed, but even then, it was broken.
    The white boarding is just to get you to think through the problem. Missing a semicolon in a high pressure situation is a poor thing for the interviewer to focus on. They should look at your approach to the problem. I also think that asking how to preform algorithms that most of us depend on the framework for is a poor way to gauge a high level programmers competence. You should have some idea of how it preforms, but not being able to produce something like quick sort in a high pressure situation isn’t a good way to judge a candidates competence in most situations.
    Additionally abstract puzzles shows how well a programmer is at problem solving, however, just because you can’t solve an abstract problem solving thought experiment doesn’t mean you are a bad developer. I am a great problem solver when immersed in the domain, or when I have some familiarity with the domain, but i don’t think working through problems like “How many golfballs fit in a school bus” does anything to prove that.
    The few good interviews I’ve had they interviewed based on having me discuss, in detail, the items on my resume. I tailored the resume to the job posting so we can discuss relevant work history.
    Additionally, one company had me pair program with each of the developers, having me write some piece of code that was related to the business domain, but not something that I had to spend a month learning the domain to do. I loved this experience. Sure, it was a bit pressured, but you got to rub shoulders and do a bit of work with the people you would be working with.
    When working on something you are not familiar with, a common tactic is to do some “research” (I know, crazy thought). When pair programming you can pull up google and quickly get your answers.
    One technique I think would be useful is have someone do a code review on some smaller piece of code. You can get an idea of their attention to detail and knowledge of best coding practices.
    One company many of my graduating class interviewed for out in California for their first job out of school had a 3 day interview process. I did not go through this process, but heard several accounts about it. They had you code on the white board. They had you learn a DSL and write a program in that DSL on paper in an 8 hour period. Of course they also had the interpersonal skills interview as well, but personally I think thats quite a bit of overkill on the technical side. It seems many interviews are geared toward entry level positions, even when interviewing for a more senior position.
    Having always been the interviewee and never the interviewer it would be nice to hear the other side of the story – how those that use the techniques we all hate and don’t see as valid metrics for a quality developer sees those techniques as adding value to the interview process.

  9. 20+ years experience

    I agree with all the comments posted here. I also have 20+ years experience and was recently laid off. I am having a tough time finding a job because on almost every interview I’m asked questions that would be more appropriate for a college grad and not someone with years of experience. (code a bubble sort, linked lists, nodes). I tell them I’m a “Web” developer with minimal Java core but still I get asked questions that seem to want to prove the interviewer’s expertise instead of mine. My last interview was a joke – I was grilled by a group of developers that together had less coding experience than me – it felt like I was in a college study group. I left feeling inadequate and frustrated. Reading these comments makes me feel good that I’m not alone in feeling this injustice. Companies are losing a lot of good people this way.

  10. This is exactly why I no longer want to talk to or work for startups, and most of the companies in Silicon Valley. Most of the people conducting the interview are on a script and have no imagination or skill in interviewing, and no interest in building on anything you say unless you’re perceived as good for pub-crawls or ping-pong. It’s more joining the club than interviewing for a job.

    The typical SV interview process is at least 4-6 hours long, and a complete waste of time, asking you to code on a whiteboard, and asking you a bunch of ridiculous questions, and then make sure you have no clue if you’re even connecting with these people–they all have a code of poker-face, never hinting that you’re making headway in the interview or if they’ve already decide to pass. Many are on a perpetual interview cycle, with no intent of hiring anyone. A lot of them say they want experienced talent, but what they really want is free consulting during the interview to check their own work for free using the candidate to validate their ideas.

    Of course this half of Silicon Valley, where all the kids have no care about capitalism, just “building something that changes the world”. What a crock of shit. No sense of urgency or care about hiring you either, they all have to make sure you’re a cultural fit, code-words for “you can’t be over 30”.

    One of the first questions you should ask your recruiter is if they are serious about hiring somebody TODAY or if it’s just going to be another revolving trail of candidates that are pre-rejected because they don’t fit all the requirements of the job. The next question you should ask is if the company has anyone over 30 and willing to hire anyone over 30. Ask these questions during your chat with the recruiter and save yourself the waste of time.

  11. Timothy Hammons

    You know the process isn’t geared toward senior developers when many articles on getting hired in our industry suggest “hitting the books” before the interview – suggesting to re-familiarize yourself with the core concepts and algorithms before starting up with the interviews.
    I had 4 or so interviews with smaller companies before I found a good fit. Half of the interviews I took were similar to your experience – seemed to be more about the technical interviewer showing off. The others were more looking for the right person and discussing past experience. Keep trying, and play the game a bit… And everyone with current jobs, try to motivate your company to use better interviewing tactics!
    My current job I found from a friends company hiring. As with all jobs it seems to be more about who you know than what you know.

  12. Scott Mudry

    20+ years experience – dont let it get to you. I know exactly what your going through. I was depressed for two weeks after I got laid off. I havent NOT had a job in 22 years. Well, that was March 8th.

    Im thinking far more clearly now. I have come up with a few ways to put us ahead of that whole crowd for positions they dont know they need yet.

    1) mention when you telecommute in your resume. Showing that you are productive offsite is big – and we can show this.

    2) We learn fast. I went back to an old friend and colleague who had an ongoing project. He hired me immediately to take charge of the all of the technology; manage outsourcing, eveything. The catch is, im doing it for living expenses only. Its value for both of us though. Im now the primary engineer, the only engineer on his Android product. Im going to have a successful commercially available product (google, iPhone)in a high demand industry in about 8 months and a real reference when i head back out into the shark infested waters.

    3) Look at some of the most understaffed positions… the ones paying astronomical salaries right now. Youve got this. We adapt. Languages are just tools.

  13. Disgruntled Interviewer

    Agree totally with this article. Modern interviewing process is as dumb as rocks.

    I remember spending a good part of my Mexican vacation writing pseudo-code as part of an interview for a reputable agency. My code was efficient (as pseudo-code can be), and easily readable. The manager said he was impressed with my code, and invited me for a verbal interview. There, I failed miserably to explain an obscure question about how Matlab handles empty elements in an array. I’ve used that language for almost 20 years. Does that count for anything?

    Needless to say I was rejected.

    I also remember vying for a financial quant position. Despite decent scores on an IQ-like test (another idiotic gate) and impressive candor, they whacked me because I couldn’t answer a math riddle about mice and wine.

    I can’t wait until Google exposes more about how jocular the criteria used for prospective candidates are.

    If you’re a firm that likes Jeopardy-formatted tubs of knowledge without creativity, maybe this is a good thing…

  14. I can relate to each one of the above comments. There appears to be a common theme here. Are Interviewers getting lazy and can interview, so they just ask you to write some code for them? It sounds like they are totally missing the mark.
    I always wondered if they simply want someone to write a function for them, which they will use.