Hiring Engineers, a Process

Disclaimer: The process outlined in this post reflects my personal approach. Please consider this as a helpful insight into what it takes to get a hiring recommendation from me. As always the law and corporate policy applies.

This post has three purposes. First, to save me the need to explain every time how I interview and hire people. Second, to inspire others to break away from the conventional and ineffective hiring process most companies use. It’s a process that fails to identify non-conforming great talent. And third, when the time comes for me to look for my next adventure (and no, I’m too happy where I am – I’m trying to hire you!), I can point hiring managers here to know how I’d like to be treated.

TL;DR

If you don’t have time to read this, we are not a good fit.

What Not to Do

In 2006 I was looking for a new job on Wall Street. I wasn’t happy at my job, but not yet ready to do my own startup. At the time I was a VP of engineering at Citi managing about 35 people. I was looking for a similar role with a smaller team and more hands-on opportunities. I was not looking for a primarily coding role. I got invited to interview with Credit Swiss for a lead role. When my first interviewer spent half an hour asking me about the order of initialization of static variable in C++, I played along. But when the second guy came in, put a laptop in front of me and said “We’re going to write some code now”, I got up and said “I’m going home now”. I didn’t get that job.

Later that year, I had an interview at Bloomberg. The interviewer greeted me at the entrance, then purposely walked me through their showy floors and world-famous curved escalator as if that’s going to motivate me. We’ve spent the first 10 minutes going over my financial products experience and then followed with 20 minutes asking me questions specifically about everything I just said I don’t have experience with. The next interviewer asked me about my coding experience which at the time was all C++, then followed up with only C questions. The last interviewer asked a riddle (“4 members of U2 are trying to cross a bridge”) which I’ve spend the entire half hour passive-aggressively not trying to solve. I didn’t get that job.

In 2009 I was approached by Google at the recommendation of some people I collaborated with. The conversation didn’t go very far when they made it clear that they will not be able to tell me what I will be working on, and more importantly, who I will be working with and for, until I start. I had serious doubts about passing their interview process, or even fitting in at Google, but I was willing to give it a try. What I wasn’t willing to do is consider a job where the only known parameter is the name of the company. I didn’t get that job.

I’m purposely focusing on companies that got it all wrong. This is not at all about settling scores. Over the years I’ve interviewed at companies like Twitter and Facebook, which resulted in offers I turned down, and companies like Mozilla and LinkedIn which did not result in offers. The point is, I don’t judge the quality of the hiring process based on my personal experience or whether an offer was extended.

I judge it based on how much I enjoyed the process and respected the people on the other side.

I – Screening

The key to a successful hiring process is a good screening filter. I look for at least one of three things when considering engaging a candidate:

  • a record of open source, standards, or other public work,
  • a recommendation from someone I respect, or
  • a cold call with an attitude

I don’t phone screen. I don’t read resumes.

The first two are self-explanatory but apparently not obvious. Too many people rely on their resume for engineering roles. I don’t care about your resume. What I’m looking for is code to look at, specifications to read, or products to play with. Nothing showcases your skills better than actual code, especially code written to satisfy your own standards and not an employer.

Without a public record or a recommendation, I’m not likely to have much interest. Which means, you need to find a way to get my attention. Sending a one line email (or a tweet) asking for more information is not going to accomplish much. I’m looking for some attitude. If you want me to spend time learning about you without having easy access to a sample of your work, you need to make it fun for me.

The obvious ways are to engage me on something I care about. Google me – it’s not hard to figure it out. Send me comments about a blog post or an open source project I published. Tell my why I’m wrong, why my project sucks, how I can make it better, or why you want to work with me. If you put an effort into it, I will do the same.

II – Greeting

Great, you got my attention. Now I am going to get yours.

Engineers are most comfortable talking to other engineers. It is always frustrating when looking for a job feels like looking for a date. We should strive not to turn the interview process into the rules of dating, making candidates think about how “needy” they might look or being afraid of asking questions that can make them look bad.

To address that, we start by assigning the candidate a buddy. This is a member of the same team the candidate is being interview for, who is not part of the decision making process. The buddy’s role is to guide the candidate through the process, offer intentionally biased opinions, relay questions, and in general be a contact person the candidate should never feel bad about reaching to. Your buddy will never fill a form saying you “seem needy”.

Our first call or meeting is going to be all about you. My job is to sell you the company, the team, and the role. The sole purpose of our first interaction is to make you want the job. To give you all the information you need to make an educated decision. After all, in the next step I’m going to make you work for it, so it’s only fair you have all the information you need before I waste your time.

III – Homework

If you made it here, it means your background is exactly what we want, and this is the job you are looking for. Now let’s write some code!

I suck at writing code on demand. I can’t really code outside my controlled environment. I got my chair, and desk, and big monitor, and IDE, and color scheme, and… you get the idea. I also do my best work when I get to “sleep on” a problem. Some people are great at writing code on a whiteboard. It’s a great interview skill to have. It’s also an idiotic skill to test for. I am also not a fan of pair programming which would be the only valid reason to live code during an interview. So no live coding.

Instead, I will present you with a few problems the team is working on right now. I’m not going to make shit up. These are all actual problems we need to solve or have solved recently. The advantages are many: you get to experience what the job is really like and we get to work with you on a problem we have been looking at recently. More often than not, these are pending items that we have not solved yet. Looking at our open issues on GitHub is a great way to get an idea what you might be tasked with.

The guidelines are simply – pick a problem and come back with something that best showcases your abilities. It can be code, a pull request, a written proposal, or an email analysis. Whatever you feel is the best way for you to shine. You can use any resources you want – the web, books, friends. You know, just like you do at work. To get an idea what past hires submitted, check out some of these proposals (some are actual answers from interviews posted with permission after a successful hire).

You also get as much time as you want. There is no deadline.

There is only one catch. Once you send your response back, we’re going to chat about it. We’ll challenge your analysis, your assumptions, your solution. We’ll try to put sticks in your wheels, introducing new constraints or share our experience if we already tried that. In other words, we’ll have an educated discussion about the problem, now that you had as much time as needed to study it. No surprises.

The homework follow up can be a 30m phone chat or part of the in-person interview. It mostly depends on your location, availability, and how blown away we are with your submission. On occasion we are left speechless.

Funny story. during my time at Citi, leading a C++ shop, I gave this homework to people: “Build a car in C++”. Most people created a set of classes and put them together to construct a car object with some methods and state. One guy sent this (actual submission!):

class Car {

private:

  void top () {
    printf("                         ________________________");
    printf("                        /             |          \");
    printf("                       /              |           \");
    printf("                      /               |            \");
    printf("                     /                |             \");
    printf("                    /                 |              \");
    printf("                   /                  |               \");
    printf("                  /___________________|________________\__________");
    printf("     ____________/                                                \");
    printf("    /                                                              \");
    printf("   /                                                                \");
  };

  void bottom () {
    printf("  /                                                                  |");
    printf(" /        ,---------,                     ,-----------,              |=====");
    printf("|        /  ______   \                   /   ________  \             |=====");
    printf("|       /  /         \                 /   /        \  \___________/");
    printf(" \_____/  /           \_______________/   /          \");
    printf("         /          |                     |            |");
    printf("        |           |                     |            |");
    printf("        |          /                      \           /");
    printf("        \         /                        \         /");
    printf("         \_______/                          \_______/");
  };

public:

  void drive () {
    top();
    bottom();
  };
};

Needless to say, he was invited for an interview. He didn’t show.

IV – Interview

The goal of the in-person interview is to evaluate the following areas:

  • Cultural fit – are you going to be a good addition to the team and work well with others? Will we enjoy working with you? Will we want to go to a conference with you? Also, as a team with remote members, evaluate your compatibility with our style and process.
  • Adaptability – we work in a mixed environment where we have control over our systems but little control over the services we consume. Working in this environment, with many legacy systems isn’t for everyone.
  • Passion – everyone should have something they are passionate about. What are you? We’ll try to find yours and chat about it a bit. It doesn’t have to be super relevant. The goal is to see how deep you dive into topics that excite you. My 2 hours interview at Yahoo! included an hour discussion about water gardens. It was awesome.

The on-site interview is usually 2-3 hours long with 3-5 team members. I usually ask each interviewer to focus on one of the areas above. I also provide them with the following list of do’s and don’ts.

The DOs:

  • Know what you are interviewing for (you’d be surprised).
  • Begin the interview describing your role in the team and allow the candidate to ask questions.
  • Ask questions based on actual team experience.
  • Focus on architecture, design, and approach to problems more than the fine details of the solution.
  • Let the candidate explain the process if their answer isn’t what you were expecting – a wrong answer can still demonstrate solid thought process.
  • Any coding question must be a follow up on the homework submitted.
  • Make it fun. Find ways to make the candidate feel at ease and enjoy the conversation.
  • Leave at least 5 minutes at the end for them to ask questions and unwind. Ending poorly sets the tone for the next interview.
  • Offer them a drink or a restroom break.

The DON’Ts:

  • Don’t ask those who interviewed the candidate before you what they think. Form your own first impression.
  • No whiteboard coding or any live coding during the interview. Some people do well coding on the fly, most don’t. It is not a skill we care about. It also takes too much time that is better spent talking.
  • No riddles, logic, trick questions, or puzzles. This is not the SAT.
  • Don’t ask questions about stuff they claim not to know. This should be obvious but sadly its not to some people. Ask them about what they know and claim to be proficient in.
  • Talk, don’t interrogate.
  • Don’t read their resume during the interview. In fact, don’t bring their resume to the interview at all. If you did not read it ahead, focus more about technology in general and our work. It is obvious when someone is interviewing you who has no clue who you are.
  • Don’t ask one long question in multiple parts. If they get the first part wrong, it sets a bad tone for the rest of the interview. Give them a chance to reset and start over with another topic. Ask and move on – don’t get stuck on getting them to reach the right answer.

V – Decision

That’s it. We meet as a team and get everyone’s feedback. We should have an answer the following day.

Final Thoughts

Like most people, I don’t like going to interviews where I don’t know the team and I am asked to “perform”. The interview should adapt to your level. Architects should not be asked to solve short algorithms, the same way managers should not be asked to write code. Your references and past work is what we should really rely on for verifying your claims. The interview is mostly about personality, compatibility, and due diligence.

A job interview goes both ways. We interview you, and you interview us. We want great talent and respect is the first step.

35 thoughts on “Hiring Engineers, a Process

  1. If only more teams used these methods. I think many small shops are starting to get it. Thanks for sharing your process.

  2. Fantastic. I worked at Microsoft for 14y and that place is king of the Random Riddle. I always sucked at them, and thought they were pointless, but somehow got hired. I still have no fucking idea how to get Bono and Adam across the river if Larry and Edge are in the canoe.

    For my entire time there I railed against the interview practices as being unrelated to work performance (in both directions!). Unfortunately, people are really lazy around hiring, they see it as a tax, so do the stuff on your “don’t” list, well, because. Thanks for putting this out there.

  3. The problem with expecting a candidate to do “homework” for FREE, is that… well… they usually get paid for it.

    In fact, they are interviewing with you so that they can GET PAID to do that “homework” in the future. So what makes you think they will do it for free?

    Oh, I get it, you are only aiming for the candidates that are so desparate that they will do anything to get this job… including wasting hours and hours doing free work for you. Good luck trying to find “quality” talent.

    • This is ridiculous. Some people don’t like this process. Some send back half baked notes or try to find the easy way out of the problem by sending back a link to someone else’s blog. Other just reply back saying it is an interesting problem they would love to work on. I am no interested in any of them. Finding compatible talent is hard, and this is a much better way than abusing someone for 5 hours with a white board. And you are going to spend the same amount of your free time at other places doing phone screens and interviews. Wouldn’t you rather spend it on writing code and solving problems? I would.

    • Actually, it just weeds out the people that are passionate about writing code. The one who are will take this setup over whiteboarding any day.

  4. Thanks for sharing; I’m forwarding this to a few of the guys I work with. Also, there were a couple of very minor typos I noticed :)

  5. I’ve read articles in a similar vein, but this is the best I’ve come across.

    One small typo: “The point it” should be “The point is”.

  6. Can you talk a little about the success rate of this approach? Weird thing is, even if you gave the absolute worst answer possible — something like “Well, actually we do things my way and *still* have a lousy team” — I’d still find the approach attractive. Not sure what that says about me.

    • I can say that over 15 years, I have yet to regret a single hire I’ve made using this process. Some who I no longer work with are still close friends. It doesn’t make it easier to find talent. The sourcing problem stays the same. But I think the people who went through this and didn’t get hired knew exactly where they stood and why they didn’t get the job. I hope they found it useful.

      • Thanks! Of course a lot of opinions have been expressed on this topic for a long time, including many similar to yours. But still I’ve always felt unsatisfied, with kind of a depressed feeling that this is an intractable problem.

        Now for the first time I have a clear picture of a process that is very commonsense in its reasons and that reflects how I would want to treat others and be treated myself, on both sides of the interaction. I don’t exaggerate when I say I see the world a little differently now.

        Personally, I would not have extended an invitation to the ASCII-art car guy, but maybe I’d feel differently if I’d talked to him or if I’d seen his answers to other questions.

        Best of luck to us all in our workplace endeavors!

  7. Sounds like the ideal hiring process to me. I WISH the jobs I’ve been interviewing for practiced this. On my last interview, I got so jittery whiteboarding a breadth-first search in front of multiple people that I must’ve looked like I had never solved a basic programming problem in my life. I just wanted to turn and say “have you guys read any of my open source code at all?” Humiliating…

  8. You are spot on with the interviews at Bloomberg. I worked there for many years and hated every day of it.
    When I finally got out, I could not figure out why I stayed there so long.

    Most of the interviewers are non-technical and get a list of “smart” questions that they get to experiment on the scape goats.

  9. Skilled developers (who are actively looking for a job) are go to have 3 more offers on the table any given week. I guess if they’re short, assignments are reasonable, but otherwise you’re just pushing yourself to the bottom of their pile. I can’t count the number of times I’ve had an interviewee offer me a 24-48 hour homework assignment, and then stare blankly at me when I ask what rate they had in mind.

    “I didn’t get that job.”

    • That’s ok. I’m not looking for all skilled developers. I’m not even looking for the smartest. I’m looking for those who are the best fit.

      The homework I give is all stuff they would be working on if they already had the job. The point of the introduction call is to make sure this is the job they way. That’s key. People KNOW what their top job is. If the best job takes a few more days, that’s nothing. And keep in mind that this entire process can take less than a week if the candidate is available.

  10. I would even go so far as to say the “culture fit” test is bogus. Great companies don’t benefit from having many workers that are alike; great companies benefit from simply adhering to the “no asshole rule” (which, if you haven’t read about it, there is a whole book on by the same name). Sometimes it’s that engineer that *doesn’t* fit with the existing culture, but also isn’t an asshole, that brings everybody around them up to a whole new level.

    • This is just semantics. Cultural fit means I want to work with you. I like small teams, where everyone is very hands on, very flat, with lots of trust. I also prefer working from home. This setup requires compatible people. I’m not looking for best friends, or people I want to drink Scotch with. I’m all set for that.

      For example, I express my views in absolutes. I have very strong, black and white opinions. Just read this blog… So I can only work successfully with people who can take it, and stand up to it. This is how I communicate. If you are going to always deffer to me, you are not a cultural fit. I think we’re on the same page.

  11. Couldn’t agree more. When is a real engineering problem solved by one person sitting in front of a whiteboard for 30 minutes?

  12. > The last interviewer asked a riddle (“4 members of U2 are trying to cross a bridge”) which I’ve spend the entire half hour passive-aggressively not trying to solve.

    Oh gods, were they being followed by Zombies? I had an interviewer ask that riddle. I didn’t _want_ the job after that (but I did get a couple of blog posts from the experience.

  13. “Tell my why I’m wrong, why my project sucks”

    Enjoy your team of confrontational macho hacker types. And forget about gender diversity — women in IT get enough abuse without emailing criticism to strangers out of the blue.

    • Taking my words out of contexts in a comment on my own blog? The point is to express an opinion, to offer a counter argument, to highlight how things can be improved, to offer a new perspective. It’s a blog post, not a book. I think you can read between the lines that what I was going for is the opposite of asking people to email ass kissing.

  14. 1. Great post and valuable share Eran, thank you.
    2. @ Jin Kwon – I agree my peers should read this to better their companies hiring processes overall. That said, I’d add it equally if not more important that every engineer and engineering manager read this post. The best engineers are drawn to engineering organizations that get it including talent in part but not just good recruiters.
    3. Overall the value of this post speaks to the “delivery” model which is so important. Many times I hear hiring parties talk about identification or sourcing as if that is the problem. In fact, finding good talent is not hard at all. Actually hiring and retaining that said talent is the hard part.

    ?Homework?
    Eran I actually agree that candidates who fulfill this step in the process and do so well are dedicated to your team and will most likely do well. My question is what candidates do you miss out on that simply cannot dedicate an extended period of time to this step? Surely you wouldn’t say any candidate that can’t do this is committed or good enough to work on your team. Is there some variant of this we could work with to find a happy medium?

    Nonetheless, what a fantastic share. Thanks again!

    • I would expect people to invest 4-6 hours in the homework. This isn’t that much investment for getting the job you want. Bringing you in for two rounds of interviews and multiple phone screens will cost you much more time.

  15. Eran, thanks for sharing!. Sorry for being a bit late to the discussion, but hope you will still have time for a couple of questions:

    1) What is your take on internships? How would you screen/interview early 3rd grade students without open source track records or public work?

    (We have been hiring compiler/runtime engineers almost exclusively through internships for fifteen years, but always looking for ways to improve our process.)

    2) How, if at all, would you extend your process to non-engineering positions? I am actually a s/w engineer turned marketer, and right now I am looking for someone to help with sales and marketing (we are a small company, so nearly everyone has to wear multiple hats.) Granted, a copywriter or designer would have a portfolio and/or references to publicly available work, but how would you screen candidates for a open position in, say, Customer Care? What would their homework be?

    Thanks in advance for sharing your opinion, Eran.

    • Hiring interns is more about potential than experience but I would apply the same process. The less experience you have the more I would expect to see in terms of school projects and hobbies. As for non-engineering – no clue.

  16. I was once asked a question so basic by an interviewer I felt insulted and didn’t bother replying. I didn’t want to work with the person after that.

  17. Nice article. I agree with almost all of it.

    I have a question. I work in an industry that is heavily regulated. All our software is considered munitions (not joking), and thus cannot be uploaded to GitHub or any public space — a foreign national might see it, and we’d be liable for an illegal export. (Yes, it is ridiculous, but the regulators are serious about it so we have to be too.) We wouldn’t be able to share our code with interviewees without first checking their citizenship and subjecting them to a rather lengthy federally mandated training.

    Is there any way your process could be modified to work in this kind of environment?

Comments are closed.