Bounding Unicorns

Interviewing For Talented Engineers

This essay is the third part of a mini-series on hiring top engineers. Today I am going to outline my interview process for engineers, aimed at efficiently determining if an engineer I am talking to is a talented one as defined in the first essay, What makes a talented engineer.

Much of my process builds on top of Joel Spolsky's excellent Guerrilla guide to interviewing, with adjustments for different technology (Ruby vs C++) and specifically focused on abstraction building. I would certainly hire an engineer who is "smart and gets things done"; however, for the purposes of making the process more clear I am specifically looking for:

  • Ability to build abstractions;
  • Ability to produce reasonable looking code in reasonable time;
  • Track record of solving problems;
  • Passion, ambition and drive.

Abstractions

While I agree with Joel Spolsky that ability to handle recursion and pointers are good indicators of an engineer's talent, the fact is that many engineers today have never written C code and the companies I work for don't use C either. Therefore pointer questions become more academic than practical, and while recursion does remain I think it's a weaker indicator.

However, neither C nor C++ offer particularly powerful tools for abstraction building, whereas Ruby for example offers plenty. Using these tools to build abstractions is something I would like engineers to do on a daily basis, and thus my questions attempt to gauge the candidate's knowledge of the tools as well as how extensively they have used them in the past.

Code

A candidate must absolutely write code during the interview, and the earlier they do so the better for everyone involved as this reduces time spent on talking to candidates, sometimes by several people, who would not be hired.

Writing code during the interview, as opposed to in a take home exercise, is critical. As Joel Spolsky observed, there is a huge productivity variance between different programmers; this variance, and the skill level of a particular candidate, is obvious when the interviewer is watching the candidate try to write code, but largely hidden when the candidate writes code at home. Having slow engineers is almost as bad as having incompetent engineers because at the end of the day they burn the company's money but do not produce. Larger companies are less concerned with this but in a startup slow engineers can be fatal.

As interviews are heavily time constrained, coding questions require careful consideration because their answers should reveal as much information about the candidate as possible in as little time as possible.

Solving Problems

This corresponds directly to the engineer being focused on the company's objectives. A candidate with any length of experience should be able to talk about at least one project from the standpoint of this project's business purpose and what the candidate's role in the project was, aligned with said business purpose.

Passion

This is a standard "tell me about a project you worked on that was particularly challenging or interesting" question. Any engineer who worked on a personal project should be able to answer this question. Experienced engineers should have no shortage of suitable projects that they worked on for hire.

In the answer I am looking for whether the candidate was personally interested in what they were working on and if they implemented anything out of their own volition that they were not required to.

Timing

Hiring takes up a lot of time, and being efficient when talking to candidates is a crucial skill. All of the above points can be covered in a one hour interview with a timeline similar to the following:

  • 10 minutes: introduction, talk about what the company is doing and my role within the company.
  • 10 minutes: ask the candidate about an interesting project they worked on. Note whether the technology that their chosen project was implemented in matches the technology we are hiring for. Figure out where the candidate's skills lie on the junior-mid-senior scale, based on how they describe their own work. Choose the coding question.
  • 5 minutes: technical questions.
  • 20-30 minutes: coding question.
  • 10 minutes: candidate asks questions.

As you might imagine, packing all of this into an hour requires carefully choosing questions so that each one provides the most new information about the candidate. Let's look at the typical questions by engineer level.

Junior Engineers

Junior engineers usually are not yet building abstractions themselves but they should understand the ones they are using. Sample junior Ruby engineer questions I would ask are:

  • What does has_many :posts do? (The answer should be more extensive than "it says that a user has many posts".)
  • What is a service object or a presenter, and what is their purpose?
  • What types of tests have you written?

When interviewing junior engineers I would spend most of the time talking about one or two projects they have done to get a sense of their depth of knowledge relating to those projects. The idea is to figure out whether the candidate copy-pasted code from Stack Overflow and tweaked it here and there or designed the system they built.

For the coding part I usually start off with a simple question like Fizz-Buzz or "capitalize first letter of every word in a string". A junior candidate will need a few minutes to write those out. If they do well they get a mid-level question along the lines of "write an application to add two numbers" which I will talk about in a moment.

Mid-Level Engineers

Mid-level engineers should either have some abstraction building experience or be quick in banging out working code.

Technical questions I might ask are along the lines of:

  • How is has_many :posts implemented in Ruby?
  • Is it better to have stateless or stateful service objects, and why?
  • How would you test a frontend application when the API backend is not built yet? (Response should cover mocking, bonus points for attempting to enforce the output of real API matching the structure of mock data that the frontend was tested against.)

Coding questions for mid-level engineers are similar to those for junior engineers but with different expectations:

  • An easy question like Fizz-Buzz, if we have time for it, should be implemented by the candidate very quickly. They should essentially produce the solution at the speed it takes them to write it down.
  • The harder question is the focus of the interview, and I expect the candidate to produce close to a working implementation in about 20 minutes. A good sample question is "build an application to add two numbers". For a backend engineer, the application should be implemented as a JSON API, taking the data and returning the result in a format appropriate for such an API. For a frontend engineer, the application should be implemented in React or their frontend technoogy of choice. The candidate is free to use the internet to look up anything they need, and in case of a frontend candidate I would accept taking a React boilerplate from the internet. The goal of this question is to see how long it takes the candidate to build a working application which involves multiple pieces.
  • For an abstraction implementation, if the candidate said that they did not use presenters for example, I might ask them to implement a presenter (presenter being just about the easiest abstraction to implement). A more difficult question might be "implement a class that logs all of its attribute changes" (I will hint that some kind of a centralized attribute writer is necessary).

The one hour interview format limits coding questions to at most 2, hence a candidate who appears strong early on may get a combination question along the lines of "implement a presenter that capitalizes the first letter of each word in a particular attribute" as their first question.

Senior Engineers

Now we are getting into people that (should) have massive output capacity and extensive knowledge of the tech they work with. The technical questions should reflect that:

  • Have you ever patched Rails/another major library or framework you worked with? (Could be an internal change or an upstreamed change.)
  • When introducing new patterns to an application or a team, what are some strategies for improving adoption? (Trick question. The answer would indicate whether the candidate had introduced new patterns, and whether the candidate worked with enough engineers to understand that there are different levels, and what the different levels of engineers' capabilities are.)
  • How can an engineer improve the number of features being completed in a particular project and the quality of those features simultaneously? (Better tests and higher level abstractions, usually, possibly a technology change that would permit one or both of these to happen.)

Each of these questions can also be expanded on to dive deeper into the candidate's technical prowess ("OK, what was the pattern you implemented?").

For the coding part, the main question to a mid-level engineer regarding the addition application can be suitable as a warmup question to a senior engineer, though in the interests of time I might prepare a skeleton so that the candidate just has to implement the logic. The expectation here is a senior engineer should be able to build an addition application at the speed they type with, in other words, the same speed that a mid-level engineer implements Fizz-Buzz with...

... bringing us to the DSL question. I believe that a senior Ruby engineer should be able to implement a DSL. Maybe not a fancy one on the spot, but something that looks like a DSL. For example, a shopping list DSL:

shopping_list do
  item :bananas, 2, 'lbs'
end

If that looks similar to RSpec tests, or XML builder, it is! DSLs permeate Ruby code and even if they have not built one yet, a senior engineer should have enough knowledge to tackle the undertaking in an interview setting.

If interviewing a senior frontend engineer, I might ask to implement something to enforce object schema as this is a very common issue in JS development. There are many libraries offering the validation itself, the challenge is meaningfully integrating the libraries into an application's workflow.

Matching Seniority And Skill

I believe it is a requirement for a candidate to demonstrate proficiency on a level corresponding to their seniority. In other words, I would tend to not hire an engineer with 10 years of experience in Ruby who performs as a mid-level engineer during an interview.

The reason is often times people with a lot of experience length-wise think of themselves as senior when they don't have the skills to match, and in fact they are not senior engineers. If given free reign, such engineers can halt the growth of less tenured but more talented engineers and if put in leadership positions, such engineers can limit the team's output by stifling better (as in more effective) solutions invented by less tenured but more talented engineers.

That said, I have also interviewed engineers who recognized their limitations and were willing to reposition themselves and improve their skills. A person like that is likely to be a good hire.

Previous: Writing effective job descriptions, next: Engineer progression, or back to software engineering section.