Having recently spent a couple of months looking for my next career opportunity, I think it’d be great to write down my interviewing experience, and my personal comments to them. Throughout the process I’ve learned some things about companies, some things about tech careers, and some things about myself. I’m hoping this will shed some light on how we’re currently interviewing software engineers as an industry. The particular experience varies a lot by the individual, but I’m sure my experience covers at least some of the common cases.
Whiteboard vs Project Based
The (infamous) whiteboard interview makes up the vast majority of my experience. I’m almost sure this has to do with my target companies being mid-sized or large ones. Throughout my search I saw 2 project based interviews, both early stage startups with less than ~30 people. Anecdotal evidence has it that this somewhat conforms with the industry-wide ratio of non-whiteboard interviews vs whiteboard ones, since we have so few companies doing non-whiteboarding that all/most of their names can fit a curated short list.
In terms of the structuring of how whiteboard vs project based interviews, an analogy is the SQL vs non-SQL databases – whiteboarding is like the SQL database, where you have standardized structure for every 45-60 minute session. Anything not conforming to that goes into the non-whiteboarding terrain, where companies in that space chooses their own adventure. Because of the flexibility, I find that project based interviews tell more about the skills and values that the team really cares about. We should do more of this!
For project based interviews, a common criticism is the asymmetrical large amount of time dedicated by the candidate, compared to the contributions from the team. A great solution to this is to have a mentor/interviewer shadow the candidate along the way – pair programming if onsite, opening up irc/slack channels if remote. This simulates the quick feedback loop of side-by-side whiteboarding, so that the candidate can get that fair share of feedback.
On the other hand, whiteboard interviews are like standardized tests. The structure, content, and communication flow are so predictable that sometimes I had that strange feeling that both the interviewer and I were just bots acting out of our Software Engineer Interview Protocol. Not saying that the problems weren’t challenging, but standardized tests are shown to be easy to crack just by spending a dedicated amount of time studying. In fact from what I’m seeing, software engineer interview prep has become a booming industry, where all those sites monetizing off of practice problems that any CS majors should have done in Algorithms 101. Remember the #1 criticism for project based interview is the favor over people who have the time to complete projects on the side. To me, it seems like whiteboarding favors the same set of people who has the privilege to spend time practicing extensively.
The Confidence Factor
As an unsolicited advice to my past/current self, you should build enough confidence in yourself before walking into interview sessions to get grilled. I cannot overrate how far confidence carries me in the job search process. I’ve had moments when it kept me motivated, and also moments when the lack of it held me back. It affects what roles I applied to, and how I put up proposals to more open ended design problems. Not to mention those behavioral questions – they’re supposed to be all about my own stories.
Knowing that I am judged through a pair of critical lenses is extremely unnerving to me. So during my interviews I was constantly battling my internal imposter syndrome plus tackling the technical problems. While I still managed to get my best out of those interview sessions, my effort to building confidence obviously shouldn’t end here.
Randomness
From the perspectives of the candidate, the interview process is literally a series of blind dates. Only that I need to impress every single date at the table. On the other side of the table, my interviewers are all over the map in terms of expectations, follow up questions(initial questions are easy to prepare, but follow up questions need to be relevant to the current conversation), answering my questions, and just general cooperativeness. So if I passed a series of interviews and got an offer, how much of it was me being a great engineer vs I got really lucky to have the exact right panel of interviewer lineup? My hypothesis is that given my skill level is constant, my interview feedback reflects more of the interviewer than of me. To experiment this, if I were to make a copy of myself, and have two different interviewers ask the same initial questions to me-copy-1 and me-copy-2, and do their follow up conversations from there, it’s pretty likely that they would provide completely different feedback. This is because interviewers differ in the following dimensions:
-
Willingness to assume best intent, and engage in a bi-directional conversation
Is the interviewer set to discover the best of the candidate, or is trying to find faults?
How much does the interviewer want to share their own opinion to contribute to the discussion?
Does the interviewer monitor the conversation to keep discussions on topic, and identify if the candidate needs a nudge or help? -
Aptitude to make as little presumptions as possible, be curious and ask why
Does the interviewer have a best/correct answer in mind for an open-ended problem?
If the interviewer disagrees with the candidate on a point, does the interviewer ask questions to explore this point further?
Does the interviewer ask questions when they don’t understand a point made by the candidate? -
Ability to ask high quality (follow up) questions to explore the boundaries of knowledge/experience of the candidate
How does the interviewer choose the content and timing to ask questions?
Does the interviewer have experience related to the candidate’s proposal/story?
These skills are especially important for open ended interviews like behavioral and system design, so that the interviewer can maximize the amount and quality of signals captured during those bounded minutes. In the field, I’ve seen open ended questions asked by mostly senior/staff engineers and managers, since they tend to be better in those dimensions above. Another ingenious solution I want to call out is to pair up interviewers in the same session. With 3 people in the room, conversations flow more naturally as adversarial-ness/adversity dilutes, and probability for high quality questions boosts by 100%. For the exact same candidate performance, the team now have two independent ratings to work with, so when the ratings differ by over a certain range, the team can do a deeper review about either the candidate or the rubric. Doing two back-to-back ratings is also how essays are graded in standardized tests (if not automated! …yet!), where inter-rater reliability was researched extensively.
Bottom Line
Overall my interview experience has been very positive – all my interviewers made efforts to make the session(s) as smooth as possible, and I’ve had many interesting conversations with my interviewers. Another thing I’m happy to see is the trend for opening up feedback, often in between the phone interview and the onsite, and sometimes after the onsite. I have some concerns about how a resume is selected for interviews vs not selected, but that’s for another day.
Now after this is all done – even with all these hours poured in from both the candidate and the team, there’s still a whole lot of uncertainty on how things will turn out after the candidate joins the team. This is fine, since now training/mentorship should be closing the expectation gap for the team (…training/mentorship is available, right?).
On a personal note: this job search took me around 4 months end to end, on an average of 4 hours per day, which to me is a pretty significant amount of time. But comparing to the next ~2 years with 40 hours per week, this is fair investment. I’d like to think about looking for the next job opportunity as if I’m looking to get into a long term relationship. Patience, dedication of a significant amount of time, and personal efforts all goes into the game.