So you find yourself in need of some software developers. Maybe your company got a big influx of funding, or you’ve had some turnover, or you’re just looking to keep expanding. So you go to leetcode, pick out some cool problems, and pair them with some good behavioral questions. But, you strike out. Months go by, and you can’t seem to get any qualified applicants. Candidates that do well at the behavioral component seem to struggle with the technical interviews. Candidates that excel at the technical side seem to do poorly on the behavioral questions. The rare gem that tackles both with ease seems to never accept your offer. What gives?
The Status Quo
You went through technical interviews like this. Your friends went through interviews like this. This is what the industry does and if you want to find smart people that’s what you should do too, right? Well, let’s take a step back and think about what it is you’re really filtering for with the technical side of your interview.
Complex algorithms can be really tricky to design on the fly. Even if you’re familiar with the problem and the important concepts to solve it, there’s still a lot of conceptualization. When presented with this kind of problem as part of normal day-to-day work, you would typically do some Googling. You would take your time to make sure you get it right and verify with coworkers as necessary. Contrast that with technical interviews where candidates are solving problems while you watch them think with limited time. Throw in little-to-no collaboration with a team, and this couldn’t be further from normal job responsibilities.
Candidate Success
So what does it mean if someone does well at this kind of challenge? Well, they could be pretty smart since those conditions are way tougher than normal job responsibilities. Of course, they could just be well prepared, too. Some people do hundreds of leetcode problems so they’re just really familiar with the algorithm problem space. It’s also possible that your interview was leaked. Plenty of websites and recruiters have a lot to gain from getting inside knowledge of your interview problems. They will do their best to get this information from previous applicants to give to future ones.
So let’s say you rotate your interview questions often. You also avoid taking anything directly off of prominent websites to avoid a candidate having done your exact problem. All you have learned is that a candidate is good at writing algorithms. Under intense time constraints. While people awkwardly stare at them. How relevant is that specific skill to their future job?
Candidate Failure
What does it mean if someone does poorly at this kind of challenge? It could mean that they’re nervous. Sure, that could be true for anything you ask in an interview, but intense algorithm design has a unique way of causing panic. For instance, if you give me a problem that involves many independent I/O operations, I’m going to make it asynchronous. I don’t need to stop and consider it as it has been ingrained in me over time to avoid I/O bottlenecks whenever possible. However, if you give me an algorithm question and I can’t wrap my head around it right away, I often start panicking as I feel like I’m already failing.
Another reason someone might do poorly is that they’re just not that good at thinking quickly. Unless you’re fixing an emergency bug in live software, most of the time 30 minutes is NOT going to make or break the timing of software release. However, not thinking of an edge case during development could lead to those emergency bug fixes, which CAN make or break software. This drives the point home that being slow and considerate is usually much more important than going quickly. It’s a skill that someone just might not excel at.
Lastly, maybe a candidate is just not familiar with the problem space of the question. Maybe they worked in a language that doesn’t typically have to solve that type of issue so they’re not practiced. Or maybe the subject matter goes farther than what they’re used to, so they struggle to absorb it. There are many more reasons, but these are common ones I’ve experienced while both taking AND giving technical interviews. You can try to mitigate this somewhat by focusing on the problem-solving process of the candidate instead of just the solution itself. Unfortunately, it’s easy for these types of things to distort even the thinking process of a candidate, so it’s difficult to isolate it.
Finding a Pattern
If we collect all these points, we start to see that you aren’t gaining much information from a candidate when they take this interview. It doesn’t even matter whether they succeed or fail. Doing well means they are good at a niche skill that is unlikely to come up very often. I mean, maybe you are a startup that has a great idea and you just really need some algorithm superstars to turn that idea into a real proof of concept to demo to investors. If that’s the case, you’re probably looking for that top talent, and you better be giving top offers.
For the other 99% of jobs out there, you really haven’t learned anything important about a candidate. How well they can actually write real software or how they collaborate would be much more important to know. Likewise, if a candidate does poorly, there are a LOT of possibilities, none of which predict how well that person would perform at their actual job.
Alternatives
See the point I’m getting at here? You’re filtering people that are good or bad at exactly what you’re testing for, no more or less. It’s fair to say that interviewing can never be 100% like a real job, but what if we could get close? I don’t know what kind of developer you need or what you’re looking for, but here are a few general ideas to get you thinking:
Architecture/design questions
Try asking a question that requires a candidate to think through designing an actual software system. This tests the important skill of software design while still allowing you to see a candidate’s problem-solving process. One of my favorite questions to ask is: “Design the software to handle an elevator system.” This can be adjusted for junior applicants by adding lots of details and constraints and possibly skeleton code. You can also leave it as a single sentence for senior applicants. That forces them to work through the ambiguity themselves and/or collaborate with the interviewer on the pros/cons of various approaches.
Take home questions
Consider creating a tough problem but giving a candidate ample time to come up with a solution and get back to you. This allows you to specifically tailor the question to the needs of the position by creating a skeleton repository with missing targeted components. You can also choose any timeframe that fits your needs as well. A short problem can get a day, or if you want to see a thorough implementation of a complex problem, you can give a week.
Live pairing sessions
The closest way to simulate a candidate on the job is to have them actually work the job. Try pairing a candidate with an experienced developer for even a few hours. This can give you massive insight into their technical ability, communication skills, and collaboration style. There is no substitute for putting them in the types of situations you expect them to be in after hiring them.
The Most Important Takeaway
There are a lot of possibilities that aren’t listed here, and this isn’t meant to be prescriptive either. You can take these as ideas and tweak them or run with them in any way you see fit. Take the needs of the role you’re hiring for into account when designing technical interviews to be as realistic as possible. If you do, you’re going to get better results than an off-the-shelf question that doesn’t tell you anything useful about a person. You might save yourself a lot of trouble by avoiding a candidate that is only good at something their role never requires. Who knows, you might just pick up a diamond in the rough developer that wasn’t picked up by other companies because those technical interviews didn’t ask the right questions.