Topology of Software Interviews

You hear that giant sucking sound in the distance? That’s thousands of your money-hungry competition furiously reading and re-reading solutions to Leetcode problems so that they can walk into an interview having spent several hundred dollars to know ahead of time what the likely assessment problems will be and nail them. After this interview they will get offers, if you applied and did not do this secret handshake you will not, and many of them will go on to uneventful careers making enormous amounts of money mostly replying to their team in a messenger of whatever form because of the geometric nature of connections among teams.

You see, it’s not literally the case that adding an additional teammate (+1) adds n (existing team) + 1 connections with the same bandwidth, but it does introduce a new asynchronous chokepoint for any work in which the new coworker is involved. It is impossible to them to add more productivity to the project than the last addition, and they will likely introduce complexity that will even slow down the onboarding of future employees. Safe to say, somewhere here is a point of diminishing returns and many large companies have probably found it, whether in a graph on a screen or just an inability to pivot with market forces bearing down. So why are we doing Leetcode questions in interviews?

In comes our antagonist: the technical software interview. These companies at or near the point of diminishing returns don’t think about refactoring teams, shedding unnecessary roles or spinning off smaller more nimble businesses with controlling interest in shares, nay! Instead they will simply eat more startups that have been performing well, they will grow ever larger in size and instead they look to the smallest atomic unit for more performance: the need for the “10x dev” is born.

How do you ensure that your programmer is incredibly capable? Simple. You ask them to synthesize an algorithm that has close to no applications in the real world, and if they can solve it they must be a genius! It’s not possible that any of them will get turned away, become disillusioned and spend enough time and resources to reduce them to wrote memorization of the algorithmic patterns. Except: spoiler alert, programmers have done precisely as described.

Maybe it works out. If you’re a mega-corp who needs programmers who will slog through an incontrovertible mountain of red tape and bureaucracy to make a tiny change, maybe the programmer that will slog through Leetcode is a precise fit.

If you actually want them to do 10x more work or 10x better work though you very well may be out of luck. Having recently started a scenic stroll through the swamp of arbitrary prompts myself (the Leetcode problem set), I can’t imagine gaining a wholistic knowledge of systems the way that I did actually learning to program starting at hosting websites on servers where I had complete and sole administration responsibility. No amount of Leetcode will teach you to the mantra “check the f***ing logs”, or that the un-linted, un-compiled SQL code nested in your software is incredibly suspect if your error happens a few lines below, or that all too often the easiest solution is an integration, not your own home-brew hacks.

Outside of all these considerations, OpenAI’s Large Language Model(s) have already demonstrated great proficiency at repeating Leetcode solutions and candidates in live coding interviews have already exhibited behaviors raising suspicion that they’re using ChatGPT. Platforms for technical interviews are already thinking about this problem

Don’t get me wrong, there are real merits to presenting your candidate with a Leetcode easy problem. Like fizz buzz or any other simple problem it can easily weed out those who can program a computer from those who can’t. My objection here is with the unflinching requirement of medium and hard problems in interviews. My experience only includes a subset of all possible types of interview but for the amount of money I make or could make the variety is pretty surprising.

So how should Interviews be done? What kind of questions are best? What kind of considerations should be made about the interview given the specific role?

I’m glad you asked.

First, interviews and the questions asked in them should never be irrelevant to the role. If they are technically irrelevant, they should be strongly behaviorally relevant. What do I mean? If you’re going to ask a candidate “You’re given an elephant, you may not kill it or give it away, what would you do?” They better be a zookeeper or literally applying for an executive position at Enron post-scandal because nobody else faces that kind of problem and so they’ve never even thought about traversing the problem space mentally, let alone under pressure hoping to secure money that they need to keep themselves and dependents alive.

Second, the interview should never fall quiet. In my first live-programming interview both of the interviewing team members gave me a prompt and fell absolutely dead silent, only responding to my questions telling me I can’t use tools and documentation that would normally be available. When they did answer clarifying questions it ended up confusing my understanding of the prompt. If I had known that would be the experience I might have declined.

Finally, I personally think the only route to take when finding your next best candidate has to be something very similar to what they’re actually going to be doing in their work.

Whether you conduct take-home tasks or live programming as part of an interview processes, all three of these recommendations are worth taking into account.

I think more than anything at all it should be at the top of an interviewer’s mind that they’re not only trying to weed out the unqualified, you’re trying to sort the proficiency of the qualified for the position. Nothing that deviates from measuring in that direction is going to be helpful unless there’s some pathology around your business like a culture of working past the end of day.

I know we can do better because I’ve seen better. I think it would be incredible if I’d never seen the worse but everyone should be on the lookout and repaying these poor hiring procedures with dropping from the process or declining offers. After all, learning if the company is a good fit for you is a part of this process, otherwise you should work somewhere else.

If you’re yelling at the screen right now “I don’t have choices I need a job!” I’m sorry and this blog post isn’t really aimed at you. I wish you the best of luck and have also taken job offers in states of duress and will never blame you for what puts food on the table. I also hope that someday the process is much better for everyone, desperate or not.

Thanks for reading!