The Transformer model is the basis for GPT-3 and ChatGPT and other systems that have been rocketing AI into mainstream consciousness lately. The ideas for the Transformer model came from a paper called “Attention Is All You Need” by Vaswani et al., published almost exactly five years ago.
AI is going to revolutionize everything, even the strangest and most outdated things. I’m not talking about weird and outdated things like neckties and bagpipes — those will last forever — I’m talking about any business processes that linger on long after their utility has been lost.
Tonight I’m thinking specifically about how AI might change the software engineering interview.
Most interview processes have a potential for bias, and a lack of real-world evidence for their effectiveness. The best businesses are finally starting to worry about bias, but many are still doing interviews the same way they did thirty years ago.
Asking someone to write a program on a whiteboard — say, to reverse a linked list — has about as much relevance to way modern software is actually written as asking candidates to put on scuba flippers and do an interpretive dance about binary trees.
Nobody worth hiring would actually write code without using an editor / IDE, and except for the most trivial chore they’d most likely use Google or Stack Overflow to see how other people do it. Learning from other people is probably the most important part of any job (aside from, you know, doing the job).
Whiteboard coding is more like a fraternity hazing than a measure of someone’s capabilities. One of the reasons people give not to stop doing it is “well, everyone else had to do it!”
To add insult to injury, interview coding / design problems are often timed. Can’t design a complete service architecture and estimate the cost of millions of dollars worth of equipment on the back of an envelope in 20 minutes? Sorry.
Some companies have finally moved on from whiteboard programming (or the online equivalents). One of the interview strategies that employers do instead is to ask candidates to do a take-home coding project, which they then review with them.
Projects are what I’ve been thinking about tonight. At first I was worried, since I just realized tonight that they are doomed. Then I realized I was thinking about them wrong.
Employers like take-home projects because it gives them a chance to verify that a candidate can do the work. But candidates hate them because they take a huge amount of time and often have little relation to the actual work they’d be doing.
There are a few things employers can do, to try to make the project more fair to candidates. Here are some that we do:
- Pay candidates for their time.
- Carefully design the projects to be relevant and similar to the work they would actually do.
- Encourage candidates to ask questions, and treat the project as much as possible like actually working together.
- Work with the candidates to make sure they have enough time. Candidates are often juggling full-time jobs, other interviews, other projects, kids, chainsaws…you don’t want to make it more stressful than it already is.
One of the problems with take-home projects is that an employer can’t tell whether the candidate did the work themselves. You wouldn’t want to hire someone who turned in someone else’s work! What that means is that for software projects, the code must compile and work — but that’s just table stakes. The review meeting helps let them demonstrate that they understand the code and can make small changes to it if a requirement changes.
Wait…back up. What was that about “someone else’s work”?
There are a lot of automated coding assistants now. GitHub Copilot has been around since June 2021. OpenAI’s ChatGPT can write astonishingly good code, along with explanations and instructions. The list of other options is growing by the day: Amazon CodeWhisperer, Kite, Codiga, Cogram, OpenAI Codex, and so on.
Someone using one of those could knock out a project in very little time, compared to doing it the old fashioned way.
I know this for sure, because I just tried that on a small project, and while I had to fix up a few things it was super easy and very fast compared to doing it all by myself.
The time is coming — very quickly — when it will be as crazy to write code by yourself as it would be to…I don’t know…smack your Honda with a buggy whip? (Metaphors failed me. Remind me to ask ChatGPT for a good one.)
But what do software engineers spend most of their time doing?
Reading code. Understanding it. Fixing it. Extending it.
A better interview, or better project, would be one that tested those skills. Those are still by far the most important technical things a software engineer does.
For now.
