Brain-teasers make bad interview questions

The other day BitTorrent posted a job posting to the BayPiggies mailing list with the following brain-teaser to solve in it:

What is the exponent of the largest power of two whose base seven representation doesn't contain three zeros in a row?

Normally, I hate "brain-teaser" type interview questions. I make it a point to never ask brain-teasers in interviews and don't put too much weight on candidates' answers when my fellow hiring managers ask them. The reason is simple: the typical brain-teaser is too easy to game.

Take BitTorrent's question, for example, the answer is 8833.

BitTorrent asked for python code to solve the problem, but that is a red-herring. You cannot solve the problem in python. Sure, you could write a program like the following that prints all of the powers of 2 that do not have three consecutive zeros in the base-7 representation:

    def base7(x):
       digits = []
       while x != 0:
           x, r = divmod(x, 7)
           digits.append(str(r))
       return ''.join(reversed(digits)) or '0'

   i, x = 0, 1
   while True:
       if '000' not in base7(x):
           print i,
       i += 1
       x += x

But BitTorrent asked for the largest power of two that meets the requirements. How do you know when a number printed by the above program is the largest? You don't! There are infinitely many numbers and this program has to test them all. That'll take a while.

The full solution to BitTorrent's problem requires you to find the upper-bound of the search -- which can only be done without a single line of python code (despite their requirement to use python): you have to solve it using math. Which implies that BitTorrent actually wants their ideal candidate to be a mathematician first and foremost. That isn't necessarily a bad thing; I don't know anything about BitTorrent's business, maybe they need people with particularly strong math backgrounds (much stronger than mine, that is).

But there is actually another way to obtain the solution: use Google. So knowing how to work a search engine will get you in the door just as effectively as having an advanced math degree. Go figure.

Neither skill says anything about the candidate's programming ability or knowledge of python in particular. In fact, I would argue that attempting to "solve" the problem by writing a python program without including the proof supporting your solution should disqualify the candidate as demonstrating that they don't understand the problem. Writing code without understanding the problem is well into Code Cowboy territory.

Anyway, the point I wanted to make was that it has been my experience with "brain-teasers" that at best they give the interviewer some vague idea of the candidate's problem-solving ability. Far more often, however, they randomly filter out people who may be perfectly good programmers but who didn't happen to have memorized (or found online) the answer to the particular brain-teaser you throw at them.

When I do interviews, I figure I only have a limited time to interview a potential employee. I would rather spend that time asking questions specific to software engineering, computer programming, or the problem domain we are hiring for. I always prepare a progression of real-world questions ranging from "easy" to "guru", with no expectation to ever ask the "guru" questions -- my intention is to see what the candidate's level of ability is and, more importantly, to see how they handle a question or two slightly beyond their ability. Maybe it's because I'm not too bright myself, but that tells me much more about the candidate's suitableness for a position than an abstract brain-teaser ever does.