The Caffeinated Penguin

musings of a crackpot hacker

Being fast vs. being correct

Posted By on October 28, 2015

I’ve not blogged about my job hunt because it’s largely been unremarkable. However, one firm threw me for a loop recently in that they had me take a skills assessment. This is different than the “normal” type of assessments, which are things like “write me some code which does X”. As a general case, all of these are a waste of the applicant’s time, so I can’t imagine that companies who have such policies have a lot of applicants who are already employed. I only have time to do it because I’m on payroll to be in the office to answer the couple of questions which arise during the day.

Anyway, the test they had me take is supposedly roughly similar to a Wonderlic test. See:

Go take one, they’re fun. I’ll wait.

They remind me as roughly similar to IQ tests and, as I understand it, seem to try to test the same thing. Unsurprisingly, I score similarly on both. Now, I have issues with IQ tests in general (more on this later), but that’s not really my main objection to this test as a pre-screening for an engineering position. My objection is that what it measures (intelligence) is not of primary importance to being a good engineer. I’d argue that your top three qualities for a good engineer are:

  1. Being correct.
  2. Being thorough.
  3. Being smart.

The scoring of both IQ and Wonderlic tests are based on time limits. However, such time limits are not reflective of real-world situations, because, only very rarely do you need to do engineering under extremely short (say, 10 minute) deadlines. And, since lives may depend on it, being correct is more important than being fast. As such, a good engineer will check his work, running the calculation several times over by different methods to ensure the same result. Do this for a couple of decades, and it becomes so automatic, it’s hard to turn off when doing one of the aforementioned speed-based tests. Therefore, such tests likely disadvantage good engineers because the prime requisite is not emphasized in favor of the tertiary requisite.

On top of that, the secondary requisite isn’t tested at all! Now, I don’t know how to test that someone is thorough, excepting to pose a situation with a pile of corner cases which need to be handled. The best vehicle for this is likely a programming problem. But, basically:

  • It doesn’t matter how fast you are, if you’re wrong.
  • It doesn’t matter how fast you are, if you miss corner cases and it all falls apart because of that.

Now, on to my problems with IQ tests (because I’m sure I’ve hooked everyone with the tease and they’ve been waiting). My problem with IQ tests is that they say I’m smart (I consistently score 140 +/- 10), except I do not believe that I’m smart. That’s it. I have no data, just an anecdote with a sample size of 1. (Publish!)

Empirically, and “getting outside my own head” as it were, this is likely a result of selection bias. If you take, as a baseline, the idea that an IQ of 70 and below constitutes “mild mental retardation”, then you’re 30 points below the median score if 100. So, if you have an IQ of 120, and work with a bunch of people with IQ’s of 150, then, compared to them, you’re mildly retarded, despite the fact that you are “superior” relative to the general population. However, since you work with these people on a day to day basis, likely select friends of similar attributes, should definitely select a mate with similar attributes, raise your children similarly, etc. you surround yourself in this bubble of smart people which can lead to a skewed perception of what the world is actually like. (Not unlike how people surround themselves with similar reinforcing viewpoints on politics, social issues, etc – the “echo chamber” effect.)

Now, all that said, even though rationally I realize that the above is likely true, I find it hard to detangle myself from the viscerally emotional feeling of “not being that smart” – likely because I am a perpetually unsatisfied type A control freak.

So, my criticisms of IQ tests are likely all in my head, but one perceives reality through ones own cognitive biases, so that doesn’t make it any less real.


Loading Facebook Comments ...

Leave a Reply

Please note: Comment moderation is currently enabled so there will be a delay between when you post your comment and when it shows up. Patience is a virtue; there is no need to re-submit your comment.