I was talking with Jacob as he was writing his blog post about "reverse" code review (asking a job candidate to review code you wrote), and told him a memory from early 2012, back when I was working at the Wikimedia Foundation.

WMF was hiring its first security engineer. Tim Starling was, at the time, the person who did the security-related code reviews at WMF. To test the candidates for the security engineer position, Tim wanted to check their code review skills. So he visited some Brazilian software-sharing sites where programmers shared software in .rar archive form, because Google didn't index that, and he found a suitable web app written in PHP, translated the variable names and comments, and audited the application himself. He found twelve vulnerabilities. Then he gave each candidate the evaluation task of finding as many vulnerabilities as they could -- and Chris Steipp was the one who found thirteen. And we hired him.

(Before publishing this, I checked with Chris and Tim, who gave their okay. Tim also said: "I can't confirm the numbers 12 and 13 but he certainly found a lot of vulnerabilities and was by far the best candidate.")

This sticks in my memory partly because of the almost fairy-tale nature of the test and the achievement. And it's a nice memory because it's how I got to work with Chris, who was a gem of a colleague. But it also speaks to the value of the reverse code review task as a work sample test, because if you're looking for someone with keen perception as a reviewer, isn't it wonderful that you can structure your interview process properly and discover just how surprisingly perceptive your candidates can be?