"Google's Broken Hiring Process"

So I saw the above headline and I just had to look. Google is supposed to be stuffed with really smart people. The article quotes Peter Norvig, "Google's director of research, former Google director of search quality and former head of the Computational Sciences division at the NASA Ames research center" as follows:

One of the interesting things we've found, when trying to predict how well somebody we've hired is going to perform when we evaluate them a year or two later, is one of the best indicators of success within the company was getting the worst possible score on one of your interviews. We rank people from one to four, and if you got a one on one of your interviews, that was a really good indicator of success.

Sounds bad, right?

But it's not. As I preach to my students: always ask who's in the sample, and why. The article, two paragraphs later, quoting Norvig, tells us:

Ninety-nine percent of the people who got a one in one of their interviews we didn't hire. But the rest of them, in order for us to hire them somebody else had to be so passionate that they pounded on the table and said, "I have to hire this person because I see something in him . . ."

One percent of the time, Google overrides its own "hiring process" because those rare applicants have something important that that process evidently misses and those unusual applicants turn out to be excellent employees. The allegedly "broken" process is simply an example of sample selection bias.

Conclusion: as always, journalists could benefit from more statistics and more economics.

Comments

[JorgXMcKie]

99% of 'journalists' could also benefit from someone finding a cure for cranio-anal inversion.

[RogerClemens]

This is exactly the same reason why findings that SAT scores don't predict GPA at top universities aren't meaningful.
Thanks for such a clear post.

Roger