Is Behavioral Interviewing bad science?
 

Most of today's larger aviation companies use behavioral interview techniques (BI, see Job interview - Wikipedia, the free encyclopedia) and heavily rely on the results to select and reject candidates for jobs. If you do not pass the B-I screen, you may not get a chance to interview in person in some cases. In others, the results are taken as conclusive, and when they contradict other forms of interview testing they receive top priority in the interview process. Fail the B-I screen and you're out.

But there is reason to suspect it is bad science. I went looking for specific academic research showing BI is a robust and valid methodology for determining within a few minutes what personality type is for a subject and could not find any. There is support for the notion that structured interviewing in general correlates to better success in determining matches to employer needs. It's the behavioral (BI) techniques I worry about.

Here are some points to debate.

1. I went through various job interviews I can recall in my own past, lots of them to recall, and looked for any obvious correlation between heavily-structured job interviews with my getting the job or not getting it. I also looked at this for the lightly-structured interviews in my past. Turns out there is a strong tie in my past for unstructured interviews with getting the job, while there is a perfect correlation in my interviews where BI techniques were used, with NOT getting the job. I have a bunch of structured interviews to draw on where about 5-6 also contained a lot of BI interview questions. This brief but clear history strongly suggests I do NOT get the job whenever BI is used. Make of it what you wish.

2. I tried to surf up academic research supporting the notion that behavioral interviews correlate to higher success in candidate selection, and could not find any. Nothing. However, there is support for the claim that structured interviewing in general produces better matches to employer needs. But the lack of academic research supporting BI techniques (specifically) is troubling. I suspect a leap of faith is going on where employers assume if it's modern psychology then it must be reliable, which may or may not be true.

3. Although there is an argument that faked responses to BI tests by job candidates (subjects) would tend to be avoided by most of them for a wide variety of reasons, most people at least want to be honest, it is inevitable that the results are inaccurate because the replies are all taken directly from the subject where error is completely unaccounted for. If the subject sees their own behavior the wrong way, they will report it wrong, and human beings are notorious for inaccurate self-perception. This could be contrasted with another scenario where a group of coworkers gives views of a subject and the collection is assimilated for accuracy and consistency. But this is not how BI tests are currently administered.

4. The idea that behavioral tests are cleverly designed to "see into the head of the subject" and the actual answers they give not do not matter individually, is not solid science. If it were that simple then courts of law would use it like they use DNA. The standard behavioral "truth trick" is to present multiple versions of the same question and look for discrepancies. But this is unreliable, because it is based only on the subject's self-perception, which is known to be unreliable. If the subject believes their own incorrect and/or fantastic views of their own behavior, then the truth test fails. All you would detect with this check is whether a subject believes their own views, NOT whether the story is true or accurate in itself. And without testing anything but self-confidence factor, what you are finding out is whether someone views their own behavior one way or another only. Detection of lies to any degree of accuracy is flawed in this test.

5. If a company reports to weak (junk) science to hire people then it will probably use the same material to fire them. That may not be any worse than complete random or subjective HR action, but it is not the way most people want to be treated in the workplace. If you do not like an employee or have serious problems with their performance you can document is ok, but basing hire/fire/promote/demote decisions on bad science would be unethical.

Please be constructive in offering your own thoughts.

I'm not sure it is science at all. No scientific method. No hypothesis or experiment.

Let's say you had 3 good hunting dogs and 3 bad ones. You notice that when you rub your eyes the good hunting dogs tilt their heads while the bad ones don't. So you start thinking of ways to test these dogs. You sing "Snoop doggy do og" and only one of the good hunting dogs barks but all the bad ones bark. Some tests help classify your dogs according to their hunting ability. Some don't. Some tests are really elaborate and weird, but they classify well. Other tests, such as when you directly as the dog "do you like to hunt", do not classify the dogs.

You find a dog for sale. You go to the seller's house to have a look. You jump around like a chimp, repeatedly slap yourself, do helicopter dick, and sing "say my name say my name". The dog's reaction isn't similar to the reaction of your good hunting dogs so you decide not to buy. The seller accuses you of bad science.

Btw: I had to change **** to tilt because this forum is a kindergarten classroom.

Are you a pilot or an HR person playing one on the Internet? It’s their train-set, play the game and see what happens.


Gf

This is pretty much it.

Personally, I think some of the elaborate stuff is weak sauce. Some of it might flag a Charley Manson, but I doubt it does that much to hone in on good pilots or employees. But it's their ball, so you have to play by their rules. Maybe get some coaching.

Best case, it might improve their selection for whatever they're looking for but it likely weeds out a lot of folks who would be great assets, ie good at filtering but not at selecting. Saves some money I guess but I'm thinking there ought to be a law...

Having watched and participated in the airline hiring over 40 years, I will say the systems used seem to be pretty good at hiring the type of candidate the HR department wants.

Four of us at my old ANG unit were rejected at AA, in the days of three-part interviews and astronaut physicals, and hired at EAL. And nearly vice-versa. The process does reject those who would be great assets, but I think it is more geared toward flagging and rejecting those who HR believes aren’t a good fit. They don’t necessarily want excellence as much as avoiding future problems. As a friend said, “every employee, but especially pilots operating $100 million assets, has a personal P&L statement”.

GF

But I don't think there's that much variance for pilot "fit" between major airlines. If you select for good leaders and good pilots, that doesn't leave you a whole lot of wiggle room for other traits.

I'd be curious as to what they think they're looking for. I could see a business case for selecting for "compliant" (ie will roll over at contract time) but that would seem to have a compatibility issue with assertive leadership. Might get away with passive seat-meat for FO's, but that wouldn't go so well at upgrade time.

But I'm no expert on this kind of stuff, all of the hiring I did was the old fashioned way (an interview or two, maybe a written or practical test).