  |
|||||||||||||||
 |
 |
 |
 |
||||||||||||
 |
|||||||||||||||
|
 |
January 2007 | Issue 21
The "Aha!" Report What's the "AHA!" REPORT all about?
This series of newsletters contains AHA! information to help people and organizations hire the best employees, make the best promotion decisions, retain the most qualified people, maintain the widest applicant pool, follow best practices, and (if you are subject to US law) remain aware of EEOC hot-spots. Testing Do’s and Don’ts One enduring mantra: if you can’t measure it, you can’t find it. (And if you can’t find it, you can’t hire it.) Use tests to measure the qualities you’re looking for. Lots of tests. Tests are good. Right? Tests are good – for some things, but not others. But I'm getting ahead of myself. Let's start by dividing tests into two groups: 1) those designed for hiring, and 2) everything else. Now, forget about the tests in the "everything else" pile. We only care about hiring tests. Hiring tests are based on a specific theory associated with job performance: no theory, no test. They use this theory to predict future job success: no prediction, no test. Developing a good hiring test requires investigating many jobs, looking for common threads underlying job performance, running controlled studies, using fancy statistics, and verifying that test scores are strongly correlated with performance ratings. This is called validation, and it basically means test scores are accurate predictors of job performance. Validation can refer to the content of the job (typing tests = typing skills, problem solving tests = problem solving skills); or, it can refer to level of job performance (typing score = processing speed, problem solving score = profitability). There is a third form called construct validity, but let’s not worry about that for now. No vendor’s claim is proof that a test is "valid" for your application. Each organization is responsible for its own validation work. If a test you use was not designed explicitly for hiring (see above), or if it is not validated, you need to find another test. Sorry to be blunt, but it saves time. Hiring tests also come in two main varieties: 1) the skills variety, 2) the self-descriptive variety. If you want to achieve accurate results, you need to know which is which and use them appropriately. Skills Tests Other tests like hand-eye coordination, spatial ability, or mechanical ability also qualify as skills tests because they measure some kind of sensory or physical ability important to the job. I once recommended a test that required putting little pegs into little holes. Why? Because the job was assembling car radios, and the test measured fine motor control and fast finger movement – precisely the kind of skill needed to place little electronic components into teensy circuit board holes. There are many other types of skills tests, but the main point is that skills tests are very hard to fake because they require a specific ability. If you do well on a skills test, a hiring organization can be pretty sure you will do better than someone who fails the same test. As a matter of fact, the only time skill tests become "shaky" is when they are either too advanced or too basic when compared with job requirements or when the applicant practiced the test ahead of time. But mostly, it's like testing basketball players for being tall. You might not always be perfect, but it sure helps play the game. Self-Descriptive Tests If you don’t see anything wrong with this idea, let's get personal. Have you ever made yourself look good on a test? Have you ever tried to second-guess what the organization was looking for? Have you ever changed your mind, and your answers, from one month to the next? Self-reports are like that: they change. Bob Hogan, one of my dissertation advisors and a leading researcher in the field of personality, calls this "socioanalytic theory:” giving answers that correspond to how you think you should present yourself to others. Bob says that before you put too much faith in self-descriptive tests, you should think about whether the data represents actual truth, or only how the test-taker wants to describe himself. Unsurprisingly, self-descriptions have very low statistical relationships with abilities. In my studies, I found that scores on a big-five personality test have about a two percent relationship with mental ability and an eight percent relationship with interpersonal skills. Pretty dismal correlations. Now, a self-descriptive test might (if validated) be a useful tool for measuring job fit, but it should never be used to make an accurate prediction of job skill (unless you can produce research showing that scores on the test have a strong relationship with criterion scores on the job – good luck). This is why reputable test vendors actively discourage the use of their self-descriptive tests for selection, or will insist on a scientifically conducted validation study prior to use. It’s not just good business practice. It is also ethical. When Self-Descriptive Tests Go Bad The HR manager built in some tests. On first blush, it seems like teamwork would be a good thing, and he must have thought so too, because he interviewed for teamwork, created simulations that measured teamwork, and made hiring decisions based on teamwork. What did he get? Three hundred employees who would not hold a meeting unless everyone could attend, could not make a business decision unless everyone agreed, and wanted to regularly socialize after work (to extreme spousal objections). Meanwhile, the company had to expand their assembly building to fix quality problems caused by people who would not confront co-workers producing poor quality work. This example illustrates the power of misused hiring tools. The HR manager wanted teamwork. He got teamwork. He just forgot about all the other things that were important, like productivity and quality. I don't know how long it took for his company to get the mix right, but it certainly didn't happen overnight. The HR manager is no longer employed at the plant. In another example, a very large organization put in hiring systems for plant start-ups at some large auto manufacturers. They selected people who were very smart, highly motivated, good with people, and hard workers. All very good when starting up a plant, right? Now, fast-forward two years when all the start-up stuff is over and production has become routine. Guess what kind of problems a plant with all leaders and no followers will have? Organizational needs change depending on growth stage. It wasn't their fault; the client believed the consulting firm knew what it was doing. Too bad. A Little Hard Work Of course, you can always use skills tests and self-descriptive tests conscientiously and follow best practices. The good news here is that you will get exactly what you measure. The bad news? Well, you just have to work a little harder to find it. |
 |
||||||||||||
 |
|||||||||||||||
Join Email List