What's the "AHA!" REPORT all about?

This series of newsletters contains AHA! information to help people and organizations hire the best employees, make the best promotion decisions, retain the most qualified people, maintain the widest applicant pool, follow best practices, and (if you are subject to US law) remain aware of EEOC hot-spots.


  Aha! Report Archives:

February 2010
December 2009
October 2009
August 2009
June 2009
May 2009
April 2009
February 2009
January 2009
October 2008
July 2008
May 2008
April 2008
March 2008
January 2008
November 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
January 2007
December 2006
October 2006
September 2006
August 2006
July 2006
June 2006
May 2006
April 2006
March 2006
February 2006
January 2006
December 2005
October 2005
August 2005
July 2005
June 2005
May 2005
April 2005
March 2005
First Issue - March 5, 2005

 
May 2007: A personality testing tutorial
 

 

 

Research tells us that overall job performance is half skills and half motivation. Basically, people want to do what they like best – and avoid doing what they don’t like. We see this in common job stereotypes: salespeople love meeting prospects, but hate paperwork; accountants excel in accuracy and timeliness, but many dislike social interaction.

Ability without willingness leads to underperforming employees. Willingness without ability leads to employees who make mistakes. While “hard skills” tests cover the first half of job performance, the role of personality testing is to evaluate the motivational side. Despite the bewildering variety of personality tests that are available in the market, this basic goal is fairly universal.

Our own Attitudes, Interests, and Motivation (AIMS) survey reflects that purpose in its very name. AIMS scores are neither magic nor pop psychobabble. They are not set arbitrarily and do not represent an ideal job image for any industry. What they do provide is a rank-ordered summary of how the candidate describes his or her self.

This simply means AIMS at the top of a candidate’s list are the most important and AIMS at the bottom are the least important. An AIMS report’s simplicity also means that interpreting the scores can be daunting. (This is true for any other personality test as well.) The tricky part is in the interpretation. But when you’ve mastered the curriculum that follows, you’ll be able to understand AIMS scores, discover hidden attitudes, interests, and motivations, and use them to predict real-life job performance.

Survey Basics

  • Remember the goal: hire fully qualified people and treat every applicant fairly.
  • Personality factors are directly related to job performance. Independent research also shows these factors are highly stable over time, so what you see is what you get.
  • These are not skills. They are attitudes, interests and motivations about whether to use the skills they have – “will do” rather than “can do.”
  • Every AIMS factor score is calculated from responses to 5 statistically related individual questions. This reduces the effect of any single question on an overall factor score.
  • Each factor score is normed by comparing it to a large applicant database. This allows employers to compare an individual applicant to a population norm. Ignore raw scores – the information is in the norms! 
  • Norming also makes the AIMS test very hard to fake. The applicant would have to know which questions load on each factor, the norms for each factor, which items were in the reliability scale, and the overall pattern required for the job. 
  • Individual AIMS factors are not-position specific. The patterns and strength of the applicant’s personality factors provide the most critical information.

First Step: Examine Answer Consistency
The AIMS report scores consistency on a bell curve; exceptionally low scores or high scores means the applicant answered at the extreme ends of the scale and their responses may not be trustworthy. To make it easier to read, the bell curve is “folded in half” so that a normed score of 15% represents either the top 15% or the bottom 15% of the bell curve (it does not matter which).

A score of 100% represents the exact middle of the bell curve. Any score above roughly 15% means the applicant answered the questions the way most applicants do. We suggest distrusting results from any survey with a reliability score below 15%.

Second Step: Look at Priorities
The AIMS priority score evaluates applicants’ willingness to put job goals ahead of personal goals. A person with a high priority score shows that he or she is willing to work overtime, work weekends, and make sacrifices for the job. Because these items are “transparent”, we expect applicants to score above 50%. Work scores below 50% indicate a lower than average job priority; we advise caution.
 
Third Step: Identify Patterns
Look first at the top three AIMS scores. These are things the applicant really “likes to do.” The bottom three AIMS are things they really dislike; those in the middle are “take it or leave it.” The following examples are stereotypical profiles intended to illustrate how patterns in these priorities work together to predict motivation

A word of warning: There can be substantial differences between one job and the next, even with the same title. Therefore, we strongly encourage every organization to research and define their own job-related standards. It’s not just a nice idea; it’s an established best practice that can make your selection process more legally defensible.

Project manager
Top three: Need to be perfect; Desire for structure; Frequent problem solving
Mid range: Innovation and creativity; Compete and win; Quick decisions
Bottom three: Willingness to change; Close personal relationships; Expressive and outgoing

This pattern tells us the applicant puts a great deal of emphasis on being perfect, following rules, and solving problems. They may dislike dealing with people, being flexible, and being outgoing. Innovation, competition and impulsiveness are ambivalent. 

Sales “hunter”
Top three: Compete and win; Expressive and outgoing; Frequent problem solving
Mid range: Close personal relationships; Quick decisions; Innovation and creativity
Bottom three: Willingness to change; Desire for structure; Need to be perfect

This pattern tells us the applicant puts a great deal of emphasis on winning, being outgoing and gregarious, and solving prospect problems. They may dislike accommodating others, following rules, and being perfect. Forming close personal relationships, impulsiveness and innovation are ambivalent. 

Inbound customer service
Top three: Expressive and outgoing; Frequent problem solving; Willingness to change
Mid range: Close personal relationships; desire for structure; Innovation and creativity
Bottom three: Quick decisions; compete and win; Need to be perfect

This pattern tells us the applicant puts a great deal of emphasis on being friendly and helpful, solving problems, and being flexible. They may dislike being assertive, being competitive, and being perfect. Forming personal relationships, being innovative, and desiring structure are reasonably high but not primary drivers. 

Conflicting Patterns
Experience shows that some factors tend to cluster together while others are dramatically polarized. For example, scores on “compete and win” and “willingness to change” tend to “avoid” each other, as do “quick decisions” and “need to be perfect.”

Caution is advised when conflicting scores are near each other. For example, a high score in impulsiveness combined with a high score in close personal relationships indicates someone who “says” they prefer making quick decisions, but is also very sensitive to others’ feelings. This could mean that the applicant is either internally conflicted or trying to fake the survey. In either case, we advise caution.

Response Strength   
The final thing to consider is the “strength of response” for each personality factor, expressed as a percentile compared to the normative database. 50% represents an average score compared to all applicants; anything above is higher than average. We usually see two different response-strength patterns:

  • A very small difference between the highest and the lowest percentile means the applicant did not use the whole four-point scale. This can mean the candidate has no strong differences in preferences (unlikely) or is trying to fake the test.
  • More typically, a broad range of percentiles means the applicant expressed strong differences in attitudes, interests, and motivations. The bigger the difference, the stronger the opinion. Really extreme scores are not necessarily bad, but they do indicate a need to match the candidate very carefully against job requirements.

Final Exam
Congratulations. By now you should have a pretty good grasp of how to interpret an AIMS report. Of course, this doesn’t make you a qualified I/O psychologist. (That would require years of study and a painful amount of mathematics.) Likewise, this tutorial doesn’t give specific instructionsabout personality tests other than AIMS. However, it should give you a firm foundation for understanding what personality testing is all about and how any personality test works.

This in turn should help you make more educated decisions about hiring processes – and recognize when people are talking nonsense. And isn’t that one of the greatest benefits of education?