Not Your Best Profile
In the world of assessments, a common sales pitch is an offer to test top performers and then create a success profile, benchmark, or template which can then be used to evaluate job candidates. It is a compelling offer. Who would not want more top performers? It sounds easy. Often, the seller will even create the profile for free. Does it sound a little too easy or too good to be true? The trap is that it is just right enough to be dangerously wrong. The purpose of this paper is to explain those dangers and to suggest an alternative approach.
THE SIMPLER THE JOB, THE BETTER IT WORKS.
The concept of profiling top performers originated with simple jobs based on rote behaviors. The concept is woefully inadequate for today’s complex jobs, particularly when knowledgework is involved. Performance depends on a complex array of traits and abilities in the individual. It also depends on the interaction with that individual and the manager, and with the facility with which they interact with the other members of the team. A simple percentage match to some template is as likely to misdirect the decision as it is to guide it.
WHAT IS A TOP PERFORMER?
The cornerstone of the profiling methodology depends upon the ability of the client to accurately and specifically define who a “top performer” is in their business. In hundreds of cases across the United States, managers were unable to agree on exactly who was a “top performer” and why they fit that description. Salespeople were “tops” for such varied metrics as total revenue, percentage of new sales, highest margins, growth of account revenue, percentage of sales of particular products, greatest number of accounts, fastest growth, and even anticipated performance. With managers or other jobs with fuzzier metrics, the puzzle is even more challenging.
HOW MANY TOP PERFORMERS DOES IT TAKE FOR A ROBUST SAMPLE?
Even assuming that top performers could be identified, a robust sample that could reasonably be expected to yield reliable results would contain fifteen or more individuals. A minimum sample for even acceptable results would be ten individuals. Since the “top performers” would typically represent the top 20% of the group, that would mean that the overall size of a work group suited to this methodology is fifty or more.
ARE SOME FACTORS MORE IMPORTANT FOR PERFORMANCE THAN OTHERS?
Absolutely. There may be many factors that contribute to success, but experience with hundreds of positions has shown that in every case, two or three of those factors are critical for even adequate performance. The problem is that none of the existing profiling methodologies have an effective process for accurately weighting those factors. In fact, the assessment product with the most sophisticated weighting system is still prone to glaring mistakes. An example of this is a candidate for a persuasive sales role whose percentage match to the success profile was 92%. A 70% match is considered acceptable; an 80% match would be a strong hiring indication; and a 90% match would be exceptionally good. The candidate was a perfect match in every factor except that of assertiveness, in which they scored in the bottom 20%, indicating an extremely submissive nature. It was an example of a single fatal flaw.
DO INDIVIDUALS PERFORM DIFFERENTLY WITH DIFFERENT MANAGERS?
That is seen over and over again in the world of professional sports. A player with a fairly lackluster record is traded to another team, and the new manager seems to unlock a world of potential in a previously unremarkable performer. The same is true in the business world. Top performers depend upon their own abilities, the direction and support of their manager, the interaction with their team, and the resources available to them. The “strong match” for one manager could easily be only “acceptable” to a different manager with a different team. The “strong match” to Manager A could even be a “weak match” with Manager B.
IF IT DOES NOT WORK THAT WELL, WHY DO THEY USE IT?
First of all, it is a sales methodology. Assessment publishers with large distributor organizations promote the profiling of top performers because it is the most effective sales approach for a non-expert sales force. Most distributors are consultants of some kind, using assessments as an entry point or add-on product for their primary business, or they are generally sales organizations, and assessments are simply one of their products. The “Test your best. Create a template. Hire more like your best.” is an appealing sales pitch. It does not require any particular expertise or knowledge of psychology or psychometrics (the science of measuring personality and cognitive abilities). A common approach is to offer to create profile for free. This is usually done with far too few individuals in the sample, and rarely is there sufficient analysis to vet the sample. It is only once the profiling program is in use for a time that the flaws in the concept become all too apparent.
WHAT ARE THE LEGAL ISSUES REGARDING HIRING WITH PROFILES?
Here is where it gets a bit sticky. The very nature of hiring templates that require a certain percentage match to that template as a part of a selection process is that the focus of the hiring decision is centered on that element of the process. A company must be strictly consistent with how that percentage match is applied. If an exception is made for some reason, then it may be difficult to defend the nondiscriminatory nature of the process if it were to be challenged. In fact, any challenge to the process by a candidate that has been excluded will center on the hiring profile. It is that profile which must be defended.
WHAT IS THE BIGGEST PROBLEM WITH USING PROFILING?
The inherent flaws of success profiling invariably results in hiring non-performers along with performers. It is the non-performers that cost money in terms of lost sales, turnover, management time, training time, and customer dissatisfaction. The greatest loss however, is in not realizing the true potential of people. Good assessments provide a clear picture of the strengths and abilities of people. When those are not masked by superfluous templates or benchmarks, it is possible to recognize how to optimize both human and organizational performance.
THE SIMPLER THE JOB, THE BETTER IT WORKS.
The concept of profiling top performers originated with simple jobs based on rote behaviors. The concept is woefully inadequate for today’s complex jobs, particularly when knowledgework is involved. Performance depends on a complex array of traits and abilities in the individual. It also depends on the interaction with that individual and the manager, and with the facility with which they interact with the other members of the team. A simple percentage match to some template is as likely to misdirect the decision as it is to guide it.
WHAT IS A TOP PERFORMER?
The cornerstone of the profiling methodology depends upon the ability of the client to accurately and specifically define who a “top performer” is in their business. In hundreds of cases across the United States, managers were unable to agree on exactly who was a “top performer” and why they fit that description. Salespeople were “tops” for such varied metrics as total revenue, percentage of new sales, highest margins, growth of account revenue, percentage of sales of particular products, greatest number of accounts, fastest growth, and even anticipated performance. With managers or other jobs with fuzzier metrics, the puzzle is even more challenging.
HOW MANY TOP PERFORMERS DOES IT TAKE FOR A ROBUST SAMPLE?
Even assuming that top performers could be identified, a robust sample that could reasonably be expected to yield reliable results would contain fifteen or more individuals. A minimum sample for even acceptable results would be ten individuals. Since the “top performers” would typically represent the top 20% of the group, that would mean that the overall size of a work group suited to this methodology is fifty or more.
ARE SOME FACTORS MORE IMPORTANT FOR PERFORMANCE THAN OTHERS?
Absolutely. There may be many factors that contribute to success, but experience with hundreds of positions has shown that in every case, two or three of those factors are critical for even adequate performance. The problem is that none of the existing profiling methodologies have an effective process for accurately weighting those factors. In fact, the assessment product with the most sophisticated weighting system is still prone to glaring mistakes. An example of this is a candidate for a persuasive sales role whose percentage match to the success profile was 92%. A 70% match is considered acceptable; an 80% match would be a strong hiring indication; and a 90% match would be exceptionally good. The candidate was a perfect match in every factor except that of assertiveness, in which they scored in the bottom 20%, indicating an extremely submissive nature. It was an example of a single fatal flaw.
DO INDIVIDUALS PERFORM DIFFERENTLY WITH DIFFERENT MANAGERS?
That is seen over and over again in the world of professional sports. A player with a fairly lackluster record is traded to another team, and the new manager seems to unlock a world of potential in a previously unremarkable performer. The same is true in the business world. Top performers depend upon their own abilities, the direction and support of their manager, the interaction with their team, and the resources available to them. The “strong match” for one manager could easily be only “acceptable” to a different manager with a different team. The “strong match” to Manager A could even be a “weak match” with Manager B.
IF IT DOES NOT WORK THAT WELL, WHY DO THEY USE IT?
First of all, it is a sales methodology. Assessment publishers with large distributor organizations promote the profiling of top performers because it is the most effective sales approach for a non-expert sales force. Most distributors are consultants of some kind, using assessments as an entry point or add-on product for their primary business, or they are generally sales organizations, and assessments are simply one of their products. The “Test your best. Create a template. Hire more like your best.” is an appealing sales pitch. It does not require any particular expertise or knowledge of psychology or psychometrics (the science of measuring personality and cognitive abilities). A common approach is to offer to create profile for free. This is usually done with far too few individuals in the sample, and rarely is there sufficient analysis to vet the sample. It is only once the profiling program is in use for a time that the flaws in the concept become all too apparent.
WHAT ARE THE LEGAL ISSUES REGARDING HIRING WITH PROFILES?
Here is where it gets a bit sticky. The very nature of hiring templates that require a certain percentage match to that template as a part of a selection process is that the focus of the hiring decision is centered on that element of the process. A company must be strictly consistent with how that percentage match is applied. If an exception is made for some reason, then it may be difficult to defend the nondiscriminatory nature of the process if it were to be challenged. In fact, any challenge to the process by a candidate that has been excluded will center on the hiring profile. It is that profile which must be defended.
WHAT IS THE BIGGEST PROBLEM WITH USING PROFILING?
The inherent flaws of success profiling invariably results in hiring non-performers along with performers. It is the non-performers that cost money in terms of lost sales, turnover, management time, training time, and customer dissatisfaction. The greatest loss however, is in not realizing the true potential of people. Good assessments provide a clear picture of the strengths and abilities of people. When those are not masked by superfluous templates or benchmarks, it is possible to recognize how to optimize both human and organizational performance.
