Paired comparison & Forced Ranking Concept and Application
Paired comparison involves pairwise comparison i.e., comparing entities in pairs to judge which is preferable or has a certain level of some property. LL Thurstone first established the scientific approach to using this approach for measurement.
The paired comparison method is particularly applicable when the jobs are significantly different from one another and where a relative measurement promises to yield insight. It is therefore useful for business situations which typically involve setting priorities in the context of limited resources.
It is a method of comparing employee and job with another one on the basis of skill sets, time required to execute tasks, knowledge etc.
The paired Comparison scaling is often used when the stimulus objects are physical products. The comparison data so obtained can be analyzed in either of the ways. First, the researcher can compute the percentage of respondents who prefer one object over another by adding the matrices for each respondent, dividing the sum by the number of respondents and then multiplying it by 100. Through this method, all the stimulus objects can be evaluated simultaneously.
Second, under the assumption of transitivity (which implies that if brand X is preferred to Brand Y, and brand Y to brand Z, then brand X is preferred to brand Z) the paired comparison data can be converted into a rank order. To determine the rank order, the researcher identifies the number of times the object is preferred by adding up all the matrices.
The paired comparison method is effective when the number of objects is limited because it requires the direct comparison. And with a large number of stimulus objects the comparison becomes cumbersome. Also, if there is a violation of the assumption of transitivity the order in which the objects are placed may bias the results.
Rater is provided with the bunch of slips, each containing a pair of names. The rater puts a tick mark against the person whom he considers better of the two, and the final ranking is determined by taking the total of number of times an employee is ranked better than another employee.
To apply Paired Comparison Method, it’s wise to use a large sheet of paper or a flip chart. Follow the steps below one by one for the analysis to work best.
Step 1: Creating table
Make a table with rows and columns and fill out the options that will be compared to one another in the first row and the first column (the headers of the rows and columns). The empty cells will stay empty for now. If there are 4 options, there are 4 rows and 4 columns and 16 cells; when there are 3 options, you get 3 rows and 3 columns and 9 cells, etcetera.
Step 2: Assigning letters
Every option is now assigned a letter (A, B, C etcetera). The options are mentioned in the headers of the rows and columns and each now has a letter so the options can be properly compared to each other.
Step 3: Blocking cells
It’s important to block out the cells in the table in which the same options overlap. Cells that contain a comparison that has been displayed earlier in the table also have to be blocked out. Every comparison should only be made once.
Step 4: Comparing options
The cells that are left will now compare the options in the rows to the options in the columns. The letter of the most important option will be noted. For example, when A is compared to C and C is a more important option, a C will be written down in that cell.
Step 5: Rating options
The difference in importance will now get a rating that will range, for example, from 0 (no difference) to 3 (important difference).
Step 6: Listing results
The results are now consolidated by adding all values for each of the options in question. If necessary, these totals can be converted to percentages.
Forced Ranking Concept and Application
Forced ranking, also known as a vitality curve, is a controversial management tool which measures, ranks and grades employees’ work performance based on their comparison with each other instead of against fixed standards.
Forced ranking process
In forced ranking process employees are divided into three into groups: A, B, or C.
- A group stands for the employees who are most engaged, motivated, passionate, open to collaboration and committed. They make up the top 20%.
- B group stands for employees who are not as engaged or motivated but are crucial to the company’s success because they are so abundant. They make up the middle 70%.
- C group stands for employees who are commonly non-producing procrastinators. They make up the bottom 10%.
Companies unwittingly give a huge boost to the competition.
From a marketplace standpoint, the bottom 10% of employees at a company such as Microsoft might be the top 10% at one of their competing companies meaning that when Microsoft jettisons their presumed lower-performing employees, they are providing the competition with fresh talent, and in many cases, their new highest-performing employees.
Forced ranking can be an engagement and innovation killer.
A force ranking system more or less tells employees that no matter how hard they work, their manager is forced to put them in the bottom 10% group if they do not produce more than their co-workers. Not only can this cause employees to feel unmotivated and disengaged, it creates unnecessary internal competition that can be destructive to synergy, creativity and innovation and pull focus from marketplace completion.
The bottom 10% isn’t always the bottom 10%.
We see significant flaws in the practice of breaking up performance rankings by department. An employee who is in the bottom 10% in a high-performing department might rank much higher when compared to employees in a different department that has lower overall performance. Why should that employee be let go when he or she outperforms those in other fucntions? This isn’t comparing apples to apples; it’s like comparing apples to bacon.