Is distributional appraisal more effective than the traditional performance appraisal method?

This study compared the distributional appraisal method (DAM) with the traditional appraisal method (TAM) on seven criteria reflecting appraisal effectiveness: Four interrater agreement measures, calculated by Cronbach's formulas, and three measures of differentiation: Ratee and dimension differentiation, and leniency. The study took place in a software company, where 45 participants, working in teams, rated all their team members on eight dimensions, both by DAM and TAM. DAM mean scores exhibited stronger agreement than TAM scores on two Cronbach's interrater agreement measures - stereotype accuracy and differential accuracy. High inverse correlations were found between measures of agreement and differentiation. The study demonstrates that DAM and TAM differ in their effectiveness and that criteria for rating effectiveness may be inversely related.