The Current Officer
Evaluation and Promotion System

Bradley A. Wayland

Abstract: The current Air Force officer evaluation and promotion system is only a "snapshot in time" rendered by an immediate supervisor. Based upon this, future advancement is impacted by subjective factors such as a supervisor's writing abilities, knowledge of the system, and personal likes and dislikes. As a result, many officers don’t perceive a direct relationship between advancement and actual duty performance. Furthermore, other inherent factors in the current system, such as inflation of ratings, may not promote the most capable leaders. It is important officers understand the current Air Force officer evaluation and promotion system and have confidence in it, and it is essential the very best leaders are promoted to the senior ranks, ensuring our continued commitment to excellence.

More and more does the 'System' tend to promote to control, men who have shown themselves efficient cogs in the machine…There are few commanders in our higher commands. And even these, since their chins usually outweigh their foreheads, are themselves outweighted by the majority—of commanders who are essentially staff officers.

Captain Sir Basil Liddell Hart
Thoughts on War, 1944 (31: 344)

How many times officer promotion board results were released has someone said, "I can't believe so-and-so made it," or "…can you believe so-and-so got passed over?" Although these cases might fall into the minority, there are times when the Air Force officer evaluation and promotion system seems to involve more than a little black magic, rather than being an objective evaluation of an officer’s performance and promotion potential. Although this may not always be the case, the evaluation and promotion system could better meet the stated purposes in Air Force Pamphlet 36-2506, You and Your Promotions—The Air Force Officer Promotion Program; which are to promote sufficient numbers as vacancies occur; provide a reasonably stable, consistent and visible progression pattern; ensure the best qualified officers are selected; and provide accelerated promotion opportunities for officers with exceptional qualities (3:1).

Problems in the current system relate more on the latter rather than former two objectives. Promoting numbers based on vacancies is routinely met and although some officers might dispute the second purpose—a known expectation of progress needed for promotion—is met, most officers know what they need to do to remain competitive. The last two objectives; ensuring the best officers are promoted and exceptional officers are promoted ahead of their peers, may not be met in our current system however. It’s said officers are promoted not on who they are but on what their records say—unfortunately, these two pictures of an Air Force officer can be drastically different. Are there some recommended changes to the current system to better meet these two objectives? This study attempts to do just that in recommending improvements to the current officer evaluation and promotion system.

A survey by the author of Air Force officers detailed concerns in the current system and indicated problems degrading from its objectives. The following chart depicts these concerns highlighted by this survey:

Over 22 percent of respondents made the number one concern in the current system the importance of a supervisor’s writing skills, which has a bigger impact than actual performance. Subjectivity and inflation of the overall system were listed next, accounting for 16 and 13 percent respectively, followed by perceived over-importance of the PRF raised by almost eight percent. Rounding out the top five issues was what many termed the "Halo Effect" and the corresponding perception that sponsorship was more significant than superior duty performance. Several respondents described the "Halo Effect" as officers with previous below-the-zone promotions or recognition by senior Air Force leaders, who continued to progress rapidly, sometimes in spite of their performance. Other responses are listed in descending order (32:NP, 33:NP):

• Competition across differing AFSCs creates uneven playing field

• Allocation of "Definitely Promote" (DP) recommendations

• Minimal second chance for Above-the-Primary Zone officers

• Outside factors appear to take precedence over duty performance

• Lack of time for promotion boards to review officer records

• Lack of realistic and objective feedback from promotion boards

Measurement Criteria for Evaluation Systems

To evaluate the current system’s effectiveness, comparison against performance measurement theory should first be done. Criteria to assess an evaluation and promotion system are how relevant the system enforces desired behavior, how valid it predicts future performance, it’s consistency, and how it discriminates differences in personnel and their performance (16:210-214). These four criteria will be looked at separately in the following paragraphs.

In assessing relevancy, the survey asked respondents how often they had seen officers not deserving of advancement get promoted. The response was slightly below "some of the time" and indicated personnel occasionally receive promotions without demonstrating the appropriate traits and thus a lack of relevance (32:NP, 33:NP).

How valid does the Air Force predict future performance and desired leadership traits? In the survey, two responses indicated concerns with the system’s validity. One question asked respondents how often people were promoted who did not deserve advancement. The response was slightly below a rating described as "some of the time" and indicated perceptions the system may not promote the best personnel as often as it should. Another question asked the opposing view of how often the system rewards the most deserving people for advancement. The response was slightly above the rating "some of the time" and again indicated perceptions that many passed-over personnel may be more deserving than others who were promoted (32:NP, 33:NP).

Is the Air Force system consistent across raters and situations? The top three responses to what respondents least liked about the current system, which accounted for over 50 percent of all answers, were inflation of OPRs, dependence upon supervisor writing skills, and system subjectivity. These responses indicated significant concerns with the system’s consistency.

Discriminating power is the final criteria, and is the ability of rating scales to distinguish differences between personnel, affected by the system’s objectivity and appraisal inflation. In the survey, respondents were asked how subjective the current system is on a scale of 1 to 5, with 1 being "very objective," 3 "well balanced" and 5 "very subjective." The average response was 3.97 and had the highest deviation from the average within the survey, showing the significant perception of subjectivity and a lack of discriminating power to distinguish between officers and their performance (32:NP, 33:NP).

In summarizing, the Air current system does not meet any of the necessary criteria defined in performance measurement theory. Minor concerns exist in the relevancy and validity of the system; however, major disconnects exist in the system’s ability to meet the criteria of consistency and discriminating power. A majority of officers did not feel the evaluation and promotion system was consistent across raters and situations nor did it to distinguish between personnel due to subjectivity, dependence on writing skills and inflation. Overall, the current system partially met the former two criteria and failed to meet the latter two.

Performance Measurement Theory Tenets

The Air Force system also needs to be assessed against the two tenets of performance measurement theory, which are to ensure evaluations give supervisors the truest assessment of subordinates and to ensure integrity is maintained in the system. As discussed earlier, officers have a lack of faith I the accuracy of OPRs based upon concerns highlighted within the survey. Additionally, officers were asked to rate the accuracy of OPRs and PRFs, and they received a grade well below the rating of "accurate" (32:NP, 33:NP). Respondents also indicated the current system strayed from the other tenet, integrity. Responses to, "What do you like least about the Air Force promotion system?" included dependence upon a supervisor's writing skills, inflation, subjectivity, perceived favoritism towards certain career fields, and fairness in allocation of "DP" recommendations. These responses all indicated concerns with the system’s integrity (32: NP, 33: NP).

Reasons for Divergence between the Air Force System and Accepted Theories

How can so many concerns in the current system exist? One explanation is inflation of OPRs. Today, virtually every officer ‘walks on water,’ making it almost impossible to detect differences between the real person, their actual duty performance, their true leadership abilities, and what their records indicate about these traits. Thus, in order to find variances in the potential of all these officers who "meet or exceed standards," OPR comments have become critical. Unfortunately, no formal training program exists for supervisors on the intricacies of these comments and instead, they must be learned through experience. As such, the importance of these comments—without formal training—creates a situation where an officer’s advancement is not only dependent upon their own abilities; but also upon their supervisor’s writing skills and knowledge of the intricacies within the OPR system.

An attempt to correct inflation has been incorporation of PRFs, which has also increased senior rater involvement in the process. However, problems with the PRF and its effects on the promotion system exist. In the survey, officers ranked the items most important to a promotion board and what they felt should be most important. The table below contains both the rankings and scores of each response.

Table 1. Rank Order of Importance of Record Items by Air Force Promotion Board

Based upon these responses, many officers felt the PRF might be over-emphasized. Although officers recognized it as the most important document for promotion, most felt it should be secondary to the OPR. The scores also indicate many respondents felt assignment progression may even be more important to assess promotion than the PRF.

In order to address these concerns in the Air Force evaluation and promotion system, the first and foremost question is: What should be changed? Due to the impact the OPR and PRF have on promotion, recommended actions will focus on these documents. Although other items have a bearing towards promotion, they are either not as subjective, or they are dependent on information contained in the OPR and PRF.

Recommended Changes to the Evaluation Process

Ultimately, many concerns with the OPR and evaluation process centered upon rating inflation. Other concerns, such as subjectivity and dependence on supervisors’ writing skills, would likely be resolved by stopping inflation within the current process. In order to resolve inflation, OPRs must better discriminate between officers, and supervisors must use the entire spectrum of ratings instead of fearing the harm an accurate rating could pose to a subordinate's career.

The current OPR contains six performance factors, but only two ratings; either an officer "meets" or "does not meet standards." To better distinguish between officers, this two-tier system should be replaced with a five-tier rating system. A change in the rating scale itself will not reduce inflation within current OPRs however. In addition to a wider range of ratings, the next step is to hold supervisors and senior raters more accountable for the accuracy of these ratings while ensuring the system does not hurt a superior performer's career. To accomplish this, it is recommended the evaluated officer’s average rating (determined by combining the numerical ratings and dividing by six) be included in the evaluation report and compared against both the supervisor’s and senior rater’s average rating against others in the same rank and competitive category (i.e. active-duty line, medical corps, etc.). This comparison system is based upon the Navy's system, which has demonstrated good inflation control. Although ratings might differ based upon a superior’s own personal rating system; the key factor is the comparison of an evaluated officer's average rating against the rater's average for other officers in the same grade and competitive category. The following table gives an example.

The supervisor marked three 3.0 ratings and three 4.0 ratings, resulting in an overall average of 3.50 for the evaluated officer. By itself, this information is difficult to assess; however, when compared against the average this supervisor gave for other officers in the same grade and competitive category, let’s say this is 3.15, it shows the evaluated officer was rated above his or her peers. Using the same example, the senior rater's average was 2.83. Again, this one rating makes it difficult to know how the officer is evaluated, but when compared against the senior rater's overall average rating for similar officers of 2.50, the rater and senior rater indicate this officer is above average. In order to easily see and compare these ratings, it is recommended the OPR include a section for both the raters' and senior raters' averages of both the evaluated officer and other officers in the same rank and competitive category. This rating system would assist in both helping to better delineate between evaluated officers and also to increase the objectivity of the system.

Another recommendation is to formalize training on rating and evaluations. Training would of course be needed with a new evaluation system, but this training should not stop once the new system is established. Instead, continued training would teach the proper methods of evaluating and rating officers and reinforce the unnecessary inflation of evaluation reports.

This recommended training should be conducted in several forums. First, this training should occur during the feedback sessions already held between an officer and supervisor. In addition to providing feedback on the evaluated officer's performance however, these sessions should also cover the evaluation system itself, in order to help better understand how the evaluation works and how past ratings compare with other officers. An additional feedback session should also be added after an OPR has been finalized and signed by the chain of command. The evaluated officer should be able to look at the final OPR, prior to incorporation into their records. This would enable truer feedback since they can immediately see the comparison between their ratings and those of their peers. It would also provide the evaluated officer an opportunity to ask questions about their performance based upon the ratings they received, and it would also stimulate questions on the OPR process itself. Once this additional feedback session is conducted, the evaluated officer would sign in the verification block included on the revised OPR and it would be ready for incorporation into their records.

To minimize dependence on supervisors’ writing skills and re-emphasize individual performance, it is recommended OPR comments be reduced to four lines only for the rater and senior rater each and only contain justification for extreme ratings (i.e. 1.0 or 5.0 ratings). This change in philosophy would require a great deal of attention to ensure these comments do not migrate back to the test of writing skills found in today's OPRs. Ensuring these comments are only brief justifications of significant problems or superior performance could be emphasized through continued training.

Another recommended change, which would help minimize comments and ease the calculation of superior's average ratings for similar officers is to accomplish all officer evaluation reports at the same time, as opposed to each officer having varying report dates. Although some exceptions would be necessary based upon cases with minimal supervision time, completing all OPRs at the same time every year would have several benefits. As briefly mentioned, it would de-emphasize OPR comments since supervisors would have several reports to complete and little time to 'word-smith' each and every report to the degree currently done. Also, by accomplishing all reports at once, it would ease in determining the overall average ratings for officers in similar grades and career fields. With all OPRs accomplished at the same time for all officers, personnel specialists and supervisors would have all the information readily available to determine raters' and senior raters' overall average ratings. Again, this change would require a major shift in philosophy, but as the importance of the comments diminishes, the emphasis should shift to the ratings themselves and increase objectivity of the system.

Recommended Changes to Promotion Recommendations

Officers appear to have a "love-hate" relationship with the PRF and promotion recommendation system. Not surprisingly, there was a direct correlation between an officer's satisfaction and how quickly they had progressed. BPZ Officers had significantly more faith and liked the system more than officers selected on-time for promotion (32:NP, 33:NP). This should not be much of a surprise; after all, it's difficult to complain about the system when it's 'the horse that brought you there.' Another positive aspect with the current promotion recommendation was the senior rater's involvement in the process. When asked, "What do you like best about the Air Force officer promotion system?," the third highest response was senior rater involvement (32:NP, 33:NP). On the opposite side of the spectrum however, when asked "What do you like least about the promotion system?," the fourth highest response was the PRF and the seventh highest response was the allocation of "DP" recommendations. The dislike many officers have with the current promotion recommendation system was also evident when almost eight percent of respondents asked to do away with PRFs and "Definitely Promote" recommendations (32:NP, 33:NP).

Based upon these concerns, it is recommended is to delete the current PRF and incorporate promotion recommendations into the OPR itself, as is done in both the Army and Navy. Senior raters would still be involved in making promotion recommendations, however, these would be accomplished every time an officer receives an OPR with addition of the following section:

Figure 3. Recommended Promotion Recommendation Section

With the concerns regarding allocation of "DP" recommendations, it is recommended no quotas be imposed; however, since this risks inflation, control over these recommendations could be done by comparison against other officers. For example, a completed section would look like this: