Air University Review, May-June 1985
THE Air Force officer effectiveness report (OER) system has been a source of chronic concern for a long time. It is widely perceived to be an organizational burden, ill-conceived to contribute to the difficult decisions required to manage the officer force well. Moreover, every time that reform is attempted, more turbulence than improvement is generated and the U.S. Air Force looks more and more like the subject of a psychology student's experiment with one blind alley after another. Is there a way out of the OER maze?
Before exploring a way out of the maze--that is, a new perspective, and a new approach to officer performance evaluation--let's examine some of the problems of rating and review the failure of the controlled OER system.
There is nothing inherently wrong with the OER forms that the Air Force has used over the years; the problem lies with what Air Force people have tried to do with those forms.
The Air Force supervisor, like any other competent supervisor, feels much more intensely the need to motivate his people and accomplish the mission than he does any imperative to establish an objective basis for deciding who is promoted and who is not. Consequently, the effective supervisor uses all tools at his command to respond to the most closely felt need. The OER is one of these tools. Its use as a motivator has led to inflation, chronic inflation, and still more inflation. Inflation has led, in turn, to extreme measures on the part of raters to ensure that their solid performers are promoted. Recognizing that the Xs are "firewalled" across the Air Force, they fall back on secret codewords as discriminators; they escalate both the level and the verbosity of their endorsements; and they insist on letter-perfect typing, which creates administrative nightmares fully capable of absorbing a dismaying share of the administrative capacity of any Air Force organization (Is our mission to fly and to fight, or to type and to proof?) The result is a burdensome process through which supervisors attempt to communicate with so confusing a mix of signals that crucial "promote early," "promote on time," and "don't promote" decisions must be made without documentary support.
The controlled OER cycle was a noble attempt to restore objectivity to the OER system so that it could once again carry its share of the load in making rational promotion decisions. Unfortunately, the controlled OER idea failed to accommodate the truth that although supporting promotion decisions was the declared function of the rating system, its primary bureaucratic utility actually was its role as the vehicle of choice for communicating psychic reward from the supervisor to the troops. The controlled rating system interrupted this function and, thereby, spread chaos across the officer corps.
The turmoil created by implementing the controlled OER system was then only exceeded by the disruption caused by its subsequent demise. This left the Air Force once more burdened as Marley's ghost with the rating system chain it had forged in an organizational life of false starts.
How can we then provide the Air Force with the tools to see over the walls and find its way out of the maze? Two such aids are needed. The first is the answer to this question: "Exactly how much do we need out of a formal rating system? There are a lot of things we don't need. We do not need an OER to document job description or level of responsibility. While these are key elements of information needed to track career progress and project potential, they can be adequately maintained as a part of objective personnel records instulated from subjective performance judgments.
Neither do we need OERs to communicate psychic reward for a job well done or to motivate air officer to do better in the future. Obviously, such functions will I become attached to air OER system if the system is vulnerable to such intrusions. However, the Air Force supervisor is quite capable of finding a variety of other vehicles for communicating the positive strokes needed to fuel the locomotive of high quality performance. Indeed, military organizations have historically institutionalized effective vehicles for rewarding their people: awards and decorations programs, formal communications, expressions of elite status both from within and without the organization, and informal communications based on the essentially paternalistic view that the organization's leadership expresses toward fellow members. Finally, supervisors are involved, all the way up through the Air Force's most senior leadership, in seeking individuals' selection for high quality, follow-on positions when it comes time for reassignment.
What we really need a rating system to do for us is to evaluate, as objectively as possible, the quality of an individual's performance of whatever job he holds as it casts light on his ability to perform at higher levels of responsibility in the organizational structure. This evaluation function must be as well protected as possible from the accretion of other bureaucratic functions, which, like barnacles, tend to proliferate on any solid rating instrument and ultimately detract from its central function.
Thus, the second thing that the Air Force needs for seeing its way out of its maze is an evaluation tool which can do what is needed while avoiding the attachment of other bureaucratic functions it was never intended to bear. Creating such a tool is a monumental task, particularly were we limited to the talents we assemble in blue suits to perform our Air Force mission. But we are not limited to in-house resources and can draw on the full resources of modern behavioral science as it is being applied every day with great success in industry and academia. Drawing on the resources available, we should be able to design a rating device with a number of questions that characterize an individual's performance, plus a variety of carefully worded responses which reflect the specific ways the raters might do his job (perhaps using the ten performance dimensions on the front side of today's OER form). The alternative responses could provide a range of performance characteristics in terms of images that allow a supervisor to match to a greater or lesser degree the way each ratee performs, without telegraphing a subjective better or worse connotation. Responses should not be listed in a worst-to-best progression that indicate value judgments or suggest how each response contributes to an aggregate rating. Rating thus becomes a pure best fit matching exercise to the greatest extent possible.
Such a rating tool might consist of, say, twenty questions designed to evaluate perhaps ten characteristics with from one to three questions targeted against each characteristic. Besides these comprehensive questions and alternative responses, the form would contain no space for rater comments. The endorser would have a small space for comments intended only to indicate Substantial disagreement with the rating. Otherwise, the endorser would only be certifying the rating as valid.
At the outset, such a form, if designed properly, should be relatively free of inflation or attempts to second-guess the system. But have no illusions that ratings would be able to escape inflationary tendencies indefinitely. The only way to keep inflation out of any rating system, even with the best discriminators, is to supersede a given edition of the form with an entirely new one on a recurring basis (perhaps every six months). This approach should avoid the chronic tendency toward inflation, which has plagued the current rating system. No ratee would have more than one rating based on the same questionnaire, and all raters would have to address subordinates' performance in new descriptive terms often enough to preclude "gaming the system" by trying to "pick the right answer" for each question and thereby resuming present-day patterns of inflation.
Translating a pattern of responses on a series of questionnaires into meaningful decision information is where this new rating system can have its greatest value to a promotion board. Once these new OERs become a matter of record and each edition has been superseded by the next, the mark-sense forms can be "graded" and individual "ratings" established. In a heavily automated process, an individual performance and potential profile can be sketched out graphically. Information on the most recent rating can then be aggregated with previous ratings to show simply, clearly, and graphically an entire career performance profile with trends or sustained performance levels on a single page. This profile should assist board members in coping more easily with the masses of data they are asked to review in arriving at their decisions, enabling them to focus more objectively on all the factors that go into the appraisal of an individual's career rather than having to spend time puzzling through the codewords and endorsers' signature blocks in today's verbose and inflated rating forms.
Obviously, this approach implies a well planned and executed preparation and implementation phase, followed by the continuing process of developing questions and responses, assembling them into questionnaires, validating them as measurement tools, and monitoring the individual and global impact of the results. Resources applied to this task would necessarily be considerable. A team of behavioral scientists or evaluation specialists would have to be assembled either within the Air Force or on a contractual basis, and its efforts would have to be integrated into the Air Force personnel community; a process for developing rating tools would have to be implemented and sustained; and teams of field workers would continually assess and validate the effort, its results, and raters' responses to candidate measurement tools. However, since the needed talents do exist, such a system could be assembled without disproportionate effort.
It is obvious that mounting the effort needed will cost money and man-hours. But whatever it costs, within the bounds of good management, the cost will be far less than the executive and administrative man-hour burden on the entire Air Force that now exists to execute our current rating system--a system of questionable utility to sound, efficient officer personnel management.
This revision to officer performance and potential rating promises to accomplish the intended function with greater objectivity than at present; avoid, or at least minimize, inflation of ratings; and reduce Air Force administrative costs associated with the current system. No rating system is perfect, nor will a given system do everything; but with the development effort the task deserves, the approach outlined here can enable us to do the job that needs to be done.
Offutt AFB, Nebraska
Colonel John J. Kohout III
(USAFA; Diplome de l'Institut, d'Etudes Politques, Paris) is Director of Programs, DCS Plans, Hq Strategic Air Command. He has served in various Air Staff positions and as an assistant base operations officer, instructor pilot and squadron flying safety officer, chairman of French courses and executive officer of the Department of Foreign Languages at the USAF Academy. Colonel Kohout is a graduate of the Armed Forces Staff College and has written numerous published articles, including previous contributions to the Review.
Disclaimer
The conclusions and opinions expressed in this document are those of the author cultivated in the freedom of expression, academic environment of Air University. They do not reflect the official position of the U.S. Government, Department of Defense, the United States Air Force or the Air University.
Air & Space Power Home Page | Feedback? Email the Editor