Air University Review, September-October 1985
Lieutenant Colonel Michael E. Thorn
HARDLY a week goes by that a critic of DOD spending does not make public an alleged cost growth or budget overrun on a weapon system. The basis for such charges can come from a wide variety of sources, but it inevitably focuses on a cost estimate performed during some phase of system development and acquisition. Regardless of the validity of the claim, the net result has almost always been the same--DOD in general and the cost analysis community in particular have entered a "defensive posture," setting forth explanations in the best case and excuses in the worst.
Is this situation of interest to those not directly involved in the cost analysis arena? Yes! While the Department of Defense currently enjoys a moderate amount of public support for its force modernization programs, that support should be viewed as only fragile and fleeting in nature. Unless there is a dramatic, sustained shift in societal priorities and public perception of the threat facing the United States and its allies, public (and congressional) support for defense budgets will continue only as long as DOD demonstrates sound fiscal responsibility in its execution of planning, programming, and budgeting activities. As an integrating function in an environment like that of DOD (i.e., where increasing needs confront budget restrictions), cost analysis has emerged as a relatively new, but vital, science of increasing importance as a management tool. This rise in stature has brought with it both problems and opportunities, requiring careful consideration of how and when this resource should be used.
Make no mistake about it--cost analysis is assuming a much greater role in the defense decision-making process. Congress has stated that the Secretary of Defense (SECDEF) may not approve the full-scale development, production, or deployment of a major defense acquisition program unless an independent cost estimate of that program has been performed and submitted for review by DOD authorities and review panels. Further, SECDEF had to submit a written report to Congress in May 1984 on the use of cost estimates in the budget process. (DOD anticipated this requirement to some extent when the Defense Resources Board decided in a November 1981 decision to have independent cost analyses to support the annual budget cycle in addition to the Defense Systems Acquisition Review Council process.) Finally, Congress has stipulated that adequate resources and personnel be allocated at all levels of DOD to ensure that the required level of cost analysis effort is maintained.
The DOD charter is thus stated clearly. The questions to be answered are why it had to be stated in the first place, what has been done to meet these explicit directions, what challenges remain, and what responses to those challenges might ensure an effective cost analysis function within DOD.
Cost growths and budget overruns directly negate the best efforts of DOD to achieve the superior fiscal responsibility that Congress and the American public desire. Not only is public support for DOD undermined, but the proverbial vicious circle is initiated. For every substantial program cost increase incurred, one or more other programs must be changed within the planning-programming-budgeting system (PPBS) to accommodate that increase, thereby setting off a chain reaction. For example, lower-priority programs that may have been established with efficient production rates are reduced in scope or stretched out, making them less efficient and more costly. This change, in turn, affects other programs, reducing the efficiency and increasing the cost of each. The end result is an accommodation of the total service/DOD budget for any given year, but at the expense of increasing unit and total program costs in the long run.
It is possible, of course, that an initial program cost increase (actual or projected) will result in a major change in the content of the priority program likely to create the budgetary difficulty, thereby avoiding an adverse impact on other acquisition efforts. However, a cursory review of the DOD track record reveals that such program changes have been the exception rather than the rule. While a certain degree of program restructuring has occurred to meet near-term budget constraints, there has been a very strong tendency to slip remaining program requirements to the relatively uncontrolled "outyears." The result has almost always been the same: increased unit and total program costs.
The "fix" for this scenario can assume many forms, ranging from technological innovation to closer program control. However, all roads eventually lead to the program cost estimate and its degrees of accuracy and reliability. Recognizing this premise, senior officials have mounted drives periodically to improve the business of cost estimation. At the highest levels, there have been flurries of proposals and initiatives, most of which, quite frankly, have met with limited success, if only because of the bureaucratic delay between the initiation of a change in the field and its receipt/implementation by those affected by it the most--the analyst force. Middle echelons of the cost analysis community have tended to react to criticism through reorganizations and a call for the development of new models and techniques. Again, with limited success.
There have been two primary reasons for the relatively minor impact of many recommended changes. First, the changes too often have been of such a broad, sweeping nature that they have tended to generate as many problems as they solve. This particularly has been the case with high-level directions which tend to be management-oriented (rather than analysis-oriented) and which frequently result in a plethora of new, time-consuming studies, reports, and requirements, adding to an already substantial workload and further diluting the effectiveness of analyst work. (There has also been some question by those in the field as to whether these additional reports and requirements are needed or used at all.)
The second reason for the limited effectiveness of command initiatives is related to (if not caused by) the first. Simply put, the pool of experienced cost analysts has been too small to allow for the gratuitous addition of new requirements. More often than not, the existing analyst work force operated essentially in are active mode to its workload, handling those projects and estimates that generated the most heat. Adding requirements without increasing qualified manning simply exacerbated the situation. Gradually, it became recognized that until the numbers and experience levels of cost analysts were increased, DOD's ability to improve its cost analysis function and produce genuine cost savings would remain limited.
Higher-level initiatives and middle-echelon efforts to improve the cost analysis business have tended to focus on organization and procedures in an ad hoc manner. As such, they have addressed the effects or, at best, surface causes of the problem. To really improve the quality of cost analysis within DOD, five major areas need to be considered.
personnel factors
As indicated earlier, there can be very little real increase in both the quantity and quality of cost analyses until the quantity and quality of cost analysts increase. Until recently, the total output of the cost analysis community represented essentially a zero sum game: the quantity of estimates could be increased (a common ploy in an attempt to avoid unexpected cost growth or budget overruns), but the quality of those estimates decreased in a proportionate manner. This situation was aggravated by the time constraints often placed on an estimate, forcing a less than in-depth analysis of requirements. A good analysis requires adequate time for a detailed review and study of the weapon system of concern, its associated requirements, and its impact on related systems. The recent environment of limited manning in relation to increased, time-sensitive requirements simply did not permit such a "luxury."
The current environment is improving, however. Commanders at all levels have recognized the importance of the cost analysis function and have committed considerably greater resources to it than in the past. In addition to enhanced recruitment efforts, specialized training has been expanded to ensure professional stature, expertise, and methodological currency. At least one service, the U.S. Air Force, has established a master's program in cost analysis. The basic issues of manpower and training have finally been addressed.
systemic factors
The overall problem of accurate, reliable cost estimation is not solved by a simple addition of analysts. Other factors need attention also. The previously mentioned time crunch was often accompanied by a request for an estimate on an incompletely defined system or a system that was years away from actual production. Given the increasing lengths of time for weapon system development and acquisition in general, and the two-year span between program office budget submission and execution in particular, difficulty in producing accurate estimates is a fact of life that the analyst must face. Still, nothing is more frustrating to a conscientious analyst than being asked to cost a system that has yet to be fully designed or that lacks a firm concept of operations for employment.
Such a situation is not uncommon and demands far more than simple calculation of possible variables. The analyst is certain that there will be changes in system direction, scope and schedule but cannot support sufficient precautionary (or, management reserve) costs to allow for such changes. If the program is so ill-defined or technologically uncertain as to demand a large management reserve, the limited total funds for all programs serve to severely constrain the analyst and decision maker in the full allotment of required funds to cover the uncertainties adequately. This tendency toward conservative estimates is especially present during the early stages of system development, when requirements and associated costs can be slipped to the "outyears" of a program. As a result, early system cost estimates have almost always been less than subsequent estimates or actual costs, providing a built-in factor for cost growth/budget overruns.
The obvious solution to this problem is to either delay the system cost estimate or provide considerably greater program definition at the time that the estimate is accomplished. Unfortunately, the requirements of the decisionmaking environment, limited resources, and the PPBS/selected acquisition report (SAR) cycles effectively preclude both alternatives. Each of these factors places great emphasis on the establishment of a cost and requirements baseline during the early stages of system development through the development of preliminary estimates. While the use of a preliminary baseline is a step in the right direction, it cannot be totally rigid. Preliminary estimates must be recognized for what they are--somewhat refined but nonetheless conceptual analyses of a weapon system, suitable primarily as an internal control mechanism. The original problem of program changes in terms of scope, direction, and schedule open to public inspection still exists.
Given the amount of money required to develop and acquire a modern weapon system, Congress and the public are certainly justified in their demands for preliminary estimates and baselines. On the other hand, it must be recognized that changes will occur for many, valid reasons--some of which originate through the annual, congressional budget approval process. The key point, though, is that there must be recognition and understanding of the need for changes and successive baselines.
A public and a Congress educated to expect budgetary revisions would not mean that the services abdicate all responsibility for those changes and the subsequent control of cost growth. Rather, each service and program management structure must be held responsible for, fully explaining the reasons for cost growth (especially those over which they can exercise control) at each major stage in the development and acquisition process. In turn, Congress and the public would need to recognize the impact of external forces on a program.
Given such a sharing of responsibility, however, the problem still exists of setting a final baseline against which all key parties can be held accountable. Ideally, this baseline would permit the development of accurate cost estimates but still provide sufficient time to cancel a program if further development or acquisition proved to be unwarranted. An appropriate time at which to establish this final cost/requirements baseline would be after system development and testing (but prior to full-scale production)--the point in the acquisition process known as DSARC III. At this juncture in the system acquisition process, it would be possible to develop a baseline composed of both firm costs and firm requirements. Use of this somewhat delayed point in the weapon system acquisition process would not eliminate all unknowns but would certainly account for the majority of them. In addition, it would offer the advantage of a limited production run on which to project actual costs.
technical factors
The cost analyst is hampered by still another set of constraints. Given enough time, data, and expertise to use them, the tools of the cost analysis trade can be formidable in their scope and depth. Analysts in both private industry and government have developed a wide array of mathematical techniques and computer models to draw on during the course of an estimate. The chief drawback is that existing models may not be appropriate for a relatively new or unique weapon system or production process. Forcing a system to fit a model, or vice versa, can result in a rather strained estimate with a questionable level of confidence. The obvious solution to this problem is to develop new techniques and models. However, doing so requires a further siphoning of scarce analyst resources--an option not available to a manager facing a series of higher headquarters' tasking deadlines.
If a suitable mathematical technique and model can be found or developed, the next hurdle to be cleared is that of data. As a preliminary step, the analyst must research and establish a data base that permits the identification or derivation of reliable relationships on which to form an estimate. This major task, requiring a detailed understanding of the program, often determines what technique or model will be used for the actual work of estimation. Unfortunately, the limited time available for research has led to a greater reliance on data bases of questionable relevance to new systems, materials, and production processes.
If a weapon system or production program is sufficiently defined or analogous to another so that a reasonable data base can be established, there is still another factor that has often caused an eventual cost growth or budget overrun: OSD inflation rates as directed by the Office of Management and Budget (OMB). While the use of these rates has permitted a high degree of standardization, they have in the past invariably reflected the goals of the executive economic program rather than the (normally higher) rates that would be realistically expected. Until inflation began declining recently, the resulting understatement of costs in outyears virtually guaranteed the grist for critics' assertions of poor government control of public funds.
task factors
As an integrating element between the activities of planning, programming, and budgeting, the cost analyst develops a rather detailed knowledge of an overall program. While this knowledge can be personally rewarding and can bring an analyst a certain degree of stature, it can also inhibit the effective performance of the analyst's primary duties. As a result of this extensive knowledge base, an analyst will frequently be required to participate in such time consuming activities as special studies, budget scrubs, program reviews, and source selection panels. While participation in these activities is both necessary and desirable, they all detract from the time available for an analyst to perform assigned cost research and estimation tasks. Furthermore, perhaps of greater long-term significance, the press of additional duties can preclude time for research and professional development, to the detriment of the individual analyst, the overall profession, and future end products--reliable, accurate estimates.
professional factors
The list of problems related to analysis work could be extended further, but the point to note is that the field of cost analysis is essentially striving to establish a solid foundation on constantly shifting sand. This situation arises because of rapidly changing programs, data bases, and duties. What is not commonly realized is that the analyst force itself also tends to be highly variable because of frequent changes in its composition.
Although the absolute number of analysts is slowly increasing in response to a rapidly increasing workload, the overall quality of the analyst work force has remained relatively constant. There are several reasons for this. For one thing, the degree of professional status attributed to a cost analyst has been very slow to develop. It has been only in the very recent past that rigorous, professional requirements for analysts have been established. Second, although the number of analysts has increased on an absolute basis, many of the newcomers are still in training. The training period required to develop a well-qualified analyst is considerable and, in some cases, has been lengthened because of an overly aggressive recruiting program that has resulted in the entry of minimally qualified personnel. The cost analysis function needs more than mere manpower--it needs skilled manpower. The third reason for the general lack of improvement in work force quality is that both the military and the civilian personnel systems have failed to treat the field of cost analysis as a separate career field. This policy has tended to generate a transitory work force en route to other areas. Inexperienced personnel enter the field of cost analysis from a variety of backgrounds, receive specialized training, and then depart at just about the time they reach a level of true competency. This sequence is particularly common on the military side of the house, at both the worker and the supervisory levels. This type of situation is discouraged in such disparate areas as engineering, budget, and flying; it should not be allowed in the critical field of cost analysis.
To alleviate the problem of obtaining skilled manning, it is frequently suggested that DOD use outside contractors on a more frequent basis. In fact, there has been a trend toward the increased use of and reliance on contracted analysts and consultants. While recourse to such sources of data and analyses can mitigate the problems of manning and time constraints significantly, there are several disadvantages in relying on outside "experts":
In short, the use of external consultants may provide an alternative to the conduct of cost analyses, especially if an independent analysis is required, but it is not a panacea to all problems. Specifically, it does not solve the overall problem of DOD manning and experience levels. Instead, it may very well complicate the problem. As these external groups develop the skills and capacities to perform more detailed and complex estimates, senior executives and decision makers tend to assume that more estimates can be tasked. The resulting increase in requested analyses, however, is bounded by the absolute number of studies that both government and external analysts can perform. To meet increased commitments (and increase their profit margins), external consultant organizations and industrial firms seek to build their corps of experienced analysts. The predictable result is a series of "raids" on a ready source of such scarce talent, government analysts. The previously mentioned exodus of trained analysts is thus exacerbated by contractor/industry lures of greater professional recognition, shorter hours, higher pay, better administrative support, and so on.
In effect, then, the unchecked use of external consultants can actually weaken the in-house cost analysis capability of DOD. Further, this drain on the analyst force is additive to the normal pull of industry in its attempt to meet the insatiable government demand for contract estimates and reports.
Meanwhile, for those analysts remaining within the DOD cost analysis community, another problem of both a personal and professional nature often occurs. The primary task of a cost analyst is to provide a comprehensive, objective estimate for use by decision makers. The analyst is theoretically not tied to a budget constraint during the conduct of an analysis, but funding limitations are never far from the analyst's mind. The net result is that the analyst is effectively caught in the squeeze between expanding needs and a relatively fixed budget. Viewed from any perspective, this is not a comfortable position. Being neither an adversary nor an advocate of a program, but supplying data for both, the analyst is in a "no-win" situation.
In an effort to match increasing needs to a relatively fixed budget, the analyst is frequently tasked to provide several program cost estimates for management consideration. This responsibility requires the assistance of a variety of functional experts to identify and tailor key parameters before an analyst can produce estimates reflecting a variety of program contents, schedules, and configurations. This task is not very difficult for an experienced analyst until the variables of conflicting interests, multiple program elements, and minimum levels of system utility are included. With the addition of factors such as these, the analyst's job is expanded to that of a referee and negotiator--a classic no-win position that pleases no one, least of all the analyst.
It should be noted also that, on occasion, the needs-budget squeeze can become exceedingly tight and require a large measure of both personal and professional integrity. Such a situation can arise in a number of ways, but most involve pressure on the analyst. One of the most common "suggestions" is to reduce or eliminate costs to fit a preconceived total or a total that will "sell," and to try to hide costs by spreading them among several program segments, thereby reducing the total under any one heading. While such situations are not an everyday event, their occasional appearance can present an analyst with some very interesting choices of a personal and professional nature.
The preceding narrative is not meant to portray cost analysts as victims of a terrible plot to drive them to an early grave. Rather, it is designed to stress the need for more analysts and better analyses. But what would be the end result of such increases, and whom would it benefit?
Both questions can be answered succinctly by stating that an enhanced cost analysis function will yield greater cost realism. In turn, cost realism can reduce perceived cost growth significantly, promote funding stability, mute the myriad critics of military spending, and permit DOD to rise from its all-too-familiar defensive posture.
However, flooding the hallways with more analysts and cost estimates will not automatically guarantee more accurate, realistic analyses. This goal can be achieved only through the involvement and assistance of all disciplines. There is, and will continue to be, a real need for technical support from others (e.g., programmers, engineers, and procurement specialists) to handle the increasing complexity of weapon systems, manufacturing processes, and the business environment. In addition, the "system" must be stabilized through the firm institutionalization of program/funding baselines, independent schedule and technical assessments, and hard-nosed decisions (e.g., program cancellations). In short, a total team effort is required if the goals of cost realism and funding stability are to be attained.
Fortunately, a coordinated, team effort to control and reduce costs, rather than merely accommodate them, has begun to materialize. Starting with the "Carlucci Initiatives" of several years ago (and subsequently streamlined by former Deputy Secretary of Defense Paul Thayer), a concerted effort at all levels has been under way to bring cost growth under control while enhancing the DOD cost analysis function. Examples of these efforts have been already mentioned, but a short listing of additional accomplishments should prove the point.
It is very easy to blame cost growth on "the other guy" or factors beyond the program manager's and DOD's control. That is a defensive strategy. DOD needs to continue its coordinated offensive to establish fiscal integrity and maintain public support for its needed programs.
CLEARLY, the role of the cost analysis function has come to the forefront of the DOD decisionmaking process. A change such as this affects far more than just the cost analysis community. Continued recognition of the importance of this function will go a long way in solving many difficult problems at many levels of decisionmaking and management. In turn, the solution of these problems will increase both the quantity and quality of future cost estimates and improve subsequent financial performance.
This matter is not a trivial concern. The long-standing conflict between increasing needs versus a fixed budget is not likely to disappear in the foreseeable future. Similarly, critics of the DOD planning-programming-budgeting process will always be ready to jump on any perceived financial irregularity as an unwarranted cost growth or budget overrun. As the middlemen between needs identification and the budget process and as one of the major groups contributing to the defense decisionmaking process, cost analysts and their technical counterparts must be ever more timely and accurate with their estimates. Continuous improvement is never easy, but DOD cannot afford anything less.
Andrews AFB, Maryland
Contributor
Lieutenant Colonel Michael E. Thorn (B.A., M.B.A., Pennsylvania State University) is Director of Plans and Resources, Hq AFSC, Andrews AFB, Maryland. He has been an instructor, flight evalutor, and functional check-flight pilot in all current USAF aircraft except T-39s and T-38s; executive officer to Commander, 71st Flying Training Wing and Wing Inspector General; cost and analyst and division chief, Hq AFSC; and Chief, Budget Investments Branch, Hq USAFE. Colonel Thorn is a Distinguished Graduate of Air Command and Staff College and DOD Professional Military Comptroller School.
Disclaimer
The conclusions and opinions expressed in this document are those of the author cultivated in the freedom of expression, academic environment of Air University. They do not reflect the official position of the U.S. Government, Department of Defense, the United States Air Force or the Air University.
Air & Space Power Home Page | Feedback? Email the Editor