Document created: 24 April 02
Published Air & Space Power Chronicles

The Challenge of Assessing Effects-Based Operations in Air Warfare

Currently, the US military is taken with the concept of effects-based operations (EBO). Reports indicate that the Air Force made EBO a centerpiece of its input to the Quadrennial Defense Review of 2001, and we find EBO mentioned prominently in both Air Force Doctrine Document (AFDD) 2, Organization and Employment of Aerospace Power and AFDD 2-1, Air Warfare.¹ Indeed, over the past few years, it has been difficult to find an issue of Aerospace Power Journal that does not feature an article or comment that extols the virtues of EBO. However, two recent books—George K. Williams’s Biplanes and Bombsights: British Bombing in World War I (Maxwell AFB, Ala.: Air University Press, 1999), and Gian P. Gentile’s How Effective Is Strategic Bombing? Lessons Learned from World War II to Kosovo (New York: New York University Press, 2001)—posit that postwar efforts to assess the effects of air warfare have been fatally flawed, thus calling into question the validity of doctrine and theory emanating from these studies. These two case studies serve as a warning light to EBO practitioners regarding the inherent challenges involved in assessing effects.

This article reviews the main findings in Williams’s and Gentile’s works in order to understand effects-assessment challenges. It also argues that other information-related challenges arise during implementation of EBO in the real world, including the prediction and observation of effects. Indeed, assessing, predicting, and observing effects accurately have been the bane of airmen since the earliest uses of combat airpower and may also be at the root of long-running debates on airpower’s effectiveness. However, before exploring these three challenges, one should review EBO’s basic concepts, advantages, and premises.

Although no formal body of literature describes EBO in detail, one can derive a few tenets from Air Force doctrine. EBO’s preeminent concept is its focus on end results. AFDD 2 states that "the focus at a given level of war is not on the specific weapons used, or on the targets attacked, but rather on the desired effects. A given airplane, dropping a given bomb, could comprise a tactical or strategic mission depending on the planned results" (emphasis in original).² A second tenet, arising out of this central tenet, is a reduction of emphasis on weapon systems. According to AFDD 2-1, "Since the focus of air warfare planning is on achieving effects on the enemy, the label placed on a given action is best determined by the combination of function performed and effect achieved, rather than by the type of weapon (information, electronic, or physical attack) used" (emphasis in original).³ Thus, the saying "the best way to kill a tank is with a tank" is anathema to EBO practitioners. A third tenet of EBO also deemphasizes destruction as a mechanism for achieving effects. Like weapons, physical destruction is secondary to the desired functional effect.⁴ Closely associated with this last principle is the strategy of parallel warfare, wherein one attempts to inflict paralysis upon an enemy by using multiple, simultaneous attacks.⁵

EBO has many advantages. Its critically important message is, Do not confuse means with ends. In addition, applying an EBO approach to strategy making and operational art encourages the strategist to be flexible and innovative in planning and executing operations. Strategists are also prompted to think of and use weapons in new and different ways. Finally, as we assess current and historical air campaigns, EBO reminds us that the ultimate measure of success is the results achieved rather than the goals set. A campaign’s perceived faults, errors, and shortfalls are important learning points but remain secondary to results.

Although EBO is a compelling concept that provides critical, strategic clarity, it is based on two premises that have proven problematic for the people faced with actually planning and assessing air campaigns. AFDD 2-1 explicitly states the first problem: "Understanding the links between cause and either physical or psychological effect is a key part of air warfare planning. Failure to properly analyze the mechanism that ties tactical results to strategic effects has historically been the shortcoming of both airpower theorists and strategists" (emphasis in original).⁶ The premises underlying this statement are as follows: one can correctly identify the mechanism, predict mechanistic effects in advance, and then accurately observe and assess them after taking action. As we shall see, assessment, prediction, and observation are walks into uncertainty.

Prediction, observation, and assessment comprise a circular loop. Where to begin in this loop presents a real "chicken or the egg" problem. However, most people generally accept that prediction is a function of past experience.⁷ Accordingly, we shall begin with assessment because it provides the foundation for prediction.

A common, although not universal, practice of the US Air Force and other nations’ air forces involves conducting surveys on the effects of air warfare in order capture lessons for the future. Williams’s and Gentile’s books, mentioned above, suggest that current airpower strategists may have cause for concern over the validity of lessons learned from the past due to problems with past assessments. Both books argue that because postwar surveys were seriously flawed, they were/are poor or misleading studies on which to base theory, doctrine, tactics, or new weapons decisions.

According to Williams’s thesis,

this study develops and substantiates a comprehensive evaluation of British long-range bombing in the First World War that runs directly counter to more generally held opinions. Natural constraints, technical limitations, and training shortfalls combined with the impact of enemy countermeasures to create a considerable disparity between the bombing results officially claimed and those actually produced. . . . The misrepresentations of the January 1920 Air Ministry report highlight the extent to which the potential of strategic bombardment was touted between the two wars to rationalize and guarantee the continued existence of the Royal Air Force as a separate service.⁸

He backs his assertions up with a painstaking reconstruction of the strategic bombing missions flown by the British during World War I, using primary-source data reported by the units themselves and carefully reviewing initial reports written by British and American bombing-survey teams immediately after the war. According to Williams, if one compares the 1920 Air Ministry report with the flying units’ own records and those submitted by the survey teams, it becomes clear that either the Air Ministry report was shoddily put together or deliberately skewed to put a positive spin on otherwise dismal results. As noted above, Williams concludes the latter. Overall, the book is well researched, and the facts appear to support the author’s assertions.

Although Williams concludes that the Air Ministry deliberately misrepresented the facts, Gentile’s criticism of the post–World War II United States Strategic Bombing Survey (USSBS) is subtler:

Two fundamental tenets formed the American [airpower] conception: strategic airpower should be used not to attack ground forces in battle directly but instead to attack the vital elements of the enemy’s war-making capacity; and the air force must be independent of and coequal with the army and navy. My study seeks to show how that conception informed and shaped the Strategic Bombing Survey’s evaluation of American air power in World War II. Since the survey accepted the American conceptual approach to strategic bombing and made it the framework for analysis, a truly impartial evaluation was never really a possibility.⁹

Gentile doesn’t attack the gathered data to the same extent he does the purpose of the survey. In short, he asserts that the only real question to which airmen wanted an answer was whether or not strategic bombing worked. Accordingly, if there were only one question, there could be only one answer, even if independent civilian analysts led USSBS—as was the case. Gentile also disputes whether or not the civilian analysts were truly independent. After supporting his assertions by discussing USSBS’s origins, methodologies, operations, and leadership, Gentile concludes by comparing USSBS to the Gulf War Air Power Survey (GWAPS). He finds that GWAPS’s wider-ranging central question (i.e., "How effective was airpower?" versus "How effective was strategic bombing?") and its truly independent and better trained civilian staff made it a much better survey.

Overall, it is difficult to accept the main thrust of Gentile’s conclusions. Certainly, USSBS has drawn criticism for making conclusions based upon interviews of German and Japanese officials who feared postwar retribution, but attacking the survey team for answering the question it was tasked to answer seems strange. The great question needing an answer after World War II was not whether airpower enjoyed success but whether independent airpower did so. Even by the end of World War I, airpower had already established its importance as an auxiliary to surface combat, and everyone appreciated the immense value of air superiority. Indeed, the importance of airpower’s role in supporting European and Pacific surface warfare was beyond dispute, as evidenced by Gen Dwight Eisenhower’s making air superiority a prerequisite for the Normandy invasion and by the aircentric nature of Gen Douglas MacArthur’s and Adm Chester Nimitz’s southwest and central Pacific campaigns, respectively.

What, then, can the effects-seeking strategist gain from these two books? First, in assessing effects, a strategist has to be cognizant of personal and institutional biases. In Essence of Decision: Explaining the Cuban Missile Crisis, a classic in the literature of political science, Graham Allison points out how bureaucracies and individuals in bureaucracies will interpret information based on organizational prerogatives and in some cases come to completely different conclusions despite having used the same facts.¹⁰ Indeed, one can attribute much of the contention between Navy and Air Force interpretations of USSBS findings mentioned in Gentile’s book to institutional and personal bias. Second, tracing higher-order effects back to their original causes is inherently difficult. In assessing higher-order effects, the strategist, like any researcher, may be faced with multiple actions that yield multiple potential causes. Third, our hard data on airpower effects comes from predominantly successful campaigns, which may give cause for reflection. Interestingly, in the US case, all of the major airpower surveys were conducted after successful wars, but no formal surveys were taken after unsuccessful wars (e.g., Korea and Vietnam). Most likely, one can attribute this discrepancy to problems with postwar access rather than some nefarious Air Force plot. However, we are left with a predominance of information on those actions that did achieve desired end effects but little on those that did not.

As noted earlier, current Air Force doctrine cautions the would-be strategist to ensure that tactical actions are tied to strategic results. However, one might accomplish the same task in several ways. For example, shooting down airplanes creates the second-order effect of being able to operate in the aerial domain without prohibitive interference. Yet, theoretically, we produce the same effect by using a more complicated mechanism (e.g., attacking aircraft factories disrupts or eliminates aircraft production, resulting in fewer aircraft and, as a consequence, a less threatening air situation). However, common sense tells us that accurately predicting the same effect in advance is probably a much easier proposition in the first case than the second. One may also explain this phenomenon by turning to complexity theory.

According to complexity theorists, complexity is a function of the number of system elements involved and the number of relationships among elements; the greater the number of elements and relationships, the greater the complexity.¹¹ Complex systems demonstrate several unique properties. The first is that they are nonlinear: "By nonlinear we mean the arrangement of nature—life and its complications, such as warfare—in which inputs and outputs are not proportional and where the whole is not quantitatively equal to its parts, or even, qualitatively, recognizable in its constituent components, and where cause and effect are not evident" (emphasis added).¹² If war is indeed a nonlinear phenomenon, the air strategist faces a conundrum. How does the strategist ensure that lower- and higher-order effects are linked if making cause-and-effect linkages is tenuous at best? We can often see the effects of nonlinearity in the confidence that the military services place in certain elements of our theory and doctrine wherein cause and effect are disputed.

Every airman knows that air superiority is "job number one" and that, historically, designing air superiority campaigns and predicting the results have been relatively easy tasks. In other words, air superiority more nearly approximates a linear situation. This is due to the relatively small number of system elements involved (aircraft, ground defenses, and support infrastructure) as well as the simplicity of relationships among the elements. It is also relatively easy to know when one has achieved success (i.e., the enemy can no longer shoot down friendly aircraft or bomb friendly troops). Finally, a hallmark of air superiority campaigns is attrition. All other things being equal, the larger air force wins. However, technology and operator skills also play major roles; indeed, these elements have grown in importance in the overall air superiority equation. Nonetheless, the essential elements of numbers, technology, and skill are known, and the effect of having air superiority is easily apparent.

It is instructive to compare an air superiority campaign with the most debated of all aerospace operations—strategic attack. In terms of air warfare, predicting strategic effects derived from tactical attacks has traditionally been a centerpiece of air warfare theory, doctrine, and/or prewar planning documents. Furthermore, this task is immensely more difficult to get right because of the complexity involved. For example, during the interwar period from 1919 to 1939, airpower theorists postulated that wars could be won by attacking the enemy population and/or economy. The suggested link was that the destruction of these two elements would devastate the enemy’s will, war-making capability, or both. This, in turn, would lead to either a popular uprising against the government or surrender via economic collapse. Looking at the "people" and "economy" threads of these early theories highlights two more properties of complexity theory in addition to nonlinearity.

The second property of complex systems that explains the difficulty of predicting strategic effects is known as "sensitivity to initial conditions," which simply means that changes in the initial environment can lead to very different outcomes, even if the same initial inputs are used.¹³ Theory that advocated attacks on the enemy populace had its roots in British civilian panics and excited government reaction to German air raids during World War I.¹⁴ In response to public demand, the British government reacted to the German threat by pulling fighter units off the front in order to defend the home island. Accordingly, theorists believed that if these small strategic attacks could cause so much public dissatisfaction, more must be better. However, by the time World War II began, civilians were much more familiar with aircraft and through widespread public discourse came to understand the likelihood of being bombed. In addition, the government instituted civil-defense measures for their protection. When the bombing recommenced, civilians were no longer easily panicked and, perhaps with the exception of those who had experienced a near miss, had become quite accustomed to bombing.¹⁵ Thus, predictions made by early airpower theorists, despite their basis in firm historical experience, simply did not come to fruition because of the difference in initial situations.

A third property of complex systems that limits a strategist’s predictive ability is "adaptation"—the notion that living systems do not simply respond to events passively but actively try to turn the situation to their advantage.¹⁶ Most historians agree that the German and Japanese economies came under increasing pressure in middle to late 1944 and began collapsing in 1945, which made continuing the war a futile endeavor. However, it is a well-known fact that for several years the German economy actually grew in spite of ever-intensifying Allied bombardment. Under Albert Speer’s direction, Germany took up the economic slack, substituted more plentiful items for scarce items, dispersed production, and—through a myriad of other means—thwarted and delayed the effects of bombing.¹⁷

As the war progressed and as bombing’s effects seemed less definitive, the Allies made several targeting adjustments in both theaters. That is, the initial focus on ball bearings switched to emphasis on oil and transportation in Germany; moreover, a major shift in tactics occurred when Gen Curtis LeMay began low-altitude area bombing in Japan. In addition, through the entire war the various wartime targeting committees provided commanders conflicting advice on target selection. This human capacity for adaptation has allowed the human targets of bombing campaigns to last beyond the time predicted for success. As noted above, this phenomenon makes us less sure that our doctrinal schema and historical interpretation are correct. Over 50 years later, soldiers, sailors, and airmen still debate airpower’s decisiveness in operations, and historians still debate the reasons for Japan’s surrender. Despite today’s massive intelligence, surveillance, and reconnaissance capabilities, does anyone really know why Serbian leader Slobodan Milosevic withdrew from Kosovo? Again, when it comes to complex systems, oftentimes cause and effect are neither readily evident nor easily traceable.

The fundamental challenges of observing the effects of air warfare have remained unchanged since the first bomb was dropped from an aerial vehicle and are well illustrated by an old philosophical question answered by a Bell Labs advertisement:

Although airpower as a combat force boasts many advantages, it must contend with some inherent limiting characteristics as well. First, because it generally lacks staying power, it is limited in its ability to verify what it has achieved over time. Second, operating at altitude can either help or hinder observation activities. Although this ability provides advantages in terms of speed, range, and flexibility, it limits the specificity of information one can acquire. Reminiscent of Bell Labs’ answer, if the strategist/analyst/sensor is not present to observe and record an effect, its value as an input to future plans is nil. As a result, we may take unneeded actions or not take needed actions. Operating at altitude also attenuates the signal received. Thus, the information gained is less precise than one might have achieved by using an observer closer to the effect.

Of course, these assertions are relative. After all, observation was the first widely recognized use of airpower in a military sense. However, altitude truly provides standoff (i.e., safety) as well as breadth and depth of observation—the very reasons why forces always seek the high ground in surface warfare. Airpower (or space power) is simply unable to replicate the level of detail that humans can provide. Aerial (and space) sensors can see the building, but they cannot see what’s inside. Sensors can listen in on conversations, but they cannot detect any "body language."

The current precision of the airpower medium has created an attendant need for more precise information to match its capabilities. Perhaps one finds proof of this assertion in today’s call for greater capability in the area of human intelligence. In addition, we have seen the great effect that airpower has achieved in Afghanistan when used in conjunction with ground-based special operations forces. Interestingly, Tenth Air Force waged a very similar—and successful—air campaign in conjunction with special forces in the China-Burma-India theater during World War II.¹⁹ Finally, is it not intriguing that while we endlessly debate airpower’s impact, we seldom argue about who won or lost a battle? Physical proximity does have its observational advantages in terms of verifying effects.

It may be time for the Air Force to adjust its schemata to develop concepts more reliant on human as opposed to air and space observation—traditionally the predilection of airmen. Imagery, a perennial favorite of airmen, is almost exclusively limited to the observation of first-order effects, while signals intelligence may assist in the observation of first- and higher-order effects. However, neither can provide the nuanced information that humans can. And human information may be the only source able to identify and report on multilink cause-and-effect chains. Soldiers and sailors have spent much time pondering how they can use airpower in support of surface operations. Given the accuracy of modern weapons and the need for more precise information to support them, perhaps it is time for airmen to think more about how surface forces can verify air and space effects.

Despite the assertion that we may need more human "sensing" to better support precision EBO, it is interesting to ponder whether or not at some deep level the human factor also lies at the root of the historical conflict between air and land forces over the relative merits of close air support (CAS) and aerial interdiction (AI). From the soldier’s point of view, CAS provides immediate and close observability. Cause and effect are immediately ascertainable, whereas AI is "somewhere over there." Frankly, if airmen do their job well, soldiers will never directly observe the effects of AI. For the airman, long lines of vehicles exploding in the open or bridges falling into the flat, blue expanse of a river are easily and immediately discernable. In short, if perception is reality, then effects are more "real" when they are easily discernable.

This article has painted a grim picture of our ability to assess, predict, and observe effects accurately. As noted earlier, it is designed as a cautionary tale. Assessment, prediction, and observation go far beyond EBO. Indeed, these processes are fundamental to human learning; as such, the challenges to improving them go beyond just the technical. As humans, we are virtually incapable of separating ourselves and our observations from our own experiences. We remain captives to interpreting data through the prism of personal experience. If we take this observation up to an organizational level, it seems highly unlikely that surface warriors and air warriors will ever see things exactly the same. In addition, as we move away from a more concrete focus on observable physical destruction (first-order effects), our observation problems become more complicated and the number of inferences expands. More inferences cause human interpretation to play a greater role, and diversity of opinion blooms.

The Air Force’s increased focus on command, control, intelligence, surveillance, and reconnaissance recognizes the fact that EBO is information intensive. Interestingly, complexity theorists come pretty close to emphasizing the same thing when they give advice about how to deal with complexity, noting that people necessarily act in a world of induction. They try to fill in gaps by forming hypotheses, making analogies, drawing from past experience, and using heuristic rules of thumb. Whatever works, works—even if they don’t totally understand why. In short, this is how people think when a problem is not fully defined or when all variables are not known. They suggest that the key element of success is to "observe, observe, observe . . . that you see reality for what it is, and realize that the game you are in keeps changing so that it’s up to you to figure out the current rules of the game as it’s being played" (emphasis added).²⁰ If the above is of any value, then it accords well with the central thrust of EBO—that a focus on results is the paramount consideration.

Notes

1. Armed Forces Journal, October 2001, on-line, Internet, 2 February 2002, available from http://www.afji.com/AFJI/Mags/2001/October/Embracing.htm ; Air Force Doctrine Document (AFDD) 2, Organization and Employment of Aerospace Power, 17 February 2000; and AFDD 2-1, Air Warfare, 22 January 2000.

2. AFDD 2, 2.

3. AFDD 2-1, 13.

4. T. W. Beagle Jr., Effects-Based Targeting: Another Empty Promise? (Maxwell AFB, Ala.: Air University Press, June 2000), 5.

5. AFDD 2, 7.

6. AFDD 2-1, 3.

7. James G. March, with the assistance of Chip Heath, A Primer on Decision Making: How Decisions Happen (New York: Free Press, 1994), 57–102.

8. George K. Williams, Biplanes and Bombsights: British Bombing in World War I (Maxwell AFB, Ala.: Air University Press, 1999), xiv.

9. Gian P. Gentile, How Effective Is Strategic Bombing? Lessons Learned from World War II to Kosovo (New York: New York University Press, 2001), 5.

10. Graham T. Allison, Essence of Decision: Explaining the Cuban Missile Crisis (Boston: Little, Brown, 1971), 175–78.

11. Arthur Battram, Navigating Complexity (London: The Industrial Society, 1998), 17–18.

12. David S. Alberts and Thomas J. Czerwinski, eds., Complexity, Global Politics, and National Security (Washington, D.C.: National Defense University, 1997), iii–xiv.

13. Battram, 152.

14. Williams, 52.

15. Irving L. Janis, Air War and Emotional Stress: Psychological Studies of Bombing and Civilian Defense (New York: McGraw-Hill, 1951), 123–24.

16. M. Mitchell Waldrop, Complexity: The Emerging Science at the Edge of Order and Chaos (New York: Simon & Schuster, 1993), 11.

17. Alfred C. Mierzejewski, The Collapse of the German War Economy, 1944–1945: Allied Air Power and the German National Railway (Chapel Hill, N.C.: University of North Carolina Press, 1988), 1–20.

18. "Understanding Analog Transmission," Bell Labs Innovations, 2001, on-line, Internet, 16 January 2002, available from http://www.bell-labs.com/technology/analog/# .

19. John F. Kries, Piercing the Fog: Intelligence and Army Air Forces Operations in World War II (Bolling AFB, Washington, D.C.: Air Force History and Museums Program, 1996), 297–302.

20. Waldrop, 253–54.

 Maj Kevin B. Glenn, USAF
Maxwell AFB, Alabama

Disclaimer

The conclusions and opinions expressed in this document are those of the author cultivated in the freedom of expression, academic environment of Air University. They do not reflect the official position of the U.S. Government, Department of Defense, the United States Air Force or the Air University.