Air University Review, November-December 1969

Orienting Today's Research
 to Tomorrow's Air Force

Lieutenant Colonel Joseph P. Martino
Lieutenant Colonel Claude D. Stephenson

It is a truism that the Air Force of tomorrow will be built on the foundation of the research of today. Research, of course, does not produce hardware. It is not even primarily aimed at solving problems. The primary goal of research is to obtain greater understanding of the universe in which we must live and work. While there are some spectacular examples of individual research having a major impact on technology, as did tie laser, such examples are actually few and far between. Over 99 percent of all the results of a research program are like the tiles in a mosaic: individually they are not very meaningful; it is only when they are arranged in the proper pattern that a picture emerges. While the spectacular results are the ones that claim attention, over the long run it is the collection of many, many isolated results into an overall pattern which permits major advances. These many individual results produce a knowledge base on which technology can build to produce solutions to problems.

This situation is not always fully appreciated. There are proposals for the management of research that would essentially restrict to the investigation of matters connected with a previously chosen technological approach to solving some problem. These proposals fail to recognize that a fully developed knowledge base might show that a particular approach is inherently infeasible and not worth investment of any work, or might show that another approach is actually better. Thus goal of research is to build a knowledge base to support the developmental problems of the future.

However, the Air Force of the future may also be a captive of the research done today. Today’s research may provide knowledge in areas that turn out to be of little importance or fail to provide knowledge in areas that turn out to be of considerable importance. To the extent that this happens, the Air Force of the future will be constrained unnecessarily. It may find itself equipped with hardware that is more expensive, or less effective, than that which could have been built on a better-chosen knowledge base. Thus the future Air Force will be determined by today’s research: it may be captive of a poorly chosen research program or beneficiary of well-chosen investigation.

Basic research programs should be relevant to Air Force needs and problems of tomorrow. The notion of relevant programs introduces the concept of mission-oriented basic research. Here selection of research programs is based on their anticipated support for the future requirements of the Air Force rather than as research done for the purpose of acquiring new knowledge per se. The goal of mission-oriented basic research is to provide a more relevant knowledge base to support future military technology.

But even so, how can the research manager be sure that his research program is the proper one? How can he be certain, ahead of time, that he is picking the winners and avoiding the losers? How can he ensure that his foresight, like his hindsight, will be 20-20? The answer is quite simple: he can’t. In his work he is faced with two uncertainties: uncertainty about what his research program will actually achieve and uncertainty about how valuable something will be even if it is achieved. In the face of these uncertainties, he can never be sure he is making the right choices.

However, his problem can be viewed from another standpoint. He is forced to make choices anyway. He is always faced with resource constraints. There are never enough manpower spaces available to do everything he would like to do; there is never enough money available to buy all the equipment and supplies be feels he needs; there are only so many hours per day available on critical facilities. Since he cannot do everything he would like to do, he must decide which projects will get extra manpower and which will not; which items of equipment will be procured this year; which programs will have priority on critical facilities. His goal is to make these choices as intelligently as possible. His decisions must be made in the light of the possible consequences of each choice and the impact of those consequences on the Air Force of the future.

One of the first considerations of the Air Force research manager is: What is the rest of the research community doing? The Air Force funds something less than ten percent of all basic research being done in the United States. Thus most of the research which the Air Force eventually applies will in fact be funded by some other source. But there will always be specialized facilities which have no application outside the Air Force—areas of science which are of little interest to the other services or to non-Defense industry; programs which are too expensive for a university. So the Air Force research manager must look for those things which will not be done at all, or will be done too slowly, unless the Air Force supports them through contract or does them in-house.

Even within these more limited areas, of course, there is still much more to be done than will ever be possible within the resources available to the Air Force research manager. He still must make intelligent choices. To make these choices, the research manager must have decision information which reduces the uncertainties about his program. Uncertainties about what will be achieved can never be eliminated, but they can be reduced by well-designed experiments. This aspect of research management is beyond the scope of this article and will not be pursued further here. Uncertainty about the value of results if they are achieved can also be reduced, to the extent that the research manager has knowledge about the environment in which the future Air Force will operate and the problems it will face in that environment. The remainder of this article will be devoted to describing approaches to minimizing the research manager’s uncertainty about the environment of the future and the problems it will pose for the Air Force.

How can the research manager choose among those research projects which he sees as being of good scientific quality, not likely to be done unless his organization does them, and of some apparent interest to the Air Force? Undoubtedly even this limited set of possible research projects will exceed one or more of the constraints on his total amount of resources. He needs to reduce in some way the uncertainty about the value of the proposed research projects, if they are successful.

A planning tool known as research analysis can be helpful to the research manager in obtaining information about the potential value of proposed research efforts. Research analysis is usually considered to involve two types of effort. Both draw heavily on the methods of operations research. However, the emphasis in one is completely reversed in the other. These two types of effort are research applications and research opportunities.

The first, research applications, addresses the question of the utility of a specific piece of completed research. Given a scientific result which makes a statement about some natural phenomenon, this statement must be translated into terms meaningful to the engineer and to the operational user. A research applications study is intended to make this translation. It takes the quantitative scientific result, as a statement about some phenomenon and converts it into a quantitative statement about some engineering parameter or into a quantitative statement about some operational capability. Thus the engineer or the operational user can directly compare the potential contribution of this scientific result with what he currently has available and determine the value of exploiting this result. Clearly, the emphasis of research applications is toward the user of research rather than the research planner, and it will not be discussed further in this article.

The other type of effort used in research analysis, namely research opportunities, is concerned directly with the problems of the research planner. It is intended to start with a statement of technological or operational deficiency and derive precise and quantitative statements about scientific research needed to overcome these deficiencies. That is, it is intended to tell the research manager which phenomena should be investigated in order to provide the knowledge base from which a solution to the stated deficiencies can ultimately be drawn. Three points should be made about this type of analysis.

First, it is important to recognize that there is more to the Air Force than combat hardware. The Air Force consists also of people and procedures. Thus the deficiencies which provide the starting point for research opportunities studies must include "people problems" and "software problems." Clearly, there is little value in conducting research which leads to excellent hardware if that hardware is rendered ineffective by problems arising from the choice of people to man it or procedures to control and integrate its operation. Also of importance, but often overlooked, are items of noncombat hardware, such as communications and logistics. Deficiencies in these areas are also important starting points for research opportunities studies.

Second, it must be recognized that a research opportunities study is not intended to indicate, ahead of time, the one area of scientific endeavor guaranteed to lead to a solution of the original deficiency. On the contrary, it is intended to indicate all those areas which appear to have a reasonable likelihood of contributing to the solution. The objective is not to lead the research manager to the one specific area which should be investigated but rather to make sure that he does not overlook some area which might make a major contribution. Clearly, since a multiplicity of scientific areas will be indicated as sources of solutions to specific deficiencies, one of the criteria the research manager will use in formulating his program is to select research areas shown to lead to the solution of a multiplicity of deficiencies. In this way his program will be less sensitive to errors in the analysis.

Third, to re-emphasize what has been stated earlier, the research cannot be based on attempting to solve today’s problems. Thus the deficiencies which provide the starting point for research analysis must be those projected for some time in the future in order that the research program can be completed in time to be useful. How are these future deficiencies to be identified? This is the subject of the next section.

Systems analysis is a tool which, when properly applied, can aid the research manager in the task of selecting areas of technological deficiency. Systems analysis, while not new, has enjoyed widespread popularity within the Defense community as a tool for making selections between competing weapon systems and operational concepts. However, many current applications of the technique use the cost-effectiveness criterion as a discriminant for selection of a specific system for the weapon system inventory. This approach, while important and satisfactory for the systems selection process, does not provide much valid decision information for the research planner.

In recent years a variant of systems analysis has been applied to develop the needed information for research planning. Here the technique is used as a vehicle to identify technological deficiencies rather than for selection of weapon systems. While cost-effectiveness considerations are employed and preferred choices are illuminated, these do not form the major thrust of the studies, which is to identify areas of potentially high payoff for technological improvements. As noted, such decision information is required to properly identify research opportunities. It is information about the possible future technological deficiencies which the planner needs to lay out a good research program.

How does one go about using the tools of systems analysis to identify technological deficiencies? How can this tool be of use to the research manager? Can all systems analysis studies be used in this way to develop decision information for the research manager?

Most systems analysis studies can be structured at the outset to give information about technological deficiencies, and many completed studies can be updated and continued to give this type of information. The key in each case is to perform a sensitivity analysis of the various components used in the overall technical solution to the problem. In a sensitivity analysis the technological parameters of various components are varied systematically to determine the effect on total system performance. The idea is to isolate those areas where modest improvements result in a high payoff in performance. While this type of analysis is frequently used in the design of specific hardware, it also applies to large-scale systems that contain elements using dissimilar components. An example of one is a ballistic missile defense system.

Let us assume that the goal of a ballistic missile defense system is to intercept the incoming warhead at a point as early in the trajectory as possible. There is an obvious tradeoff between the interceptor and the sensor system. That is to say, with a better sensor one could tolerate some reduced interceptor performance and still achieve the desired goal. Likewise, increased interceptor performance would permit a reduction in the required sensor performance. Or, if an increase in the intercept altitude were desired, one could determine whether it could be obtained more easily and cheaply through improving interceptors or sensors. By using these approaches, one could arrive at specific technological tradeoffs, such as relating the specific impulse of the interceptor propellant to the power output or receiver sensitivity of the sensor radar.

This approach, of course, has historically been used to obtain near-term information for the design of a specific system. Its use as a research planning tool, however, is relatively new. In this application, future known or assumed military mission tasks are studied and possible technological approaches are postulated for their solution. These approaches can usually be stated in broad terms. Since the analysis is not intended to pick out a uniquely best solution, exact projection of costs and performance is not required. The only thing necessary is a consistent analysis that produces a base line of cost and performance data for sensitivity analysis. The purpose is to identify areas of possible technological deficiency, not to obtain the cost and performance data that would enable a choice of the best possible system. Once a base line has been obtained, it is possible to vary the estimates of performance of the various possible system components to illuminate technological problem areas. It is also possible to consider components or specific techniques which appear totally incapable of reaching adequate levels of performance. The technological barriers identified in this way, as well as the estimated payoffs from increases in component performance, then serve as inputs to a research analysis study.

The results of a systems analysis, however, are of value to the research planner only if they address tomorrow’s problems. The mission tasks selected for study by systems analysis must be relevant to the needs of the environment in which the Air Force of the future will operate. It is not sufficient to assume that because we have always done a certain thing we will continue to do it. Nor is it sufficient to assume that since we have always had weapons of a certain type we will continue to use them. The mission tasks selected for study must be based on sound estimates of the requirements of the time period 10 to 20 years into the future. The tools and techniques for obtaining these estimates are discussed in the next section.

Mission analysis, or mission identification as it is sometimes called, is one of the newest techniques available to the decision-maker. Man has long studied the various phenomena with which he is confronted, at first learning to explain and then to predict future occurrences of the phenomena. In recent years scholars have approached the realm of human affairs with the same inquiring analysis that has characterized studies of the natural sciences. While still in its elementary stages, some progress is being made, and new insights and techniques are being developed which have utility to the planner and the decision-maker.

Mission analysis involves two steps: the first is to develop a range of potential conflicts for the future time period, and the second is to transform these conflicts into mission tasks useful for their solution. Again there is nothing new about this approach; the improvements are in the manner of doing the study and how the information is developed.

The reliability and credibility of the list of possible mission tasks used for planning are strongly related to the reliability and credibility of the future conflict picture as it is understood at the present time. To be sure, some improvements are needed in the techniques for transforming conflict data into mission tasks, but the key to successful mission identification continues to be in developing better conflict information.

One approach is to postulate a spectrum of conflicts. In the crudest form, it is assumed that all the conflicts are equally probable. In some studies these various forms of conflict are developed into scenarios in order to give the information a feel of plausibility. Probability estimates or rankings can also be developed to ease the decision-maker’s task in selecting from this array of potential conflicts. However, it should be clear that the validity of the selected spectrum of conflicts has direct bearing on the validity of the derived array of mission tasks. These, in turn, have a bearing on the relevance of the technological deficiencies that are derived from the systems analysis studies using these various mission tasks as input data.

Recent investigations at some leading universities indicate that systematic studies can be made of the international sociopolitical environment to identify areas of potential conflict. Many of these studies apply quantitative techniques, which have the advantage that they can be duplicated and understood by other investigators. While the validity and reliability of these quantitative techniques remain to be proved, the potential does exist for the capability of developing improved conflict data.

One observation is important at this time. While planning for the long range (which is characteristic of planning for research) does require valid, reliable, and relevant estimates of future conflict, it does not follow that the group which does the planning must also develop the conflict data. The important factor is that valid estimates of future conflict patterns are essential to developing a plausible mission task structure necessary for research planning. The essential factor for the research manager is to have assurance that the input to the planning process is valid. For research planning this means that the mission task structure used for developing technological needs does represent a valid estimate of the future.

What has been described here is a process or method whereby research managers can get better information on which to base their decisions for the allocation of today’s resources to solve tomorrow’s problems. Many of the techniques are being used today, while others require further refinements before they are a part of the everyday tool kit of the research manager.

As is indicated by these descriptions, the areas of research analysis, systems analysis, and mission analysis are each important in its own right. A single laboratory, for instance, may carry out a program of research analysis based on statements of deficiencies obtained from higher headquarters or other organizations. An organization concerned with systems development may well carry out a program of systems analysis utilizing information about missions and environments obtained from various service or joint planning documents. Mission analysis could well be carried out by major operating commands interested in preparing long-range plans or by Hq USAF. In fact, there is a considerable amount of effort in each of the three areas in various Air Force organizations. Nevertheless, there are distinct advantages to carrying out all three activities in the same organization.

One important consideration is the ability to judge the quality of work done elsewhere. Even if an organization does both systems analysis and research analysis, for instance, it is unlikely to do all the systems analysis required as input to its research analysis efforts. Even if it had a large enough staff, some of its work would inevitably duplicate work done elsewhere. Hence any organization will normally attempt to take advantage of information produced elsewhere. However, the presence of an in-house capability in the same field, with adequate depth of experience gained through conduct of the same kind of work, will allow the organization to identify the limitations and strengths of studies conducted elsewhere. These studies can then be used with more confidence as information sources.

A second advantage, somewhat less obvious, is the ability to modify studies done elsewhere, to make them more compatible with the internal requirements of the organization; A study done elsewhere for some specific purpose may not be exactly suitable for the purpose intended by the organization planning to use it. An in-house capability to do the same type of study will enable the organization to make the desired modifications in initial assumptions, mathematical models, or data sources quickly and easily. Otherwise, the organization will have no alternatives except to use or reject an externally produced study as is.

A final advantage, although somewhat subtle, is extremely important—perhaps more important than the two already listed. This arises from the synergistic interaction of the three activities when carried out in the same organization. The results of a research analysis study, for instance, may completely alter the initial assumptions of the systems analysis study which preceded it by uncovering a previously unsuspected capability. Similarly, a systems analysis study may produce results which significantly alter the inputs to a preceding mission analysis study by showing a previously unsuspected impact of an offense-defense interaction. When these feedback effects are encountered, it is very valuable to have all three capabilities available in-house so that the sequence of studies can be iterated until a stable solution is reached. It is virtually impossible to obtain this synergistic effect when an organization produces only one type of study and uses as inputs studies done by other organizations for their own purposes.

The Office of Research Analyses (ORA) is an element of the Office of Aerospace Research (OAR). Its assigned mission is to develop and maintain an in-house capability to conduct all three of these types of studies. Much of the OAR effort in these areas is carried out by ORA, which also serves as a source of planning information for the other elements of OAR.

The Office of Research Analyses has at least one effort under way at all times in each of the areas of research analysis, systems analysis, and mission analysis. Many of these studies are done in response to requests from outside organizations. Others are generated internally as a result of a need uncovered by one or more members of the staff. Results of all studies are available to interested user organizations. However, the emphasis in all studies, whether done in response to an outside request or internally generated, is on providing planning information and guidance to the various OAR research planning staffs and research managers.

One important product of the ORA research program is new methodology for conducting each of the types of studies. Advances in analysis methods, new and more effective techniques, and more accurate or more general models are major outputs of the ORA program. Thus ORA provides a significant input to the planning activities of the OAR research manager, both through conducting studies whose results are useful to him and by developing new methodology for research analysis, systems analysis, and mission analysis. Since all these activities are carried out in the same organization, OAR benefits from the availability of an in-house capability in each activity and from the synergistic interactions of the three activities carried out simultaneously.

Research analysis, systems analysis, and mission analysis can provide the research manager with information about which areas of scientific endeavor are likely to have a high payoff in terms of providing the knowledge base to solve the problems of the Air Force of tomorrow. This information may still not be sufficient, however, to enable him to make decisions about allocating his resources. It may happen that an area of research which appears to be important is not yet ripe for activity. Before the needed research can be carried out, there may be a requirement for instrumentation advances, additional knowledge from another field, or some other step. Or there may be only a limited number of scientists of the required specialty available in the country. Or the work might require an expensive and specialized facility, identical to one already in existence elsewhere. A whole host of considerations must be taken into account before deciding to carry out research which has been shown to be of value to the future Air Force.

Thus the information obtained from the cycle of mission analysis, systems analysis, and research analysis is simply one input among many which the research planner must consider. It by no means tells the whole story or forces the decision in a certain direction. The function of this type of input is to reduce the uncertainty about the value of research in various areas of science. The research planner must still consider other factors. Nevertheless, this reduction of uncertainty is a very valuable asset. When properly used by the research planner, it can help ensure that his research program will expand, rather than constrain, the possibilities open to the Air Force of tomorrow.