Approved for public release; distribution is unlimited.
Published: 1 December 2009
Air & Space Power Journal - Winter 2009
Views and Analyses
Col William A. Thomas Jr., USAF, MC, FS*
Losing a student from Air Force undergraduate pilot training (UPT) incurs both direct and indirect costs to the service. In addition to the fact that it wastes an opportunity that another Air Force asset could have used, all or part of the approximately $750,000 that a UPT training slot costs will see no return on investment.1 Seeking to minimize attrition from all causes, Air Education and Training Command (AETC) uses a variety of screening tools for selecting students. The largest category of student loss from UPT is voluntary attrition, called “dropping on request” (DOR). Attempting to minimize this type of attrition, in 2004 the Air Force changed the syllabus for evaluating students prior to attendance at UPT by replacing Initial Flight Training (IFT)—a decentralized course that provided 50 hours of flight instruction and a private pilot’s license—with Initial Flight Screening (IFS), a more centralized program that offers only 25 hours of flight instruction but that demands more rigorous training and emphasizes officership. Now, five years later, we need to evaluate the effectiveness of this change.
Since the beginning of military aviation, the Army Air Corps and then the US Air Force have outlined requirements for qualification of student pilots and have sought effective screening of training applicants to select those who would become the most successful. Selection criteria and the number of student pilots needed by the service have changed substantially over time as political situations altered, as the physiology of humans in the flying environment became better understood, and as the performance of aircraft developed.2
A variety of methods have been used to actively manage the volume and capacity of pilot training. As early as 1938, student pilots completed initial training requirements under the tutelage of civilian instructors before continuing their training at Brooks and Kelly Fields, Texas.3 Just as the number of required pilots varied through the conflicts of the last century, so did civilian screening and training programs. Analysis of these methods validated their efficacy as well as their shortfalls. In 1955 the Flying Training Air Force, a forerunner of AETC, conducted a study that compared attrition rates of 538 students who had received preflight training to those of 541 who had not. They found similar overall attrition rates but a smaller rate of voluntary attrition from subsequent training in the group that had undergone the preflight program.4 Additionally, the latter students “scored higher in . . . attitude, motivation levels, knowledge of service, and practical experience.”5 Later, between 1956 and 1958, a study found an attrition rate from UPT of 6.3 percent for Air Force Reserve Officer Training Corps (AFROTC) cadets who had received civilian instruction in light planes as part of the Flight Instruction Program prior to UPT, compared to 24.7 percent for those who had not.6 However, participants in the program “had to unlearn a variety of bad flying habits during primary training.”7 Since then, programs including military and civilian courses have offered pre-UPT training to Air Force Academy and ROTC cadets. Most recently the Air Force has used a centralized and standardized syllabus for contracted instruction to provide flying-orientation and training programs to students interested in proceeding to UPT.8
Over the years, the Air Force has also studied physical and psychological variables, employing them as tools for screening pilot candidates. Methods for aptitude testing, used as early as 1928, include a variety of psychological evaluations, psychomotor testing, and standards for physical examination.9 A board for training selects today’s UPT candidates, based on a combination of factors such as academic performance, letters of recommendation, and Pilot Candidate Selection Method scores—generally predictive of success in UPT.10 The latter scores include the Test of Basic Aviation Skills and the Air Force Officer Qualification Test, as well as the number of flying hours that the candidate may have accumulated. Prior to beginning UPT, students complete a battery of neuropsychological tests called the Medical Flight Screening-Neuropsychiatric (MFS-N)—standard for all UPT candidates since 1994—which includes verbal and performance IQ testing; personality testing; and cognitive testing for attention, concentration, and psychomotor skills.11 Though not part of the UPT selection criteria, these data are a rich source of information on the attributes of the candidate and have been used to construct a composite neuropsychological picture of the successful Air Force aviator. The results of neuropsychological testing may also serve as a baseline study for the individual aviator in the event that a medical evaluation necessitates repeated testing.
As mentioned above, in 2004 the Air Force changed the method, locations, and requirements for pre-UPT training from IFT (a 50-flying-hour program) to IFS (a 25-flying-hour program). IFT began in 1998 after Air Force–wide grounding of the T-3 aircraft due to several fatal mishaps, which halted the Enhanced Flight Screening UPT training program.12 In accordance with federal guidance for pilot preparation, a civilian-only staff conducted the IFT program.13 The training enjoyed wide latitude in methods, focusing mainly on the end state—successful completion of requirements for obtaining a private pilot’s license. To provide IFT for UPT candidates, AETC contracted with flight schools that conducted training at over 200 locations nationwide. On average, most students (civilian and military) needed 70–80 flying hours of instruction to attain the license. AETC accelerated training requirements to mandate that students successfully solo by 25 hours, pass a check ride with a Federal Aviation Administration examiner, and earn their private pilot’s license by the 50-hour training point (compared to the average of 70–80 hours of flight time needed to attain licensure for general aviation students). This compressed requirement served as an indicator of the candidate’s potential for successful completion of UPT.14 After finishing IFT, candidates underwent Medical Flight Screening and, if cleared, joined a UPT class.
Over time, there arose widespread perception that the Air Force lacked sufficient oversight of the IFT program. Leaders at Headquarters US Air Force and AETC felt that the decentralized training was not rigorous enough to prepare students adequately for UPT and that its content varied too much.15 They surmised that the absence of a UPT-like environment for flight training and discipline could be the cause of increased rates of voluntary attrition at UPT.
These concerns prompted a search for other options to meet the needs of the Air Force. Developed to correct problems, minimize attrition, and provide a more UPT-like training environment, the IFS program would limit training sites and enhance Air Force oversight by centralizing the training at a single location over the course of several years. AETC developed a structured syllabus and contracted with a single agency (Doss Aviation) to execute the program at its facility in Pueblo, Colorado. IFS focuses less on training and more on screening to identify the most appropriate candidates to continue to UPT. Toward that end, it includes 18 hours of flight academics, 12 hours of ground training, and 28 hours of officer development—but just 25 hours of flight time.16 Importantly, Medical Flight Screening occurs before IFS begins; the requirement for solo flight moves up to the 17-flying-hour point, with a check ride by a military or civilian pilot; and the program offers no pilot’s license.17 As IFT drew down and IFS ramped up, the programs overlapped somewhat, and significantly fewer civilian schools participated in IFS since the Air Force intended to limit the program to the Pueblo facility.
In light of the fact that (1) the Air Force wishes to minimize attrition from UPT, (2) the IFS program has significantly decreased the number of flying hours completed by students before entering UPT, and (3) all entrants to UPT have satisfactorily completed Medical Flight Screening but have less flying experience, it is time to assess the impact of the program change. This article reports the findings of a study in which the author compared the two pre-entry training programs to determine if a significant difference exists between their UPT attrition rates (due to medical issues, failure to progress, or voluntary withdrawal). These findings should prove useful to the AETC Operations Directorate (AETC/A3) and might help guide planning for future programs in pilot training.
As a preliminary step, the Institutional Review Board of the Air Force Research Laboratory reviewed and approved the research outline, assuring the existence of appropriate safeguards for the confidentiality of personal information.18 The author then queried the flight-training database at AETC/A3 for the total number of students who had completed the 25-hour IFS program since its inception, either at Pueblo or at a civilian flight school, from 2005 through late 2008. Students who had completed the 50-hour IFT program at any location between 2004 and 2006 (the time frame just preceding the change) served as a comparison group. Cross-referencing of these rosters through the Training Information Management System database revealed which of these students had officially started UPT. Rosters for both programs listed the outcome for all students, indicating whether they had completed UPT through phase two (during which they train in a specific airframe) or attrited from the program. The study excluded students who had attrited prior to the rest of their classmates’ graduation from phase two. Categories of attrition included DOR, medical reasons (MED), failure in academic or flying performance (Flying/Academic), and lack of adaptability (LOA)—which includes students who withdrew due to fear of flying, persistent airsickness, or manifestations of apprehension. When the data noted no outcome for a student or listed the category of attrition as “other,” the author contacted the registrar at the UPT base for clarification. The defined and validated data then underwent statistical analysis.
Next, the study evaluated reasons for DOR attrition. The author contacted the UPT bases again to gather information on students’ underlying reasons for their DOR. The registrars do not have access to such specific information; neither is it forwarded to AETC. In all cases, either the registrar or squadron leadership reviewed paper or electronic files maintained locally to determine why the student requested release from training. The author grouped these reasons into broad categories and completed a statistical analysis, according to whether the student completed IFS or IFT. Finally, he compared in aggregate the MFS-N test scores for the group who had attrited by DOR to composite scores that characterize the successful Air Force pilot.
Raw data received from AETC/A3 included information on students who had completed IFS but whose classes had not yet graduated from phase two of UPT. These entire classes were eliminated from the analysis, including those in that group who had already attrited, leaving 40 students with undefined outcomes or with attritions categorized as “other.” Registrars at the UPT bases clarified these outcomes. Completion of the data collection and validation yielded 1,649 students with defined outcomes (630 from IFS and 1,019 from IFT). Only one student remained categorized as “other” due to closure of the UPT training programs at Moody AFB, Georgia, in 2005. The author then sorted the data by program type and completion status for the initial evaluation (table 1).
UPT attrition according to type of pre-UPT training
The study utilized chi-square analysis to determine if a significant difference existed between the overall attrition rates for the two programs. Subsequently, sorting of the data by specific type of attrition for further analysis proceeded in two steps: (1) a comparison of students in the category of attrition under consideration to those in all other categories (attrited and completed) combined, and (2) analysis using only the category of attrition under consideration versus the graduates but not including the other types of attrition (i.e., attrition versus graduates only).
The evaluation showed no significant difference in attrition across all causes between the IFS and IFT programs.19 When we consider specific causes, it is apparent that the shift to the IFS curriculum has not improved the DOR rate. The 25-hour program reflects a statistically significant increase in attrition due to DOR, compared to the 50-hour IFT program.20 Even with 40 percent fewer students, IFS had a higher LOA rate, so that kind of attrition may also be related to the type of training, though we have less confidence in this relationship.21 This LOA finding remained consistent when compared both to graduates only and to all UPT starts; it may become more well defined as the number of IFS trainees increases. Medical Flight Screening prior to IFS may contribute to the lower rate of medical attrition.
Since IFS offers only half the number of flying training hours, we may surmise that the decreased exposure to flying may influence more students to enter UPT, especially those who are perhaps unsure of their commitment to flying or less motivated to pursue an Air Force flying career. Registrars at the UPT bases compiled the reasons for DOR among members of the group considered in this analysis. Table 2 summarizes the broad categories of attrition.
Reasons for DOR attrition by base and type of pre-UPT training
Students who DOR from UPT because they did not enjoy flying or did not have the desire to fly accounted for half of the total DOR attritions. The number of students who DOR from UPT after having completed IFS showed a significant statistical increase over the number who DOR for the same reason after finishing IFT.22 The decreased number of flying hours that these students experienced prior to starting UPT may have some bearing on this finding.
An aviation psychologist and a biostatistician at the Air Force School of Aerospace Medicine Consultation Service compared the aggregate MFS-N data from the DOR group to the composite data of the successful Air Force pilot. Despite the existence of statistically significant differences with adequate statistical power on a few of the 45 categories of the test profiles, “the effect sizes were not large enough to warrant viewing the differences as clinically meaningful.”23 We might still determine the relationship between an applicant’s test results and the likelihood of DOR from UPT by utilizing a more thorough characterization of the underlying reason for DOR attrition. This could prove useful in helping guide a future programmatic change.
Limitations of this analysis include, first, lack of detail on the students’ background and flying history. We could expect candidates who have held a private pilot’s license or have had significant flying experience (military or civilian) prior to attending UPT to be more motivated to fly and to display better performance during training. Second, the data and the categorization of reasons for DOR may not accurately reflect the students’ true motivation for their attrition. The author had varying levels of access to the “show cause” letters and categorized them subjectively as a “best fit” into potentially overlapping categories. Despite the possibility of multiple causes for DOR, the study placed the individual in only a single category of DOR attrition. Further, closure of one of the UPT training bases prevented the gathering of specific reasons for DOR among students located there. Similarly, specifics of the medical diagnoses leading to attrition and the reasons for categorization as LOA lack clarity. Such details could make the study more meaningful and help define relationships that may exist between the MFS-N scores and UPT attrition.
Finally, changes in Air Force policies during the period under consideration may have affected the results. AETC’s Initial Flight Training Branch (AETC/A3FI) reports that for a period of time in 2006 and part of 2007, the Air Force separated lieutenants who failed their initial flight training. Those who DOR were also required to pay back any scholarship money the Air Force had given them. The number of such students remains unknown, as does the amount of money actually recouped by the Air Force—but the DOR rate dropped when the policy was in effect.24
The Air Force may be able to minimize DOR and LOA attrition by implementing additional screening processes to assess students’ adaptability and motivation for flying. Moreover, specific clarification of the reasons for DOR may help outline the programmatic actions needed to lessen this type of attrition. For example, when requesting DOR from training, students must supply a “show cause” letter, provided to the wing commander through the chain of command. Additionally, requiring UPT students to categorize more specifically their reason for DOR by selecting from a list of common options on a worksheet would allow tracking at the major command level. This data would prove useful in determining specific underlying causes for DOR that we might anticipate on the basis of the MFS-N scores or address by implementing programmatic changes.
Travis AFB, California
*The author is commander of the 60th Aerospace Medicine Squadron, Travis AFB, California.
1. “There are no ‘good’ figures [on costs per student] for these courses as the variables and conditions for analysis are too numerous. . . . These [are] approximations: IFS—$14,000, IFT—$9,000, UPT—$750,000.” Wayne Mudge, AETC/A3FI, to the author, e-mail, 7 May 2009.
2. For a detailed history of the Air Force’s programs for candidate screening and flight training, see Anne Krueger Hussey, Air Force Flight Screening: Evolutionary Changes, 1917–2003 (Randolph AFB, TX: Office of History and Research, Headquarters AETC, 2004), http://www.aetc.af.mil/shared/media/document/AFD-061109-020.pdf.
3. Ibid., 5.
4. Ibid., 25–26.
5. Ibid., 26.
6. Ibid., 26–27.
7. Ibid., 27.
8. “Flying Training: Initial Flight Screening,” AETC Syllabus S-V8A-S (Randolph AFB, TX: Headquarters AETC, 2006), http://www.dossifs.com/usaf/Docs/SyllabusAug06.pdf.
9. Hussey, Air Force Flight Screening, 5.
10. See the frequently asked questions regarding the Pilot Candidate Selection Method: “PCSM Program,” Air Education and Training Command, https://pcsm.aetc.af.mil/FAQS/FAQS2.HTM#ONE (accessed May 2009).
11. Wayne Chappelle, aerospace psychologist, USAF Aeromedical Consultation Service (USAFSAM/FEC), personal communication with the author, 27 April 2009.
12. Hussey, Air Force Flight Screening, 60–61.
13. “Flying Training: Initial Flight Screening”; and 14 Code of Federal Regulations, part 61, subpart C.
14. Wayne Mudge, AETC/A3FI, to the author, e‑mail, 18 November 2008.
15. Hussey, Air Force Flight Screening, 68.
16. “Flying Training: Initial Flight Screening,” 1.
17. Mudge, e-mail, 18 November 2008.
18. Air Force Research Laboratory, Institutional Review Board Protocol no. F-BW-2008-0004-H, 17 September 2008–17 September 2009.
19. p = 0.05. (This means that there is a 95 percent likelihood that the finding is not due to chance alone.)
20. p = 0.005.
21. p = 0.15. (This means that there is a 15 percent possibility that the finding is due to chance alone.)
22. p < 0.05.
23. Furthermore, “it is possible in the instances where power was sufficient that increasing sample size might increase power, but in those outcomes where power was sufficient, the only thing that would increase effect size would be increased separation in the mean scores. . . . Miniscule differences in mean scores could result in significantly different findings at p < .05 with a power = > .80. The effect size . . . is a function of mean and standard deviation in its simplest form: effect size = (mean of sample - mean of population) / pooled standard deviation). In order to increase effect size, the difference in means must increase or standard deviation change or some combination thereof. Again, we have determined that an effect size of .7 or greater would be clinically meaningful, and, as such, our outcomes don’t meet that criterion.” Bill Thompson, USAFSAM/FEC biostatistician, to the author, e-mail, 1 May 2009.
24. Wayne Mudge, AETC/A3FI, to the author, e‑mail, 1 December 2008.
The conclusions and opinions expressed in this document are those of the author cultivated in the freedom of expression, academic environment of Air University. They do not reflect the official position of the U.S. Government, Department of Defense, the United States Air Force or the Air University
[ Home Page | Feedback? Email the Editor]