Timeline of Activities Leading to the Certification of the Boeing 737 MAX 8

Aircraft and Actions Taken After the October 2018 Lion Air Accident

Report No. AV2020037

June 29, 2020

Timeline of Activities Leading to the Certification of the Boeing 737 MAX 8 Aircraft and Actions Taken After the October 2018 Lion Air Accident Requested by the Secretary of Transportation; the Chairmen of the House Committee on Transportation and Infrastructure and its Subcommittee on Aviation; the Chairman and Ranking Member of the Senate Committee on Appropriations, Subcommittee on Transportation, Housing and Urban Development, and Related Agencies; and Senator Richard Blumenthal

Federal Aviation Administration | AV2020037 | June 29, 2020

What We Looked At The Federal Aviation Administration (FAA) is responsible for the safety and certification of all civilian aircraft manufactured and operated in the United States. However, two accidents in late 2018 and early 2019 involving Boeing 737 MAX 8 aircraft raised significant safety concerns about FAA’s certification of this aircraft. On March 19, 2019, Secretary of Transportation Elaine L. Chao requested that we compile an objective and detailed factual history of the activities that resulted in the certification of the 737 MAX 8. We also received similar requests from the Chairmen of the House Committee on Transportation and Infrastructure and its Subcommittee on Aviation; the Chairman and Ranking Member of the Senate Committee on Appropriations, Subcommittee on Transportation, Housing and Urban Development, and Related Agencies; and Senator Richard Blumenthal. They requested that we review aspects of FAA’s approach to certifying the MAX series of aircraft, its reliance on the Organization Designation Authorization (ODA) program, and the Agency’s actions following the two accidents. Our overall audit objective was to determine and evaluate FAA’s process for certifying the Boeing 737 MAX series of aircraft.

What We Found In this report, we provide a detailed timeline of the activities resulting in the certification of the 737 MAX 8, beginning in January 2012, when Boeing submitted its initial application for an Amended Type Certificate to FAA. This report also compiles a timeline of events following the October 29, 2018, crash of Lion Air Flight 610 up until the crash of Ethiopian Air Flight 302 on March 10, 2019. In addition, during the same time period as FAA’s certification efforts, Boeing, FAA, and our office were identifying issues that—although not specific to the 737 MAX 8—may have impacted the original certification of the aircraft. As such, we also provided a timeline of concurrent related oversight actions and events related to FAA’s ODA program.

Our Recommendations We are not making recommendations in this report. The data gathered are informational and represent our observations in response to the Secretary’s and other congressional requests. We will report further on FAA’s oversight of the certification process and other related matters, as well as make recommendations as applicable, in future reports.

All OIG audit reports are available on our website at www.oig.dot.gov.

For inquiries about this report, please contact our Office of Government and Public Affairs at (202) 366-8751.

AV2020037

Contents Memorandum 1

Background 3

Results in Brief 8

Timeline of Activities Leading to the Certification of the Boeing 737 MAX 8 Aircraft 10

Timeline of Events Between the Lion Air and Ethiopian Airlines Crashes 29

Timeline of Concurrent FAA Organization Designation Authorization Oversight Actions and Events 34

Conclusion 38

Recommendations 38

Agency Comments and OIG Response 38

Exhibit A. Scope and Methodology 39

Exhibit B. Organizations Visited or Contacted 41

Exhibit C. Glossary of Terms 42

Exhibit D. Major Contributors to This Report 45

Appendix. Agency Comments 46

AV2020037 1

U.S. DEPARTMENT OF TRANSPORTATION OFFICE OF INSPECTOR GENERAL

Memorandum Date: June 29, 2020

Subject: INFORMATION: Timeline of Activities Leading to the Certification of the Boeing 737 MAX 8 Aircraft and Actions Taken After the October 2018 Lion Air Accident Report No. AV2020037

From: Howard R. “Skip” Elliott Acting Inspector General

To: Federal Aviation Administrator

The Federal Aviation Administration (FAA) is responsible for the safety and certification of all civilian aircraft manufactured and operated in the United States. While FAA has an excellent safety record, two accidents in late 2018 and early 2019 involving Boeing 737 MAX 81 aircraft have raised significant safety concerns about FAA’s certification of this aircraft.

On October 29, 2018, Lion Air Flight 610 crashed into the Java Sea shortly after departing Soekarno-Hatt International Airport, Jakarta, resulting in 189 fatalities. Just over 4 months later, on March 10, 2019, Ethiopian Air Flight 302 crashed shortly after departing Addis Ababa Bole International Airport, resulting in 157 fatalities, including 8 Americans.

On March 19, 2019, Secretary of Transportation Elaine L. Chao requested that we compile an objective and detailed factual history of the activities that resulted in the certification of the 737 MAX 8. We also received similar requests from the Chairmen of the House Committee on Transportation and Infrastructure and its Subcommittee on Aviation; the Chairman and Ranking Member of the Senate Committee on Appropriations, Subcommittee on Transportation, Housing and Urban Development, and Related Agencies; and Senator Richard Blumenthal. They requested that we review aspects of FAA’s approach to certifying the MAX series of aircraft, its reliance on the Organization Designation Authorization (ODA) program,2 and the Agency’s actions following each of the two accidents.

1 The official model number of the Boeing 737 MAX 8 is the 737-8. 2 FAA created the ODA program in 2005 to standardize its oversight of organizational designees (e.g., aircraft manufacturers) that have been approved to perform certain functions on the Agency’s behalf, such as determining compliance with aircraft certification regulations.

AV2020037 2

This is the first report that the Office of Inspector General (OIG) is providing related to these requests. Our overall audit objective was to determine and evaluate FAA’s process for certifying the Boeing 737 MAX series of aircraft. In this report, in response to the Secretary’s request, we provide a detailed timeline of the activities resulting in the certification of the 737 MAX 8. In addition, in response to multiple congressional requests, this report includes timelines of events following the October 2018 Lion Air crash up until the March 2019 Ethiopian Air crash and concurrent related oversight actions and events related to FAA’s ODA program. We are also undertaking additional analyses of FAA’s processes for certifying the 737 MAX 8 aircraft, including its use of the ODA program, as well as examining FAA’s actions following the Ethiopian Air crash. We will report on the results of these and other related reviews in future reports.

We conducted this audit in accordance with generally accepted Government auditing standards. Exhibit A details our scope and methodology, and exhibit B lists the organizations we visited or contacted. For a glossary of terms used in this report, see exhibit C.

We appreciate the courtesies and cooperation of Department of Transportation (DOT) representatives during this audit. If you have any questions concerning this report, please call me at (202) 366-1959.

cc: The Secretary DOT Audit Liaison, M-1 FAA Audit Liaison, AAE-100

AV2020037 3

Background FAA is charged with overseeing the safety and certification of all civilian aircraft manufactured and operated in the United States.3 This is a significant undertaking given that the U.S. civil aviation industry encompasses almost 292,000 aircraft, nearly 1,600 approved manufacturers, and more than 5,400 aircraft operators, among others. Recognizing that it is not possible for FAA employees to oversee every facet of such a large industry, Federal law4 allows the Agency to delegate certain functions to private individuals or organizations, such as determining compliance with aircraft certification regulations. Designees can perform a substantial amount of critical certification work on FAA’s behalf. For example, according to FAA data, in 2018 four U.S. aircraft manufacturers approved about 94 percent of the certification activities for their own aircraft.

In 2009, FAA fully implemented the ODA program to standardize its oversight of organizations (e.g., aircraft manufacturers) that are approved to perform certain delegated functions on its behalf. While delegation is an essential part of meeting FAA’s certification goals, the Agency faces challenges in providing ODA oversight. For example, in 2015 we reported5 that FAA’s oversight of ODA program controls was not systems- and risk-based,6 as recommended by an aviation rulemaking committee.7 Rather, the oversight was more focused on individual engineering projects and areas that we determined were low risk. Under FAA’s ODA program, the Agency’s Boeing Aviation Safety Oversight Office (BASOO) provides oversight of authorized functions granted to Boeing. The BASOO is comprised of 42 FAA employees who oversee Boeing’s ODA.8 The Boeing ODA unit includes approximately 1,500 Boeing-designated ODA representatives. FAA’s oversight program is based on managing and supervising an organization, rather than overseeing individual designees.

When undertaking certification activities for a manufacturer with an ODA, FAA typically retains some level of involvement in significant design changes, novel

3 49 U.S.C. § 44702. 4 49 U.S.C. § 44702(d). 5 FAA Lacks an Effective Staffing Model and Risk-Based Oversight Process for Organization Designation Authorization (OIG Report No. AV2016001), October 15, 2015. OIG reports are available on our website at http://www.oig.dot.gov/. 6 Systems-based oversight shifts from focusing on individual project engineering work to holistically assessing whether ODA companies have the people, processes, procedures, and facilities in place to produce safe products, thus allowing FAA to focus its oversight on the highest-risk areas, such as new, innovative aircraft designs. 7 Aircraft Certification Process Review and Reform Aviation Rulemaking Committee, a joint FAA and industry group, formed in response to a congressional mandate to study the aircraft certification process. 8 The BASOO includes 23 engineers who perform both certification work as well as oversight, 3 inspectors that perform oversight, and additional project manager engineers and support staff.

AV2020037 4

designs, and critical compliance activities, based on the ODA’s experience and FAA’s judgment of the potential risk. According to FAA, the Agency always retains inherently governmental functions such as regulatory exemptions and functions for which an ODA is not authorized.

FAA’s process for determining the certification basis of aircraft models is set forth in regulations (14 CFR Part 21) and guidance (FAA Orders 8110.48 and 8110.4c). Under this guidance, FAA can either award a type certificate9 (TC) for new aircraft models or an amended type certificate (ATC) for aircraft models that are derivatives of already-certificated aircraft10 (see figure 1 for a flowchart of the certification process).

Figure 1. Key Phases in the Certification Process

Source: OIG analysis of FAA Order 8110.4C

9 An approval document issued by FAA that states a specific aircraft model is compliant with airworthiness regulations. 10 This is known as the “baseline aircraft.”

AV2020037 5

The Boeing 737 MAX series11 is the fourth-generation model of Boeing’s 737 aircraft series (see figure 2). The first Boeing 737, the 737-100, received its type certificate on December 15, 1967—49 years before the Boeing 737 MAX 8. The 737 MAX 8 was certified as an ATC with the 737-800 (certified on March 13, 1998) as the baseline, part of the 737 Next Generation (NG) series.

Figure 2. Boeing 737 Family of Aircraft – 1967 to 2017

Source: OIG analysis of FAA and Boeing data

According to FAA regulations, once applicants file for a new or amended type certificate, they have 5 years to complete the process. During the certification process, manufacturers are required to demonstrate compliance to the relevant standards. Those standards are largely contained in 14 CFR Part 25 and are amended as needed due to new technologies, in response to operational data, or because of legislative mandates. The major milestones and requirements of the certification process for a new or amended type certificate are similar. However, if an aircraft model is certified under the ATC process, only systems or areas that have been significantly changed need to be brought up to current regulatory standards,12 and other exceptions can be applied.13

The 737 MAX 8 included a function in the flight control software—the Maneuvering Characteristics Augmentation System (MCAS)—that was new to commercial aircraft. MCAS modifies aircraft handling characteristics as an additional function of the existing aircraft speed trim system. The speed trim system is a flight control system designed to improve the airplane’s flight stability

11 The 737 MAX series includes the 7, 8, 9, 10, and 8200. The MAX 7, 10, and 8200 have not yet been certified by FAA. 12 14 CFR § 21.101 and Advisory Circular 21.101-A, more commonly known as the “Changed Product Rule.” 13 Applicants can also comply with earlier requirements when (1) an area, system, component, equipment, or appliance are not affected by the change; (2) compliance with a later amendment does not materially improve safety; or (3) compliance with the latest amendment is impractical.

AV2020037 6

during operations in certain conditions when the autopilot is not engaged.14 Boeing developed MCAS for the 737 MAX 815 to compensate for changes in aerodynamics from the previous model caused by the MAX’s larger engines and the placement of those engines on the wing (see figure 3).

Figure 3. Engine Size and Placement: 737 NG (pictured left) vs. 737 MAX (pictured right)

Source: Boeing

More specifically, MCAS can cause the airplane’s horizontal stabilizer16 to move without pilot input in certain, limited aircraft configurations17 related to airspeed and the angle of the aircraft in the air. This has the effect of moving the plane’s nose down during flight (see figure 4) to compensate for the aircraft’s tendency to pitch up. The accident report for the October 29, 2018, Lion Air accident states that MCAS was a significant contributing factor for the accident, after activating 24 times during the flight.18 MCAS activated after receiving faulty data from one of the aircraft’s two Angle-of-Attack (AOA) sensors—external sensors that measure the angle of the aircraft in the air. While the accident investigation for the March 10, 2019, Ethiopian Air accident is still ongoing, the preliminary and interim reports19 also point to MCAS as a potential contributing factor to the accident.

14 The speed trim system monitors airspeed, thrust lever position, and vertical speed, and then provides inputs to adjust the aircraft’s horizontal stabilizer as needed. As the airplane speed increases or decreases, the system automatically commands the stabilizer in the direction needed. 15 While MCAS is included on some military versions of the 767 refueling tanker, the system has different features on that model, including additional redundancy of input data. 16 A control surface near the tail of the airplane that controls up and down movement of the airplane. 17 These configurations include the plane being in manual flight (autopilot off) and the flaps being in an up position. 18 Komite Nasional Keselamatan Transpotasi Republic of Indonesia. KNKT.18.10.35.04. FINAL. 2019. 19 Federal Democratic Republic of Ethiopia, Ministry of Transport, Aircraft Accident Investigation Bureau. Preliminary Report. Interim Report, March 2020.

AV2020037 7

Figure 4. How MCAS Works on the 737 MAX

Source: OIG analysis of FAA and Boeing data

AV2020037 8

Results in Brief Summary of Events Leading up to FAA’s Certification of the Boeing 737 MAX 8

Overall, FAA followed its established certification process for the 737 MAX 8, which began in early 2012 when Boeing submitted its initial application for an ATC. Under an ATC, as agreed to by FAA and Boeing, only the significant differences between the 737 MAX 8 and the previous model—in this case, the 737-800 aircraft—must be certified to current regulatory standards as of the application date. Early in the process, Boeing included limited information in initial briefings to FAA on the MAX’s flight control software, MCAS, which subsequently has been cited as a contributing or potentially contributing factor in both accidents. However, Boeing presented the software as a modification to the existing speed trim system that would only activate under certain limited conditions. As such, MCAS was not an area of emphasis in FAA’s certification efforts and therefore did not receive a more detailed review or discussion between FAA engineers and Boeing. Instead, FAA focused its efforts on areas it identified as potentially high risk, such as the aircraft’s larger engines, fly-by-wire spoilers, and landing gear changes. As a result, FAA was not well positioned to mitigate any risks related to MCAS.

From 2012 to 2014, Boeing and FAA collaborated to develop and implement an overall certification plan, including determining which aspects of the certification process would be delegated to the Boeing ODA. Throughout 2015 and 2016, FAA and the Boeing ODA conducted certification activities that evolved over the course of the project. During this timeframe, Boeing also began modifying MCAS as a result of flight testing, including significantly increasing MCAS’s ability to lower the aircraft’s nose automatically under certain conditions. However, Boeing did not submit certification documents to FAA detailing the change. FAA flight test personnel were aware of this change, but key FAA certification engineers and personnel responsible for approving the level of airline pilot training told us they were unaware of the revision to MCAS. Boeing did not communicate to FAA the formal safety risk assessments related to MCAS until November 2016 and January 2017, more than 4 years into the 5-year certification process. According to FAA management, it is not unusual for manufacturers to complete and submit the safety assessments towards the end of the certification process. Moreover, Boeing’s safety analysis did not assess system-level safety risks as catastrophic; thus, Boeing designed MCAS to rely on data from a single aircraft sensor rather than including redundancy, which would have reduced risk.

In 2016, FAA and Boeing began certification flight testing to determine the aircraft’s compliance with FAA’s requirements. In addition, FAA’s Flight Standards Service conducted separate tests and subsequently approved a training plan

AV2020037 9

proposed by Boeing—known as Level B training—for 737 MAX pilots who were already qualified to fly the Boeing 737-800. This outcome aligned with Boeing’s overarching goal of achieving a common type rating20 for pilots moving from the NG series to the MAX and keeping costs down by avoiding simulator training for MAX pilots. Pilot response to automated MCAS activation was not included in the required training. In March 2017, FAA issued an ATC to Boeing for the 737 MAX 8, which began flying passengers later that year.

Summary of Events Between the Lion Air and Ethiopian Airlines Crashes

On October 29, 2018, Lion Air Flight 610 crashed, resulting in 189 fatalities. According to the accident report, MCAS activated based on erroneous AOA data more than 20 times, automatically pushing down the aircraft’s nose, before the flight crew lost control. According to Boeing, while its engineers and test pilots had anticipated multiple MCAS activations to be possible, they decided it would be no worse than a single activation because pilots would be able to recognize and counteract any downward movement of the aircraft’s nose. On November 6, 2018, Boeing generated a bulletin21 to operators, prompting FAA to issue an Emergency Airworthiness Directive (AD) the next day. Although the bulletin and the Emergency AD emphasized pilot procedures for handling repeated nose-down movements, neither specifically mentioned MCAS. At this time FAA also began reviewing the MCAS certification process. This was the first time that FAA’s certification engineers had performed a detailed review of MCAS, and according to several FAA certification engineers, it was also the first time they were presented with a full picture of how MCAS worked. As a result of FAA’s risk analysis following the crash, Boeing proposed, and FAA accepted, a redesign of MCAS. In February 2019, FAA and Boeing formally agreed on a schedule to implement the recommended MCAS software fix. Based on FAA’s risk analysis and existing risk guidelines, Boeing was to complete the software update by April 12, 2019. On March 10, 2019, Ethiopian Air Flight 302, operating a 737 MAX 8 aircraft, crashed shortly after departing Addis Ababa Bole International Airport, resulting in 157 fatalities, including 8 Americans. The interim accident report shows that MCAS activated based on erroneous AOA data before the crew lost control.

Summary of Concurrent FAA ODA Oversight Actions and Events

During the same time period as FAA’s certification efforts, Boeing, FAA, and our office were identifying issues that—although not specific to the 737 MAX 8—may have impacted the original certification of the aircraft. More specifically, in 2015, we reported on FAA’s lack of a risk-based oversight approach to ODA. In

20 A type rating is an endorsement on the pilot certificate indicating that the pilot has completed the required training and testing for a specific make, type, and/or series of aircraft (for example Boeing 747-400). 21 Flight Crew Operations Manual (FCOM) Bulletin TBC-19, dated November 6, 2018.

AV2020037 10

addition, FAA identified concerns regarding the quality of ODA certification documents that needed to be addressed. In December 2015, FAA and Boeing signed a Settlement Agreement, and Boeing paid a civil penalty of $12 million regarding violations of Boeing’s quality control system and insufficient certification documents. FAA’s subsequent oversight found that Boeing has not yet resolved all the identified issues, including improving its identification and resolution of the root causes of non-compliances with FAA requirements. During this time period, Boeing and FAA also identified concerns about undue pressure on ODA personnel at multiple Boeing facilities, which culminated in FAA issuing a formal compliance action against Boeing in November 2018. Boeing’s response to this compliance action remains ongoing.

Given that we have an open recommendation for FAA related to ODA and that we are planning additional analysis of FAA’s certification process and the use of the ODA program for the 737 MAX 8 aircraft, we are not making recommendations in this report. The data gathered are informational and represent our observations in response to the Secretary’s and other congressional requests. We will report further on FAA’s oversight of the certification process and other related matters, as well as make recommendations as applicable, in future reports.

Timeline of Activities Leading to the Certification of the Boeing 737 MAX 8 Aircraft

The following presents a detailed timeline of the events leading to the certification of the 737 MAX 8, which began in January 2012 and culminated with the issuance of an ATC in March 2017. Figure 5 presents an overview of the events; detailed descriptions follow.

AV2020037 11

Figure 5. Timeline of Major Events for the Certification of the Boeing 737 MAX 8

Source: OIG analysis of FAA and Boeing data

AV2020037 12

2012–2013: Boeing and FAA Initiate Certification Process; MCAS Was Not an Area of FAA Emphasis

The certification process for the 737 MAX 8 officially began in early 2012, when Boeing submitted its initial application for an ATC. In November 2013, FAA accepted Boeing’s 737 MAX 8 Master Certification Plan, which established the means of compliance Boeing planned to use to get the aircraft design certified. While Boeing’s flight control software, MCAS, was included in an early technical briefing presented to FAA, MCAS was not an area of emphasis because Boeing presented it to FAA as a modification to the existing speed trim system, with limited range and use. According to FAA, the Agency focused its involvement on potentially higher risk areas such as the aircraft’s larger engines, fly-by-wire spoilers, and landing gear changes.

January 27, 2012 Boeing files an Amended Type Certificate application with FAA for the 737 MAX 8. In submitting its application for certification, Boeing used the 737-800 aircraft design as the basis for the 737 MAX 8 model. Under FAA’s aircraft certification processes,22 FAA can prescribe a special condition for new or novel technology when no applicable standards exist. Systems identified as new or novel receive extra scrutiny from FAA. According to Boeing, the company did not need to identify MCAS as new or novel and MCAS did not require a special condition. The company stated this was because the design feature had been covered under existing regulations23 relating to flight control systems, in addition to being included on the military Boeing 767 refueling tanker. However, the version of MCAS installed on the 767 tanker differed from the version of MCAS installed on the 737 MAX. For example, while both aircraft have two AOA sensors, the tanker version uses a median input value of both sensors, while the 737 MAX version of MCAS relied on the input from one sensor.24 Moreover, the Joint Authorities

22 14 CFR § 21.16 states that if applicable regulations do not contain adequate or appropriate safety standards for an aircraft due to a novel or unusual design feature, FAA can prescribe a special condition to the aircraft to ensure an equivalent level of safety to requirements in the regulations. 23 Existing regulations including 14 CFR §§ 25.671, 25.672, 25.1309, and 25.1329. 24 The MCAS software resides in both of the aircraft’s flight control computers, each of which receives data input from its respective AOA sensor. The 737 MAX is designed to rely on only one of its two flight control computers per flight, alternating from one to the other after each flight. As a result, MCAS receives data from just one AOA sensor.

AV2020037 13

Technical Review (JATR)25 team stated in its 2019 report that MCAS on the 737 MAX controlled the aircraft’s movements in a new way.

According to internal Boeing meeting minutes from 2013,26 the company made the decision to portray MCAS as a modification to an existing flight control system in part because if MCAS “was emphasized as a new function, there may be a greater certification and training impact.” An ODA representative working on FAA’s behalf also agreed with portraying MCAS as a modification and not a new function. According to an FAA Flight Standards representative and an internal Boeing email, an early Boeing program goal was to keep a common type rating for the aircraft—which would minimize additional training requirements for 737 MAX pilots previously certified on the NG series—and to avoid the need for 737 MAX pilots to train in simulators, which can add costs for airlines that purchase the aircraft. References to MCAS were later removed from flight crew training requirements; therefore, any simulator training, while not proposed, probably would not have included MCAS.

March 21, 2012 FAA and Boeing hold a General Familiarization Meeting. During General Familiarization Meetings, the certificate applicant introduces FAA to the changes or new systems and features of an aircraft design and reviews the general architecture. Under an ATC, only the significant differences from the baseline model must be certified to the regulations applicable on the date of the application. (See figure 6 for a diagram of what FAA identified as the significant changes to the aircraft. MCAS was included in “system revisions.”) One FAA flight control engineer we interviewed recalled that during the 737 MAX General Familiarization meeting he participated in, MCAS information was not an area of focus, but it was presented briefly with limited details. Technical familiarization documents we reviewed supported this evidence.

25 The JATR is a team consisting of representatives of regulators from 10 countries (including the United States) that was chartered by FAA on June 1, 2019, to examine the Agency’s certification of the 737 MAX 8. The JATR issued a report on October 11, 2019. 26 This particular meeting was held June 7, 2013, but represents an example of how Boeing presented the system to FAA and other regulators in order to meet program goals.

AV2020037 14

Figure 6. Significant Changes From the 737 NG to the 737 MAX Aircraft

Source: OIG analysis of FAA and Boeing data

March 22, 2012 FAA accepts Boeing’s Amended Type Certificate application. One day after FAA and Boeing had their General Familiarization Meeting, FAA acknowledged Boeing’s proposal for the 737 MAX 8 and allowed the applicant to proceed. MCAS was included in the accepted ATC application as a modification to the previous aircraft model’s flight control system software.

May 1–2, 2012 FAA and Boeing hold a Technical Familiarization Meeting. Similar to General Familiarization Meetings, Technical Familiarization Meetings are opportunities for FAA to learn about technical designs and changes from the applicant. During the meetings, FAA typically looks at the design changes between the new derivative aircraft and the baseline aircraft and determines what issues must be addressed. In addition, FAA began evaluating whether to delegate or retain authority for assessing whether specific areas, features, or systems comply with Federal regulations.

In Boeing’s Technical Familiarization Meeting presentations, MCAS was included as a provisional modification to address the plane’s tendency to pitch upwards at high speeds. However, according to FAA representatives present at the meeting, it was not an area of emphasis. Based on our review of the Technical

AV2020037 15

Familiarization Meeting briefing slides, we determined that 23 of 482 slides covered primary aircraft flight controls. However, there were only 2 lines of text within those almost 500 slides—covered over a 2-day period— that referenced MCAS. According to FAA representatives, flight control engineers on the project focused on other issues identified as potentially higher risk, such as fly-by-wire spoilers.27 This demonstrates that FAA relied heavily on the information that Boeing provided in the early stages of the aircraft’s certification, which in turn drove the Agency’s decisions on areas of involvement.

July 21, 2012 FAA begins the initial certification basis evaluation. This evaluation is FAA’s initial review of Boeing’s proposed certification basis—i.e., how the company proposed to satisfy the certification requirements for its 737 MAX 8. Within the certification basis document, FAA requires that applicants provide an adequate overview of the project scope to support early certification activities.

February 15–November 14, 2013 FAA reviews and accepts the Master Certification Plan. The Master Certification Plan is a key step in the certification process, as this document describes how FAA and Boeing planned to certify the 737 MAX 8, including the method for testing key items. This document establishes which project areas and documents Boeing proposes its ODA will be responsible for reviewing and assessing for compliance (i.e., which items FAA will delegate to the ODA) and which areas and documents FAA will retain or remain involved in. According to our review, FAA and Boeing communicated questions, changes, and clarifications with each other as Boeing worked to develop the Master Certification Plan. In November 2013, the Agency accepted the Master Certification Plan. FAA initially retained the sections of the Master Certification Plan related to flight controls and the stabilizer, including MCAS. At this time, according to Boeing data, FAA had delegated 28 out of 87 (32 percent) detailed certification plans for the aircraft.

Notably, the number of certification plans that FAA delegated and retained changed throughout the certification process, which, according to FAA, is typical. For example, according to Boeing data, as of November 2016 FAA had delegated 79 of 91 (87 percent) detailed certification plans back to Boeing’s ODA, including the flight controls and stabilizer plans containing MCAS (see figure 7).28

27 In a fly-by-wire system, a computer collects sensor data from the pilot’s controls and uses those signals to move the corresponding aircraft control surfaces; this replaces an older system relying on physical cables connecting pilot controls to control surfaces. 28 According to Boeing data, between November 2016 and March 2017, FAA eventually delegated all 91 certification plans to Boeing’s ODA.

AV2020037 16

Furthermore, under FAA’s ODA program, FAA can delegate specific deliverables within each certification plan, such as system safety assessments, even if FAA retains the plan itself. These can also change over the course of the project, as was the case for the over 1,700 Boeing 737 MAX deliverables.

Figure 7. Delegation and Retention of Certification Plans

2832%

5968%

Certification Plans at Master Certification (November 2013)

Certification Plans Delegated

Certification Plans Retained

7987%

1213%

Certification Plans as of November 2016

Certification Plans Delegated

Certification Plans Retained

Source: OIG analysis of Boeing data

2014: FAA Establishes Initial Certification Basis February 6, 2014 FAA establishes the initial certification basis for the 737 MAX 8. Over approximately 2 years between 2012 and 2014, Boeing and FAA collaborated to establish the initial certification basis for the 737 MAX 8, having agreed in March 2012 that an ATC would be appropriate for the aircraft. During this same time period, FAA and Boeing collaborated through Issue Papers, which provide a continuous way of communicating and working through differences about the means of compliance with relevant standards and regulations.29 FAA established the certification basis for the model 737 MAX 8 within the G-1 Issue Paper, a document that FAA and Boeing used to collaborate about design requirements and conditions. The initial certification basis specifies the applicable regulations and special conditions that must be complied with for the project. Between this date and March 2, 2017, FAA and Boeing continued formally

29 In Order 8110.112A, FAA defines Issue Papers as the method to document the negotiation and resolution of certification issues with the applicant.

AV2020037 17

discussing the certification process through the G-1 Issue Paper, refining the certification basis accordingly.

2015: Certification Continues, Including Revisions to MCAS

Throughout 2015, FAA and Boeing’s ODA unit members30 continued certification activities for the 737 MAX 8. Boeing also began revising MCAS based on the results of aircraft analyses and testing. However, MCAS was still not a major focus of FAA’s certification efforts, which continued to emphasize areas such as the aircraft’s new larger engines, fly-by-wire spoilers, and changes to the landing gear.

September 17–18, 2015 Joint Operational Evaluation Board reviews Boeing’s Flight Crew Operations Manual (FCOM), including MCAS. The FCOM is an aircraft-specific manual that manufacturers provide to operators, which contains necessary operating limitations and other procedures the flight crew needs to safely operate that aircraft. As part of the certification process, a panel of representatives from FAA, Transport Canada, and the European Aviation Safety Agency, known as the Joint Operational Evaluation Board, reviewed the draft FCOM on September 17–18, 2015. The draft FCOM at the time included a brief description of MCAS indicating that it would only activate at high speed and high load factors.31 It did not include any references to repeated activations. According to Boeing, MCAS was removed from the FCOM in March 2016. Subsequent versions that we reviewed did not include a description of MCAS but retained MCAS in the abbreviations section.

2016: Boeing and FAA Conduct Flight Tests; Boeing Completes Failure Analysis and Continues To Revise MCAS

Flight testing of the 737 MAX 8 began in 2016. Boeing also completed failure analyses during the same time period. Flight testing is a critical component of the certification process. Typically, flight testing involves a series of tests in order to verify engineering assumptions, assess design decisions, and check for

30 ODA unit members are organization employees authorized, as part of the ODA and on FAA’s behalf, to perform functions necessary for FAA approval of that project as outlined in FAA Order 8100.15B. 31 Load factor is the ratio of aerodynamic forces divided by the weight of the aircraft.

AV2020037 18

compliance with requirements related to flight handling, stall identification, and control system malfunctions.

January 19, 2016 Boeing completes the first Single and Multiple Failure document on the 737 MAX 8. Boeing uses the Single and Multiple Failure document to “prevent simultaneous failure from a single threat event which causes loss of continued safe flight and landing.” Boeing considered this failure probability analysis an internal document only and did not submit it as a required certification deliverable. Therefore, Boeing did not provide it to FAA, nor did FAA have to review or approve it as part of the certification process. However, according to FAA, some aspects of Boeing’s analysis from the Single and Multiple Failure document should be included in system safety assessments later provided to the Agency as certification deliverables.

Boeing’s analysis identified 75 failure cases to assess the potential impacts of those failure scenarios on the aircraft and flight crew. Boeing deems failures to be acceptable under certain circumstances, including if:

• the equipment and systems perform as intended during operating conditions;

• catastrophic failure conditions are not caused by a single failure;

• catastrophic failures are determined to be extremely improbable;

• information concerning unsafe system operating conditions is provided to the crew.

Boeing’s Single and Multiple Failure analysis found all 75 potential failure cases to be acceptable.

One potential failure case involved the loss of one AOA sensor—an external sensor that measures the angle of the aircraft in the air32—followed by faulty AOA data in the other sensor. (See figure 8, which shows the location of AOA on the 737 MAX aircraft.)

32 AOA sensors are attached to the outside of the aircraft.

AV2020037 19

Figure 8. 737 MAX Aircraft External Probes, Including the AOA Sensor

The AOA sensor is the lower one of the two probes.

Source: Boeing

However, despite identifying this failure case and deeming it catastrophic, Boeing determined this failure case was acceptable because the probability of occurrence was determined to be extremely remote, and it was assumed the crew would recognize the situation and take appropriate action. While this failure test case may not be exactly the same as the circumstances encountered in the Lion and Ethiopian Air accidents, erroneous AOA data—potentially caused by the failure of one AOA sensor—was a factor present in both accident scenarios.

During its Single and Multiple Failure analysis, Boeing rated this potential failure case as “catastrophic”33 but also determined that the low probability of occurrence meant it qualified as acceptable. Boeing also rated 11 other potential failure cases as catastrophic but ultimately deemed them acceptable based on probability and engineering judgement.

33 FAA Advisory Circular 25.1309.1A classifies risk ratings as: Minor (failure conditions which would not significantly reduce airplane safety and which involve crew actions that are well within their capabilities), Major (failure conditions which would reduce the capability of the airplane or the ability of the crew to cope with adverse conditions), Hazardous (failure conditions which would reduce the capability of the airplane or the ability of the crew to cope due to physical distress or excessive workload), and Catastrophic (failure conditions which would prevent continued safe flight and landing).

AV2020037 20

January 29, 2016 Boeing conducts first flight test of the Boeing 737 MAX 8. Boeing’s 737 MAX 8 flight testing began in January 2016. However, per FAA’s standard certification process, FAA does not actively participate in flight testing until the Agency issues the Type Inspection Authorization (TIA).

March 14, 2016 FAA issues initial Type Inspection Authorization. The initial TIA authorized Boeing to begin certification test flights for the 737 MAX 8 using FAA flight test pilots, and defined which certification test flights FAA needed to perform. The flight test protocol included 129 certification flight test34 plans to be completed as part of the testing. Under this TIA, FAA delegated 62 of the 129 (48 percent) flight test plans to Boeing’s ODA.

The TIA also defined the tests and analysis that would be performed by the Seattle Aircraft Evaluation Group (AEG)—a group of aviation safety inspectors in FAA’s Flight Standards Service tasked with determining the appropriate type rating and levels of training for aircraft that are undergoing evaluation for an ATC. For the 737 MAX 8, as required, the AEG assessed and determined it needed to participate in its own testing. The testing that the AEG performed was handling and differences testing, which determines the type rating and the differences between the baseline aircraft (737-800) and related aircraft (737 MAX 8) on which pilots must be trained, evaluated, and remain current.

March 30, 2016 Boeing completes MCAS Revision D, a significant change. Boeing continued to revise and refine MCAS during the flight testing process. Revision D was a significant change that updated MCAS software technical and data requirements based on pilot assessments and flight test results. In this revision, Boeing changed the parameters under which MCAS would activate to include much slower airspeeds.35 It also increased the maximum range of MCAS from 0.55 degrees to 2.5 degrees, an increase of over 300 percent. This meant that each time MCAS activated, it could push the nose of the aircraft downward with a maximum range of 2.5 degrees of movement.36

In its MCAS Revision D, Boeing also included an assessment of functional hazards related to the software, describing hazard descriptions, failure conditions, and

34 Guidance for flight testing of the 737 MAX 8 is contained in FAA Advisory Circular 25-7C. 35 Following this revision, MCAS could now activate at speeds of 0.2 to 0.84 Mach, whereas it could previously only activate at speeds above 0.67 Mach. Mach is calculated by dividing the speed of the aircraft by the speed of sound. 36 Specifically, MCAS uses the aircraft’s horizontal stabilizer (near the tail of the aircraft) to control the angle of the aircraft, and the range of MCAS is the angle of movement of the stabilizer.

AV2020037 21

associated effects. One of the noted hazards was an uncommanded or automatic MCAS activation that continued until the pilot took action. When developing this risk assessment, Boeing tested unintended MCAS activation in the simulator and assumed that commercial pilots would recognize the effect as a runaway stabilizer37—a scenario which is covered in basic commercial pilot training—and react accordingly. Boeing assumed the average pilot reaction time in this scenario to be 4 seconds, which Boeing classified as a hazardous38 event. However, if a pilot’s reaction time was greater than 10 seconds, the event would be classified as catastrophic due to the pilot’s inability to regain control of the aircraft. Despite these significant revisions, Boeing did not provide internal coordination documents for Revision D,39 noting the increased MCAS range, to FAA certification engineers. Because these revision documents were not required certification deliverables, the company did not submit them to FAA for review or acceptance.

March 30, 2016 Boeing removes MCAS from the Flight Crew Operations Manual and training differences tables. Boeing requested (and received) permission from the FAA AEG to remove any references to MCAS from its FCOM40 and Other Differences Requirements tables, which help aircraft operators manage their training on differences between related aircraft. Boeing justified its request by stating that MCAS would be “transparent”41 to the flight crew. This decision was not documented via official correspondence; rather, Boeing requested it via email and FAA approved it verbally in a subsequent meeting, according to the responsible FAA AEG representative.

However, the FAA AEG representative who concurred with the request told us that FAA based the decision on the understanding that MCAS was still as originally designed—i.e., that it would only activate in situations that included high speeds and high load factors, and was limited to a range of 0.55 degrees. An earlier version of the FCOM that this representative would have reviewed contained only three sentences specifically referencing MCAS, and it stated that the system would only activate when the aircraft was at speeds of “0.7 Mach or

37 A technical fault resulting in continuous unintended movement of the horizontal stabilizer. 38 Boeing added a statistical credit in its evaluation of this scenario that reduced the effect from Hazardous to Major, based on the assumption that it was unlikely that a typical flight would be operating outside of normal aircraft parameters. 39 Revision D is where the major changes to MCAS first occurred; subsequent MCAS Revision E (dated July 5, 2016) configurations were the versions actually installed on the aircraft as of the date of ATC issuance. 40 FAA does not formally approve the FCOM. However, Agency inspectors do review and accept the document before it is issued to individual operators. 41 In this context, Boeing and FAA use the word “transparent” to mean the system or function would be invisible to the flight crew—they would not be aware or have any indication that the system was present or in operation.

AV2020037 22

greater.” However, on this same date, Boeing revised MCAS to activate at much slower speeds.

While AEG and FAA certification engineers were unaware of the revisions to MCAS, FAA flight test personnel were aware of the increased maximum range of MCAS in the flight control computer actually installed on the 737 MAX 8 test aircraft. This varied understanding of the final flight control design of the 737 MAX among different FAA offices demonstrates a lack of consistent and transparent communication both between Boeing and FAA, as well as within FAA.

April 13, 2016 FAA conducts first flight test of the 737 MAX 8. Following the issuance of the TIA, FAA conducted flight tests using the Agency’s test pilots from the Aircraft Certification Service. The flight crews for tests typically included one FAA test pilot, one FAA test flight engineer, Boeing engineers, and one Boeing pilot.

May 6, 2016 Flight Control System (including MCAS) flight testing begins. FAA began a series of seven flight tests to test stall speed performance for the 737 MAX 8. While MCAS was not tested as a stand-alone item, FAA stated that it included MCAS in its tests of the flight control system and stall functions. For example, according to FAA, the Agency tested several high-speed maneuvers—such as upset recovery42 and wind-up turns43—in which MCAS would have activated as intended, but was not the focus of the test.

August 15, 2016 Boeing releases Flight Control Computer software revision. Boeing released the version of the flight control computer software that it intended to use as the final version on the MAX 8 on this date. This version is known as the “Black Label Equivalent,” meaning the software has undergone additional testing but requires a final flight test conducted by FAA. This software revision included the version of MCAS that had the ability to push down the nose of the aircraft with a maximum movement of 2.5 degrees.

42 Upset recovery is the ability to correct the aircraft after unintentionally exceeding normal flight parameters. 43 A wind-up turn is a constant altitude, constant speed turn with increasing normal acceleration or angle of attack.

AV2020037 23

August 16, 2016 Boeing completes requirements for aircraft Level B training. Boeing successfully completed the testing requirements for Level B training on August 16, 2016. FAA’s Boeing 737 Flight Standardization Board (FSB)44 approved Level B training45 for the 737 MAX 8 for pilots who were qualified to fly the 737-800.46 This action meant that pilots would be eligible to fly the 737 MAX 8 following completion of classroom or computer-based training on FAA-mandated topics. Level B training also meant that no simulator training was required (see table 1 for pilot training differences levels). FAA granted this approval after using domestic, commercial pilots—in addition to FAA operational test pilots—during its training-related flight testing, a practice that is allowed but not frequently done.

The approved training did not include material on MCAS. Internal Boeing emails show company officials congratulating staff for the accomplishment of receiving FAA approval for Level B training, providing further evidence that this reduced level of training was a Boeing program goal for the 737 MAX.

44 FAA typically establishes an FSB when certificating large jet or propeller aircraft. It consists of members of the AEG, FAA operations inspectors for the initial operator of the aircraft, representatives from the Office of Safety Standards, and other technical advisors if necessary. One of the FSB’s mandates is to develop training objectives for normal and emergency procedures and maneuvers. 45 FAA uses AC 120-53B to determine the level of training necessary for pilots that hold a type rating in a particular aircraft to be able to obtain a type rating in a related derivative aircraft. 46 FAA issued a provisional training validation letter on February 22, 2017, and formally approved the training on March 7, 2017.

AV2020037 24

Table 1. Pilot Training Difference Levels

Level Training Training Methods Checking

A Self-instruction Operating manual page revisions, operating bulletins, handouts, etc.

Not applicable (or next proficiency check)

B Aided instruction Slide/tape presentations, computer-based instruction, stand-up lecturers, or video tapes, etc.

(Note: This was the level of training approved for the Boeing 737 MAX.)

Task or system check

C Systems devices Training devices to supplement instruction, including interactive computer-based training, cockpit procedure trainers, or part task trainers.

Partial proficiency check using device

D Partial flight simulator

Flight training device that is accurate, capable of performing flight maneuvers in a dynamic real time environment, high fidelity integration of systems and controls, and realistic instrument indications.

Partial proficiency check using Flight Simulation Training Device

E Full flight simulator or aircraft

Requires full flight simulator or aircraft training. New type rating is normally assigned.

Proficiency check on full flight simulator, or aircraft

Source: OIG analysis of FAA policy

September 28, 2016 FAA delegates Flight Control Computer Certification Plan, which includes MCAS, to Boeing’s ODA. Initially, the Master Certification Plan stated that FAA engineers would be responsible for reviewing a key safety assessment of the flight control system, including MCAS. According to FAA representatives, however, Boeing did not present the formal version of this system safety assessment to FAA for the first time until January 2017—more than 4 years into the 5-year certification process. According to FAA management, it is typical for manufacturers to complete and submit the safety assessments towards the end of the certification process. Prior to this, on September 28, 2016, FAA engineers decided to delegate back approval of future certification plan revisions to the Boeing ODA, but FAA continued to retain the safety assessment deliverable until the Agency reviewed it in February 2017.

November 10, 2016 FAA delegates Stabilizer Certification Plan to Boeing’s ODA. While initially retained, FAA delegated to Boeing’s ODA the Stabilizer Certification Plan, which details how the manufacturer will demonstrate compliance for the

AV2020037 25

aircraft’s horizontal stabilizer—a control surface near the tail of the airplane that controls up and down movement of the airplane. When MCAS activates, it adjusts the angle of the horizontal stabilizer on the plane, resulting in pushing down the nose of the aircraft. While Boeing’s Stabilizer Certification Plan document included some details regarding MCAS functions, it did not include an interrelated view of how MCAS interacted with other systems, which was spread throughout several documents. FAA delegated the system safety assessment for this certification plan back to Boeing the following month, in December 2016. The FAA engineer who reviewed the system safety assessment stated that he was not aware of the increased MCAS range from 0.55 degrees to 2.5 degrees on the aircraft’s horizontal stabilizer when he recommended approval of Boeing’s delegation request. In addition, Boeing did not update the hazard assessment table within the safety assessment to reflect the expanded MCAS use and range.

In this system safety assessment, Boeing identified potential failure scenarios related to the horizontal stabilizer and evaluated their risk. Notably, Boeing included a scenario in which there would be an “unintended MCAS activation.” However, Boeing assigned this failure scenario the risk rating of “Major” under normal flight operations, which meant that there was no requirement to provide design redundancy (i.e., a requirement for MCAS to pull data from both external AOA sensors on the 737 MAX 8, rather than relying on a single AOA sensor as the system was designed). Such redundancy is required for the higher-risk rating of catastrophic.

Boeing recognized that the risk of unintended MCAS activation could be more severe under certain circumstances if the aircraft was operating outside of normal flight parameters. However, the company adjusted its evaluation of this risk based on statistical analysis showing it was unlikely that a typical flight would be operating in those circumstances, and therefore unlikely that MCAS would activate under these conditions. In its 2019 report following the two accidents, the JATR47 questioned Boeing’s assumption, stating that the statistical credit Boeing used was intended to be used in selecting test cases for flight handling qualities evaluation, not for showing compliance with systems safety regulations.

While Boeing tested a single, unintended activation of MCAS, it did not test repeated MCAS activations. Boeing engineers and test pilots, in discussions, deemed multiple activations of MCAS to be no worse than a single activation of MCAS. However, Boeing did not include this untested conclusion in certification deliverables provided to FAA. Further, Boeing’s safety assessments did not fully account for how pilots would react to a multi-failure scenario. Boeing noted in

47 Joint Authorities Technical Review (JATR), Boeing 737 MAX Flight Control System: Observations, Findings, and Recommendations. Submitted to the Associate Administrator for Aviation Safety, U.S. Federal Aviation Administration on October 11, 2019.

AV2020037 26

these assessments that it did not simulate an accumulation or combination of failures leading to unintended MCAS activation, nor their combined flight deck effects.

November 20, 2016 FAA and Boeing complete flight testing of Flight Control System (including MCAS). This was the final certification test flight for the control system that includes MCAS. In sum, Boeing and FAA completed 58 tests of the airplane’s flight control systems. FAA flight test pilots participated in 47 of the 58 flight control system tests, and FAA flight test engineers participated in 34 of these tests. According to FAA flight test personnel, some of the tests included MCAS operation with a range of 0.55 degrees, while other tests were at 2.5 degrees. This varied depending on which version of MCAS was installed on the aircraft used for the test flight, as Boeing was revising MCAS during flight testing.

2017: FAA Certifies the Boeing 737 MAX 8 In March 2017, FAA issued an ATC to Boeing for the 737 MAX 8 aircraft, which meant that the company could begin delivering the MAX 8 to customers. The aircraft began flying commercially on May 22, 2017.

March 8, 2017 FAA issues the Amended Type Certificate, certifying the Boeing 737 MAX 8 meets transport category airplane requirements. FAA completed the certification of the 737 MAX 8 and issued an ATC for the aircraft.48 Subsequently, FAA issued an ATC for the 737 MAX 9 on February 15, 2018. The other 737 MAX variants that have yet to be certified include the smaller 737 MAX 7, the larger 737 MAX 10, and the higher-seating capacity 737 MAX 8200.

May 16, 2017 Boeing delivers the first Boeing 737 MAX 8. Boeing delivered the first Boeing 737 MAX 8 to Malindo Air, a subsidiary of Lion Air Group. Malindo Air then flew the first Boeing 737 MAX 8 commercial flight on May 22, 2017, from Kuala Lumpur to Singapore. Until the grounding order in March 2019, Southwest, United, and American Airlines operated the 737 MAX 8 and 9 in the United States. Southwest received the first domestic delivery of a 737

48 Section 21.101e states if type certificate approval does not occur within 5 years, the applicant may select a new application date, which revises the date of the applicable 14 CFR Part 25 regulations. The applicant then must comply with any new or revised regulations as of the amended date. In December 2016, Boeing amended the application date to June 30, 2012; thus, the certification was in compliance with the 5-year requirement.

AV2020037 27

MAX 8 on August 26, 2017. As of December 2019, a combined total of 387 Boeing 737 MAX 8 and MAX 9 aircraft had been delivered worldwide and 72 specifically for U.S. commercial air carriers (see tables 2 and 3).

Table 2. Boeing 737 MAX Aircraft, in Fleet and on Order, as of December 2019

U.S. Commercial

Air Carriers U.S. Non-Commercial

Operators Foreign Carriers Total

In Fleet 72 46 269 387

On Order 528 463 3,554 4,545

Source: OIG analysis of Boeing documents

Table 3. Boeing 737 MAX Aircraft, in Fleet and on Order for U.S. Commercial Air Carriers, as of December 2019

Alaska American Southwest United Total

In Fleet 0 24 34 14 72

On Order 32 76 249 171 528

Source: OIG analysis of Boeing documents

August 10, 2017 Boeing identifies AOA disagree alert issue. Boeing design engineers identified an issue with a cockpit alert designed to notify pilots when the two AOA sensors disagree by more than 10 degrees for at least 10 seconds. Disagreeing AOA sensors can indicate that one or more sensors has failed or is providing unreliable information. However, Boeing discovered that not all 737 MAX aircraft were equipped with the alert. According to Boeing representatives, Boeing had intended this cockpit alert message to be standard on all 737 MAX 8 aircraft within the flight control computer system. However, Boeing stated that its software contractor inadvertently paired the alert with an optional feature—an AOA indicator—which only approximately 20 percent of MAX customers purchased49 (see figure 9). Neither of the accident aircraft had this AOA disagree cockpit alert.

49 Komite Nasional Keselamatan Transpotasi Republic of Indonesia. KNKT.18.10.35.04. FINAL. 2019.

AV2020037 28

Figure 9. AOA Disagree and AOA Indicator

Source: FAA

When first implemented on the 737 NG (and as carried over into the MAX), there were no established Boeing operational procedures for the AOA cockpit alert feature. Instead, it was a source of supplemental information, and the checklist for the AOA disagree did not require any pilot action as a result of the alert. Therefore, according to Boeing representatives, they analyzed the issue and determined that the cockpit alert was not “necessary for the safe operation of the airplane.” Boeing documented the problem in August 2017 and planned to have the problem corrected for the entire MAX fleet by late 2020.

Boeing did not directly notify FAA of the AOA alert issue50 since its analysis had determined that there was not an “operational impact,” nor did the company notify MAX operators at the time the issue was discovered. According to Boeing representatives, the company will generally inform its customers of issues such as this one. In this case, however, Boeing failed to do so upon discovering the omission, and, according to Boeing representatives, the company has been unable to determine why it did not notify operators about the AOA alert issue.

50 According to Boeing representatives, the AOA disagree cockpit alert message issue was included in updated certification documents in October 2017; however, Boeing did not submit a formal notification of the issue directly to FAA nor could the company confirm that FAA had reviewed the documents containing the first notification. According to Boeing, the Agency agreed in February 2019 with Boeing’s disposition of the problem and the determination that it was “not an unsafe condition.”

AV2020037 29

Boeing representatives stated that the company has since notified the operators of the AOA disagree alert issue.

Timeline of Events Between the Lion Air and Ethiopian Airlines Crashes

Following the fatal Lion Air crash on October 29, 2018—the first accident of a 737 MAX 8 aircraft—Boeing generated a bulletin to operators, prompting FAA to issue an Emergency Airworthiness Directive (AD). FAA issues an Emergency AD when an unsafe condition exists that requires immediate action by an owner/operator. However, neither the bulletin nor the Emergency AD specifically mentioned MCAS. Just over 4 months after the first accident, on March 10, 2019, Ethiopian Airlines Flight 302 crashed, resulting in 157 fatalities.

October 29, 2018 Lion Air Flight 610 crashes, resulting in 189 fatalities. Lion Air Flight 610 crashed into the Java Sea shortly after departing Soekarno-Hatt International Airport, Jakarta, resulting in 189 fatalities. According to the accident report, multiple cockpit alerts activated during takeoff, including a potential stall warning51 as well as airspeed and altitude disagreement messages.52 As the flight progressed, MCAS activated based on faulty data from the aircraft’s external AOA sensor over 20 times, which led to loss of control of the aircraft.

Notably, the accident aircraft experienced unintended MCAS activation the day before with a different outcome. Shortly after departure on October 28, 2018, Lion Air Flight 043, traveling from Denpasar to Jakarta, experienced a series of failures similar to that of Lion Air 610. During the incident, the flight crew successfully counteracted MCAS and used the stabilizer trim53 cutout switches54 to effectively “turn off” MCAS (see figure 10). The flight crew continued to Jakarta, but with continuous cockpit alerts for the duration of the flight.

51 Specifically, the stick shaker warning was activated, an alert that warns the flight crew when the aircraft is close to a wing stall condition. 52 These messages indicate that the airspeed and altitude data being reported by the Captain’s instruments and the First Officer’s instruments do not match. 53 Trim systems help minimize a pilot’s workload by aerodynamically assisting in the movement and position of the flight control surfaces. These systems can be manipulated manually but typically feature an electrically powered system to assist pilots. 54 The stabilizer trim cutout switches remove power from the stabilizer trim motor when positioned to cutout.

AV2020037 30

Figure 10. 737 MAX Stabilizer Trim Cutout Switches

The stabilizer trim cutout switches are marked in the red circle above. Source: Boeing

The Indonesian accident investigators found that the left AOA sensor on the 737 MAX 8 was replaced prior to Lion Air 043, but this replacement sensor subsequently reported faulty data. Investigators were unable to determine if required testing of the replacement sensor had been performed properly.55 MCAS was designed to rely on a single AOA sensor, making it vulnerable to failure from a sole source of erroneous input. Further, upon landing, the flight crew did not fully report all of the issues experienced, making a complete evaluation by maintenance technicians difficult.

55 Two types of testing on the replacement sensor were required: (1) functional testing of the replacement sensor by the air carrier upon installation and (2) maintenance testing performed by the repair station prior to providing the part to Lion Air. FAA has since revoked the vendor’s repair station certificate.

AV2020037 31

November 4–7, 2018 FAA conducts initial risk analysis following the Lion Air crash. FAA conducted an initial review of events through its Continued Operational Safety Program.56 The review included a quantitative analysis of the service history of the Boeing 737 MAX 8. Based on this review, FAA determined that while urgent57 action was necessary, the ongoing risk did not meet the threshold for aircraft grounding. This analysis was the basis for issuing an Emergency AD.

November 6, 2018 Boeing issues bulletin to operators regarding the 737 MAX 8 and 737 MAX 9. The bulletin informed 737 MAX 8 and MAX 9 operators that erroneous AOA data could result in uncommanded nose-down movement of the aircraft and that this action can repeat until the related system is deactivated. The bulletin emphasized pilot procedures to perform, including returning the aircraft to a neutral trim position, following the runaway stabilizer checklist, using the appropriate switches (e.g., figure 10 above) to remove power from the related system, and using manual trim once they turned the stabilizer off. The bulletin further reminded pilots that they can experience additional indications and effects, including but not limited to altitude and airspeed disagreement alerts.

November 7, 2018 FAA issues Emergency Airworthiness Directive. One day after Boeing’s bulletin, FAA issued an AD58 to all air carriers operating the 737 MAX 8 and MAX 9. The AD identified that if the aircraft’s AOA sensor fails or sends erroneous data to the flight control system, there is potential for repeated nose-down movement that could lead to difficulty controlling the airplane. The AD required all owners and operators of the 737 MAX 8 and MAX 9 aircraft to—within 3 days of the receipt of the AD—revise the Airplane Flight Manual to provide flight crews with procedures to follow under certain conditions that would counteract the aircraft’s nose-down movements. While neither the bulletin nor the AD specifically named MCAS, Boeing issued a message to operators of the aircraft on November 10, 2018, with a brief MCAS description.

56 FAA’s Continued Operational Safety Program is a data-driven program intended to manage risks associated with specific aircraft fleets. It uses qualitative and quantitative analysis to determine the appropriate course of action following potential safety events. 57 FAA uses a risk model with recommended levels of response including the probability of individual injury per flight hour, such as pursuing immediate actions (1 fatal injury in 1 million flight hours) and grounding the aircraft (1 fatal injury per 100,000 flight hours). 58 FAA issues ADs to aircraft owners and operators and transmits ADs to foreign aviation authorities. Emergency AD 2018-23-51 was issued on November 7, 2018.

AV2020037 32

November 28, 2018 FAA completes a risk analysis of the Lion Air Flight 610 accident. FAA completed its initial Continued Operational Safety risk analysis and subsequent review by the Corrective Action Review Board.59 The risk analysis results supported the original decision to issue the Emergency AD and stated that additional action was required to further reduce risk. Specifically, the Board found that the uncorrected individual risk60 was 2.68 fatalities per 1 million flight hours. This exceeds FAA’s Transport Airplane Risk Analysis Methodology (TARAM)61 risk guidelines of 1 fatality per 10 million flight hours.

Based on the risk analysis, Boeing proposed and FAA accepted a redesign of MCAS software that would include additional safeguards against unintended MCAS activation. FAA completed an additional analysis on December 12, 2018, for the risk post-implementation of the AD. This analysis determined a risk of about 15 accidents occurring over the life of the entire 737 MAX fleet if the software fix was not implemented.62 FAA’s risk analysis also indicated that the AD mitigated the risk sufficiently enough to allow continued aircraft operation for a limited period of time, until July 2019, while the software fix was being developed and implemented on the existing fleet. As a result of the Lion Air accident, Boeing agreed to begin developing software design changes to MCAS. The initial proposal for the software fix would revise MCAS to compare data from both AOA sensors and limit its ability to activate multiple times.

January 9, 2019 FAA begins review of MCAS certification process. In January 2019, FAA initiated an internal review of the original MCAS certification process. This was the first time FAA performed its own detailed analysis of MCAS, and according to several FAA certification engineers, it was also the first time that they were presented with a full picture of how MCAS worked. This review resulted in documentation that was never finalized.63 In the draft, FAA did not find any non-compliances, but the Agency noted Boeing’s document traceability and

59 The Corrective Action Review Board is a panel of FAA experts who formally recommend the action to be taken, which could include the issuance of an AD or grounding a specific model or fleet of aircraft. 60 Uncorrected individual risk is the probability of individual fatal injury per flight hour if no action is taken to address an identified condition. 61 FAA’s TARAM handbook outlines a process for determining the numerical risk associated with the continued operation of passenger carrying aircraft, and guidance for identifying unsafe conditions and corresponding regulatory actions. 62 This figure of 15 accidents assumes an estimated fleet size of 4,800 aircraft in operation; there were approximately 250 aircraft in operation at the time of the Lion Air accident. 63 This documentation included both required supervision records and a draft report. According to FAA management, the report was going through management review and comment at the time of the Ethiopian accident, at which time the Agency considered it overtaken by events.

AV2020037 33

clarity of explanations were lacking in its revisions to MCAS and other system certification documents. FAA’s post-accident review determined that an independent reviewer would not have been able to effectively review the safety assessment as a standalone compliance document or understand the full system functionality and linkage with other systems and functions.

February 13, 2019 FAA and Boeing formally agree to a schedule for implementation of the MCAS software fix. Based on the analysis performed in December 2018, FAA determined that Boeing and operators had until July 2019 to develop and implement the MCAS software update in order to remain within the allowable risk guidelines contained in the TARAM Handbook.64 FAA and Boeing agreed to an implementation plan to meet that date. Under the agreement, Boeing would develop the software update by April 12, 2019; FAA would issue an AD requiring implementation of the new software by April 19, 2019; and operators would have until June 18, 2019, to install the software. According to FAA, the Agency calculated these milestones based on the best information and data available at the time about the aircraft and the Lion Air accident.

March 10, 2019 Ethiopian Airlines Flight 302 crashes, resulting in 157 fatalities. Just over 4 months after the Lion Air Flight 610 crashed into the Java Sea, Ethiopian Airlines Flight 302 crashed shortly after departing Addis Ababa Bole International Airport, resulting in 157 fatalities, including 8 Americans. The interim accident report65 found that shortly after takeoff, the left AOA sensor failed. MCAS again activated multiple times based on erroneous AOA data, resulting in uncommanded nose-down movement of the aircraft. According to the report, the flight crew used the stabilizer trim cutout switches. However, according to the interim report, flight data from the investigation indicate the crew subsequently returned power to the stabilizer trim system. This was not in compliance with FAA’s Emergency AD, which states that the stabilizer trim cutout switch remain set to the cutout position for the reminder of the flight. Ultimately, the crew lost control of the aircraft. The interim investigation results show that the pilots did not reduce power from takeoff thrust during the duration of the flight, resulting in excessive speed. This can lead to aerodynamic forces that exceed the ability of the flight crew to counteract these forces through manual adjustments.

64 TARAM Handbook and FAA Policy Statement. PS-ANM-25-05. November 4, 2011. 65 On March 9, 2020, the Federal Democratic Republic of Ethiopia, Ministry of Transport, Aircraft Accident Investigation Bureau, released Interim Investigation Report of accident 737-8 MAX ET-AJV, ET-302.

AV2020037 34

Timeline of Concurrent FAA Organization Designation Authorization Oversight Actions and Events

During the same timeframe of the 737 MAX 8 certification and the Lion Air accident, Boeing, FAA, and our office were identifying some significant problems—although not specific to the 737 MAX 8—with the Boeing ODA, as well as FAA’s ODA oversight. These included undue pressure on ODA unit members, quality and timeliness of certification documentation, and the effectiveness of FAA oversight.

2013–2019 Boeing ODA self-audits and surveys identify employee concerns about undue pressure. In two ODA self-audits—one in the Seattle, WA, area (2013) and another in Charleston, SC (2014), Boeing identified employee concerns related to undue pressure within the ODA. The term “undue pressure” describes situations in which an ODA unit member or other designee—i.e., a company employee working on behalf of FAA—faces conflicting non-ODA duties or interference from other company or organizational elements regarding how to effectively administer pertinent regulations.

Subsequently, in 2016, Boeing conducted an undue pressure survey of its ODA unit members, obtaining 523 responses as of November 2016. While 97 percent of the respondents agreed that they understood the process for reporting undue pressure, almost 40 percent had encountered situations where they perceived potential undue pressure, and almost a quarter of respondents had experienced undue pressure beyond their direct reporting structure while performing their ODA function. Further, respondent comments included common themes such as pressure from high workloads, confusion and potential undue pressure due to the dual roles of a unit member,66 and a desire for the company to share information about other undue pressure cases to help other unit members learn and understand from those cases.

According to FAA, Agency discussion with Boeing and analysis of the survey results indicated there may have been a distinction between “potential” undue pressure and undue pressure that would rise to the level of formal reporting,

66 The Boeing ODA has nearly 1,500 personnel; however, ODA administrators and unit members perform those duties only part-time. The same engineer can work for the company on a particular design and then approve that same design as an ODA unit member. Boeing ODA managers who administer the program also have concurrent roles within Boeing.

AV2020037 35

which was not differentiated in the survey. Interviews of ODA unit members found that all formally reported instances of undue pressure were satisfactorily addressed. However, according to the Agency, FAA observations indicated a need for further oversight of the undue pressure systems and processes.

Over the course of 2018 and 2019, the Boeing ODA completed seven internal audits that covered the undue pressure reporting process for ODA unit members at selected Boeing facilities—not specific to the MAX. None of the audits found non-conformities (e.g., violations of FAA regulations) related to undue pressure, and the reports remarked that the process for reporting concerns about undue pressure was well communicated and supported. However, the ODA’s self-audits reported that while unit members in interviews demonstrated awareness and knowledge of the undue pressure reporting process, one audit noted a perception of “inadequate protection from actions by leadership outside of ODA.” Although not a formal audit finding, another internal audit noted a “general lack of confidence that the [undue pressure reporting] process would reach a satisfactory conclusion and/or protect the Unit Members.”

October 15, 2015 DOT OIG issues report citing issues with staffing and processes for ODA oversight. While delegation is an essential part of meeting FAA’s certification goals, our office has reported since 201167 that the Agency faces challenges in overseeing ODA companies, including Boeing. For example, in October 2015, during the timeframe of the Boeing 737 MAX 8 certification, we reported68 that FAA’s oversight of ODA program controls was not systems- and risk-based,69 as recommended by an aviation rulemaking committee.70 Instead, FAA’s oversight was more focused on individual engineering products and areas that we determined were low risk. We also found that FAA lacked a comprehensive process for determining staffing levels needed to provide ODA oversight and that FAA did not conduct sufficient oversight of ODA personnel who performed certification work at companies that supply components to manufacturers. We made nine recommendations aimed at improving FAA’s staffing and oversight of the ODA program.

67 FAA Needs To Strengthen Its Risk Assessment and Oversight Approach for Organization Designation Authorization and Risk-Based Resource Targeting Programs (OIG Report No. AV2011136), June 29, 2011. 68 FAA Lacks an Effective Staffing Model and Risk-Based Oversight Process for Organization Designation Authorization (OIG Report No. AV2016001), October 15, 2015. 69 Systems-based oversight shifts from focusing on individual project engineering work to holistically assessing whether ODA companies have the people, processes, procedures, and facilities in place to produce safe products, thus allowing FAA to focus its oversight on the highest-risk areas, such as new, innovative aircraft designs. 70 Aircraft Certification Process Review and Reform Aviation Rulemaking Committee, a joint FAA and industry group, formed in response to a congressional mandate to study the aircraft certification process.

AV2020037 36

FAA has since addressed most recommendations from our report, but the Agency has not yet implemented a risk-based approach to ODA oversight as we recommended. This approach would allow FAA to assess the greatest risks and target its oversight accordingly. FAA’s current plan is to implement its new system for ODA oversight by December 2020.

During our current review, we found that engineers in FAA’s Boeing oversight office still face challenges in balancing certification and oversight responsibilities. While we have not found any evidence of an inappropriately close relationship between FAA and Boeing to date, some FAA personnel expressed concern that FAA executives are too deferential to Boeing.

2015–2019 FAA and Boeing sign a Settlement Agreement, and Boeing pays a $12 million fine related to ongoing ODA manufacturing and certification issues. During the same time period as the MAX 8 certification, FAA was performing ODA oversight activities and enforcement actions related to manufacturing and certification issues at Boeing, such as documentation quality, timeliness, and corrective actions. In culmination of these efforts, FAA and Boeing signed a Settlement Agreement on December 18, 2015, wherein Boeing agreed to take actions in specified regulatory compliance areas and acknowledged obligations to meet performance metrics. These actions would resolve allegations documented in 13 FAA Enforcement Investigative Reports (EIR) spanning from 2009 through 2015.

All 13 EIRs cited violations of Boeing’s approved production, delegation, and certification systems. Each EIR also cited violations of more specific regulations, such as those governing completed aircraft conformity, fuel tank flammability, compliance plans, and compliance with ODA procedures. Although not specifically tied to the 737 MAX 8 aircraft, these allegations pertained to issues regarding the quality of ODA certification documents and Boeing’s processes to identify the root causes of its non-compliances71 and establish corrective actions to resolve them.

Upon signing the agreement, Boeing paid a $12 million civil penalty to FAA and could face civil penalties up to $24 million if it fails to meet the settlement agreement commitments by December 31, 2020.

71 Root causes are the contributory or initiating underlying causal factors of a nonconformity or undesirable event. A causal factor is considered the root cause if its removal from the event sequence prevents the undesirable event from recurring.

AV2020037 37

Between February 2017 and March 2019, FAA initiated 17 ODA oversight activities72 (called supervision records) related to the compliance findings in the 2015 Settlement Agreement. FAA deemed six (35 percent) of the activities to be unsatisfactory, related to incomplete information and/or insufficient justification provided in certification project documentation.

According to FAA and Boeing officials, the company is still working on improving its documentation and processes, particularly in the area of identifying and resolving root causes to prevent non-conformances and non-compliances from recurring.

November 21, 2018 FAA initiates a formal compliance action against Boeing related to ODA oversight. During the same time period of the Lion Air accident, FAA was in the process of completing its oversight related to potential undue pressure on engineering unit members, not directly related to the 737 MAX. In November 2018, FAA initiated a formal compliance action73 against Boeing, citing five engineering unit members who had conveyed to FAA instances of interference or conflicting duties with their unit member roles. According to FAA, one of these five individuals reported the instances of undue pressure through the formal Boeing process for resolution.

In subsequent months, Boeing requested three extensions from FAA before providing its response to the compliance action, including a corrective action plan. FAA did not accept Boeing’s response to this compliance action. FAA also issued two separate letters of investigation74 in June 2019 and March 2020 against Boeing, related to potential undue pressure of unit members. FAA did not accept Boeing’s response to the June 2019 letter of investigation and is currently evaluating that letter of investigation and the formal compliance action together. The Agency is still awaiting Boeing’s response to the more recent March 2020 letter of investigation.

72 FAA ODA oversight employees actually initiated 26 of these supervision records, but only 17 were accepted upon review by FAA management. Of the 26 records initially submitted, 5 were deleted, 3 were rejected, and 1 had not yet been reviewed as of July 25, 2019. 73 In contrast to a legal enforcement action, such as a civil penalty, compliance actions allow a manufacturer to address a non-compliance in accordance with a corrective action plan agreed upon with FAA. According to FAA, an insufficient response to a compliance action can result in enforcement action. 74 A Letter of Investigation, as part of FAA compliance and enforcement program, serves the dual purposes of notifying an apparent violator that they are under investigation for a potential violation and providing the factual details about the activities being investigated. It also gives the apparent violator an opportunity to provide input and respond to the Agency. According to FAA, an insufficient response to a Letter of Investigation can result in enforcement action.

AV2020037 38

Conclusion FAA is charged with overseeing the safety and certification of all civilian aircraft manufactured and operated in the United States. The tragic accidents in 2018 and 2019 involving the Boeing 737 MAX 8 aircraft have raised important questions about FAA’s certification process, including its oversight of the ODA program. The accidents, including FAA’s response following the Lion Air crash, have also drawn attention to the Agency’s processes for determining certification basis, assessing pilot training needs, and conducting risk analyses. While investigations and related reviews are still ongoing, FAA’s sustained management attention will be essential in identifying and monitoring the highest-risk and safety-critical areas of aircraft certification, while also working to restore public confidence in its aircraft certification processes.

Recommendations Given that we have an open recommendation for FAA related to ODA and that we are planning additional analysis of FAA’s certification process and the use of the ODA program for the 737 MAX 8 aircraft, we are not making recommendations in this report. In our 2015 report, we recommended FAA develop and implement risk-based tools to aid ODA team members in targeting their oversight. By December 31, 2020, FAA plans to implement a risk-based approach to ODA oversight.

The data gathered for this report are informational and meant to be responsive to the Secretary’s request. We will report further on FAA’s oversight of the certification process, ODA, and other related matters in future reports.

Agency Comments and OIG Response We provided FAA with our draft report on April 28, 2020, and received its response on June 8, 2020, which is included as an appendix to this report. As our report did not contain recommendations, no further actions are required.

Exhibit A. Scope and Methodology 39

Exhibit A. Scope and Methodology We conducted this performance audit between April 2019 and April 2020 in accordance with generally accepted Government auditing standards as prescribed by the Comptroller General of the United States. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives.

We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. This report is in response to the Secretary’s request to compile an objective and detailed factual history of the activities that resulted in the certification of the 737 MAX 8 and is the initial product to be issued related to FAA’s certification of the Boeing 737 MAX.

To determine the reliability of the data, we compared dates regarding Boeing 737 MAX certification documentation received from both FAA and Boeing and obtained source documentation to confirm and resolve discrepancies from respective presentations. We also sought and obtained source documentation to verify information obtained from testimonial evidence. In addition, we assessed the completeness and integrity of FAA’s ODA oversight records by reviewing the content and accuracy of the data and determining FAA’s processes for assessing data quality.

To obtain detailed, factual information regarding FAA’s aircraft certification process and the historical certification of Boeing’s 737 MAX, we met with FAA aircraft certification officials in both Washington, DC, and Oklahoma City, OK, to discuss the evolution of FAA’s certification and ODA policies and guidance. We also collected current and historical ODA policy and guidance documents and internal policy office analyses, including data tracking metrics for Boeing’s ODA.

We received multiple briefings from FAA’s Aircraft Certification Service, System Oversight Division and Compliance and Airworthiness Division, as well as the Flight Standards Service, Aircraft Evaluation Group, located at the Northwest Mountain Regional Office. We conducted interviews of FAA safety inspectors; flight test and control engineers; standards staff engineers; and certification, oversight, and flight test management personnel. We reviewed and analyzed certification plans and associated deliverables, issue papers, internal correspondence, internal safety analyses conducted during the certification process and following the 2018 and 2019 accidents, and flight test documents pertaining to the Boeing 737 MAX. We also interviewed a National Air Traffic Controllers Association representative for FAA aircraft certification engineers to look at issues raised over the course of the MAX’s certification.

Exhibit A. Scope and Methodology 40

We analyzed the 2015 Settlement Agreement between FAA and Boeing and collected and reviewed supervisory records, pilot reports, ODA audits, and compliance and enforcement actions that FAA initiated on Boeing during the timeframe of the MAX’s certification. We also collected information related to recent compliance actions regarding undue pressure of Boeing ODA employees.

We visited Boeing facilities in Everett, Renton, and Seattle, WA, and interviewed Boeing management about the Boeing 737 MAX’s certification. We collected further documentation from Boeing regarding certification plans, internal system safety analyses, MCAS-specific requirements and testing documents, internal flight test reports, and updates regarding return-to-service actions and MCAS software revisions. We also interviewed ODA management and collected and reviewed internal ODA procedure manuals and self-audits. Interviews of Boeing certification personnel were limited in scope because of liability concerns raised by Boeing. Individual interviews of Boeing ODA staff to obtain information about undue pressure and other climate issues were conducted within agreed-upon parameters with Boeing, such as OIG not asking the two ODA staff interviewed about specific certification decisions or certification programs such as the 737 MAX.

Finally, we had several coordination meetings with the National Transportation Safety Board over the course of our audit, to both avoid duplication of effort in our respective reviews and to also receive updates on the status of its accident investigations.

Exhibit B. Organizations Visited or Contacted 41

Exhibit B. Organizations Visited or Contacted

Federal Aviation Administration Aircraft Certification Service:

System Oversight Division

Boeing Aviation Safety Oversight Office, Des Moines, WA

Boeing Certificate Management Office, Des Moines, WA

Compliance and Airworthiness Division

Northwest Flight Test Section, Des Moines, WA

Seattle Aircraft Certification Office, Des Moines, WA

Policy and Innovation Division

Transport Standards Branch – Des Moines, WA

Certification Procedures Branch – Washington DC

Delegation and Organizational Procedures Branch – Oklahoma City, OK

Flight Standards Service (AFX):

Seattle Aircraft Evaluation Group, Des Moines, WA

Other Organizations Boeing Commercial Airplanes

Everett, WA

Renton, WA

Seattle, WA

National Air Traffic Controllers Association

National Transportation Safety Board

Exhibit C. Glossary of Terms 42

Exhibit C. Glossary of Terms Term Definition

AC Advisory Circular Advisory Circulars are guidance documents produced by FAA to inform and guide entities within the aviation industry, as well as the general public, and describe actions or advice that FAA expects to be implemented or followed.

AD Airworthiness Directive ADs are legally enforceable rules issued by FAA to correct an unsafe condition in a product. 14 CFR Part 39 defines a product as an aircraft, aircraft engine, propeller, or appliance.

AEG Aircraft Evaluation Group

A group in FAA’s Flight Standards Service tasked with determining the appropriate types of training for aircraft that are undergoing evaluation for an ATC.

AFM Airplane Flight Manual An Airplane Flight Manual’s primary purpose is to provide an authoritative source of information considered necessary for safely operating the airplane. AFMs have specific information that must be provided to satisfy airworthiness regulations.

AOA Angle of Attack The difference between the pitch angle (nose direction) of the airplane and the angle of the oncoming wind. AOA sensors measure the angle between an airplane’s wing and the oncoming air.

ATC Amended Type Certificate

An ATC is issued by FAA when the holder of a type certificate receives FAA approval to modify an aircraft design from its original design. An ATC approves not only the modification but also how that modification affects the original design.

BASOO Boeing Aviation Safety Oversight Office

Provides oversight of designee authority granted to Boeing.

COS Continued Operational Safety

A data-driven, risk-based approach for safety assurance and safety risk management.

FAA Federal Aviation Administration

The Agency responsible for overseeing numerous aviation activities designed to ensure the safety of the flying public.

FCC Flight Control Computer

The component of digital flight control software that provides several functions integral to flight, including autopilot, flight director, and speed trim.

FCOM Flight Crew Operations Manual

The FCOM contains operations information and provides the necessary operating limitations, procedures, performance, and systems information the flight crew needs to safely and efficiently operate the aircraft.

Exhibit C. Glossary of Terms 43

Term Definition

FSB Flight Standardization Board

FAA typically establishes an FSB when certificating large jet or propeller aircraft. One of the FSB’s mandates is to develop training objectives for normal and emergency procedures and maneuvers.

JATR Joint Authorities Technical Review

The JATR is a team consisting of representatives of regulators from 10 civil aviation authorities that was chartered by FAA on June 1, 2019, to examine the Agency’s certification of the 737 MAX 8. The JATR issued a report on October 11, 2019.

JOEB Joint Operational Evaluation Board

A multi-regulatory body that conducts a multi-day session with global regulatory and airline pilots to validate training requirements.

MDR Master Differences Requirements

Specifies the highest training and checking difference levels between a pair of related aircraft derived from the Differences Tables.

MCAS Maneuvering Characteristics Augmentation System

Flight control law implemented on the 737 MAX to improve aircraft handling characteristics and decrease pitch-up tendency at elevated angles of attack.

MoC Means of Compliance The means by which an applicant shows compliance with the flight requirements for an airworthiness or type certificate.

NTSB National Transportation Safety Board

The NTSB conducts independent accident investigations, advocates safety improvements, and decides pilots’ and mariners’ certification appeals.

ODA Organization Designation Authorization

FAA created the ODA program in 2005 to standardize its oversight of organizational designees (e.g., aircraft manufacturers) that have been approved to perform certain functions on the Agency’s behalf, such as determining compliance with aircraft certification regulations.

S&MF Single and Multiple Failure

Boeing uses the S&MF document to analyze the probability of potential failures of key systems and equipment, the probability that those failures will interact, and the impact of multiple failures on continued safe flight and landing.

SSA System Safety Assessment

An assessment of the process to identify and classify failure conditions and ensuing means for regulatory compliance.

TARAM Transport Airplane Risk Analysis Methodology

Outlines a process for calculating risk associated with continued-operational-safety (COS) issues in the transport-airplane fleet. It explains how to use such risk-analysis calculations when making determinations of unsafe conditions and selecting and implementing corrective actions.

TC Type Certificate An approval document issued by FAA that states a specific aircraft model is compliant with airworthiness regulations.

Exhibit C. Glossary of Terms 44

Term Definition

TIA Type Inspection Authorization

Issued after FAA reviews the applicant’s test results package, the TIA authorizes official conformity, airworthiness inspections, and ground and flight tests necessary to fulfill TC certification requirements.

Exhibit D. Major Contributors to This Report 45

Exhibit D. Major Contributors to This Report ROBIN KOCH PROGRAM DIRECTOR

MARSHALL JACKSON PROGRAM DIRECTOR

STEFANIE MCCANS PROJECT MANAGER

CHRISTOPHER FRANK PROJECT MANAGER

MELISSA PYRON SENIOR AUDITOR

ANDREW FARNSWORTH SENIOR ANALYST

KEVIN MONTGOMERY SENIOR ANALYST

HENNING THIEL SENIOR ANALYST

AIESHA MCKENZIE SENIOR ANALYST

JASON LEWIS ANALYST

RACHEL MENCIAS AUDITOR

GRACE ITA-CICCHELLI ANALYST

SETH KAUFMAN DEPUTY CHIEF COUNSEL

AUDRE AZUOLAS SENIOR TECHNICAL WRITER

SHAWN SALES VISUAL COMMUNICATIONS SPECIALIST

MAKESI ORMOND STATISTICIAN

Appendix. Agency Comments 46

Appendix. Agency Comments

U.S. Department of Transportation

1200 New Jersey Avenue, S.E. Washington, D.C. 20590

Office of the Secretary of Transportation

MEMORANDUM

Date: June 8, 2020

Subject: INFORMATION: Management Response to Office of Inspector General (OIG) Draft Report on FAA’s Oversight of Boeing 737 MAX Certification

From: Steven G. Bradbury General Counsel (and performing the functions and duties of Deputy Secretary)

To: Howard R. Elliott

Acting Inspector General

The Department of Transportation (DOT) appreciates the opportunity to review the draft report by the Office of Inspector General (OIG) documenting the timeline of the certification of the Boeing 737 MAX aircraft, the use of Organization Designation Authorization (ODA) programs at the Federal Aviation Administration (FAA), and FAA’s actions after the Lion Air Flight 610 and Ethiopian Airlines Flight 302 accidents. The Department’s top priority is safety. OIG’s review will help FAA to better understand some of the factors that may have contributed to the crashes and ensure these types of accidents never occur again.

Background on Ongoing MAX Re-certification Process

The Department of Transportation (DOT) and FAA continue to extend our deepest sympathy and condolences to the families of the victims of Lion Air Flight 610 and Ethiopian Airlines Flight 302. We honor the memory of those 346 lives by striving for the highest possible margin of safety in the global aviation system.

FAA’s aviation safety professionals have our unequivocal support in carrying out their critical mission. They are following a thorough process for returning the 737 MAX to service—a process that is not driven by a timeline, but by safety. As Administrator Dickson has testified, “The FAA is continuing to follow a data-driven, methodical analysis, review, and validation of the modified flight control systems and pilot training required to safely return the 737 MAX to commercial service.” He has directed FAA employees to take whatever time is necessary to

Appendix. Agency Comments 47

complete their work. In addition, FAA will continue to coordinate with foreign airworthiness authorities around the world as work to return the 737 MAX to service proceeds.

FAA’s Approach to Certifying the MAX Series

The timeline prepared by OIG reveals some strengths in FAA’s aircraft certification process, as well as areas for improvement. This and other reviews, both completed and ongoing, will inform important reforms of FAA’s aircraft certification process. Although OIG determined that FAA followed its certification process for the MAX, OIG’s review also makes clear that FAA’s certification of the 737 MAX was hampered by a lack of effective communication, both between Boeing and FAA and within FAA, which led to an incomplete understanding of the scope and potential safety impacts of changes to the flight control system. For example, OIG noted that during the original certification process, “key FAA certification engineers and personnel responsible for approving the level of airline pilot training were unaware of the revision to [the Maneuvering Characteristic Augmentation System (MCAS)].” FAA’s certification process relies on receiving complete, candid information from manufacturers. The agency will be taking further steps to ensure integrity and transparency with regard to information sharing, assumptions, and validation, all of which are integral to the overall certification system. Additionally, FAA anticipates strengthening coordination among the lines of business with certification responsibilities, as well as enhancing its human factors, flight controls, and system safety expertise to address weaknesses that led to an incomplete understanding of MCAS prior to certification.

FAA’s ongoing work to improve its certification process includes moving toward holistic review and oversight from initial application to final certification, as well as coordinating a flexible information flow throughout the oversight process. It also includes promoting an environment where the proactive self-disclosure of errors is expected and appreciated, and where the reporting of safety issues is encouraged. Safety Management Systems (SMS) for all industries involved in the aerospace system and “Just Culture” concepts that allow for the consideration of honest mistakes and incentivize openness and transparency will help achieve these goals.

Use of Organization Designation Authorization (ODA)

FAA is currently in the process of developing a new policy to ensure appropriate FAA oversight of ODA programs. Delegation in the aviation industry has existed in some form since the 1920s, and since 1958 for aircraft. Congress established the current ODA program, which sought to streamline aspects of the certification process, as part of the FAA Modernization and Reform Act of 2012 (Pub. L. 112-95, sec. 312, codified at 49 U.S.C. § 44704(e)). As Administrator Dickson testified in December, the ODA “construct is based on trust . . . it’s a privilege . . . it’s not a right.” FAA’s new policy will be based on that theme—building upon the successes of ODA while ensuring the appropriate level of FAA oversight based on the risks associated with each ODA’s authorized functions, the size and complexity of the ODA’s organization, any history of undue influence on ODA decision-making, and other performance and risk factors. FAA will also base the policy on recommendations from the evaluations currently underway by OIG and others, such as the Expert Review Panel recently formed per section 213 of the FAA

Appendix. Agency Comments 48

Reauthorization Act of 2018. FAA expects to have the draft policy prepared for public comment in the coming months. Additionally, FAA has established the Organization Designation Authorization Office within the Aviation Safety Organization. This office will provide a system-level focus on multiple areas, including ODA utilization, establishment of ODA limitations, ODA oversight, and performance of ODAs. By looking at the ODA system from an integrated certification and operational perspective, this office will support standardized outcomes and drive improvement across all our ODAs while coordinating national program policy. FAA is currently in the process of operationalizing this office with the goal of having permanent staff by the end of calendar year 2020. Currently, the office is developing a detailed implementation plan to further operationalize the office while defining actions needed to address continuous improvement of the ODA Program. Post-Accident

Within a matter of days following the Lion Air crash, FAA issued an emergency Airworthiness Directive (AD) in response to the accident. The AD reminded pilots how to handle a runaway stabilizer scenario, since an unintended MCAS activation was understood to manifest to pilots as a runaway stabilizer. FAA made the decision to issue the emergency AD based on the information available at the time. After the accident, FAA also initiated a review of the original MCAS certification process and began certification work on the initial changes Boeing proposed to address preliminary concerns about the 737 MAX. That work has been incorporated into FAA’s ongoing recertification process. Conclusion

The events noted on the OIG timeline are all important steps that are being evaluated with the goal of identifying potential improvements to FAA’s risk-assessment methodology, development of assumptions, decision-making, and information reliance. DOT looks forward to the results of the OIG’s continued review including its recommendations to FAA later this year. Department and FAA leadership appreciate the work of OIG, the Secretary’s Special Committee to Review the FAA’s Certification Process for the 737 MAX, the National Transportation Safety Board, the 737 MAX Technical Advisory Board, the Joint Authorities Technical Review, and the accident investigation authorities to help FAA improve aviation safety both domestically and abroad. FAA welcomes feedback from its international peers, intergovernmental partners, governmental auditors, Congress, and industry experts. There will never be a risk-free mode of transportation, but it is the dedication and hard work of aviation safety professionals within FAA and throughout the industry that have made commercial aviation in the U.S. the safest mode of transportation in the world. We appreciate this opportunity to respond to the draft report. Please contact Madeline Chulumovich at (202) 366-6512, with questions or if you require additional information.

Our Mission OIG conducts audits and investigations on

behalf of the American public to improve the performance and integrity of DOT’s programs

to ensure a safe, efficient, and effective national transportation system.