AEROSPACE REPORT NO. TOR-2015-02542

Process Approach to Determining Quality Inspection Deployment Product Overview

May 7, 2015

Eric S. Richter and Arthur L. McClellan Systems and Operations Assurance Department Mission Assurance Subdivision

Prepared for:

National Reconnaissance Office 14675 Lee Road Chantilly, VA 20151-1715

Contract No. FA8802-14-C-0001

Authorized by: Engineering and Technology Group

Developed in conjunction with Government and Industry contributions as part of the U.S. Space Programs Mission Assurance Improvement Workshop.

Distribution Statement A: Approved for public release; Distribution unlimited.

Report Documentation Page Form ApprovedOMB No. 0704-0188

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.

1. REPORT DATE 07 MAY 2015

2. REPORT TYPE Final

3. DATES COVERED -

4. TITLE AND SUBTITLE Process Approach to Determining Quality Inspection DeploymentProduct Overview

5a. CONTRACT NUMBER FA8802-14-C-0001

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) Eric S. Richter and Arthur L. McClellan

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) The Aerospace Corporation 2310 E. El Segundo Blvd. El Segundo, CA 90245-4609

8. PERFORMING ORGANIZATION REPORT NUMBER TOR-2015-02542

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) National Reconnaissance Office 14675 Lee Road Chantilly, VA20151-1715

10. SPONSOR/MONITOR’S ACRONYM(S) NRO

11. SPONSOR/MONITOR’S REPORT NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release, distribution unlimited

13. SUPPLEMENTARY NOTES The original document contains color images.

14. ABSTRACT 2015 MAIW Topic Product Overview

15. SUBJECT TERMS

16. SECURITY CLASSIFICATION OF: 17. LIMITATIONOF ABSTRACT

UU

18. NUMBEROF PAGES

23

19a. NAME OFRESPONSIBLE PERSON

a. REPORT unclassified

b. ABSTRACT unclassified

c. THIS PAGE unclassified

Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18

i

Acknowledgements

This document was created by multiple authors throughout the government and the aerospace industry. For their content contributions, we thank the following contributing authors for making this collaborative effort possible:

Jack Harrington – The Boeing Company Dave Martin – Raytheon Space and Airborne Systems Jeanne Kerr – Lockheed Martin Corporation Art McClellan – The Aerospace Corporation Dan Gresham – Orbital Sciences Brian Reilly – Defense Contract Management Agency

A special thank you for co-leading this team and efforts to ensure completeness and quality of this document goes to:

Eli Minson – Ball Aerospace and Technologies Corporation Frank Pastizzo – SSL Eric Richter – The Aerospace Corporation

The Topic Team would like to acknowledge the contributions and feedback from the following organizations:

The Aerospace Corporation Northrop Grumman Aerospace Systems Ball Aerospace and Technologies Corporation Orbital Sciences The Boeing Company Raytheon Defense Contract Management Agency (DCMA) SSL Lockheed Martin Corporation

The authors deeply appreciate the contributions of the subject matter experts who reviewed the document:

Kathy Augason – Lockheed Martin Corporation David Newton – Northrop Grumman Kevin Craig – SSL Ethan Nguyen – Raytheon Ken Dodson – SSL Michael Phelan – DCMA Ed Gaitley – The Aerospace Corporation Robert Pollard – Ball Aerospace and Technologies Corporation Neil Limpanukorn – SSL Thomas Reinsel – Raytheon

1

U.S. SPACE PROGRAM MISSION ASSURANCE IMPROVEMENT WORKSHOPLOCKHEED-MARTIN | SUNNYVALE, CA | MAY 5 - 7, 2015

Process Approach to Determining Quality Inspection Deployment

Eli Minson, Ball AerospaceFrank Pastizzo, SSL

Eric Richter, The Aerospace Corporation

Product Overview

May 7, 2015

U.S. SPACE PROGRAM MISSION ASSURANCE IMPROVEMENT WORKSHOPLOCKHEED-MARTIN | SUNNYVALE, CA | MAY 5 - 7, 2015 2

Agenda

• Motivation and Team Charter• Product Overview• Examples• Topic Details• Topic Follow-on Recommendations• Team Membership and Recognition

U.S. SPACE PROGRAM MISSION ASSURANCE IMPROVEMENT WORKSHOPLOCKHEED-MARTIN | SUNNYVALE, CA | MAY 5 - 7, 2015 3

Motivation for Topic

• DOD issued 55 years agoMIL-Q-9858A and MIL-I-45208A– Emphasis on complete and

frequent visual inspection

• Technology has improved sincethen– Process controls– Product quality– Inspection capabilities

• Inspection change versus riskguidance is lacking

Team Charter

• Develop a tool for determining ifa change in inspection approachis warranted– Review industry data and

feedback from DCMA toidentify candidate processes

– Identify best practices foroptimal quality inspectionplanning and deployment

– Evaluate candidate processesusing new tool

U.S. SPACE PROGRAM MISSION ASSURANCE IMPROVEMENT WORKSHOPLOCKHEED-MARTIN | SUNNYVALE, CA | MAY 5 - 7, 2015 4

Decision Tree Manufacturing Process Change Inspection Process Change Data Driven Inspection Change

• Shift inspection ofPWB from manualinspection to flyinghead automated probe– False errors manual

inspection reduced– Time study of the

same board showssignificant timereduction

– Output of machine listspart non-conformities

– Manual Inspectioncovers10-20% of partsnot covered by themachine

ExampleICT via Flying Probe

U.S. SPACE PROGRAM MISSION ASSURANCE IMPROVEMENT WORKSHOPLOCKHEED-MARTIN | SUNNYVALE, CA | MAY 5 - 7, 2015 5

In-Circuit Test via Flying Head ProbeAnalyses Performed

Critical Process

• Reviewed historicalinspection processoutput

• Reviewed customerrequirements

• Identified potential toolsuppliers

• Performed riskanalysis againstexisting processes

• Study of cost vs.CAPEX vs. inspectionperformancecompleted

Process Capability

• Reviewed supplier toolsets

• Performed bench testusing EDU boards

• Verified results againstexisting inspectionmethod

• Identified processaccuracy andrepeatability issues

• Compared results torisk and cost analyses

U.S. SPACE PROGRAM MISSION ASSURANCE IMPROVEMENT WORKSHOPLOCKHEED-MARTIN | SUNNYVALE, CA | MAY 5 - 7, 2015 6

In-Circuit Test via Flying Head ProbeAnalyses Performed

Effective Inspection

• Test board coverageand issues reviewed

• Identifiedrequirements againsttypical part usage

• Identified part typesand applicationswhere ICT not able tocapture all issues

ROI

• Performed study forpurchasing unit vs.outsourcing

• Identified multiplesuppliers andreviewed capabilitiesagainst requirements

U.S. SPACE PROGRAM MISSION ASSURANCE IMPROVEMENT WORKSHOPLOCKHEED-MARTIN | SUNNYVALE, CA | MAY 5 - 7, 2015 7

BACKLOG QUADRANT

JUST DO IT QUADRANT

STRATEGIC QUADRANT

FORGET IT QUADRANT

LOW HIGH

0% 50% 100%

HIGH

LOW

0%

50%

100%

INVESTMENT

RETU

RN

Analysis Results into Tool

Analysis Category Entries in tool ManufacturingProcess Change

Inspection Process Change

Data Driven Insp. Change

Manufacturing Lines 1-5 40% 22% 22%Inspection Lines 6-8 30% 45% 38%Cost andCustomer

Lines 9-10 30% 33% 40%

Strategic

U.S. SPACE PROGRAM MISSION ASSURANCE IMPROVEMENT WORKSHOPLOCKHEED-MARTIN | SUNNYVALE, CA | MAY 5 - 7, 2015 8

Tool DesignAnalyses

1. Do the results of a PFMEA showpotential for improved quality?

2. Is the process qualified andcapable?

3. Does the first article indicate lessinspection is required?

4. Does the current process have a lowlevel of nonconformities?

5. Does the proposed process outputrate affect inspection capabilities?

6. Was a gage R&R performed withpersonnel performing the inspectionfunction?

7. Will the improved inspector processincrease the ability to findnonconformities?

8. Will the process change reduceinspector escapes?

9. Has a cost analysis been performed(p<k1/k2, see Appendix B)?

10.Will the customer allow the change?

Fixed by Tool

Just

ifica

tion

Weight

• ManufacturingProcessChange

• InspectionProcessChange

• Management orCustomer Input

User Modifiable

Return

1. Does not justifyremoval ofinspectionprocess

2. Additional datarequired beforedecision can bemade

3. Data Justifiescapabilities studyfor processmodification

4. Justifiesmodification ofinspectionprocess

5. Justifies removalof inspectionprocess

User Modifiable

Investment

1. Low Effort (Easyor completed,limitedpersonnel, <3months)

2. Between Lowand Medium

3. Medium Effort(Hurdles,somewhatdifficult, >6months)

4. BetweenMedium andHigh

5. High Effort(Complex, lots ofpeople, >1 yr)

User Modifiable

Weighted results

U.S. SPACE PROGRAM MISSION ASSURANCE IMPROVEMENT WORKSHOPLOCKHEED-MARTIN | SUNNYVALE, CA | MAY 5 - 7, 2015 9

Additional Examples in ProductTorque Witness by Inspection Personnel

Forget It Just Do It

Evaluating whether or not to eliminate Inspection witness of "Torque" operations

Test to flight (class 2) electrical mates

Elimination of a secondary inspection(by QA) for test to flight connector mates

U.S. SPACE PROGRAM MISSION ASSURANCE IMPROVEMENT WORKSHOPLOCKHEED-MARTIN | SUNNYVALE, CA | MAY 5 - 7, 2015 10

Additional Examples in Product

Backlog

Evaluating reduction in duplicative inspection efforts upon receipt for items that are Final Source Inspected

Receiving Inspection of subcontracted products (QSI-1002)

BACKLOG QUADRANT

JUST DO IT QUADRANT

STRATEGIC QUADRANT

FORGET IT QUADRANT

LOW HIGH

0% 50% 100%

HIG

HLO

W

0%

50%

100%

INVESTMENT

RETU

RN

Examples of Each Potential Outcome

U.S. SPACE PROGRAM MISSION ASSURANCE IMPROVEMENT WORKSHOPLOCKHEED-MARTIN | SUNNYVALE, CA | MAY 5 - 7, 2015 11

Target Audience and Intended Product Use

• Target Audience – Quality organizations looking for efficiencies– Manufacturing organizations pursuing new technology – Stakeholders seeking ways to reduce non-value added costs

• How Used – Best applied early in change evaluation decision– Useful when many trades are possible • Provides best indication of tradeoffs resulting from a proposed

process change

U.S. SPACE PROGRAM MISSION ASSURANCE IMPROVEMENT WORKSHOPLOCKHEED-MARTIN | SUNNYVALE, CA | MAY 5 - 7, 2015 12

Quality Deployment Team MembershipCore Team

First Name Last Name Organization

Kathy Augason Lockheed MartinKevin Craig SSLKen Dodson SSLFrank Fieldson HarrisEdward Gaitley The Aerospace CorporationAnthony Gritsavage NASAMichael Kelly NASANeil Limpanukorn SSLMichael Phelan DCMARobert Pollard Ball AerospaceThomas J. Reinsel Raytheon

Ric Alvarez Northrop Grumman

Dave Newton Northrop Grumman

Ethan Nguyen Raytheon

First Name Last Name OrganizationArt McClellan The Aerospace CorporationEli Minson Ball AerospaceFrank Pastizzo SSLEric Richter The Aerospace Corporation

Jack Harrington BoeingJeanne Kerr Lockheed MartinDan Gresham OrbitalDave Martin RaytheonBrian Reilly DCMADaniel Hyatt MDA

Bold – co-leads

SME Team

Approved Electronically by:

AEROSPACE REPORT NO.TOR-2015-02542

Process Approach to Determining Quality Inspection Deployment Product Overview

Todd M. Nygren, GENERAL MANAGERSYSTEMS ENGINEERING DIVISIONENGINEERING & TECHNOLOGYGROUP

Jacqueline M. Wyrwitzke, PRINCDIRECTORMISSION ASSURANCE SUBDIVISIONSYSTEMS ENGINEERING DIVISIONENGINEERING & TECHNOLOGYGROUP

Catherine J. Steele, SR VP NATL SYSNATIONAL SYSTEMS GROUP

Jackie M. Webb-Larkin, SECURITYSPECIALIST IIIGOVERNMENT SECURITYSECURITY OPERATIONSOPERATIONS & SUPPORT GROUP

© The Aerospace Corporation, 2015.

All trademarks, service marks, and trade names are the property of their respective owners.

SK0789

Technical Peer Review Performed by:

AEROSPACE REPORT NO.TOR-2015-02542

Process Approach to Determining Quality Inspection Deployment Product Overview

Jacqueline M. Wyrwitzke, PRINCDIRECTORMISSION ASSURANCE SUBDIVISIONSYSTEMS ENGINEERING DIVISIONENGINEERING & TECHNOLOGYGROUP

Eric S. Richter, ENGRG SPCLST SRSYSTEMS AND OPERATIONSASSURANCE DEPTMISSION ASSURANCE SUBDIVISIONENGINEERING & TECHNOLOGYGROUP

Arthur L. McClellan, DIRECTOR DEPTSYSTEMS AND OPERATIONSASSURANCE DEPTMISSION ASSURANCE SUBDIVISIONENGINEERING & TECHNOLOGYGROUP

Cheryl L. Sakaizawa, ADMINISTRATIVESPEC IIIMISSION ASSURANCE SUBDIVISIONSYSTEMS ENGINEERING DIVISIONENGINEERING & TECHNOLOGYGROUP

© The Aerospace Corporation, 2015.

All trademarks, service marks, and trade names are the property of their respective owners.

SK0789

External Distribution

REPORT TITLE

Process Approach to Determining Quality Inspection Deployment Product Overview

REPORT NO. PUBLICATION DATE SECURITY CLASSIFICATION

TOR-2015-02542 June 26, 2015 UNCLASSIFIED

Craig WesserNorthrop Grummancraig.wesser@ngc.com

Richard FinkNROfinkrich@nro.mil

Marvin LeBlancNOAAMarvin.LeBlanc@noaa.gov

Robert AdkissonBoeingrobert.w.adkisson@boeing.com

Mark BaldwinRaytheonMark.L.Baldwin@raytheon.com

Richard BennettFlight Microwavebennett@flightmw.com

Dennis BoiterIntelsatDennis.Boiter@intelsatgeneral.com

Silva BouchardNorthrop GrummanSilvia.Bouchard@ngc.com

Mark BraunRaytheonmark.j.braun@raytheon.com

Marvin CandeeLockheed Martinmarvin.candee@lmco.com

Larry CapotsLockheed Martinlarry.capots@lmco.com

Steve CarlsonCadencecarlson@cadence.com

Danny ChanRaytheondanny_s_chan@raytheon.com

Janica CheneyATKjanica.cheney@jtk.com

Kevin ChisholmUnited Technologies, ISR SystemsKevin.Chisholm@utas.com

David DavisSMC/ENDavid.Davis.3@us.af.mil

Ken DodsonSSLken.dodson@sslmda.com

Deanna DonerLockheed Martindeanna.e.doner@lmco.com

Jason EmeryRaytheonBrent.Emery@raytheon.com

Dave ErstadHoneywelldave.erstad@honeywell.com

James FarrellBoeingjames.t.farrell@boeing.com

James FieberLockheed MartinJames.r.fieber@lmco.com

Sherri FikeBallsfike@ball.com

Bruce FlanickNorthrop Grummanbruce.flanick@ngc.com

Will CavenSSLwill.caven@sslmda.com

Helen GjerdeLockheed Martinhelen.gjerde@lmco.com

Steven GoldNaval Research Labsteven.gold@nrl.navy.mil

Claude GoldsmithLockheed Martinclaude.goldsmith@lmco.com

Jonathan GrafMacAulay-Brown, Inc.jonathan.graf@macb.com

Mohinder GuruIntelsatMohinder.Guru@intelsat.com

Tom HanhauserLockheed MartinThomas.j.hanhauser@lmco.com

Bill HansenLockheed Martinbill.hansen@lmco.com

John HarringtonBoeingjack.harrington@boeing.com

Keith HenderlongMIT Lincoln LabsKeith.Henderlong.ffrdc@mda.mil

Mike HerzogPacific Scientificmherzog@psemc.com

Bill HoehnRaytheonwkhoehn@raytheon.com

Jerry HolsombackRaytheonjerry.b.holsomback@raytheon.com

Eric HolzmanNorthrop Grummaneric.holzman@ngc.com

Paul HopkinsLockheed Martinpaul.c.hopkins@lmco.com

Daniel HyattMDADaniel.Hyatt@mda.mil

Ed JopsonNorthrop Grummanedward.jopson@ngc.com

Frederick KelsoMDAFrederick.Kelso@mda.mil

Jeanne KerrLockheed MartinJeanne.R.Kerr@lmco.com

Kurt KetolaRaytheonketola@raytheon.com

Rolf KichFMCkich@flightmw.com

Mark KingMicropacmarkking@micropac.com

Brian KosinskiSSLBrian.Kosinski@sslmda.com

John KowalchikLockheed MartinJohn.J.Kowalchik@lmco.com

Debbie SchreiberLockheed Martindebbie.schreiber@lmco.com

C. J. LandHarriscland@harris.com

Jim LarosaBAE Systemsjames.larosa@baesystems.com

Neil LimpanukornSSLNeil.Limpanukorn@sslmda.com

Louie LombardoLockheed Martinlouie.lombardo@lmco.com

Rob LyonSSLrobert.lyon@sslmda.com

Ronand MandelLockheed Martinronald.h.mandel@lmco.com

Patrick MartinNASApartick.martin@nasa.gov

Steven McNeilXilinxstevem@xilinx.com

Kevin MeadowsNorthrop Grummankevin.meadows@ngc.com

Eli MinsonBalleminson@ball.com

Tom MusselmanBoeingthomas.e.musselman@boeing.com

Helen NarcisoLockheed Martinhelen.narciso@lmco.com

John NelsonLockheed Martinjohn.d.nelson@lmco.com

David NewtonNorthrop Grummandavid.a.newton@ngc.com

Ethan NguyenRaytheonethan_m_nguyen@raytheon.com

Jeff OberstLockheed Martinjeff.oberst@lmco.com

Frank PastizzoSSLfrank.pastizzo@sslmda.com

Kevin PaxtonBoeingkevin.r.paxton@boeing.com

David PinkleyBalldpinkley@ball.com

Rob PollardBallrpollard@ball.com

Anne RamseyHarrisaramsey@harris.com

Brian ReillyDCMABrian.Reilly@dmca.mil

Robert RiccoNorthrop Grummanbob.ricco@ngc.com

John RobinsonAerojet Rocketdynejohn.robinson@rocket.com

Frank RollerLockheed Martinfrank.d.roller@lmco.com

Joseph RoubalAeroflex, a Cobham Companyjoseph.roubal@aeroflex.com

Rabindra SinghSSLRob.Singh@sslmda.com

Melanie SloaneLockheed Martinmelanie.sloane@lmco.com

Homer D. StevensLockheed Martinhomer.d.stevens@lmco.com

Norman StrampachLockheed Martinnorman.strampach@lmco.com

David SwansonOrbital ATKDavid.Swanson@orbitalatk.com

Alfred TadrosSSLAlfred.Tadros@sslmda.com

Jeffrey TateRaytheonjeffrey_tate@raytheon.com

Paul ThompsonIntelsatPaulAlex.Thompson@intelsat.com

Brett TobeyLockheed Martinbrett.f.tobey@lmco.com

Mike TolmasoffBoeingmike.w.tolmasoff@boeing.com

Ghislain TurgeonSSLGhislain.Turgeon@sslmda.com

Deborah ValleyMIT Lincoln Labsdeborah.valley@ll.mit.edu

Richard VeresHoneywellrichard.veres@honewell.com

Brynn WatsonLockheed Martinbrynn.a.watson@lmco.com

Lance WerthmanLockheed Martinlance.werthman@lmco.com

Dan YasukawaLockheed Martindan.yasukawa@lmco.com

Thomas FitzgeraldSMCthomas.fitzgerald.5@us.as.mil

Hal BellNASAharold.m.bell@nasa.gov

Mike WadzinskiMDAmike.wadzinski@mda.mil

Brig Gen Anthony CottonNROcottonan@nro.mil

Kevin CraigSSLkevin.craig@sslmda.com

Thomas ReinselRaytheonthomas_j_reinsel@raytheon.com

Michael PhelanDCMAmichael.phelan@dcma.mil

APPROVED BY_____________________________________________________________________________________      DATE___________________________________(AF OFFICE)