Planes, Trains, and Automobiles: A Data Scientist’s Guide to Modeling Engine Degradation

Planes, Trains, and Automobiles

A Data Scientist’s Guide to Modeling Engine Degradation

April Song @aprilsongg Sarah Aerni @itweetsarah

Gene Sequencing

Smart Grids

COST TO SEQUENCE ONE GENOME HAS FALLEN FROM

$100M IN 2001

TO $10K IN 2011 TO $1K IN 2014

READING SMART METERS EVERY 15 MINUTES IS 3000X MORE DATA INTENSIVE

Stock Market

Social Media

FACEBOOK UPLOADS 250 MILLION

PHOTOS EACH DAY

Oil Exploration

Video Surveillance

OIL RIGS GENERATE

25000 DATA POINTS PER SECOND

Medical Imaging

Mobile Sensors

All industries need technology to process and store data

How can connected devices in our home be smart enough to

make daily life easier?

How can we know a tree has fallen on a power line before the

residents complain?

How can we use data to prevent airplane accidents?

Aerospace Industry is Embracing IoT

!  Engines are being fitted with more and more sensors

!  Aircraft data networks are improving data transfer speeds

!  Real time analytics is improving efficiency and performance

Pratt & Whitney’s Geared Turbo Fan Engine

!  5,000 sensors

!  10 GB data per second

!  12 hours of flight = 844 TB data

WHY IS THIS A DATA SCIENCE PROBLEM?

How does this…

…become this?

How does this…

…become this?

By recognizing this

HOW CAN IT SOLVE JET ENGINE CHALLENGES?

But what can we do with this much data?

Predict thrust demands of an engine Reduction in fuel consumption

Monitor engine health and degradation

Reduced maintenance costs with increased performance, efficiency, and engine lifetime

Detect faults and anomalies during a flight

Prevention of equipment failures and accidents

What We Will Cover Today

!  Jet Engine Sensor Data

!  Enabling Technologies for Data Science

!  Building Models on Large-Scale Datasets –  Detecting Engine “end-of-life” Signal via Clustering –  Tracing Engine Health Degradation using Classification

Commercial Modular Aero-Propulsion System Simulation

Introduction to C-MAPSS

C-MAPSS a Matlab program that simulates a large high-bypass commercial turbofan engine capable of ~90k lbs thrust –  GUI allows point-and-click

operation of engine models –  simulates deterioration and

faults

Simplified diagram of 90k engine

Overview of Flights

!  6,875 flights –  5,244 flights from

nominal engines –  1,631 flights from

fault engines

!  Flight lengths range from 74 to 85 minutes

!  Average length of flight is ~80 minutes

Length of Flight (Seconds)

Flight Parameters

Parameter Name Description Units

Flight Conditions

time Flight time sec

alt Altitude ft

MN Mach number pct

TRA Trottle resolver angle deg

Wf Fuel flow pps

Fn Net thrust lbf

Measurement Temperatures

T48 Total temperature at HPT outlet R

T2 Total temperature at fan outlet R

T24 Total temperature at LPC outlet R

T30 Total temperature at HPC outlet R

T50 Total temperature at LPT outlet R

Other Measurements Nf Physical fan speed rpm

Nc Physical core speed rpm

epr Engine pressure ratio (P50/P2) --

phi Ratio of fuel flow to Ps30

pps/psiu

Ps30 Static pressure at HPC outlet psia

NfR Corrected fan speed rpm

NcR Corrected core speed rpm

BPR Bypass ratio --

farB Burner fuel-air ratio --

htBleed Bleed enthalpy --

PCNfRdmd Percent corrected fan speed pct

W31 HPT coolant bleed lmb/s

W32 LPT coolant bleed lmb/s

Health Indicators

SmHPC HPC stall margin --

SmLPC LPC stall margin --

SmFan Fan stall margin --

Pressure Measurements

P2 Pressure at fan inlet psia

P15 Total pressure in bypass-duct psia

P30 Total pressure at HPC outlet psia

LARGE DATASETS REQUIRE NEW TECHNOLOGIES

At-Scale Modeling

Need for new environments to process big data?

HDFS STORAGE AND MPP ARCHITECTURES DISTRIBUTE STORAGE

AND PREVENT DATA MOVEMENT VARIETY/VELOCITY

DISTRIBUTED COMPUTATION FOR PARALLELIZATION

PETABYTES OF DATA

OPEN-SOURCE LIBRARY FOR MACHINE LEARNING AT SCALE AND FRAMEWORK

TO ACCESS COMMON LANGUAGES

RAPIDLY EVOLVING FIELD OF DATA SCIENCE AND

SQL ENGINE AND ODBC/JDBC CONNECTIONS TO HADOOP

MANY EXISTING LIBRARIES, TOOLS AND

EXPERTISE

FLEXIBLE

SCALABLE

ENABLING

ACCESSIBLE

A single address for everything analytics Analytics with Pivotal

Time-to-Insights FORECASTING CLUSTERING

REGRESSION

CLASSIFICATION

OPTIMIZATION

Pivotal Greenplum MPP DB Think of it as multiple PostGreSQL

servers

Rows are distributed across segments by a particular field (or

randomly)

Segments/Workers

Master

Greenplum Database Features for Data Scientists

•  Window functions: Perform calculations across a set of table rows that are somehow related to the current row

•  Analytics extensions: In-database machine learning at scale using MADlib

•  Procedural language extensions: Extended functionality using non-SQL programming languages and packages (e.g. Python and R) !  Client Access: ODBC and JDBC

access to support connections to 3rd party tools

* Only a subset of Greenplum Database features

MADlib: Scalable, In-database ML

•  Open Source https://github.com/madlib/madlib •  Works on Greenplum DB, HAWQ and PostgreSQL •  In active development by Pivotal •  Downloads and Docs: http://madlib.net/

•  For embarrassingly parallel tasks, we can use procedural languages to easily parallelize any stand-alone library in Java, Python, R or C/C++

•  The interpreter/VM of the language ‘X’ is installed on each node of the MPP environment

Standby Master

Master Host

Interconnect

Segment Host Segment Segment

Data Parallelism through PL/X

CREATE FUNCTION pymax ( a integer, b integer) RETURNS integer AS $$ if a > b: return a return b $$ LANGUAGE plpythonu;

SQL wrapper

Source language code

Source language

declaration

User Defined Functions

Altitude over time for some example flights

What does a typical flight look like?

!  Flight consists of series of ascents, cruises, and descents

!  Average cruise at 35,000 Ft is for ~ 21 minutes –  Engine health is

calculated from a snapshot of parameters during this cruise

Time Series: Pressure Parameters

!  P2, P15, and P30 appear to be positively correlated except during the middle cruise –  correlation may

differ depending on regime

P2 = Pressure at Fan Inlet P15 = Total pressure in bypass-duct P30 = Total pressure at HPC outlet

Life of a Nominal Engine

!  Engine health is modeled to degrade exponentially over time

!  5,244 flights from 25 nominal engines

!  Median number of flights for a nominal engine is 201

!  Median health score of nominal engines across all flights is ~.81

Opportunity for Clustering of Engines

!  Nominal engines seem to degrade in at least 4 different ways –  cluster engines

based on degradation trend

–  caveat: small sample size (35 engines)

!  Additional Modeling Opportunity: –  Predict engine

health score

Life of a Fault Engine !  Significant drop in

engine health is apparent after a fault flight

!  1,631 flights from 10 fault engines

!  Median number flights of fault engines is 137 flights

!  Median health score of fault engines across all flights is ~.72

Fault Flight

Example: Engine Pressure Ratio (EPR) for flight 32-15, a flight with a fan fault

What happens when there is a fault?

At first glance, fault’s effects are not noticeable –  Need to zoom in to see the effects of a fault

Feature Engineering: Transforming Timeseries

!  Many modeling approaches require feature extraction –  Clustering of engines –  Regression to reverse-engineer engine

health

Engineering Features From Time Series

!  Goal: Represent timeseries data as variables

!  Approach: 1.  Identify the different phases

of the flight: takeoff, climbs, cruises, descents, landing

2.  For each phase and parameter calculate:

3.  Summary stats on rate of change for features

▪  mean ▪  min ▪  max ▪  stddev

▪  max – min ▪  median

mean: 13,674 stddev: 0 max: 13,674 min: 13,674 max-min: 0 median: 13,674

mean: 33,596 stddev: 5,732 max: 45,575 min: 25,959 max-min: 19,616 median: 32,556

Calculating Correlations between Sensors

!  How correlated are two sensors?

!  Are correlations between the sensors different flight to flight?

!  Approach: –  1) Calculate correlations over entire flight data set and observe

trends –  2) Calculate correlations over each flight and observe trends

Sensor Parameter Correlations

!  Correlations calculated on entire flight data set

!  435 total unique parameter pairs –  162 pairs are strongly

positively correlated (>.8) –  45 pairs are strongly

negatively correlated (<-.8) –  228 pairs are weakly

correlated

Correlation

Top Correlated Parameter Pairs

Parameter 1 Parameter 2 Correlation

p2 alt -0.985

t2 alt -0.974

p15 alt -0.972

w31 alt -0.931

w32 alt -0.931

Parameter 1 Parameter 2 Correlation

nc htbleed .999

t30 nc .999

t30 htbleed .999

ps30 p30 .999

w31 w32 .999

Negatively Correlated Positively Correlated

p2 pressure at fan inlet t2 total temp at fan inlet p15 total pressure in bypass-duct w31 HPT cooland bleed w32 LPT cooland bleed

nc physical core speed htbleed bleed enthalpy t30 total temperature at HPC outlet ps30 total pressure at HPC outlet

Top Negatively Correlated Sensors

p2 pressure at fan inlet t2 total temp at fan inlet p15 total pressure in bypass-duct w31 HPT cooland bleed w32 LPT cooland bleed

!  Potential Analysis: Calculating correlations at a regime level may reveal anomalies

Top Positively Correlated Sensors

nc physical core speed htbleed bleed enthalpy t30 total temperature at HPC outlet ps30 total pressure at HPC outlet

!  Potential Analysis: Calculating correlations at a regime level may reveal anomalies

Correlation Between Altitude and P2 Flight ID

Clustering Flights Insights on engine degradation and end of life

Feature Reduction using VIF

K-Means Clustering Algorithm Objective: Group flights based on their parameter time series

Time Series for Single Sensor Data

Extract Summary Statistics for All Phases

Cluster using K-means algorithm in MADlib with Summary Statistics as Feature Vector

K-Means Clustering Algorithm Objective: Group flights based on their parameter time series

Cluster using K-means algorithm in MADlib with Summary Statistics as Feature Vector

Param 1

Extract Features

K-Means Clustering Algorithm

Source: http://www.naftaliharris.com/

For each Cluster using K-means algorithm in MADlib with Summary Statistics as Feature Vector

K-Means Clustering Algorithm

For each Cluster using K-means algorithm in MADlib with Summary Statistics as Feature Vector

Repeat process for 29 parameters

Flights in Cluster 4 Indicate Engine’s end of life

Smfan Timeseries Features Clustering Results

Classification-Based Similarity Metric Understanding similarities between flights

Classification-Based Distance Metric

!  Binary classification methods to build models to differentiate between two groups using available attributes –  Algorithms allow us to use

optimal subset of attributes to differentiate classes (feature selection)

–  Ability to differentiate becomes a proxy for dissimilarity

Class 1 Class 2 Class 3

Classes differentiated by size and color

These classes are indistinguishable

Model accuracy HIGH : able to predict class

Model accuracy LOW: unable to predict classes

Classification-Based Flight Similarity Metric

!  For a given pre-takeoff phase –  Create a non-overlapping set of all 5-second windows –  Extract features

▪  Summary statistic (402) for each parameter in the time-window ▪  Correlations between all pairs of parameters in the time-window used for

propulsion data only

Flight 1, Flight 2

Flight 1, Flight 3

Flight m, Flight n

Train Classifier for

Classification Accuracy Score

Engine 1

Expected Results

!  745,281 total models built –  For each flight, classifier to

each other flight for the same engine

–  Modeling run-time ~11 min on 128-segment cluster

!  As engines begin to degrade, adjacent flights should be similar (low accuracy)

Class 1 Class 2 Class 3

Classes differentiated by size and color

These classes are indistinguishable

Model accuracy HIGH : able to predict class

Model accuracy LOW: unable to predict classes

Expected Results

!  745,281 total models built –  For each flight, classifier to

each other flight for the same engine

–  Modeling run-time ~11 min on 128-segment cluster

!  As engines begin to degrade, adjacent flights should be similar (low accuracy)

Model accuracy HIGH : able to distinguish flights that occur after degradation

Model accuracy LOW: unable to predict distinguish adjacent flights (little difference)

Flight number

REFERENCE FLIGHT

Engine 1 results Model accuracy HIGH : able to distinguish flights that occur after degradation

REFERENCE FLIGHT

Engine 1 results Model accuracy HIGH : able to distinguish flights that occur after degradation

REFERENCE FLIGHT

Engine 1 results

Model accuracy HIGH : able to distinguish flights that occur before and after degradation

REFERENCE FLIGHT

Engine 1 results

Model accuracy HIGH : able to distinguish flights that occur before and after degradation

REFERENCE FLIGHT

Logistic Regression Results

!  Earlier flights are more similar to each other

!  Earlier flights are more dissimilar to later flights

!  Flights up until 50th are similar to each other

!  Flights after 50th are only similar to neighboring flights but start to differ from earlier flights

!  Indicates change/degradation over time

Similar Dissimilar

Examining Engine Degradation Over Time

!  Summary statistics over flights provide insights into degradation patterns –  Median/mean accuracies over

PRECEDING flights indicates what degradation occurred since the engine start

–  Observations over adjacent windows may be of interest

–  Detecting anomalies

Flight number

REFERENCE FLIGHT

Engine Health and Engine Classification-based Similarity

!  Median accuracy score of a flight to prior flights increases as engine health decreases

!  Abrupt changes in engine health can be found using future flights (to find an inflection)

Accuracy Scores Show both Time and Degradation

!  With many more flights median accuracy increases

!  Degradation in engine causes median accuracy to drop faster

Example of Fault: engine 32

!  Flight before fault occurs

!  avg scores of flights before fault flight is slightly higher

!  flight after fault: more flights with score > .8

HPT Fault

Classification-Based Similarity Changes at Faults

LPC Fault – low engine health change still detected

Engine Health and Median Accuracy Correlations

HPT fault flights

Engine Health and Median Accuracy Correlations

fan hpc hpt

lpc lpt

What Did We Learn? What is next?

•  Through technology, data exploration and feature generation becomes easier –  What we learned: Rapidly transforming large volumes of

sensor data –  What’s next: Timeseries analysis, interpolation on missing

data •  Experimentation with building models to predict engine decay

and faults –  What we learned: unsupervised techniques for clustering

and distance metrics enable us to discover signals of decay

–  What’s next: supervised approaches to detect known faults

Opportunities in the Digital Brain

CONNECTED CARS

PERSONALIZED MEDICINE

SMART METERS

SECURITY

PREDICTIVE MAINTENANCE

SPORT TRACKING

OPTIMIZATION AND EFFICIENCY

Appendix

•  Propulsion dataset can be downloaded at: https://c3.nasa.gov/dashlink/resources/140/

Planes, Trains, and Automobiles: A Data Scientist’s Guide to Modeling Engine Degradation

Data & Analytics

Transcript of Planes, Trains, and Automobiles: A Data Scientist’s Guide to Modeling Engine Degradation

Scientist’s Perspective Dr. Nick Bontis

Acoustic Conditions during Interpretive Programs at ... · Potentially interfering sounds of propeller planes, helicopters, jets and automobiles were heard only during the outside

Planes Planes and Planes

English - AMSImedia.amsi.ae/documents/school_isas/2015-2016/SummerWorks/Gr… · transportation. “We have planes, trains, automobiles, and boats,” he explained. “What if there

Constructing S.T.E.A.M.: a PHYSICSian scientist’s perspective

Padbury Catholic Primary School - OSHClub › wp-content › uploads › 2017 › 11 › ... · 2019-11-12 · GNOME SWEET GNOME PLANES, TRAINS & AUTOMOBILES MYSTICAL MAGIC ODD SOCKS

China’s Hunger for U.S. Planes and Cars: Assessing the ......2014/03/27 · 3 China’s Demand for Planes and Automobiles Greater Reliance on China’s Market The U.S. aerospace

A Scientist’s View of Open Access

Digital Advertising: An Information Scientist’s Perspectivecourses.ischool.berkeley.edu/i296a-dsa/s12/files/ESSIR-Shanahan... · Digital Advertising: An Information Scientist’s

Planes, Homes and Automobiles: The Role of

2014 SFO Climate Action Planmedia.flysfo.com › ... › 2014-sfo-climate-action-plan_0.pdf · 2016-01-13 · connections among planes, trains, automobiles, waterborne, and human-powered

A Mad Scientist’s Chemistry Presentation CHEMISTRY TAKS.

ACFrOgCg1x0kg9aNQUoBMNTV22KK6tvTts3AefAVO3wmu ......author of Planes, Canes, and Automobiles: Connecting with Your Aging Parents through Travel (Greenleaf, 2015) and Clash of the Generations:

Planes, Trains and Automobiles - THOTCON · PDF filePlanes, Trains and Automobiles ... Mercedes S600 interface ... Don’t forget the language variants for the config files!

TARAS SHEVCHENKO NATIONAL UNIVERSITY OF KYIV Faculty of Economics Young Scientist’s Association (YSA) Young Scientist’s Association (YSA)

Planes, trains, and automobiles! A Vehicles Guessing Game Planes, trains, and automobiles! A Vehicles Guessing Game Train Conductor: Dina Peters, M.S.

Climate Change: A Scientist’s Perspective

Planes, Trains and Automobiles - VPPPA, Inc. · PDF fileOSHA VOLUNTARY PROTECTION PROGRAM (VPP) Employee Participation Drives Safety Excellence OMAHA, NE WHITE SANDS, NM ALBUQUERQUE

D-Brief Edition 10 - Trains, Planes and Automobiles

Hamlet Rehabhamletrehab.com/wp-content/uploads/2019/10/Hamlet.pdf · 12pm Friendsgiving followed by Afternoon Matinee' "Planes, Trains and Automobiles" 6:45pm Computer BINGO 2:15pm