N. S. Reddy -...
Transcript of N. S. Reddy -...
N. S. Reddy
School of Materials Science and EngineeringDepartment of Metallurgical & Materials Engineering
Gyeongsang National University, Jinju, Gyeongnam 660-701, Korea
Artificial Neural Networks in Materials Science
Summer School for Integrated Computational Materials Education, Ann Arbor, June 5-16, 2017
NSR
Charpy
Fatigue
Corrosion
Tensile
Critical stress intensity Why
Axioms
• All properties can be measured.• Measurements can be used in safe design.• Measurements can be used in control.
Conclusions• There are useful ways of expressing properties• Limited models relating properties to microstructure
• No method for predicting properties in general
WhySummer School for Integrated Computational Materials Education, Ann Arbor, June 5-16, 2017
Source: http://www.msm.cam.ac.uk/phase-trans/abstracts/neural.review.html
Solidification conditions
Mechanical Treatment
Alloy contents: C, Si, Mn, S, P, Ni, Cr, Mo….
Heat treatment
Physical and Mechanical Properties
Materials
Complex Materials System
Microstructure of
Why
Pickering linear equations (1978)
σY = 53.9 + 32.3WMn + 83.2Wsi + 354.2(W Nf) 0.5 + 17.4(dα)-0.5
σU = 294.1 + 27.7WMn + 83.2Wsi + 3.85(%pearlite) + 7.7(dα)-0.5
σY is predicted yield strength in MPa and σU is predicted ultimate tensile strength in MPa, WMn, WSi and WNf are the contents of manganese, silicon and free nitrogen in weight percent respectively, and dα is the ferrite grain size in millimeters.
Source: http://www.msm.cam.ac.uk/phase-trans/2006/NNGA.html
Source: http://www.msm.cam.ac.uk/phase-trans/2006/NNGA.html
Contents Artificial Neural Networks (ANN)• What• Why• When
Materials Science Problems• Grain refinement in Al-7Si alloy (3 1)• Phase volume fraction in Ti – 6Al – 4V alloy (6 2)• Mechanical Properties in Steels (10 5)
• How• Where
Summer School for Integrated Computational Materials Education, Ann Arbor, June 5-16, 2017
A Brief history• Early stages
– 1943 McCulloch-Pitts: Neuron as computing element – 1949 Hebb: Learning rule– 1958 Rosenblatt: Perceptron– 1960 Widrow-Hoff: Least mean square algorithm
• Recession– 1969 Minsky-Papert: Limitations perceptron model
• Revival– 1982 Hopfield: Recurrent network model– 1982 Kohonen: Self-organizing maps– 1986 Rumelhart et. al.: Backpropagation
When
Artificial Neural Networks• 65 journals
• 413 Conferences and workshops are presently dedicated
exclusively to artificial neural networks
• About 67138 articles are being published on the current trends
in artificial neural networks.
• USA, India, Japan, Brazil and Canada are some of the leading
countries where maximum studies are being carried out.
• Artificial Neural Networks (ANN) are computational models
inspired by an animal's central nervous systems
http://www.omicsonline.org/artificial‐neural‐network‐journals‐conferences‐list.php
A few Neural Networks Journals1. Neural Networks
2. Advances in Artificial Neural Systems
3. Applied Intelligence
4. IEEE Transactions on Neural Networks
5. Journal of Artificial Intelligence and Soft Computing
6. Neural Computing and Applications
7. International Journal of Neural Networks
8. Neural processing Letters
9. Applied Soft computing
IMPORTANCE OF ANN in Materials Modeling
0 0 ' 0 1 ' 0 2 ' 0 3 ' 0 4 ' 0 5 ' 0 6 ' 0 7 ' 0 8 ' 0 9 '0
1 0 0 0
2 0 0 0
3 0 0 0
4 0 0 0
5 0 0 0
6 0 0 0
7 0 0 06 3 6 1
5 0 2 14 4 6 3
3 8 9 1
3 2 7 02 8 8 6
2 6 5 0
1 9 3 11 5 9 1AN
N P
ublic
atio
ns
Y e a r
1 4 5 0
Number of ANN papers published in Materials Science Journals from Jan 2001 – Dec 2009. (Elsevier publishers only)
Keyword Search criteria: Neural networks and materials science in allfields. www.sciencedirect.com Why
Where are ANN used?• Recognizing and matching complicated,
vague, or incomplete patterns• Data is unreliable• Problems with noisy data
– Prediction– Classification– Data association– Data conceptualization– Filtering– Planning– **** …
Where
What’s a neural network?
What
Ariana Grande, US Singer, Actress, Manchester
What’s a neural network?
What
What’s a neural network?
Ravindra Jadeja and BoltWhat
Sherif Ismail, Prime Minister of Egypt
What’s a neural network?
What
Yao Ming, Basket Ball Player, China
What’s a neural network?
What
Saudi King Salman bin Abdulaziz Al Saud
What’s a neural network?
What
Son ga-in
What’s a neural network?
Shirin Ebadi, First Iranian woman , Nobel peace prize in 2003
What’s a neural network?
What
Aziz Sancar, Nobel Laureate in Chemistry 2015
What’s a neural network?
What
What’s a neural network?
What
What
What’s a neural network?
This is a dog This is a dog
This is a cat
This is a dog
This is a cat
This is a cat
This is a dog
What is this?
What
Artificial Neural networks (ANN) are modeling system, which mimics the human brain.
• Knowledge is acquired by the process of learning
• Storing the knowledge (like brain).• Generalizations capability of the situation
based on the acquired Knowledge
Neural Networks: Lessons from human brain
What
Simple Artificial Neuron
Inputvalues
weights
Summingfunction
Biasb
Activationfunction
u Outputy
x1
x2
xm
w2
wm
w1
)(u
)e 1/(1 y
xwu
u-
m
1jj
bj
Linear, Hyperbolic, Sigmoid
weighted sum
Inputvector x
weightvector w
The n-dimensional input vector x is mapped into variable y by means of the scalar product and a nonlinear function mapping What
Feed Forward Neural Networks (FFNN)• Neurons are arranged in layers.• Each unit is linked only in the unit in next layer, no
units are linked between the same layer, back to theprevious layer or skipping a layer.
• Computations can proceed uniformly from input tooutput units.
Inputlayer
Outputlayer
Hidden Layer
ParametersLearning RateMomentum TermHidden LayersHidden NeuronsIterations
Information
Error Propagation What
Updating Weights
ij
k
Wji
Wkj
WkjWji
)1()()()1( tjiWtjiWiOjtjiWtjiW
)1()()()1( tjiWtjiWiOjtjiWtjiW
)()( kkkk netfOT )()( kkkk netfOT
k
kjkjj Wnetf )( k
kjkjj Wnetf )(
Output Layer
Hidden Layer
Learning Rate
Momentum Rate
Xi
Ok Function SignalsError Signals
ANN Model Flowchart
Stop
Select Input and Outputs best represent the model
Y
Define Range & Distribution of Data
Data preprocessing
Neural Network Structure SelectionTraining Algorithm
Perform Training
Is Validation good Accuracy
Test the quality of neural ModelYes
No
How
yd is the desired response, yo is the output response from the ANN, and p is the of patterns presented
2
1
1 [ ]p
d oj j
jMSE y y
p
Criteria for model selection
where Etr(y) = average error in prediction of training andtesting data set for output parameter y, N = number ofdata sets, Ti(y) = targeted output, Oi(y) = outputcalculated.
N.S. Reddy et al., “Computational Materials Science”, Vol. 107, 2015
The sensitivity curve & the instantaneous sensitivity
N.S. Reddy et al., “Materials Science and Engineering A”, Vol. 434, 2006
Objectives
• To investigate the suitability of neural networks tocomplex Materials Systems.
• To predict properties/microstructure at newinstances.
• To examine the Effect of Individual Elements onoutput parameters keeping other elements unaltered.
• To design the Optimum Alloy for desired properties.
• To validate the model predictions with experiments
Why
1. Grain refinement in Al-7Si alloy(3 1)
2. Microstrucure in Titanium alloys(6 2)
3. Mechanical Properties in Steels(10 5)
Example.1: Grain Refinement of Al-7Si Alloy Importance of Al-Si alloys
Need of grain refinement
How to achieve grain refinement
Master alloys for Grain refinement of Al –7Si alloyBinary Master Alloys
• Al – 3B • Al – 3Ti
Ternary Master Alloys•Al – 1Ti – 3B•Al – 3Ti – 1B•Al – 3Ti – 3B•Al – 3Ti – 3B•Al – 3Ti – 3B
Where
“Al-Si alloys are frequently used in castingsdue to their combination of good physicalproperties and excellent castability”
• Light weight• Energy and Environmental
considerations• Castability, weldability• Corrosion resistive• Good at room temperatures & fairly
upto 200oC• High strength to weight ratio
Al-Si alloys are frequently used in castings due to their combination of good physical properties and excellent castability.
Problem?Coarse columnar grains
Al-5Ti-1B Master alloy
Example1. Grain refinement of Aluminum and its alloys
Further Processing
Rolling Forging Extrusion
Al with grain refiner
Al without grain refiner
Applications of Aluminium
Al
Slabs
Grain refiner
Mould Box
Aerospace Packaging Electrical
Molten
Aluminum
Molten Aluminium
Aluminium Castings
Tundish
Al-5Ti-1B Master alloy Grain refiner
(Columnar Grains)
(Fine Equiaxed grain)
Example 1. Statistics of Grain refinement dataSystem(Train + Test)Sets
Variables Minimum Maximum Mean StandardDeviation
Binary Masteralloy addition
(48 +12)
Ti (%) 0 0.10 0.021 0.03B (%) 0 0.10 0.021 0.03Time (min)
0 120.00 40.69 43.86
GS (m) 98 610.00 276.08 158.16
Ternary Masteralloy addition
(120 + 30)
Ti (%) 0 0.10 0.029 0.028B (%) 0 0.10 0.029 0.028Time(min)
0 120 42.27 43.74
GS (m) 68 610.00 186.74 116.24
Total data(binary +Ternary alloy)
(168 +42)
Ti (%) 0 0.10 0.023 0.029B (%) 0 0.10 0.022 0.028Time(min)
0 120.00 36.18 43.06
GS (m) 68 610.00 266.54 185.55
N.S. Reddy et al., “Materials Science and Engineering A”, Vol. 391, 2005 Where
Validation of ANN model predictions with Al-1Ti-3B alloy
0 2 30
60 120 150
5
(0.1% )
N.S. Reddy et al., “Materials Science and Engineering A”, Vol. 391, 2005
0 30 60 90 120 150400
450
500
550
600
650
700
(0.1% M13)
Exp.Model 2Model 3%E
Holding Time (min)
Gra
in S
ize
(m
)
0
1
2
3
4
5
6
%E
(Mod
el 3
)
Where
(B=0.003 & T=0.001)
2 5 30
15012060
0
Validation of ANN model predictions with Al-1Ti-3B alloy1.67%
N.S. Reddy et al., “Materials Science and Engineering A”, Vol. 391, 2005
0 30 60 90 120 1506065707580859095
100(1.67% M13) Exp.
Model 2Model 3%E
Holding Time (min)
Gra
in S
ize
(m
)
0
2
4
6
8
%E
(Mod
el 3
)
Where
(B=0.05 & T=0.0167)
2 5 30
15012060
0
Predicted Grain refinement map of Al-7Si alloy
0.00 0.05 0.10 0.15 0.20 0.25 0.300.00
0.05
0.10
0.15
0.20
0.25
0.30
Al3Ti + TiB2
Ideal
grain
refin
er
(Ti:B
=1:1.
67)
Al3Ti
Al3Ti + TiB2AlB2
M51
M31
M13
M15
% Titanium
% B
oron 0
80.00160.0240.0320.0400.0480.0560.0640.0700.0
N.S. Reddy et al., Journal of Materials Performance, Vol 22, 2013
Color band numbers indicates grain size in m
Master AlloysAl – 1Ti – 3B (M13)Al – 3Ti – 1B (M31)Al – 3Ti – 3B (M33)Al – 1Ti – 5B (M15)Al – 5Ti – 1B (M51)
WhereIdeal Grain Refiner
Validation of Predictions with Experiments
0.000 0.005 0.010 0.0150.00
0.01
0.02
0.03
0.04
0.05(74m)1.67% M13
(654m) 0.1% M13
0.3% M03
1.67% M03(96m)
(241m)
M51M31
080.00160.0240.0320.0400.0480.0560.0
% Titanium
% B
oron
M13M15
WhereN.S. Reddy et al. Journal of Materials Engineering and Performance, 696,Vol 22(3), March 2013, 696-699
M2A3 BradleyM2A3 Bradley TomahawkTomahawk
Few Applications of Titanium alloyFew Applications of Titanium alloy
Golf-headGolf-head
F-22 RaptorF-22 Raptor Guggenheim MuseumGuggenheim MuseumBiomaterialBiomaterial
Ti-6Al-4V alloy as a Biomaterial
Where
Alloy contents: Al, V, Fe,
O & N
Heat treatment
Volume fraction of α-β phases
Ti ALLOY
(a)
Beta Volume Fraction is 29%
(b)
Beta Volume Fraction is 55%
(c)
Beta Volume Fraction is 23%
(d)
Beta Volume Fraction is 54%
Ti-6.19Al-4.05V-0.19Fe-0.12O-0.01N
[ 750 oC ] [ 900 oC ]
Ti-6.3Al-4.1V-0.21Fe-0.168O-0.005N
Example 2. : Phase volume fraction in Ti-6Al-4V alloy
Where
N.S. Reddy et al., “Computational Materials Science”, Vol. 107, 2015
Statistics of data used for modeling (Ti Alloys)
Test Samples Variables Minimum Maximum Mean
Al (%) 5.72 7 6.245V (%) 1.5 5 3.939Fe (%) 0.01 3.04 0.462O (%) 0.08 0.3 0.149N (%) 0.003 0.02 0.007Temperature ( oC) 600 1000 865.75
104 Training + 15 validation + 9 test data sets
Vol. α and β 0 100 47.79
N.S. Reddy et al., “Materials Science and Engineering A”, Vol. 434, 2006 Where
Performance of ANN Model
0 20 40 60 80 1000
20
40
60
80
100
0 20 40 60 80 1000
20
40
60
80
100
0 20 40 60 80 1000
20
40
60
80
100
0 20 40 60 80 1000
20
40
60
80
100
Predicted vs Experimental volume fraction of phase
(a)All data
R = 0.9982
Data Points Best Linear Fit
Expe
rimen
tal
Predicted
(b)Training
R = 0.9997
Data Points Best Linear Fit
Expe
rimen
tal
Predicted
(c)Testing
R = 0.9965
Data Points Best linear Fit
Expe
rimen
tal
Predicted
(d) Data Points Best Linear Fit
Validation
R = 0.9802Expe
rimen
tal
Predicted
WhereN.S. Reddy et al., “Materials Science and Engineering A”, Vol. 434, 2006
N.S. Reddy et al., “Computational Materials Science”, Vol. 107, 2015
Comparison of experiment & calculation in Ti–6.3Al–4.1V–0.21Fe–0.17–0.005N alloy quenched at (a) 700 C, (b) 815 C, (c) 900 C, and (d) in Ti–6.85Al–1.6V– 0.13Fe–0.17–0.001N quenched at 900 C.
N.S. Reddy et al., “Computational Materials Science”, Vol. 107, 2015
81/81.8
46/50.5
70/67.5
91/90.8
Unexpected Trend: Uncertainty
600 700 800 900 1000 11000
102030405060708090
100 (h)( tr = 933oC)
5.88Al-4.35V-3.04Fe-0.084O-0.004N Volu
me
fract
ion
of
pha
se
Tem perature oC
N.S. Reddy et al., “Computational Materials Science”, Vol. 107, 2015
Effect of alloying elements on phase volume fraction
Volume Fraction of Beta map of Ti-Al- V at different temperatures
Calculating Index of Relative Importance
N.S. Reddy et al., “Computational Materials Science”, Vol. 107, 2015
Ti–6.34Al–4V–0.19Fe–0.17O–0.01N at 980 C.
Experimental value for the alpha phase of the alloy 10.8% and predicted value is 11.8%
Validation of ANN model predictions
9 0 0
9 2 5
9 5 0
9 7 5
1 0 0 0
1 0 2 5
1 0 5 0
1 0 7 5
T i- 7 A l- 1 .5 V T i- 6 .8 5 A l- 1 .6 V T i- 6 .1 9 A l- 4 .0 5 V T i- 6 .3 3 A l- 4 V
Tr
ansu
s Te
mpe
ratu
re E x p e r im e n ta l P re d ic te d
Comparison of predicted and measured beta-transus temperatures for single-phase-alpha, near-alpha, and alpha/beta titanium alloys.
N.S. Reddy et al., “Materials Science and Engineering A”, Vol. 434, 2006
Thanks to Dr. Jeoung Han Kim for conducting experiments in POSCO
Where
Example 3.Low alloy steels data examples
Where
The range of low alloy steels (EN100 steels)
YS: Yield strength, UTS: Ultimate tensile strength, %El: % Elongation, %RA: %Reduction in Area. Total data sets are 140 and 112 were used for training and 28used for testing.
Composition (in wt.%)
Min. Max.
C 0.32 0.44
Si 0.19 0.37
Mn 0.33 1.51 S 0.01 0.042 P 0.02 0.038 Ni 0.56 1.08 Cr 0.21 0.57 Mo 0.11 0.25
Heat treatment variables
Cooling rate (oC/S) 2.8 118 Tempering Temperature (oC)
400 700
Mechanical properties
Y.S (MPa) 542 1194 UTS (MPa) 707 1295 % El 13 29 % RA 31 67 Impact Strength (J) 15 94
N.S. Reddy et al., “Materials Science and Engineering A”, Vol. 508, 2009 Where
Model Predictions with Test data (EN100 Steels)
1 2 3 4 5 6 7 8 9 10 11 120
200400600800
100012001400
YS
Samples
Predicted Actual
1 2 3 4 5 6 7 8 9 10 11 120.00.51.01.52.02.53.03.54.0
YS
Samples
%Error
1 2 3 4 5 6 7 8 9 10 11 12012345678
%EL
Samples
%Error
1 2 3 4 5 6 7 8 9 10 11 120
5
10
15
20
25
30
%EL
Samples
Predicted Actual
Where
Hypothetical Alloy YS UTS % EL %RA ISS5 (Actual) 760 931 19.50 49.50 20.00Carbon (0.4) 753 921 18.93 47.73 17.80Silicon (0.3) 737 906 19.78 48.90 20.80Manganese (1.51) 738 905 19.67 48.8 20.00Phosphorous (0.034) 736 901 19.89 48.95 21.94Nickel (0.89) 738 906 19.78 48.90 20.80Chromium (0.21) 737 904 19.00 48.00 11.36Molybdenum (0.16) 734 904 19.49 48.45 16.00Mn/S ratio (41.94) 739 907 19.45 48.38 18.75Cooling rate (3.1) 724 916 19.19 49.82 22.37Tempering Temp (550oC) 737 904 19.64 48.64 20.06
Comparison of properties for hypothetical alloy of a sample.
N.S. Reddy et al., “Materials Science and Engineering A”, Vol. 508, 2009 Where
Simulated effect of heat treatment parameters
0 20 40 60 80 100 120400
450
500
550
600
650
700(YS)
Cooling Rate (oC/s)
Tem
perin
g Te
mpe
ratu
re (o C
) 500.0660.0820.0980.011401300
0 20 40 60 80 100 120400
450
500
550
600
650
700(UTS)
Cooling Rate (oC/s)
700.0840.0980.0112012601400
0 20 40 60 80 100 120400
450
500
550
600
650
700(EL)
Cooling Rate (oC/s)
Tem
perin
g Te
mpe
ratu
re (o C
) 10.0014.0018.0022.0026.0030.00
5
12
34
6
12
34
6
12
34
65 5
N.S. Reddy et al., “Materials Science and Engineering A”, Vol. 508, 2009 Where
N.S. Reddy et al., “Computational Materials Science”, Vol. 101, 2015
Disadvantages of ANN modelingO
utpu
t
Input0
Over-fitted model “Real” model
cycles
Error
Overfitted model
holdout
training
Alternative Technology Lab, Graduate Institute for Ferrous Technology (GIFT), POSTECH, Korea
Difficult to design: There are no clear design rulesHard or impossible to train: When to stop and Over trainingDifficult to evaluate internal operation: It is difficult to find out whether, and if so what tasks are performed by different parts of the net
Variation of weights with varying iterations
Hidden layers
Iterations
Momentum Term and Learning rate
Artificial Neural Networks LiteratureMain text books:
– “Neural Networks: A Comprehensive Foundation”, S. Haykin(very good -theoretical)
– “Pattern Recognition with Neural Networks”, C. Bishop (very good-more accessible)
– “Practical Neural Network Recipe's in C++”’ T. Masters(emphasizing the practical aspects)
Review Articles: – R. P. Lippman, “An introduction to Computing with Neural Nets”’ IEEE
ASP Magazine, 4-22, April 1987.– A. K. Jain, J. Mao, K. Mohuiddin, “Artificial Neural Networks: A
Tutorial”’ IEEE Computer, March 1996’ p. 31-44.– H. K. D. H. Bhadeshia, "Neural Networks in Materials Science" ISIJ
International, Vol. 39, 1999, 966-979 – H.K.D.H. Bhadeshia: Performance of Neural Networks in Materials
Science, MST
Some Internet Resources for ANN
• http://www.faqs.org/faqs/ai-faq/neural-nets/part1/preamble.html(This FAQ provides Very Useful basic information for individuals who are new to the field of neural networks. The FAQ is divided into seven parts i.e., 1.Introduction, 2. Learning, 3. Generalization, 4. Books & data etc, 5. Free software, 6. commercial software and 7. hardware)
• www.inference.phy.cam.ac.uk/mackay/Bayes_FAQ.html (Bayesian methods for neural networks – FAQ)
• http://www.brain.riken.jp/labs/mns/geczy/Links-E.html (ANN around the world)• http://www.cs.nthu.edu.tw/~jang/nfsc.htm• http://www.msm.cam.ac.uk/phase-trans/abstracts/neural.review.html (Materials Science
& NN related literature) • http://calculations.ewi.org/vjp/secure/FNCoolingRate.asp (online Ferrite Number
calculations for stainless steel based on chemical composition)
Summary (broad)Ø Neural Networks are capable to predicting complex systems.
Ø Sensitivity Analysis examine the effects of input variables onthe output parameters, which is incredibly difficult to doexperimentally.
Ø Developed model is able to map the relation between
Ø inputs and outputs and
Ø between the output parameters though this information is notfed to the model.
Ø The model helps in reducing the experiments required and thereby saving a lot of money, material and manpower for designingthe new alloys for desired properties
I would like to acknowledge• Prof. Chong Soo Lee
Department of Materials Science and Engineering, POSTECH, Korea
• Dr. Jeoung Han Kim, Dr. You Hwan Lee, and Dr. Chan Hee Park, & Dr. YeomFormer students of Prof. C S Lee, Department of Materials Science and Engineering, POSTECH, Korea
• Dr. J. Krishnaiah, BHEL, Hyderabad, India• Yandra Kiran Kumar, Software Engineer,
Minneapolis
Swami Vivekananda Quote on Newton
The mind of the man is an infinite reservoir of knowledge, and all knowledge, present, past or future, is within man, manifested or non manifested.The object of every system of education
should be to help the mind to manifest it. For instance, the law of gravitation was within
man, and the fall of the apple helped Newtonto think upon it, and bring it out from within his mind
Thank you very much for your kind attention…