Data Analytics National Academy of Indian Railways Workshop on 'New Financial Initiatives' in Indian...
-
Upload
barnaby-jenkins -
Category
Documents
-
view
222 -
download
1
Transcript of Data Analytics National Academy of Indian Railways Workshop on 'New Financial Initiatives' in Indian...
Data Analytics
National Academy of Indian RailwaysWorkshop on 'New Financial Initiatives' in Indian Railways
ForPrincipal/ Coordinating FA&CAOs
23 December 2015
Data Analytics - NAIR 2
There areLies
Damned lies; and?Statistics
23-Dec-15
Data Analytics - NAIR 3
Data AnalyticsWhat do you understand by it?
23-Dec-15
Data Analytics - NAIR 423-Dec-15
An example
Data Analytics
Digital Dashboard
Use of Numbers
Data on Indian Railways
Reorganising Statistical Units
Analytics Ecosystem on IR
A word of caution
Plan for the session
Data Analytics - NAIR 5
An example from RailwaysNot Financial Data
23-Dec-15
Data Analytics - NAIR 6
Accidents on Indian RailwaysAn illustration
23-Dec-15
Data Analytics - NAIR 7
Cause 2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15Failure of Railway
staff84 87 83 63 56 52 46 51 62
Failure of other than Railway staff
87 81 68 75 57 63 59 56 58
Failure of equipment
7 9 0 6 5 5 6 4 2
Sabotage 8 7 13 14 16 6 3 3 3Combination of
factors1 0 4 1 3 1 0 0 0
Incidental 7 8 5 4 4 3 7 3 8Could not establish
1 2 4 2 0 1 0 0 0
None Held 0 0 0 0 0 0 1 0 2Awaited 0 0 0 0 0 0 0 1 0
Total 195 194 177 165 141 131 122 118 135
Cause wise Analysis of Consequential Train Accidents over IR(2006-07 to 2014-15)
23-Dec-15
Data Analytics - NAIR 8
2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15
195 194
177
165
141
131
122118
135
Accidents on Indian Railways
23-Dec-15
Data Analytics - NAIR 9
2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15
195 194
177
165
141
131
122118
135
Accidents on Indian Railways
23-Dec-15
Data Analytics - NAIR 10
2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15
195 194
177
165
141
131
122118
135
Accidents on Indian Railways
23-Dec-15
Data Analytics - NAIR 11
2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15
8 8
139
59
64 5
96100
85
80 80
55
4953
63
4 53 2 2
4
9 7 67
12
75 5
75 4
68
47
41 2
03
5
72
6562
65
48
54 53
4750
Collisions Derailments Fire Manned Level Crossing Accidents Miscellaneous Accidents Unmanned Level Crossing Accidents
Consequential Accidents over the years
23-Dec-15
Data Analytics - NAIR 12
Collisions5%
Derailment48%
Fire3%
Manned Level Crossing Accidents
4%
Unmanned Level Crossing Accidents
37%
Miscellaneous Accidents2%
Accidents by type in 2006-2015
23-Dec-15
Data Analytics - NAIR 13
2006-07
2007-08
2008-09
2009-10
2010-11
2011-12
2012-13
2013-14
2014-15
96100
8580 80
55
4953
63
Derailments
2006-07
2007-08
2008-09
2009-10
2010-11
2011-12
2012-13
2013-14
2014-15
72
6562
65
48
54 53
4750
Unmanned Level Crossing Accidents
Consequential Accidents over the years
23-Dec-15
Data Analytics - NAIR 14
2006-07
2007-08
2008-09
2009-10
2010-11
2011-12
2012-13
2013-14
2014-15
96100
8580 80
55
4953
63
Derailments
2006-07
2007-08
2008-09
2009-10
2010-11
2011-12
2012-13
2013-14
2014-15
72
6562
65
48
54 53
4750
Unmanned Level Crossing Accidents
Consequential Accidents over the years
23-Dec-15
Data Analytics - NAIR 15
Seasonal variations?
A widely prevalent belief
Specific types of accidents have higher frequency in different times of the year
23-Dec-15
Data Analytics - NAIR 16
April May June July August September October November December January February March
120
131
115109
136
102
125
103
119 117
10398
Month-wise distribution of Total Accidents in the period 2006-2015
23-Dec-15
Data Analytics - NAIR 17
April May June July August September October November December January February March
120
131
115109
136
102
125
103
119 117
10398
Month-wise distribution of Total Accidents in the period 2006-2015
23-Dec-15
Data Analytics - NAIR 18
April May June July August September October November December January February March
120
131
115109
136
102
125
103
119 117
10398
Month-wise distribution of Total Accidents in the period 2006-2015
23-Dec-15
Data Analytics - NAIR 19
January February March April May June July August September October November December
117
10398
120
131
115
109
136
102
125
103
119
Month-wise distribution of Total Accidents in the period 2006-2015
23-Dec-15
Data Analytics - NAIR 20
July August September October November December January February March April May June
109
136
102
125
103
119 117
10398
120
131
115
Month-wise distribution of Total Accidents in the period 2006-2015
23-Dec-15
Data Analytics - NAIR 21
May-June
23-Dec-15
Data Analytics - NAIR 22
July-September
23-Dec-15
Data Analytics - NAIR 23
December - January
23-Dec-15
Data Analytics - NAIR 24
March – AprilOctober - November
23-Dec-15
Data Analytics - NAIR 25
Month-wise distribution of Different types of Accidents in the period 2006-2015
AprilMay
JuneJuly
August
Septem
ber
October
November
December
January
Febru
aryMarc
h
46
65
52
39
46
27
33
45
3937
45
42
Unmanned Level Crossing Accidents
AprilMay
JuneJuly
August
Septem
ber
October
November
December
January
Febru
aryMarc
h
52
46
5457
76
60
79
41
5452
4446
Derailment
23-Dec-15
Data Analytics - NAIR 26
Month-wise distribution of Different types of Accidents in the period 2006-2015
AprilMay
JuneJuly
August
Septem
ber
October
November
December
January
Febru
aryMarc
h
46
65
52
39
46
27
33
45
3937
45
42
Unmanned Level Crossing Accidents
AprilMay
JuneJuly
August
Septem
ber
October
November
December
January
Febru
aryMarc
h
52
46
5457
76
60
79
41
5452
4446
Derailment
23-Dec-15
Data Analytics - NAIR 27
Collision 3%
Derailment 52%
Fire 3%
MLC Accident 8%
Miscellaneous Accident 2%
UMLC Accident 32%
Monsoon-Gangetic Plain
Collisions5%
Derailment48%
Fire3%
Manned Level Crossing Ac-cidents
4%
Unmanned Level Crossing Accidents
37%
Miscellaneous Accidents2%
Accidents by type in 2006-2015
Derailment 72%
Fire 6% UMLC Accident
22%
Monsoon-East
Collision 3%
Derailment 41%
Fire 6%
MLC Accident 3%
UMLC Accident 47%
Monsoon-West
Collision 8%
Derailment 39%
Fire 3%
MLC Accident 11%
Miscellaneous Accident 3%
UMLC Accident 37%
Fog-Gangetic Plain
Collision4%
Derailment49%
Fire4%
MLC Accident4%
Miscellaneous Ac-cidents
2%
UMLC Accidents37%
Harvest Seasons
23-Dec-15
Data Analytics - NAIR 28
Failure of Railway staff50%
Failures other than of Railway staff35%
Failure of equipment 5%
Sabotage4%
Combination of factors1%
Incidental4%
Could not be estb.1%
None Held0%
Causes of Accidents 2006-15
23-Dec-15
Data Analytics - NAIR 29
2000-2001 2001-2002 2002-2003 2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 2008- 2009 2009- 2010
2010 - 2011
2011 - 2012
2012-2013 2013-2014 2014-2015
293
249
186161
119 12085 88 75 63 56 52 46 51 60
109
103
118
107
78 86
84 8176
7557 63
59 5758
33
24
18
18
14 8
9 9
06
5 56 3
3
19
14
10
18
4 6
8 7
13 14
16 63 3
3
4
0
2
2
1 0
1 0
4 1
31
0 00
11
20
15
17
16 11
7 85
4
43
7 48
4
5
2
2
2 3
1 14
2
01
1 01
0
0
0
0
2
Causes of Accidents 2006-2015: Number wise break-up
Failure of Railway staff
Failure of other than Railway staff
Failure of equipment
Sabotage
Combination of factors
Incidental
Could not be estb.
None Held
23-Dec-15
Data Analytics - NAIR 30
2000-2001 2001-2002 2002-2003 2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 2008- 2009 2009- 2010
2010 - 2011
2011 - 2012
2012-2013 2013-2014 2014-2015
293 249186
161 119 12085 88 75
63 56 52 4651 60
109 103118
107 7886
84 8176
75 5763
59
57 58
33 24 1818
148 9 9
0 65
5 63
3
19 14 1018 4 6 8 7
1314
16
6 33
3
40 2 2
10 1 0 4
1 3 10
0
0
11 20 15 17 16 11 7 8 5 4 4 37
48
4 5 2 2 2 3 1 1 4 2 0 1 1 01
0 0 0 2
Causes of Accidents 2006-2015: Percentage break-upFailure of Railway staff Failure of other than Railway staff Failure of equipment Sabotage Combination of factors Incidental Could not be estb. None Held
23-Dec-15
Data Analytics - NAIR 31
Causes of Accidents 2006-2015: Trends
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008- 2009
2009-
2010
2010 -
2011
2011 -
2012
2012-2013
2013-2014
2014-2015
23%25%
34% 33% 33%
37%
43%42%
43%
45%
40%
48% 48% 48%
43%
Failure of other than Railway staff
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008- 2009
2009-
2010
2010 -
2011
2011 -
2012
2012-2013
2013-2014
2014-2015
62%60%
53%
50%51% 51%
44%45%
42%
38%40% 40%
38%
43%44%
Failure of Railway staff
23-Dec-15
Data Analytics - NAIR 32
Causes of Accidents 2006-2015: Trends
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008- 2009
2009-
2010
2010 -
2011
2011 -
2012
2012-2013
2013-2014
2014-2015
23%25%
34% 33% 33%
37%
43%42%
43%
45%
40%
48% 48% 48%
43%
Failure of other than Railway staff
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008- 2009
2009-
2010
2010 -
2011
2011 -
2012
2012-2013
2013-2014
2014-2015
62%60%
53%
50%51% 51%
44%45%
42%
38%40% 40%
38%
43%44%
Failure of Railway staff
23-Dec-15
Data Analytics - NAIR 33
Failure of Railway staff67%
Failure of other than Railway staff
7%
Failure of equipment 7%
Sabotage11%
Combination of factors
1%
Incidental7%
Could not estb.1%
None Held0%
Awaited0%
DerailmentsCause-wise Analysis(2006-07 to 2014-15)
23-Dec-15
Data Analytics - NAIR 34
Failure of Railway staff
79%
Failures other than of Railway staff
19%
Combination of factors
2%
Manned LC AccidentsCause wise Analysis(2006-07 to 2014-15)
23-Dec-15
Data Analytics - NAIR 35
Unmanned LC AccidentsCause wise Analysis(2006-07 to 2014-15)
23-Dec-15
Failure of Railway Staff1%
Failure of other than Railway Staff99%
Data Analytics - NAIR 36
Preventive Measures To Curb UMLCs
UMLC By Closure/ Merger/Subway By Manning Total
2010-11 800 434 12342011-12 481 777 12582012-13 700 463 11632013-14 777 325 11022014-15 721 427 1148
Total 2010-15 3479 2426 5905
2015-16 (Up to July’15) 206 65 271
As on 01.04.2015, there were approximately 29487 LC on IR out of which 19047 (64.6%) are Manned and 10440 (35.4%) are Unmanned.
Progress made in elimination of LC in last 5 years & up to July, 2015 by Closure, Merger, Subway and Manning are as under
MLC 2010-11 2011-12 2012-13 2013-14 2014-15 Total 2010-15 2015-16 (Up to July’15)
By Closure 133 225 257 301 310 1226 89
23-Dec-15
Data Analytics - NAIR 37
Data Analytics
23-Dec-15
Data Analytics - NAIR 38
What is data analytics?
Derive meaning from data by incorporating
Statistics
Mining of data
Visualisation
23-Dec-15
Extracting actionable data in a manner that supports
decision-making
Data Analytics - NAIR 39
What is data analytics?
Massive expansion in the ability of computers to handle data leading to certain
other crucial items now becoming possible:
Machine Learning
Database engineering
All this to solve complex problems
23-Dec-15
Data Analytics - NAIR 40
What is data analytics?
Caters to Big Data with a view to capturing major and seemingly minor relationships of performance indices
Caters to day to day reporting needs
Caters to ad hoc querying
Provides analytical dashboards and alerts
Provides comprehensive information and actionable insights for taking informed decisions
23-Dec-15
Data Analytics - NAIR 41
What has changed?
More powerful computers
Methods to obtain data directly from source
Data feeds available from diverse sources
Algorithms to extract information from unstructured data
23-Dec-15
Data Analytics - NAIR 42
4 Vs of Big Data
23-Dec-15
• Scale of data • Different forms of data
• Uncertainty of data• Analysis of data
Volume Variety
VeracityVelocity
Data Analytics - NAIR 43
Insights into the data
Traditional business intelligence
What happened?
Diagnostic analytics
Why is it happening?
Predictive
What will happen in future?
Prescriptive
What should we do?
23-Dec-15
Communication Flow in Analytics
Service request by User
Request assigned a priority number
Analyst, with assistance from
User, creates functional
requirements
Design team selects dashboard
format
Implementation team implements
the selected model
Feedback is sent to Analyst for
recalibration/modification
Continuous monitoring and
upgradation
Analytics Team
Functional User
Functional Analyst
providing liaison between User and Technology Team
Technology team
Software Programmers
to extract data from databases and prepare it for analytical models
Data Scientists
to decide choice of model and provide interpretation of analytical output in functional terms
Statistical Analysts/Econometricians
for developing appropriate logical and physical data models
Quantico Analysis Team
testing the quality of product from non-functional point of view
Data Analytics - NAIR 46
Present
23-Dec-15
Extract data Decide action
Delays in access to data
Limited electronic filing and diary
Limited integration with policy and financials
Multiple manual processes to enter, correct, extract and analyze data
Delayed decision making based on limited information and insights
Action?
Data Analytics - NAIR 47
Possibilities in future
23-Dec-15
Extract data Decide action
Shorter duration for analysis
Shorter time-lapse for decision
Quality of data analysis provides decision support
Continuous feedback on decisions taken
Rapid course correction if needed
Easier launching of new initiatives
Action
Features determining analytics maturity
Use of real time data
Electronic filing and diary in a centralised model for each business
Information integrated across the organization
Advanced modeling techniques used to evaluate across functional areas
Fully simulated business operations to evaluate decisions
Data Analytics - NAIR 49
Digital DashboardA crucial tool
23-Dec-15
Data Analytics - NAIR 5023-Dec-15
Data Analytics - NAIR 5123-Dec-15
Data Analytics - NAIR 5223-Dec-15
Data Analytics - NAIR 53
The Digital Dashboard
Visual depictionTimely alerts
Multi deviceDrilldown capability
23-Dec-15
Data Analytics - NAIR 54
The Digital Dashboard
Relevance
Right insight at the right time
Convenience
Readily available when needed
Validation
Checked for errors and model validity
23-Dec-15
Data Analytics - NAIR 5523-Dec-15
Data Analytics - NAIR 56
Use of numbersStatistician’s delight
23-Dec-15
Data Analytics - NAIR 57
la loi des grands nombres(Law of Large Numbers)
If the expected result of an experiment is random
And the experiment is repeated a large number of times
Then the results tend to stabilise over a period of time
23-Dec-15
Data Analytics - NAIR 5823-Dec-15
Data Analytics - NAIR 5923-Dec-15
Data Analytics - NAIR 60
Statistician’s Delight
23-Dec-15
Data Analytics - NAIR 61
Dependent and Independent variables
y = a0x0+ a1x1+ a2x2+ a3x3+ ε
23-Dec-15
Data Analytics - NAIR 62
Dependent and Independent variables
Throughput = f (Availability ofpowerreliable signalling,Crew,TXR cleared RS,efficient controllers,motivated staff and officers,reliable equipment of all types,
etc.)23-Dec-15
Data Analytics - NAIR 63
Optimisation
Max. [y – (ax+ by+ cz)]
23-Dec-15
Data Analytics - NAIR 64
Optimisation
Maximise Throughput
Subject to: loco failures, signal failures, crew non-availability, failing rolling stock, absence of crucial staff, demotivated staff and officers, other asset failures, etc.
23-Dec-15
Data Analytics - NAIR 65
Data on Indian RailwaysOur position
23-Dec-15
Data Analytics - NAIR 66
IDigitally CapturedDigitally Reported
IIManually CapturedDigitally Reported
IVDigitally CapturedManually Reported
IIIManually CapturedManually Reported
Quadrant I is ideal.However, on IR data can be available to varying degrees in all Quadrants.
Recording of data on IR
23-Dec-15
Data Analytics - NAIR 67
Data on Indian Railways
Several IT projects on IR have potential for data analytics:
Expenditure side application of PRIME/AFRES/IPAS/ e-Recon/ARPAN etc.
Data Warehouse for PRS
Data Warehouse for UTS
Data Warehouse for FOIS
Control Office application
Integrated Coach Monitoring System (ICMS)
Loco Sheds Management System (LSMS)
Software for Locomotive Asset Management
(SLAM) for Electric Loco Sheds
Track Management System (TMS)
MIS for Land & Amenities on IR
Traction Distribution Management System (TDMS)
Signaling Maintenance Management System (SMMS)
23-Dec-15
Data Analytics - NAIR 68
Data on Indian Railways
Tickets
reserved / unreserved
Freight
RRs
Train movement
control office
Crew
movement and lobbies
Maintenance
P Way and fixed assets
Maintenance
Rolling stock
Materials
purchase and depots
HR
manpower deployment and staff welfare
23-Dec-15
Costing and accounting – all of the above
Data Analytics - NAIR 69
Reorganising Statistical UnitsChange
23-Dec-15
Data Analytics - NAIR 70
Reorganizing Statistical Unit to Analytics Unit
Dynamic officers with adequate field experience and IT knowledge should lead the team
Identify comparatively younger staff and train them in data handling and analysis
Involve young JE level staff of EDP centres as part of the Unit
Unit should provide analyzed inputs to all departments
23-Dec-15
Data Analytics - NAIR 71
Analytics Ecosystem on IRSomething new
23-Dec-15
Data Analytics - NAIR 72
An Analytics Ecosystem for IR
23-Dec-15
Data Analytics - NAIR 73
Possible Areas for Analytics
Predictive maintenance
Dynamic pricing
Evaluating different marketing strategies
Improving capacity utilization/route congestion
Real time management of linear and rolling stock assets
23-Dec-15
Data Analytics - NAIR 74
Possible Areas for Analytics
Expenditure on Fuel consumption
Correlation with changes in fuel prices, Specific Fuel Consumption of locomotives, route electrification and transport output
RCD wise energy rate and consumption data
Automatic alerts in case the inventory of fuel increases to more than 10 days
Expenditure on Traction Bills
Correlation with prices of traction, Loco-wise and EMU coach wise energy consumption and transport output
TSS-wise energy rate and consumption data
Automatic alerts in cases of diesel locos running under wires23-Dec-15
Data Analytics - NAIR 75
Don’t rely too much on predictive capacity of current data
Some theoretical stuff
23-Dec-15
Data Analytics - NAIR 76
An economist can affect the economyas much as
The weatherman the weather
23-Dec-15
Data Analytics - NAIR 77
Logical fallacy
Post hoc ergo propter hoc?
The rooster crows before sunrise.
Ergo the rooster causes sunrise.
Cum hoc ergo propter hoc?
Rate of deaths in India due to TB increased even as civilian deaths during war in Iraq was increasing
Ergo War in Iraq was the cause of increase in TB death rate in India
23-Dec-15
Data Analytics - NAIR 78
Chaos Theory
Chaos
When the present determines the future,
but the approximate present does not approximately determine the future
23-Dec-15
Data Analytics - NAIR 79
Chaos Theory
Does the flap of a butterfly’s wings in Brazil set off a tornado in Texas?
The butterfly does not power or directly create the tornado
The flapping of wings by the butterfly is a set of initial conditions which are followed by the tornado – the final result
Had the butterfly not flapped its wings, the trajectory of the system might have been vastly different
23-Dec-15
For want of a nail the shoe was lost.For want of a shoe the horse was lost.For want of a horse the rider was lost.
For want of a rider the message was lost.For want of a message the battle was lost.For want of a battle the kingdom was lost.And all for the want of a horseshoe nail.
Data Analytics - NAIR 80
Thank You
23-Dec-15