Post on 13-Jan-2016
description
Retail SLA Proposed Changes
RMS/TDTWG August 2008
Trey FeltonIT Account Manager
Weekend Transaction Volume vs. Current/Proposed Outage Windows
12
36
12 12
18 18
12
36
0
5
10
15
20
25
30
35
40
1 2 3 4
Week of Month
Ho
urs
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
To
tal
Tra
nsa
ctio
ns
Current Outage Window
Proposed Outage Window
Transactions: Sat - Sun (48-hr)
Transactions: 8am Sat - 8pm Sun(36-hr)
Transactions: 12pm Sat - 12amMon (36-hr)
Apr – Jun 2008
Sample Data
TRANS NAME COUNT(*)
TRANS NAME COUNT(*)
TRANS NAME COUNT(*)
TRANS NAME COUNT(*)
810_02 - 810_02 - 810_02 - 810_02 - 814_01 861 814_01 1383 814_01 1,683 814_01 822 814_02 148 814_02 156 814_02 212 814_02 64 814_03 2,737 814_03 3072 814_03 4,067 814_03 1,330 814_04 1,677 814_04 1165 814_04 3,879 814_04 2,202 814_05 1,775 814_05 964 814_05 3,244 814_05 1,469 814_06 210 814_06 72 814_06 379 814_06 169 814_07 599 814_07 338 814_07 642 814_07 43 814_08 295 814_08 259 814_08 505 814_08 210 814_09 248 814_09 163 814_09 491 814_09 243 814_11 - 814_11 - 814_11 - 814_11 - 814_12 54 814_12 59 814_12 68 814_12 14 814_13 34 814_13 28 814_13 70 814_13 36 814_14 - 814_14 - 814_14 - 814_14 - 814_15 - 814_15 - 814_15 - 814_15 - 814_16 1,500 814_16 1296 814_16 1,890 814_16 390 814_17 136 814_17 134 814_17 186 814_17 50 814_18 4 814_18 4 814_18 4 814_18 - 814_19 4 814_19 4 814_19 4 814_19 - 814_20 6,271 814_20 4786 814_20 9,038 814_20 2,767 814_21 7,297 814_21 5576 814_21 9,024 814_21 1,727 814_22 18 814_22 6 814_22 25 814_22 7 814_23 25 814_23 6 814_23 25 814_23 - 814_24 2,105 814_24 1997 814_24 2,589 814_24 484 814_25 1,209 814_25 792 814_25 2,606 814_25 1,397 814_26 - 814_26 - 814_26 - 814_26 - 814_27 - 814_27 - 814_27 - 814_27 - 814_28 268 814_28 167 814_28 459 814_28 191 814_29 400 814_29 173 814_29 451 814_29 51 824_02 833 824_02 63 824_02 905 824_02 72 867_02 4,242 867_02 3092 867_02 5,814 867_02 1,572 867_03 281,855 867_03 15053 867_03 558,249 867_03 276,394 867_04 3,835 867_04 830 867_04 8,062 867_04 4,227
318,640 41,638 614,571 295,931
May 3 and 4, 2008
Saturday 8 a.m. to Sunday 8 p.m.
Saturday 12:01 a.m. to 8:00 a.m.and Sunday 8:00 p.m.
to midnight
In Period 48 Hour Period Outside Period
Saturday 12 a.m. to Sunday 12 p.m.
In Period
For complete data set see: Retail SLA_Transaction Stats.xls
Weekend Transaction Volume during Outage Window Timeframes (1-year average vs 3-month average)
0
100,000
200,000
300,000
400,000
500,000
1 2 3 4
Week
# o
f T
ra
ns
ac
tio
ns
1-year Average (8 to 8)
3-month Avg (8 to 8)
3-month Avg (12 to 12)
Current Release Window
Proposed Release Window
Summary
• SLA Proposed Changes– Add 6 hours to Week 1
• Currently 12 hours
– Move 36-hr release window to Week 4 • Currently in Week 2—busiest week
– Add a 12-hour outage window in Week 5s (Mar/May/Aug/Nov)• Not in current SLA
• November will not be used for a release or week 5 outage
• Rationale– 5-week Months become general outline for major releases
(Mar/May/Aug/Dec)• Extra weekend gives additional outage window opportunity
– Remaining months will be reserved for minor releases
Additional Considerations
• For 2009– Move November Week 4 Outage to December Week 2
• To accommodate for Thanksgiving Holiday
– Move May Release up 1 week (to Week 3)• To accommodate for Memorial Day
– Analyze Quarterly Release concept
• For 2010– Reduce monthly minor release windows if possible
TDTWG Recommendations
• Limit Maintenance Window request to 15 hrs– Yes
• Reduce Release Outage to 30 hrs– Cannot Support
• Marketrak available 8am – Noon on Saturday– Researching, along with other data for MT Task Force
• Have 1 weekend a month with zero outages– With our current systems, ERCOT cannot support– Targeted for 2010
Proposed Release Calendar (2009)
Month Release DateJanuary 24/25February 21/22March 21/22April 25/26May 16/17June 27/28July 25/26August 22/23September 26/27October 24/25November N/ADecember 12/13
Proposed Maintenance/Release Windows for 2009
Week 1
Week 2
0
1200
0600
1800
2400
Mon ThursWedTue Fri Sat Sun
Availability – 99% 1
Maintenance
Week 3
Week 5
0
1200
0600
1800
2400
Mon ThursWedTue Fri Sat Sun
Availability – 99% 1
Maintenance
2
2
MAINTENANCE1st and 2nd Sunday– 6:00am until 9:00pm Sunday (15 hours)
Option 2: Sunday– 9:00am until 12:00am Monday (15 hours)3rd and 5th Sunday—8:00am until 8:00pm (12 hrs)
Option 2: Sunday—10:00am until 10:00pm (12 hrs)
RELEASE: 4th weekend– 12:00pm Saturday until 12:00am Monday (36 hours)Option 2: 10:00am Saturday until 10:00pm Sunday (36 hours)
Week 4
0
1200
0600
1800
2400
Mon ThursWedTue Fri Sat Sun
Availability – 99%
1
1
Release
2
2
Month Release DateJanuary 24/25February 21/22March 21/22April 25/26May 16/17June 27/28July 25/26August 22/23September 26/27October 24/25November N/ADecember 12/13
Actual Maintenance Outages (2008)
0
5
10
15
20
25
Jan (1) Jan (3) Feb (1) Mar (1) Mar (4) Apr (1) May(1)
May(3)
Jun (1) Jun (3) Jun (4)
Month (Wk)
Ho
urs
Hrs
Maintenance Outages (since Jan 1, 2008)
ERCOT Proposed SLA (18 hours)
Current SLA (12 hours)
Average = 9.8 hrs
40SQL Server DBs
70Linux servers
32Oracle DBs
449Windows servers
40SQL Server DBs
70Linux servers
32Oracle DBs
449Windows servers
TDTWG Proposed (15 hours)
0
5
10
15
20
25
30
35
40
45
50
Feb Aug Sep_2 Nov Jan Mar Jun
Current / Proposed SLA (36 hours)
TDTWG Proposed (30 hours)
Releases (2007-2008)
Average = 26.5 hrs
37% took more than 36 hrs
50% took more than 30 hrs
Marketrak
Marketrak Saturday Availability (2008)
0
20
40
60
80
100
120
2/16 3/8 3/29 4/19 5/10 5/31 6/21 7/12 8/2
Saturday Availability
Overall Availability
Backup Slides
Release Outage (apportioned time)
Start Time Sat 12:00pm
Shut down apps and clear queues 1.5 hrs
New Code Migration 4 - 6 hrs
Bring Systems Back Up / Tech Verification 1 hr
Production Verification 4 hrs
Rollback Contingency 1 - 4 hrs
Minimum Time Needed 11.5 – 16.5 hrs
Variations
• June Release: Planned 11 hrs, Actual 24.8 hrs (Prod verification failure; rollback)
• July Release: Planned 9 hrs, Actual 9 hrs
• Average Release has been 24 hrs (since Jan 1, 2008)
2008 Net Service Availability
99.00%
98.00%
97.00%
94.00%
95.00%
96.00%
93.00%
92.00% and below
100.0%
2008 Net Service Availability Through June 30, 2008
Transaction Processing TML MarkeTrak Frequency ControlRealtime Balancing Market
99.03% 98.79%99.14% 99.999%99.868%
Wholesale Total
94.730%
Transaction Processing SLA Target 99.9%
TML Availability Target99%
MarkeTrak Availability Target – 98%
Retail & Wholesale Operations System Operations
15 July, 2008 ERCOT Confidential
Releases (2008)
Actual Release Outages (2008)
0
5
10
15
20
25
30
35
40
Jan Feb Mar Apr May Jun Jul
Hrs Outage (Hrs)
Current / Proposed SLA (36 hours)
Average = 24 hrs
TDTWG Proposed (30 hours)