Change Control Process and Guidelines_V1 1

8/10/2019 Change Control Process and Guidelines_V1 1

http://slidepdf.com/reader/full/change-control-process-and-guidelinesv1-1 1/8

BIAS CLOUD SERVICES

Change Control Processes and

Guidelines

Author: Sri Vuyyuru

Creation Date: Aug 6, 2014Last Updated: Oct 6, 2014

Version: 1.1



CHANGE CONTROL PROCESSES AND GUIDELINES

1 Document Control

Change Record8

Date Author Version Change Reference

08/6/2014 Sri Vuyyuru 1.0 Initial Draft

10/06/2014 Anand Devaraj 1.1 Updates to Outage process

Reviewers

Date Author Version Change Reference




1 DOCUMENT CONTROL ................................................................................................................................... 2

CHANGE RECORD ........................................................................................................................................................... 2

REVIEWERS ................................................................................................................................................................... 2

2 INTRODUCTION ............................................................................................................................................. 4

3 CHANGE CONTROL PROCESS AND GUIDELINES .............................................................................................. 4

3.1 WHEN IS A CHANGE REQUEST (CR) CREATION REQUIRED? ........................................................................................ 4

3.2 STANDARD CHANGE WINDOWS ........................................................................................................................... 4

3.3 PLANNING ....................................................................................................................................................... 4

3.4 SUBMITTING THE CR .......................................................................................................................................... 6

3.5 EXECUTING THE CR ........................................................................................................................................... 7

3.6 VALIDATION OF CR ............................................................................................................................................ 7

3.6.1 Implementer/Primary Validator............................................................................................................... 7

3.6.2 Application Team Validator ..................................................................................................................... 7

3.6.3 Secondary DBA Validator ......................................................................................................................... 7




2 IntroductionThis document explains in the complete process to be followed while creating, submitting and executing

the Change requests at Home Depot Exadata and Non-Exadata environments.

3 Change Control Process and GuidelinesThis document is to help minimize inconsistency while planning, submitting and executing on Change

Requests.

3.1

When is a Change Request (CR) creation required?

1. Any change in production needs a CR with approvals. May it be a database bounce or resizing a

datafile, CR is mandatory. If the situation is in eleventh hour, a different type of CR(Break fix)

can be opened. In these cases, we have to open a HPSM ticket first, to keep a track of the

activity, complete the fix in production and then inform Sri/Ajeet the next day so that they canopen a break fix CR within the next 48 hours window. It would be considered a process

violation if a break fix CR is not opened for a fix in Production within 48 hours

2. Any changes pertaining to process improvements, say for example crontabs/backups and any

other scripts can be made by BIAS, but only after informing the DBA owner of that database and

taking his approval. FYI, I have included the DBA contact information in the password sheet.

3. Any observations/inferences that are made on a daily basis needs to be delivered to the DBA

owner of that particular database. They would definitely appreciate it if we work on some fixes

and let them know what analysis we did. Say for example, we could not start the cluster sincethere is some issue with asmlib drivers. In those cases we work with the UNIX team and get the

issue fixed and let the THD DBA team know what was done to fix the issue

3.2 Standard Change Windows

Standard change windows is 00:00 ET to 06:00 ET Monday through Thursday.

Exception: Basic infrastructure, i.e. mainframe changes can go in starting at 16:00 on Sunday.

Anything outside these standard change windows will need director level approval.

3.3 Planning

Please plan for your change to follow the normal approval process. This may add a little lead time, but

will minimize the need for minute approvals via Hi Priority emails.

The following are the key times to note.




TIME

(ET)

11:00 AM Report on CR’s pending approval is collected for review at 2pm ET.

01:00 PM Cut off time for any changes in the next change window starting at 8 pm Today.

All changes must be fully approved by this time.

02:00 PM Dept. 876 Meets to review/approve pending changes. As this is past the 1 PM cut

off for today’s 8pm change window, so any changes approved in this meeting can

be implemented after 8pm tomorrow.08:00 PM Start of the next Change window. Runs through 7:59 PM next day.

NOTE : Changes approved for the Friday 8pm change windows covers any change

from 8:00pm Friday thorough 7:59pm Monday

Day Of

Implementation

Monday Tuesday Wednesday Thursday Friday Saturday Sunday

Before 8 PM ET By 1pm

ET

Friday

By 1pm

ET

Monday

By 1pm ET

Tuesday

By 1pm ET

Wednesday

By 1pm

ET

Thursday

By 1pm

ET

Previous

Friday

By 1pm

ET

Previous

Friday

After 8 PM ET By 1 PM

ET

Monday

By 1PM

ET

Tuesday

By 1PM ET

Wednesday

By 1pm ET

Thursday

By 1pm

ET

Friday

By 1pm

ET

Previous

Friday

By 1pm

ET

Previous

Friday

Table 1 IT Dead Lines for CR to be FULLY approved.

Day Of

Implementation

Monday Tuesday Wednesday Thursday Friday Saturday Sunday

Before 8 PM ET By 9am

ET

Thursday

By 9am

ET

Friday

By 9am ET

Monday

By 9am

ET

Tuesday

By 9am ET

Wednesday

By 9am

ET

Thursday

By 9am

ET

Thursday

After 8 PM ET By 5 PM

ETThursday

By 5 PM

ETFriday

By 5 PM ET

Monday

By 5 pm

ETTuesday

By 5 pm ET

Wednesday

By 5 pm

ETThursday

By 5 pm

ETThursday

Table 2 DBA Guide Lines for CR Approvals – You changes should be submitted for approval by




All implementation plans should be reviewed by a second DBA before being submitted for approval.

During implementation plan review you should be validating/checking the steps, methodology and back

out steps. Validators’ email approval should be attached to the CR when submitted.

3.4

Submitting the CR

Please ensure the following guidelines are followed while creating a change request.

1. The Short description describe the change being made at a high level

2. The Describe field contains more details about the change including

a. The Application Name.

b. If CR is supporting another change or application rollout, please make note of this as

well.

3. The Justification field should contain the business reason behind for the change.

a. If this is an emergency, then it should contain the emergency justification

b. Include details around WHY are we making this change.

c.

If the CR is a last minute request or a freeze breaker situation being pushed by theapplication team, then make note of the details and also attach the justification email

from the application teams Management.

4. The Watchers List should include

a. The Application Team Management (NOT Lead or Contractors)

b. If on Exadata then add “[email protected]”.

c. The primary DBA for that application if you are not the submitter or validator.

d. The application team Manager.

5. Down Time

a. “YES” – if the change will cause an application downtime that is outside its normal

maintenance window.b. “NO” – if no outage is required or if the required application outage will be within a

maintenance window and the business does not expect the application to be available

6. Configuration Items

a. Please include the Application as the primary Configuration Item

b. Server/Database/Instance/Schema can also be added to the degree you feel

appropriate. At least the server should be added.

When you are requesting approval for the CR, there are some standard questions you can expect to be

asked. If you can proactively answer these questions then you should have smooth sailing to

Approvalville.

1. Has this been tested in lower lifecycles

2. Is the application team aware of the change and doing validation?

3. If down time required, has the business approved the outage

4. New Code or features being enabled

a. Is there any new SQL to be reviewed

b. Capture performance information before and after the change




c. Was there any performance testing

3.5 Executing the CR

Always send the START, VALIDATE and COMPLETE emails to ITO_CHANGE_COORDINATION

group ([email protected]) and copy all the people in the

watchers list including Frank Rusconi. For emergency CRs (Breakfix), always explicitly copyFrank Rusconi along with the standard THD DBA distribution lists.

Stick to the implementation plan.

Stay within the planned window

Before executing the CR send an email with subject “Start of CR <CR#>” to ITO Change

Coordination and copy Frank Rusconi.

Back out if things do not go as expected. No flying by the seat of your, or anyone else’ pants.

At end of the change validation emails should be sent to your manager.

DBA’s can forward the application validation email.

Include in validation the actual start/stop time of the change.

Please include “CRVAL:” and CR number in subject line.

If application or anyone wants to deviate from the plan, engage your manager

3.6 Validation of CR

Before validating the CR send an email with subject “Validation of CR <CR#>” to ITO Change

Coordination and copy Frank Rusconi.

As we have been discussing we need to ensure we thoroughly validate any change request. We require

a minimum of two DBA’s to validate every change. The second validator could be THD or Contractor but

will be held equally accountable for the change.

If the activity is on the Exadata Platform, then one of the DMA’s should be validating any

create/change/drop of database environments.

3.6.1

Implementer/Primary Validator

The implementer is primarily responsible for the validation of the change. They should have any

changes also validated by the application team to ensure the health and successful roll out of any

changes.

3.6.2 Application Team Validator

When changes are made that can impact the application, or changes that required the application to be

stopped for the change to be made there must be some application validation performed. The

application team member should be identified in the CR. Completion of validation from the application

team should be via email.

3.6.3

Secondary DBA Validator

The secondary DBA validator is responsible for ensuring the environment that underwent the change is

in a healthy, accessible, and valid state at the end of the change.

mailto:[email protected]

mailto:[email protected]




The following set of Validation steps should be included in every Change Request and performed by at

least the Secondary Validator.

Validation Steps

1. Confirm the implementation plan is completed successful and/or backed out.

2. Ensure the database environment and objects are connectable/accessible

3. Validate state of objects in the environment. Check for invalid objects etc.

a. Indexes Valid

b. Packages, Views, Procedures etc….

c. Ensure that any object warning statuses/messages have been resolved or there is an

appropriate plan to resolve at the earliest possible window

4. No errors showing up in log files or audit files

a. Database and server error reports as appropriate

b. Ensure there are no errors in monitoring software

5. Replication Running where applicable (SQL, Golden Gate, ADG …)

a. Ensure replication is transferring data.

Change Control Process and Guidelines_V1 1

Documents

Transcript of Change Control Process and Guidelines_V1 1