Lucas TOI – CUC Upgrade and Rollback

© 2006 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 1

Lucas TOI –CUC Upgrade and Rollback

Monica Price Cohen


Agenda

L2 Upgrades Upgrade Switchover

Rollback Switchover Partial Data Migration, gotchas

Troubleshooting


L2 Upgrade L2 Upgrade consists of 2 Phases Upgrade and switchover, same a CCM

GUI Upgrade - On the OS Administrator page, pick “Software Upgrades | Install/Upgrade”

CLI Upgrade - utils system upgrade*

Manual Switchover Needed if you choose not to do automatic reboot GUI – On the OS Administrator page, pick “Setting |

Version”. Click button to switch version. CLI Switchover – utils system switch-version Beware – A result is returned almost immediately

from both the CLI and GUI as successful. In reality, switchover is still in progress.


Upgrade Fresh install on passive partition Most of the system is function during this phase

For Cores: The majority of the CCM user data is migrated at

this time No User CRUD should be performed at this time.

New binaries, new database installed Unlike CCM, there is no CUC data migration during this

phase Locales will be carried forward for ES and maintenance

releases only Major and minor upgrades will require all locales be

reapplied (similar to Unity)


L2 Switchover Connection services are stopped A full CUC database migration is performed between the

active and passive partitions Different than CCM, which migrates only dynamic data at

this time On a co-res system connection will stop it’s services

first, do the migration and then CCM will stop it’s services for it’s data migration.

Expected down time is about 20 mintues. Results may vary depending on user count, etc.

A grub swap is performed at the very end and the box is rebooted to the new partition


Rollback Reverts the system back to the most recent version prior

to upgrade. Only mailbox (message) data is migrated. User data is

not migrated during Rollback Mostly the same code used for switchover during

upgrade. Only possible to revert one version in the past


Rollback Example: System is upgraded

from version A -> B After the reboot A will

on the passive partition and B will be on the active partition.

During upgrade from B->C, B will be still be active and C will be installed on the partition where A used to reside.

After reboot, a rollback to B is your only option.

C

After Reboot

B

C

During Upgrade B->C

B

A

After Reboot

B

A

During Upgrade A->B

B

Active

Passive

Rollback Possible

Rollback Possible


Rollback gotchas CUC’s priorities during rollback

1. Get the system back to a working state on previous version.

2. Make a best effort to retain messages and message states (New, Saved, Deleted)

Two types of successful rollback: Gold Star Rollback - Successful CUC mail store

database migration (achieves 1 and 2) Silver Star Rollback - Failed CUC mail store

migration, but rollback completed (achieves 1)


Gold Star Rollback Users inboxes will contain messages in the same state

as before the rollback No mailbox data (messages, message state) should be

lost One caveat – If a user was created after upgrade, then

their messages will exist on the rolled back system, but their mailbox, subscriber objects will not. So these messages are forward to the “Undeliverable Messages” mailbox.


Silver Star Rollback In the event that an error is encountered while migrating

message data, the mail store migration will be aborted and the message database will be restored to it’s original state at the time before upgrade.

The users mailbox size, and counts will be restored to the pre-upgrade state. Message received after upgrade will be lost Message deleted after upgrade will seem to reappear Many of these “reappearing” messages will be

missing their wav attachment. The message header has been returned, but the wav attachment was discarded after the user deleted the message on the upgraded system. The TUI handles this case, but it still can be unpleasant for the end user.

Not a desirable situation, but better than a busted system.


Troubleshooting With root access: Install log are located at

/common/log/install for both CCM and CUC install logs Access these files through RTMT.

Trace and Log Central | Remote Browse | Trace Files. Click next until you see “Select System

Service/Applications” on the tab Scroll down towards the bottom of list Check Upgrade and Install logs and click finish

cuc-*.* files show the details for upgrade and switchover


Troubleshooting - Upgrade install_log_<date/time>.log shows the overall progress

of the upgrade upgrade-results.xml shows the results of the upgrade

(not switchover) and a good place to start

Failed upgrade-result.xml

<upgrade-result> <result>error</result> <description>Unity installation failed. RPM

returned 15</description> <keytag>errors.system</keytag> <result>error</result> <description>Failed to exec command: "/

partB/usr/local/bin/base_scripts/env.sh …"</description>

…


Troubleshooting – Upgrade (cont) Same as troubleshooting a fresh install If errors are pointing at cuc, cuc-install.log and cuc-

dbinstall.log show the details for the upgrade (installing rpm packages and the database install respectively)


Troubleshooting - Switchover GUI returns with “Update successful”, even though

the switchover is still in progress Watch the contents in the install log directory for

updates In case of error during switchover, review the

install_log<date/time>.log file for details If errors are pointing at cuc, cuc-version-switch*.*

files should be reviewed for errors


Troubleshooting – Switchover (cont) Ex: Successful Rollback (Gold or Silver Start)

cuc-version-switch_<date/time>.log

Monday June 18 2007 03 15:04:39 started /common/…/cuc-version-switch…06/18/2007 15:04:39 - CUC Source version = 2.0.0.331.06/18/2007 15:04:39 - CUC Destination version = 2.0.1.1000.06/18/2007 15:04:39 - UCOS Source version = 6.0.1.1000-7.06/18/2007 15:04:39 - UCOS Destination version = 6.0.1.1000-18.…Connection DB[NOTRUNNIG]06/18/2007 15:10:19 - Removing the data migration directory /common/datamigration.06/18/2007 15:10:19 - Unity Connection data migration completed.06/18/2007 15:10:19 - Total seconds to run CUC switchover: 340.515178919


Troubleshooting – Switchover (cont) Ex: Silver Star Rollback (rollback succeeds, but mailbox

store data is not migrated) This should only the case when a defect is present.

cuc-version-switch_<date/time>.log

06/18/2007 14:16:38 - Executing: UPDATE tbl_FolderItem ...; 06/18/2007 14:16:38 - Failed to reassign orphaned messages06/18/2007 14:16:38 - Exception caught. Details = SQLCODE -346 in EXECUTE: IX000: Could not update a row in the table.

06/18/2007 14:16:38 - Failure occurred during import for [email protected]/18/2007 14:16:38 - Failure during message import. Reverting

to orginal backup:unitymbxdb1@ciscounityb.

Lucas TOI – CUC Upgrade and Rollback

Documents

Transcript of Lucas TOI – CUC Upgrade and Rollback