AVAMAR UTILITY NODE REPLACEMENT PROCEDURE · User=admin 2. Log into the replacement utility node as...

1

T E C H N I C A L N O T E S

Table of Contents

Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Optional followup considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

AVAMAR UTILITY NODE REPLACEMENTPROCEDURE

AVAMAR V5.X PRODUCTS AND EARLIER

TECHNICAL NOTEP/N 300-007-551

REV A04APRIL 14, 2011

Procedure

AVAMAR UTILITY NODE REPLACEMENT PROCEDURE TECHNICAL NOTE 2

ScopeThe procedure documented in this publication is only appropriate for replacing a failed utility node in a multi-node Avamar server that also contains a live spare node. This procedure does not apply to single-node servers.

Procedure

IMPORTANT: If a utility node fails without corrupting thedata on the disk drives, the first remedy you should exploreis to determine whether the disk drives can be physicallytransferred from the utility node to the live spare node. Ifpossible, then go directly to Configure, initialize and startManagement Console Server (page 23) and proceed fromthere.

IMPORTANT: You must install the same version Avamarsoftware on the replacement utility node that is installed onother nodes in the system. If an upgrade is desired, performthis procedure first and then upgrade the entire system.

The entire utility node replacement procedure comprises the following individual tasks in the following order:

1. Save probe.out/probe.xml file of defective node (page 3)

2. Configure replacement node as a utility node (page 3)

3. Restore or fix probe.out/probe.xml (page 5)

4. Copy .ssh directories to new utility node (page 6)

5. Change OS passwords (page 8)

6. Test the .ssh keys (page 9)

7. Install Avamar software (page 8)

8. Examine server configuration files for site customizations (page 11)

9. Configure and start web services (page 12)

10.Verify/Start login manager (page 13)

11.Set up and configure NTP (run asktime) (page 14)

12.Create permanent license key file (page 16)

13.Configure, initialize and start Management Console Server (page 23)

14.Configure, initialize and start Enterprise Manager Server (page 25)

15.Start/Resume cron activity (page 27)

16.Take and validate system checkpoint (page 28)

Procedure


Save probe.out/probe.xml file of defective node

If the defective utility node is still accessible, save a copy of the existing probe.out file (for all pre-v5.0 installations) or probe.xml file (v5.x installations) prior to shutting down and removing the node. The file is located in the /usr/local/avamar/var directory. If you are unable to save a copy of the existing probe.*** file, then later in this process, you will be prompted to manually restore all entries in the file that comes with the replacement utility node.

Configure replacement node as a utility node

All replacement nodes are initially configured as “super nodes” so that they can be quickly converted to any system node type (that is, utility or storage). You must now configure the replacement node to function as a utility node.

1. Open a command shell.

User=admin 2. Log into the replacement utility node as user admin.

3. When prompted for a password, enter the admin password and press ENTER.

4. Load the admin OpenSSH key by entering:

ssh-agent bash

ssh-add ~admin/.ssh/admin_key

You are prompted to enter a passphrase (P3t3rPan on new servers) .

5. Enter the admin user account passphrase and press ENTER.

User=root 6. Switch user to root by entering:

su -

7. When prompted for a password, enter the root password and press ENTER.

8. Enter:

cd /usr/local/avamar/src

ls

change_nodetype, config_info and create_newconfigs are present.

9. Enter:

./change_nodetype --utility

10.Verify that step 9 worked.

cat /usr/local/avamar/etc/node.cfg

Output should look like:

node type=utility

Procedure


11.Verify that /etc/resolv.conf and /etc/hosts contain the correct information by entering the following:

cat /etc/resolv.conf

cat /etc/hosts

TIP: It is useful to log into one of the storage nodes, inorder to compare the correct files on the storage nodes withthe files on the live spare node.

If these files are incorrect and require editing, use a text editor such as vi.

12.Modify /etc/sysconfig/network file, to set the hostname and gateway as follows:

vi /etc/sysconfig/network

13.Modify /etc/sysconfig/network-scripts/ifcfg-eth0 file to set the IP address and netmask

vi /etc/sysconfig/network-scripts/ifcfg-eth0

Also modify the following files to set the IP address and netmask:

• ifcfg-bond0• ifcfg-eth2• Any other similar bonding and ethPortNum files (if other Ethernet ports

were previously bonded)

14.Reboot the node by entering:

reboot

User=admin 15. Log into the replacement utility node as user admin.

16.When prompted for a password, enter the admin password and press ENTER.

17. Load the admin OpenSSH key by entering:

ssh-agent bash

ssh-add ~admin/.ssh/admin_key

You are prompted to enter a passphrase (P3t3rPan on new servers).

18. Enter the admin user account passphrase and press ENTER.

Procedure


19.Modify the usersettings.cfg file to ensure correct server, ID, password, and hfsport settings by entering:

vi /usr/local/avamar/etc/usersettings.cfg

Settings must be the following:

--server=UTILNODE--vardir=/usr/local/avamar/var--bindir=/usr/local/avamar/bin--id=root--password=PWD--hfsport=27000

Where UTILNODE is the hostname of the utility node and PWD is the admin user password. The default setting for UTILNODE in the usersettings.cfg file is dpn99; you must change dpn99 to the hostname of the utility node.

IMPORTANT: If the --hfsport=27000 entry is missingfrom /usr/local/avamar/etc/usersettings.cfg, a warningappears when the server automatically restarts followingrollback.

Restore or fix probe.out/probe.xml

If you were able to save a copy of the existing probe.out file (for all pre-v5.0 installations) or probe.xml file (v5.x installations) prior to removing the defective utility node, then copy that probe.*** file to the replacement utility node /usr/local/avamar/var directory.

If you were unable to save a copy of the existing probe.*** file, then you must manually restore all the entries in the /usr/local/avamar/var/probe.*** file on the replacement utility node.

Run the following commands to correct permissions:

For pre-v5.0 installations

chmod 664 /usr/local/avamar/var/probe.out

chown admin:admin /usr/local/avamar/var/probe.out

For v5.x installations

chmod 664 /usr/local/avamar/var/probe.xml

chown admin:admin /usr/local/avamar/var/probe.xml

Procedure


Copy .ssh directories to new utility node

If you changed your admin user password before the utility node failed, then first you must perform the following procedures for all three accounts (admin, dpn and root). These procedures ensure the .ssh keys on the new utility node match those on the existing data nodes.

IMPORTANT: If you did not previously change the OS loginpassword on any user account, skip the following sectionsand continue with Install Avamar software (page 8).

Rename existing .ssh directories on the new utility node

For each changed user account, perform the following procedure on the new utility node.

User=accountname

1. Login as a user of the account (root, admin or dpn).

When prompted, enter the account password.

2. Rename the existing hidden .ssh directory to .x-ssh by entering:

mv .ssh .x-ssh

Backup and copy .ssh directories from an existing data node

Create a tarball of the .ssh directory of an existing data node for each changed user account.

User=accountname

1. Login to an existing data node as a user of the account.

When prompted, enter the account password.

2. Go to the correct directory location for the account.

• root account: /root/• admin account: /home/admin/• dpn account: /home/dpn/

3. Backup the existing .ssh directory by entering:

tar cvf ssh.tar .ssh/

4. Confirm files are in the tarball by entering:

tar tvf ssh.tar

Procedure


5. Copy the tarball to the new utility node by entering:

scp ssh.tar ACCOUNT@IP:PATH.

Where ACCOUNT is the user account (root, admin or dpn), IP is the IP address of the new utility node, and PATH is the directory location listed in step 2.

Examples:

scp ssh.tar [email protected]:/root/

scp ssh.tar [email protected]:/home/admin/

scp ssh.tar [email protected]:/home/dpn/

A message similar to the following might appear:

Warning: Permanently added XXX.XXX.XXX.XXX (RSA) to the list of known hosts.

When prompted, enter the account password on the new utility node. A progress message similar to the following appears:

ssh.tar 100% 10KB 37.4MB/s 00:00

6. If needed, repeat this procedure for other accounts.

(a) Switch to another user account by entering one of the following:

root account: su -

admin account: su - admin

dpn account: su - dpn

When prompted, enter the user account password on the new node.

(b) Repeat steps 2 through 5.

Extract .ssh directories on the new utility node

Extract the tarball of the .ssh directory on the new utility node for each changed user account.

User=accountname

1. Login to the new utility node as a user of the account.

2. Go to the correct directory location listed in step 2 of the previous section.

3. Extract the ssh.tar file by entering:

tar xvf ssh.tar .ssh/

Information similar to the following appears:

.ssh/

.ssh/known_hosts

.ssh/authorized_keys2

.ssh/config

4. Confirm the files have been extracted by entering:

tar tvf ssh.tar

Information similar to the following might appear:

drwx------ root/root 0 2008-03-14 19:08:45 .ssh/-rw-r--r-- root/root 2002 2008-03-17 11:20:23 .ssh/known_hosts-r-------- root/root 1209 2008-03-14 19:08:45 .ssh/authorized_keys2-r-------- root/root 27 2008-03-14 19:08:45 .ssh/config

Procedure


5. If needed, repeat this procedure for other accounts. Otherwise, go to the next section.

(a) Switch to another user account by entering one of the following:

root account: su -

admin account: su - admin

dpn account: su - dpn

When prompted, enter the user account password on the new node.

(b) Repeat steps 2 through 4.

Change OS passwords

Use the following procedure to change the root, admin and dpn OS passwords to match the passwords on the data nodes, if needed.

User=root 1. Login to the new utility node as root user.

When prompted, enter the root account password.

2. Enter the following commands:

passwd admin

passwd dpn

passwd root

Install Avamar software

It is essential that you install the same version Avamar software on this replacement utility node that is installed on other nodes in the system. The easiest way to ensure that you install the correct version is to obtain an Avamar software customer tarball from one of the other nodes in the system. Nodes 0.0 and 0.1 usually retain a copy of the Avamar software customer tarball in /usr/local/avamar/src.


su -

2. Use WinSCP or an equivalent program to copy the customer tar ball (VERSION.BUILD-customer.tar.gz) to /usr/local/avamar/src.

Where VERSION.BUILD is the actual Avamar server software version and build you are installing.

3. Change directory by entering:

cd /usr/local/avamar/src/

4. Extract the axion_install script by entering:

tar zxvf VERSION.BUILD-customer.tar.gz RHELVER/axion_install

Where RHELVER is either RHEL3 or RHEL4_64, whichever directory applies.

Procedure


5. Do one of the following:

6. Restore the install.conf file in /usr/local/avamar/var/ to its original content on the failed utility node.

Do this by either copying the one from the failed utility node to the replacement, if you are able, or restoring the content of the install.conf file on the replacement utility node. Ensure that NTP server address, time zone, and lockbox password are correct.

7. Run axion_install by entering:

./RHEL3/axion_install

axion_install scans the current working directory structure for a customer tar ball. When a customer tar ball is located, you will be asked to confirm that it is the correct one.

8. Enter y and press ENTER to confirm that the correct customer tar ball has been located.

axion_install begins installing the Avamar server software RPMs.

9. Wait for axion_install to complete.

10.Remain logged into the root user account for the next task.

Test the .ssh keys

User=admin 1. Login as admin and load the admin keys by entering:

su - admin

ssh-agent bash

ssh-add .ssh/admin_key

When prompted, enter the correct admin key password for your site.

IF DO THIS

A pre-v5.0 installation

Go to step 7.

v5.x installation Go to step 6.

Procedure


2. Test the admin .ssh keys by entering:

mapall --all date

This command ensures the utility node and all data nodes reply and do not prompt for passphrase or admin password.

Example of success for pre-v5.0 installation:

Using /usr/local/avamar/var/probe.out (0.s) ssh -x [email protected] 'date' Wed Sep 7 14:41:26 PDT 2005 (0.0) ssh -x [email protected] 'date' Wed Sep 7 21:41:27 UTC 2005 (0.1) ssh -x [email protected] 'date' Wed Sep 7 21:41:27 UTC 2005 (0.2) ssh -x [email protected] 'date' Wed Sep 7 21:41:27 UTC 2005

Example of success for v5.x installation:

Using /usr/local/avamar/var/probe.xml (0.s) ssh -x [email protected] 'date' Wed Sep 7 14:41:26 PDT 2005 (0.0) ssh -x [email protected] 'date' Wed Sep 7 21:41:27 UTC 2005 (0.1) ssh -x [email protected] 'date' Wed Sep 7 21:41:27 UTC 2005 (0.2) ssh -x [email protected] 'date' Wed Sep 7 21:41:27 UTC 2005

Example of failure for pre-v5.0 installation:

admin@dpe28:~/>: mapall --all date Using /usr/local/avamar/var/probe.out (0.0) ssh -x [email protected] 'date' Enter passphrase for key '/home/admin/.ssh/id_rsa': [email protected]'s password:

Example of failure for v5.x installation:

admin@dpe28:~/>: mapall --all date Using /usr/local/avamar/var/probe.xml (0.0) ssh -x [email protected] 'date' Enter passphrase for key '/home/admin/.ssh/id_rsa': [email protected]'s password:

3. Exit the admin user account.

4. Login again as admin and load just the public dpn key by entering:

ssh-agent bash

ssh-add .ssh/dpnid

The following information appears:

Identity added: .ssh/dpnid (.ssh/dpnid)

5. Run mapall by entering:

mapall --all date

Procedure


6. Verify that the mapall command works if you have privileges of root by entering:

export SYSPROBEUSER=root

mapall --all date

If an error occurs just this time, then the root login might have an out of date key in ~root/.ssh/authorized_keys on node 0.1. Ensure you copied the .ssh directory for the user root (Refer to Copy .ssh directories to new utility node (page 6)).

7. Exit the admin user account.

8. Login as dpn and load the dpnid keys by entering:

su - dpn

ssh-agent bash

ssh-add .ssh/dpnid

9. Test the dpn .ssh keys by entering:

mapall --all date

This command ensures the utility node and all data nodes reply and do not prompt for passphrase or admin password.

10.Verify that the mapall command works if you have privileges of root by entering:

export SYSPROBEUSER=root

mapall --all date

11.Confirm your keys are loaded if any of the mapall steps fail by entering:

ssh-add -l

Below is example of keys not loaded.

Could not open a connection to your authentication agent.

12. If you have any failures, note and capture these and contact EMC Technical Support.

Examine server configuration files for site customizations

In order to ensure that no customer-specific server configuration settings are lost during this procedure, you must run the backup_upgrade_files utility, which will list any changes to the default settings.

User=admin 1. Ensure that you are still logged into the new utility node as user admin and that the admin OpenSSH key is still loaded.


su -

When prompted for a password, enter the root password and press ENTER.

Procedure


3. Enter the following on a single command line:

perl /usr/local/avamar/bin/backup_upgrade_files --diffdir=backups_DATE-TIME

Where backups_DATE-TIME is a user-defined temporary directory for the output files.

4. Reconcile any differences between the original files and the new files with default settings.

5. If the Avamar server root password was previously changed, edit the/usr/local/avamar/etc/usersetting.cfg file.

Change the following line to read:

usersettings.cfg:--password=PWD

Where PWD is the root password.

6. If the MCUser password was previously changed, edit the /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml file.

Change MCUser1 in the following line to the new password:

<entry key="MCUSERAP" value="MCUser1" />

7. Remain logged into the root user account for the next task.

Configure and start web services

User=root 1. Ensure that you are still logged into the replacement utility node as root.

Configure 2. Configure web services settings by entering:

website create-cfg

The following information appears in your command shell:============Defining HFSADDR for Webserver================Creating /etc/avamar/avamar.cfgFile already exists. All information in /etc/avamar/avamar.cfg will be removed.Are you sure you want to proceed (y/n)?

3. Enter y and press ENTER.

The following information appears in your command shell:--hfsaddr=xx.x.xxx.xxx--vardir=/usr/local/avamar/var--singleconn===Done

Where x.x.xxx.xxx is your actual Avamar server IP address.

Procedure


Initialize 4. Initialize web services by entering:

website init

The following information appears in your command shell:============Initialize Webserver===============Shutting down websiteShutting down httpd: [FAILED]===Adding web-managers aliasweb-managers: [email protected] web-managers alias===Done

NOTE: Ignore any FAILED status indications during webserver shutdown. This is normal.

Restart 5. Restart web services by entering:

website restart

The following information appears in your command shell:

NOTE: Ignore any FAILED status indications during webserver shutdown. This is normal.

============Initialize Webserver===============Shutting down websiteShutting down httpd: [FAILED]===Adding web-managers aliasweb-managers: [email protected] web-managers alias===DoneBe sure to setup /etc/avamar/avamar.cfg (/usr/local/avamar/bin/website create-cfg)Then restart webserver with SSL enabled (/usr/local/avamar/bin/website restart)

6. Verify web services by pointing your web browser at http://UTILITY-NODE-IP

Where UTILITY-NODE-IP is the IP address of the Avamar server utility node.

The Please click here to transfer to the secure login page appears.

Verify/Start login manager

User=root 1. Ensure that you are still logged in as root.

2. Start login manager by entering:

/etc/init.d/lm start


Starting login manager server: [ OK ]

Procedure


3. Verify that login manager is running by entering:

/etc/init.d/lm status

The following information appears in your command shell:lm (pid 13364 13363 13362) is running...

User=admin 4. Switch back the admin user account by entering:

exit

Set up and configure NTP (run asktime)

IMPORTANT: The procedure described in this section is notrequired for v5.0 installations. You should have set up NTP instep 6 of Install Avamar software (page 8). Go to the nexttopic.

User=admin 1. Ensure that you are still logged into the replacement utility node as user admin and that the admin OpenSSH key is still loaded.

User=dpn 2. Switch to the dpn user account by entering:

su - dpn

3. When prompted for a password, enter the password and press ENTER.

4. Load the dpn OpenSSH key by entering:

ssh-agent bash

ssh-add ~dpn/.ssh/dpnid

NOTE: The dpnid OpenSSH key does not require a pass-phrase.

5. Enter:

asktime

NOTE: The following example asktime prompts and userresponses are the suggested ones for most sites. However,some customer configurations might require differentresponses. Contact EMC Technical Support for additionalinformation.

If you are upgrading an existing Avamar server, asktime will detect the previous NTP settings and prompt you as follows:

Do you want to make use of your previous answers?(You will be given the chance to review and to change them.) y(es), n(o), q(uit/exit):

Procedure


(a) Enter y and press ENTER to accept the previous settings as default settings for the remainder of this asktime session.


Are external time servers available?

(b) Enter y and press ENTER.


Do you want to use U.S. public time servers out on the wider Internet, such as those offered by NIST or the U.S. Naval Observatory?

(c) Enter n and press ENTER.


Do you have access to other external time servers either on-site or on the wider Internet?

(d) Enter y and press ENTER.


Use these NTP servers (defined in DNS as ntp.example.com)?

(e) Enter y and press ENTER.


Are there other external time servers that you would like to use?

(f) Enter n and press ENTER.


Please enter the name of the local time zone, using one of the file names under /usr/share/zoneinfo/. Examples: US/Alaska US/Central US/Eastern US/Mountain US/PacificNote: this is a case-sensitive file name that must exist in /usr/share/zoneinfo.

(g) Enter your time zone and press ENTER.


Do you want to proceed with installation of these files on the selected node?

(h) Enter y and press ENTER.


Is this approximately correct (within a minute or two)?

(i) Enter y and press ENTER.

Older versions of asktime will terminate at this point and return you to the command prompt. Newer versions of asktime will continue to run.

Procedure



User=admin 7. Switch back to user admin by entering:

exit

Create permanent license key file

IMPORTANT: The following procedure assumes you havealready obtained "assigned license key" information, which isrequired to create a permanent license key file.

If this information is not available at this time, you may skipthis entire section and continue with Configure, initialize andstart Management Console Server (page 23).

However, it is critical that you create a permanent licensekey file within 30 days. If you fail to do so, the Avamar sys-tem will cease to function properly.

The Avamar server assigned license key is available from the Avamar download center: Subscribenet. To access your Subscribenet account, enter the login credentials provided in the email sent to you from [email protected].

If you cannot find the email from [email protected], send an email to [email protected] to request the Avamar assigned license key. Include the EMC

IF DO THIS

asktime terminates and returns you to the command prompt.

Verify proper NTP configuration by entering:

mapall --all ’/usr/sbin/ntpq -pn’

Current date and time are returned for each node in the server. If configuring a multi-node server, all times must be within one second of one another.

asktime continues to run.

The following information appears in your command shell:Do you want to wait and watch for time synchronization?

Enter y and press ENTER.The following information appears in your command shell:We appear to have time synchronization. Do you want to see results?

Enter y and press ENTER.NTP results appear in your command shell.If configuring a multi-node server, all times must be within one second of one another.

Procedure


product SO number in the email. (The EMC product SO number is required.) The return time for an email response is 48 hours.

The following is an example of an assigned license key, which includes data required in the procedure that follows:

EMC Avamar Software License Key Information Avamar System Customer Account ID: CN-10062734404Avamar System Asset ID: A-2010014578

To create the permanent license key file, perform the following:


2. Type:

gathergsankeydata

The following information appears in the command shell:

Enter your Avamar system customer account number:

IMPORTANT: Use the same assigned license keys for eachAvamar server for licenses received after June 1, 2010.

3. Type the Avamar System Customer Account ID number and press ENTER.

A valid Avamar System Customer Account ID number (account ID) conforms to the following format:

CN-YYMMDDNNNNN

where:• YY is a two-digit year.• MM is a two-digit month.• DD is a two-digit day of the month.• NNNNN is a five-digit numerical sequence.

The following information appears in the command shell:Enter your Avamar system asset ID number:

4. Type the Avamar System Asset ID number and press ENTER.

A valid Avamar System Asset ID number (asset reference ID) conforms to the following format:

A-YYYYNNNNNN

where:

• YYYY is a four-digit year.• NNNNNN is a six-digit numerical sequence.

The following information appears in the command shell:

Please enter the Internet domain for this account:

Procedure


5. Type your corporate Internet domain and press ENTER.

Information similar to the following appears in the command shell:Your answers were:Customer account ID: [CN-10062212345] Customer asset ID: [A-2010123456] Internet domain: [emc.com]Is this correct? [y(es), n(o), e(xit)]:

6. Type y and press ENTER.

The local directory now contains the gsankeydata.xml license key information file. This file is used to generate the permanent license key.

IMPORTANT: To generate a permanent license key file, youmust have a login and password to the Avamar License Por-tal. Only users who have completed the Avamar installationtraining class receive login credentials to the Avamar LicensePortal.


IF DO THIS

You have login credentials to the Avamar License Portal

1. Open a web browser and go to http://www.avamar.com/support1.html.The EMC Avamar License Portal page appears.

2. Type your username and password in the Username and Password text boxes, respectively.

3. Click Submit.The EMC Avamar Support page appears.

4. Continue with step 8.

You do not have login credentials to the Avamar License Portal

1. Contact EMC Customer Support and request a license key generation.EMC Customer Support provides you with an XML file that contains an activated license key. The XML file uses the same naming convention as shown in step 16.

2. After you receive the XML file from EMC Customer Support, skip steps 8 through 17 and continue with step 18.

Procedure


8. On the EMC Avamar Support page, click Avamar Key Generation.

The EMC Avamar Key Generation page appears.

9. Click Browse...

The Choose File dialog box appears.

NOTE: If you are using the Mozilla Firefox browser, the FileUpload dialog box appears.

10.Navigate to the license key information file, select it, and then click Open.

The Choose File dialog box closes.

The Upload Key Data XML text box now contains the license key information file.

11. Type your email address in the Send To Email Address text box.

EMC sends the license key information file to this email address.

IMPORTANT: The email account must be an account fromwhich you can save the attached license key file and copy itto the Avamar server.

Procedure


12.Click Generate Key.

The programs displays either the EMC Avamar Support final key generation page or the EMC Avamar Support Enter Terabytes page.The EMC Avamar Support Enter Terabytes page (shown in the following figure) appears for license keys issued after June 1, 2010:

This page enables you to enter the number of capacity terabyte licenses you want to allocate to the Avamar server.

IMPORTANT: Use the same assigned license keys for eachAvamar server. Enter only the number of Capacity TerabyteLicenses you have purchased as specified in your contractualagreement and purchase quantity with EMC. Licensing issubject to audit. Exceeding the software TB license pur-chased, requires contacting EMC Sales for order compliance.

Procedure


13.Do one of the following:

14.Click Generate Key.

The EMC Avamar Support page displays a confirmation message that the license key file has been emailed to the address you entered in step 11.

15. Log in to the email account you entered in step 11.

You should have an email message from [email protected] with the subject line: EMC Avamar Key Information.

16.Open the email message from [email protected].

The email message contains the permanent license key file as an attachment. The file uses the following naming convention: ASSET-NAME_Key.xml

where ASSET-NAME is typically the Avamar server hostname as defined in the corporate DNS.

IF DO THIS

The EMC Avamar Support final key generation page appears, confirming that the license key has been emailed to the address you entered in step 11.

Continue with step 18.

The EMC Avamar Support Enter Terabytes page appears

Type the number of capacity terabyte licenses in the Terabytes text box:

• For Avamar capacity-based licenses, type the number of terabytes. For example, type 10 for a ten-terabyte capacity license.

• For an upgrade, add the original terabytes with the add-on terabytes. For example, if the original order was 10 TB and the add-on is 2 TB, type 12.

TIP: To find the original number of terabytes, open a command shell and type: avmaint license --avamaronly

License information appears in the command shell.If you did not regenerate a license key after an upgrade, the output from avmaint license --avamaronly might not be accurate. In this case, check all purchase orders with EMC for verification.NOTE: If additional storage capacity is purchased, a new license key file must be generated.

Procedure


17.Save the ASSET-NAME.xml email attachment to a temporary directory or folder.

18.Use WinSCP or an equivalent program to copy the ASSET-NAME.xml license key file from the temporary directory or folder to the /tmp directory on the Avamar server utility node.

IMPORTANT: To activate the license by using the avmaintlicense command, the Avamar server subsystem (alsoknown as GSAN) must be running.

User=admin 19.Switch to the command shell session and ensure that you are still logged in as user admin, and the admin OpenSSH key is loaded.

20.Show the status of the Avamar server subsystem by typing:

dpnctl status gsan

If gsan is running, the following information appears in the command shell:

dpnctl: INFO: gsan status: ready

21.Do one of the following:

22.Verify that the server license is correctly installed by typing:

avmaint license --avamaronly

License information appears in the command shell.

IF DO THIS

The Avamar server is running

Change file permissions on the ASSET-NAME.xml license key file and activate the license by typing the following commands:chmod 644 /tmp/ASSET-NAME.xml

avmaint license /tmp/ASSET-NAME.xml --avamaronly

where ASSET-NAME.xml is the license key file you copied to the Avamar server in steps 17 through 18.

The Avamar server is not running

1. Change file permissions on the ASSET-NAME.xml license key file and activate the license by typing the following commands:cd /usr/local/avamar/etc

mv license.xml license.xml.old

cp /tmp/ASSET-NAME.xml license.xml

chmod 644 license.xml

where ASSET-NAME.xml is the license key file you copied to the Avamar server in steps 17 through 18.

2. Start the Avamar server.

Procedure


Configure, initialize and start Management Console Server

IMPORTANT: Only perform this task on new Avamar serv-ers that have never been deployed at a customer site.


ConfigureManagement

Console Server

2. Configure the Management Console Server by entering:

avsetup_mcs

Information similar to the following appears in your command shell (Java version depends on specific installation):Enter location of your Java 1.4.2 installation [/usr/java/j2re1.4.2_02]:

NOTE: The following example avsetup_mcs prompts anduser responses are the suggested ones for most sites. How-ever, some customer configurations might require differentresponses. Contact EMC Technical Support for additionalinformation.

3. Press ENTER to accept the default Java location.

The following information appears in your command shell:Enter the Avamar server IP address or fully qualified host name to be used by backup clients (i.e. server_name.your_company.com - This is normally the address or alias to the services node that is resolvable/reachable by the backup clients):

4. Enter your fully-qualified Avamar server name or IP address and press ENTER.

avamar-1.example.com is used as an example fully-qualified Avamar server name for the remainder of this procedure.

The following information appears in your command shell:'avamar-1.example.com' resolvable and pingable from utility node.

Enter the Avamar server IP address or fully qualified host name to be used for Administrator server to Avamar server communication. Use 'avamar-1.local.example.com' or its IP address to rely on internal Avamar server name resolution and network or use 'avamar-1.example.com', its IP address, or another name to rely on external name resolution and network [avamar-1.local.example.com]:

Enter the IP port used to communicate with the Avamar Server [27000]:

5. Press ENTER to accept the default port.

The following information appears in your command shell:Using port '27000'.

Is Network Address Translation (NAT) used on this utility node Y/N? [N]:

Procedure



The following information appears in your command shell:Enter the Avamar server accounting system root user password for Avamar server avamar-1.example.com. avsetup_mcs will log on and create the required Administrator server accounts:

7. Enter the Avamar root password and press ENTER.

The following information appears in your command shell:Enter a password to assign to the MCUser account on avamar-1.example.com:

8. Enter the password you want to assign to the MCUser account and press ENTER.

The following information appears in your command shell:Enter the hostname or IP address of the SMTP server used for email home:

9. Enter the outgoing SMTP mail server host name as defined in corporate DNS or IP address and press ENTER.

The following information appears in your command shell:Attempting to modify mcserver.xml file...mcserver.xml file updated.Exiting avsetup_mcs.

RestoreManagement

Console Server

10. Initialize the Management Console Server by entering:

mcserver.sh --restore

Information similar to the following appears in your command shell. Provide the requested information.

--restore will modify your Administrator Server database and preferences.Do you want to proceed with the restore Y/N? [Y]:

Enter the Avamar Server IP address or fully qualified domain name torestore from (i.e. dpn.your_company.com):Enter the Avamar Server IP port to restore from [27000]:Using port '27000'.Enter password for MCUser: MCUser1

When finished, the following lines will be shown:DONE: Restoring database dump file.Performing restore administrative tasks...Restore administrative tasks complete.Timestamp of flush restored: 2010-12-27 03:18:52 UTC

IF DO THIS

NAT is used. Enter y and press ENTER.The following information appears in your command shell:Enter the external IP address for this utility node. This is the IP address that administrator clients can contact the administrator server from:

Enter the actual (external) IP address of the Management Console Server and press ENTER.

NAT is not used. Press ENTER to accept the default selection (no) and proceed directly to the next step.

Procedure


Start ManagementConsole Server

11.Enter:

mcserver.sh --start

12.Verify Management Console Server operation by entering:

mcserver.sh --status

Configure, initialize and start Enterprise Manager Server



su root

ConfigureEnterprise

Manager Server

3. Configure the Enterprise Manager server by entering:

avsetup_ems

4. Wait for avsetup_ems to complete.

NOTE: The following example avsetup_ems prompts anduser responses are the suggested ones for most sites. How-ever, some customer configurations might require differentresponses. Contact EMC Technical Support for additionalinformation.

Switch Back toadmin

5. Switch back to user admin by entering:

exit

Initialize EnterpriseManager Server

6. Initialize the Management Console Server by entering:

emserver.sh --init

The following information appears in your command shell:Use the local Administrator Server to authenticate logins to the Enterprise Manager? [Y]:

IMPORTANT: EMC strongly recommends that you use thelocal Management Console Server to authenticate logins.

Procedure




Enter the Avamar server fully qualified host name or IP address to be used by the Enterprise Manager Server (i.e. server_name.your_company.com - This is an address that is resolvable/reachable by the Enterprise Manager Server):

8. Enter your fully qualified Avamar server host name and press ENTER.

avamar-1.example.com is used as an example fully qualified Avamar server host name for the remainder of this topic.

The following information appears in your command shell:'avamar-1.example.com' resolvable and pingable from utility node.

Enter the IP port used to communicate with the Administrator Server [7778]:



Enter the IP port used to communicate with the Avamar Server [27000]:



The following information appears in your command shell:Enter the MCUser password for Avamar server avamar-1.example.com.: MCUser1

11.Enter the MCUser1 account password and press ENTER.

Start EnterpriseManager Server

12.Enter:

emserver.sh --start

13.Wait for emserver.sh --start to complete.

User=root 14.Switch user to root by entering:

su - root

15.Enter:

emwebapp.sh --start

IF DO THIS

You want to use the local Management Console Server to authenticate Enterprise Manager server logins.

Press ENTER to accept the default selection (yes).No other information will appear in your command shell; proceed directly to step 14.

You want to use another Management Console Server to authenticate Enterprise Manager server logins.

Enter n and press ENTER.Perform all the remaining steps.

Procedure


Set up Desktop/Laptop

This procedure applies only to Avamar v5.0 or later software installations.

1. Install Desktop/Laptop by entering:

avsetup_dtlt.pl

The following information appears in your command shell:INFO: Tomcat java application server installed.

INFO: creating /usr/local/avamar/var/dtlt with permissions 0755

INFO: Installed Tomcat

2. Start Desktop/Laptop by entering:

dtltwebapp.pl --start

The following information appears in your command shell:Starting tomcat /usr/local/apache-tomcat-5.5.27/bin/startup.sh

Using CATALINA_BASE: /usr/local/apache-tomcat-5.5.27

Using CATALINA_HOME: /usr/local/apache-tomcat-5.5.27

Using CATALINA_TMPDIR: /usr/local/apache-tomcat-5.5.27/temp

Using JRE_HOME: /usr/java/jre1.6.0_12

User=admin 3. Switch to admin user by entering:

exit

Start/Resume cron activity


2. Enter:

resume_crons

Correct version information

In the /usr/local/avamar/var/ directory, you must now create a file named dpnctl-state:VERSION that contains correct version information. Do this by typing the following command:

/bin/rpm -q --queryformat='%{version}-%{release}' dpnmcs > /usr/local/avamar/var/dpnctl-state\:version

The previous command must be entered onto a single command line. To ensure accuracy, EMC recommends that you copy/paste it into your command shell.

Optional followup considerations


Take and validate system checkpoint


Take SystemCheckpoint

2. Enter the following:

cd /usr/local/avamar/bin

cp_cron

This creates a cp.DATE directory in the /data01 thru /data04, where DATE is the date and time of this checkpoint.

3. Validate the checkpoint using Management Console Server.

Optional followup considerationsAfter completing the previous procedures, determine whether any of the optional features listed below were configured on the original failed utility node. If so, the new utility node must be configured to support that feature.

• External authentication scheme such as LDAP• Replication• Morning cron run

Also, if a client-only upgrade was done previously, you must reinstall the client-only upgrade on the new utility node for it to be compatible with the upgraded clients.


Copyright c 2002 - 2011 EMC Corporation. All rights reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use,copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date regulatory document for your product line, go to the Technical Documentation and Advisories section on EMC Powerlink. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. All other trademarks used herein are theproperty of their respective owners.

AVAMAR UTILITY NODE REPLACEMENT PROCEDURE · User=admin 2. Log into the replacement utility node as...

Documents

Transcript of AVAMAR UTILITY NODE REPLACEMENT PROCEDURE · User=admin 2. Log into the replacement utility node as...