SymantecNetBackup 5200 SeriesTroubleshootingGuide WBT/NBU 5220... · 10 9 8 7 6 5 4 3 2 1. ... See...

Symantec NetBackup™ 5200Series Troubleshooting Guide

Release 2.0

NetBackup 5200 and 5220

The software described in this book is furnished under a license agreement andmay be usedonly in accordance with the terms of the agreement.

Documentation version:

PN:

Legal NoticeCopyright © 2011 Symantec Corporation. All rights reserved.

Symantec and the Symantec Logo are trademarks or registered trademarks of SymantecCorporation or its affiliates in theU.S. and other countries. Other namesmaybe trademarksof their respective owners.

This Symantec product may contain third party software for which Symantec is requiredto provide attribution to the third party (“Third Party Programs”). Some of the Third PartyPrograms are available under open source or free software licenses. The LicenseAgreementaccompanying the Software does not alter any rights or obligations you may have underthose open source or free software licenses. Please see theThird Party LegalNoticeAppendixto this Documentation or TPIP ReadMe File accompanying this Symantec product for moreinformation on the Third Party Programs.

The product described in this document is distributed under licenses restricting its use,copying, distribution, and decompilation/reverse engineering. No part of this documentmay be reproduced in any form by any means without prior written authorization ofSymantec Corporation and its licensors, if any.

THEDOCUMENTATIONISPROVIDED"ASIS"ANDALLEXPRESSORIMPLIEDCONDITIONS,REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT,ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TOBELEGALLYINVALID.SYMANTECCORPORATIONSHALLNOTBELIABLEFORINCIDENTALOR CONSEQUENTIAL DAMAGES IN CONNECTION WITH THE FURNISHING,PERFORMANCE, OR USE OF THIS DOCUMENTATION. THE INFORMATION CONTAINEDIN THIS DOCUMENTATION IS SUBJECT TO CHANGE WITHOUT NOTICE.

The Licensed Software andDocumentation are deemed to be commercial computer softwareas defined in FAR12.212 and subject to restricted rights as defined in FARSection 52.227-19"Commercial Computer Software - Restricted Rights" and DFARS 227.7202, "Rights inCommercial Computer Software or Commercial Computer Software Documentation", asapplicable, and any successor regulations. Any use, modification, reproduction release,performance, display or disclosure of the Licensed Software andDocumentation by theU.S.Government shall be solely in accordance with the terms of this Agreement.

Symantec Corporation350 Ellis StreetMountain View, CA 94043

http://www.symantec.com

Printed in the United States of America.

10 9 8 7 6 5 4 3 2 1

http://www.symantec.com

Technical SupportSymantec Technical Support maintains support centers globally. TechnicalSupport’s primary role is to respond to specific queries about product featuresand functionality. TheTechnical Support group also creates content for our onlineKnowledge Base. The Technical Support group works collaboratively with theother functional areas within Symantec to answer your questions in a timelyfashion. For example, theTechnical Support groupworkswithProductEngineeringand Symantec Security Response to provide alerting services and virus definitionupdates.

Symantec’s support offerings include the following:

■ A range of support options that give you the flexibility to select the rightamount of service for any size organization

■ Telephone and/or Web-based support that provides rapid response andup-to-the-minute information

■ Upgrade assurance that delivers software upgrades

■ Global support purchased on a regional business hours or 24 hours a day, 7days a week basis

■ Premium service offerings that include Account Management Services

For information about Symantec’s support offerings, you can visit our Web siteat the following URL:

www.symantec.com/business/support/

All support services will be delivered in accordance with your support agreementand the then-current enterprise technical support policy.

Contacting Technical SupportCustomers with a current support agreement may access Technical Supportinformation at the following URL:


Before contacting Technical Support, make sure you have satisfied the systemrequirements that are listed in your product documentation. Also, you should beat the computer onwhich theproblemoccurred, in case it is necessary to replicatethe problem.

When you contact Technical Support, please have the following informationavailable:

■ Product release level



■ Hardware information

■ Available memory, disk space, and NIC information

■ Operating system

■ Version and patch level

■ Network topology

■ Router, gateway, and IP address information

■ Problem description:

■ Error messages and log files

■ Troubleshooting that was performed before contacting Symantec

■ Recent software configuration changes and network changes

Licensing and registrationIf yourSymantecproduct requires registrationor a licensekey, access our technicalsupport Web page at the following URL:


Customer serviceCustomer service information is available at the following URL:


Customer Service is available to assist with non-technical questions, such as thefollowing types of issues:

■ Questions regarding product licensing or serialization

■ Product registration updates, such as address or name changes

■ General product information (features, language availability, local dealers)

■ Latest information about product updates and upgrades

■ Information about upgrade assurance and support contracts

■ Information about the Symantec Buying Programs

■ Advice about Symantec's technical support options

■ Nontechnical presales questions

■ Issues that are related to CD-ROMs, DVDs, or manuals



Support agreement resourcesIf youwant to contact Symantec regarding an existing support agreement, pleasecontact the support agreement administration team for your region as follows:

[email protected] and Japan

[email protected], Middle-East, and Africa

[email protected] America and Latin America

mailto:[email protected]



Technical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Chapter 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

About troubleshooting the NetBackup appliance .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9NetBackup Appliance log file location information .... . . . . . . . . . . . . . . . . . . . . . . . . . 21About gathering information for NetBackup-Java applications .... . . . . . . . . . 13

Chapter 2 Troubleshooting procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

About troubleshooting procedures ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Troubleshooting NetBackup problems .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Troubleshooting NetBackup Appliance problems .... . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

About troubleshooting appliance installation and upgradeproblems .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Troubleshooting appliance configuration problems .... . . . . . . . . . . . . . . . . . . 19About troubleshooting client installations .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20NetBackup Appliance log file location information .... . . . . . . . . . . . . . . . . . . . 21About troubleshooting decommissioning issues ... . . . . . . . . . . . . . . . . . . . . . . . . 22About troubleshooting FactoryReset problems .... . . . . . . . . . . . . . . . . . . . . . . . . 23Performing a factory reset on an appliance with a Symantec

Storage Shelf attached .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25About controlling the NetBackup and administration console

processes ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27About changing VxAT properties ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27About a login error message that does not go away .... . . . . . . . . . . . . . . . . . . . 28About NetBackup support utilities ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28About Fibre Channel HBA card configuration verification .... . . . . . . . . . . 30About Fibre Transport media server verification .... . . . . . . . . . . . . . . . . . . . . . . 31

About troubleshooting storage expansion unit problems .... . . . . . . . . . . . . . . . . . 37About NetBackup 5220 and Symantec Storage Shelf matched

pairs ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38Resolving an expansion hang problem .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Resolving a problem that prevents expansion .... . . . . . . . . . . . . . . . . . . . . . . . . . . 40Resolving a problem that causes failed backups when disk space

is available ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Contents

Resolving a problem that causes a factory reset to migrate datafrom a storage expansion unit ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Resolving an expansionproblemcausedbypressingCtrl C duringexpansion .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Resolving a boot order change problem .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Contents8

Introduction

This chapter includes the following topics:

■ About troubleshooting the NetBackup appliance

■ NetBackup Appliance log file location information

■ About gathering information for NetBackup-Java applications

About troubleshooting the NetBackup applianceIf you experience trouble with your appliance, it is important that you can definethe problem and collect any supporting information. When you reach this point,you should contact Symantec Technical Support. A technical supportrepresentative works with you to diagnose the trouble and produce a satisfactoryresolution.

The following steps offer general guidelines to help you resolve any problems youmay encounter while you useNetBackup. The steps provide links tomore specifictroubleshooting information.

1Chapter

Table 1-1 Steps for troubleshooting NetBackup appliance problems

DescriptionActionStep

Error messages are usually the vehicle for telling you something wentwrong. If you don’t see an error message in an interface, but still suspecta problem, check the reports and logs. The NetBackup appliance providesreporting and logging facilities.

The logs show you what went right and the operation that was ongoingwhen the problemoccurred. Logs and reports are essential troubleshootingtools.

See “NetBackup Appliance log file location information” on page 21.

You can also seemore information onusing error logs forNetBackup issues.

See, Chapter 3, "Using logs" in the Symantec NetBackup TroubleshootingGuide.

Remember theerrormessageStep 1

Ask the following questions:

■ What operation was tried?

■ What method did you use?

For example, more than one way exists to install software on a client.Alsomore than one possible interface exists to use formany operations.Some operations can be performed with a script.

■ What type of server platform and operating system was involved?

■ If your site uses both the master server and the media server, was it amaster server or a media server?

■ If a client was involved, what type of client was it?

■ Have you performed the operation successfully in the past? If so, whatis different now?

■ What is the software version level?

■ Do you use operating system software with the latest fixes supplied,?

■ Is your device firmware at a level, or higher than the level, at which ithas been tested according to the posted device compatibility lists?

Identifywhat youwere doingwhen the problem occurred

Step 2

Capture potentially valuable information:

■ Progress logs

■ Reports

■ Utility Reports

■ Debug logs

■ Check for error or status messages in the system log and Event ViewerApplication.

■ Error or status messages in dialog boxes


Record all informationStep 3

IntroductionAbout troubleshooting the NetBackup appliance

10

Table 1-1 Steps for troubleshooting NetBackup appliance problems (continued)


If you define the issue as a NetBackup issue, you can use the followinginformation to correct it:

■ Take the corrective action as recommended by the status code ormessage.

See the NetBackup Status Code Reference Guide.

■ If no status code or message exists, or the actions for the status codedo not solve the problem, use additional troubleshooting procedures toisolate common problems.

Correct the problemStep 4

If your troubleshooting is unsuccessful, prepare to contact TechnicalSupport by filling out a problem report.

See “About gathering information for NetBackup-Java applications”on page 13.

For NetBackup issues on UNIX systems, the/usr/openv/netbackup/bin/goodies/support script creates a filethat contains data necessary for Technical Support to debug any problemsyou encounter.

See the Symantec NetBackup Troubleshoting Guide.

Complete a problem reportfor Technical Support

Step 5

The Symantec Technical SupportWeb site has awealth of information thatcan help you solve NetBackup problems.

Access Technical Support at the following URL:


Contact Technical SupportStep 6


See “About gathering information for NetBackup-Java applications” on page 13.

NetBackup Appliance log file location informationAs you define and troubleshoot a problem, always try to capture potentiallyvaluable information. The NetBackup appliance has the ability to capture data inthe log locations that are shown in the Table 1-2.

Browse and view the log files as follows:

■ Enter browse mode by running the Main_Menu > Support > Logs followed bythe Browse command in the appliance shell menu. The LOGROOT/> promptappears.

11IntroductionNetBackup Appliance log file location information

http://www.symantec.com/business/support/

■ Todisplay the available log directories on your appliance, type ls at LOGROOT/>prompt.

■ To see the available log files in any of the log directories, use the cd commandto change directories to the log directory of your choice. The prompt changesto show the directory that you are in. For example, if you changed directoriesto the GUI directory, the prompt appears as LOGROOT/GUI/>. From that promptyou can use the ls command to display the available log files in the GUI logdirectory.

■ To view the files, use the less <FILE> or tail <FILE> command. Files aremarked with <FILE> and directories with <DIR>.

You can also use the Main_Menu > Support > Logs commands to do the following:

■ Upload the log files to Symantec Technical Support.

■ Set log levels.

■ Export or remove CIFS and NFS shares.

Refer to the SymantecNetBackup5200SeriesCommandReferenceGuide formoreinformation on how to use the Logs commands.

Table 1-2 NetBackup 5200-log file locations

Log file locationNetBackup appliance logs

<DIR> APPLIANCE

config_nb_factory.log

NetBackup52xx appliance configuration log

<DIR> APPLIANCE

app_change_control.log

NetBackup 52xx appliance command log

<DIR> APPLIANCE

app_debug.log

NetBackup 52xx appliance debug log

<DIR> NBU

■ <DIR> netbackup

■ <DIR> openv

■ <DIR> volmgr

NetBackup logs, Volume Manager logs, andtheNetBackup logs that are contained in theopenv directory

IntroductionNetBackup Appliance log file location information

12

Table 1-2 NetBackup 5200-log file locations (continued)


<DIR> OS

boot.log

boot.msg

boot.omsg

messages

NetBackup52xx appliance operating system(OS) installation log

<DIR> WEBSERVER

■ <DIR> SYMCOpsCenterGUI

■ <DIR> SYMCOpsCenterServer

■ <DIR> SYMCOpsCenterWebServer

NetBackup Administrative Web userinterface log and theNetBackupWeb serverlog

You can view additional information about the NetBackup troubleshooting errorcodes from the appliance user interface. Click Monitor > Jobs and then click astatus code link in the jobs table to view the error code details.

To view NetBackup 52xx error information, download the following documentfrom the Symantec Support Web site:

http://www.symantec.com/docs/TECH136888

Note: At the time of the release of the NetBackup appliance, the error codeinformation was not complete. This Technote contains the most up-to-date errorcode information for the NetBackup appliance. At the next major release, theseerrors will be available as online Help from the administrative Web UI.

About gathering information for NetBackup-Javaapplications

If you encounter problemswith theNetBackup-Java applications, use the followingmethods to gather data for support.

The following scripts are available for gathering information:

13IntroductionAbout gathering information for NetBackup-Java applications

Logs data in a log file in/usr/openv/netbackup/logs/user_ops/nbjlogs. At startup,the script tells you which file in this directory it logs to. Normally,this file does not become very large (usually less than 2 KB). Consultthe file/usr/openv/java/Debug.properties for theoptions thatcan affect the contents of this log file.

jnbSA

( NetBackup-Java administration applicationstartup script)

Logs data in a log file if NetBackup is installed on the computer wherethe application was started. It logs oninstall_path\NetBackup\logs\user_ops\nbjlogs. IfNetBackup was not installed on this computer, then no log file iscreated. To produce a log file, modify the last “java.exe” line in thefollowing to redirect output to a file:install_path\java\nbjava.bat.

.

NetBackup-Java administration applicationon Windows

Provides a Java virtual machine stack trace for support to analyze.This stack trace is written to the log file that is associated with theinstance of execution.

/usr/openv/java/get_trace

Creates a file containing data necessary for customer support to debugany problems you encounter. For more details, consult the usageinformation of the script by using support -h.

/usr/openv/netbackup/bin/goodies/support

The following example describes how you can gather troubleshooting data forSymantec Technical Support to analyze.

Wait for several minutes before you assume that theoperation is hung. Some operations can take quite a whileto complete, especially operations in the Activity Monitorand Reports applications.

An application does notrespond.

Run /usr/openv/java/get_trace under the accountwhere you started the Java application. This script causesa stack trace to write to the log file.

For example, if you started jnbSA from the root account,start /usr/openv/java/get_trace as root. Otherwise,the command runs without error, but fails to add the stacktrace to the debug log. This failure occurs because root isthe only account that has permission to run the commandthat dumps the stack trace.

Still no response afterseveral minutes.

Run/usr/openv/netbackup/bin/goodies/support.Run this script after you complete the NetBackupinstallation and every time you change the NetBackupconfiguration.

Get data about yourconfiguration.

IntroductionAbout gathering information for NetBackup-Java applications

14

Provide the log file and the output of the support scriptfor analysis.

Contact Symantec TechnicalSupport

15IntroductionAbout gathering information for NetBackup-Java applications

IntroductionAbout gathering information for NetBackup-Java applications

16

Troubleshooting procedures

This chapter includes the following topics:

■ About troubleshooting procedures

■ Troubleshooting NetBackup problems

■ Troubleshooting NetBackup Appliance problems

■ About troubleshooting storage expansion unit problems

About troubleshooting proceduresThis chapter contains descriptions, and in some cases, procedures and for findingthe cause ofNetBackup appliance errors.Whenprocedures are provided, they aregeneral in nature and do not try to cover every problem that can occur.

When you perform these procedures, try each step in sequence. If you alreadyperformed the action or it does not apply, skip to the next step. If it branches youto another topic, use the solutions that are suggested there. If you still have aproblem, go to the next step in the procedure. Also, alter your approach accordingto your configuration and what you have already tried.

See “Troubleshooting NetBackup problems” on page 17.

See “Troubleshooting NetBackup Appliance problems” on page 18.

Troubleshooting NetBackup problemsThe Symantec Technical Support site has a wealth of information that can helpyou solve NetBackup problems. See the following site for comprehensivetroubleshooting details:

http://www.symantec.com/business/support/

2Chapter


If you encounter problems with NetBackup, refer to the traditional NetBackupdocumentation for additional information. You can locate NetBackup-specifictroubleshooting information on the following topics.

■ Preliminary troubleshooting

■ Configuration troubleshooting

■ General test and troubleshooting

■ Other troubleshooting procedures

For complete information, see the Symantec NetBackup Troubleshooting Guide.


Troubleshooting NetBackup Appliance problemsThese procedures for finding the cause of NetBackup errors are general in natureand do not try to cover every problem that can occur.

About troubleshooting appliance installation and upgrade problemsUse the following steps to troubleshoot appliance installation and upgradeproblems.

Table 2-1 Steps for troubleshooting installation problems.


Some reasons for failure are as follows:

■ Not logged on as an administrator.

■ Bad media (contact Technical Support)

■ Defectivedrive (replace thedrive or refer to vendor’shardwaredocumentation)

■ Improperly configured drive (refer to the system and the vendordocumentation)

Determine if you caninstall the software onthe appliance by usingthe release media.

Step 1

Determine if the problem is related to general network communications.Resolve networkproblems.

Step 2

Troubleshooting proceduresTroubleshooting NetBackup Appliance problems

18

Table 2-1 Steps for troubleshooting installation problems. (continued)


Note: Before you install or useNetBackup on a Linux client, verify that theinetd(or xinetd) service is started on that computer. This service ensures propercommunication between the NetBackup master and the Linux client.

Note: You cannot install NetBackup client software on a Windows client from aUNIX NetBackup server.

Do the following:

■ For an install to a trusting UNIX client, verify the following:

■ The correct client name is in your policy configuration

■ The correct server name is in the client /.rhosts file

If the installationhangs, check for problemswith the shell or the environmentvariables for the root user on the client. The files that you check depend onthe platform, operating system, and shell you use. For example, your .loginon a Sun system runs an stty (such as stty ^erase) before it defines yourterminal type. If this action causes the install process to hang, you canmodifythe .login file to define the terminal before you run the stty. Or, move theclient .login to another file until the install is complete.

■ For an installation to a secureUNIX client, check your ftp configuration. Forexample, you must use a user name and password that the client considersvalid.

Determine if you caninstall NetBackupclient software on theclients.

Step 3

The following topics describe specific problems that you may encounter.

See “Troubleshooting appliance configuration problems” on page 19.

See “About troubleshooting client installations” on page 20.


See “About troubleshooting decommissioning issues” on page 22.

Troubleshooting appliance configuration problemsUse the following steps to check for problems after an initial configuration orafter changes are made to an existing configuration.

19Troubleshooting proceduresTroubleshooting NetBackup Appliance problems

Table 2-2 Steps for troubleshooting configuration problems


Begin, by verifying the parameters that you entered during the initialconfiguration process are correct. Refer to Chapter 3, Configuration, in theNeBackup 5200 Series Getting Started Guide and review the "About applianceconfiguration guidelines" topic. This topic steps you through the required IPaddresses, firewall port usage, licenses, and so forth, to successfully configureyour appliance.

See the NeBackup 5200 Series Getting Started Guide.

Check the applianceconfigurationparameters

Step 1

If you found and corrected any configuration problems, retry the operation andcheck for status codes or messages in the following:

■ Check the log files. The contents of the logs can provide specific information,that is useful when the error can result from a variety of problems.

If you find a status code or message, perform the recommended correctiveactions.

See the Status Codes Reference Guide.

■ Check the appropriate enabled debug logs. Correct any problems you detect.

If these logs are not enabled, enable them before your next try.

Retry the operationand check for statuscodes and messages.

Step 2

If you performed corrective actions, retry the operation. If you did not performcorrective actions or the problem persists, go to one of the following procedures.

■ If the NetBackup installation directory fills up, such as with logging files, anumber of problems can result. NetBackup may become unresponsive. Forexample, NetBackup jobs may remain queued for long periods, even thoughall NetBackup processes and services are running.

See, "Resolving full diskproblems" in theSymantecNetBackupTroubleshootingGuide.

■ If the backup or restore jobs are running slowly, verify that the networkinterface cards (NIC) are set to full duplex. Half duplex often causes poorperformance.

See, "Troubleshooting network interface card performance" in the SymantecNetBackup Troubleshooting Guide.

Retry the operationand do additionaltroubleshooting.

Step 3


About troubleshooting client installationsWhen you install client software on a Windows system, the installation processdisplays an, "Install complete"message. Thismessage appears even if an erroroccurs during the installation. To ensure that the installation completed correctly,you must look at the script in the installation dialog and verify that a failure did


20

not occur. If a failure occurs, you can identify the failure by searching for a failureexit code.

NetBackup Appliance log file location informationAs you define and troubleshoot a problem, always try to capture potentiallyvaluable information. The NetBackup appliance has the ability to capture data inthe log locations that are shown in the Table 1-2.

Browse and view the log files as follows:

■ Enter browse mode by running the Main_Menu > Support > Logs followed bythe Browse command in the appliance shell menu. The LOGROOT/> promptappears.

■ Todisplay the available log directories on your appliance, type ls at LOGROOT/>prompt.

■ To see the available log files in any of the log directories, use the cd commandto change directories to the log directory of your choice. The prompt changesto show the directory that you are in. For example, if you changed directoriesto the GUI directory, the prompt appears as LOGROOT/GUI/>. From that promptyou can use the ls command to display the available log files in the GUI logdirectory.

■ To view the files, use the less <FILE> or tail <FILE> command. Files aremarked with <FILE> and directories with <DIR>.

You can also use the Main_Menu > Support > Logs commands to do the following:

■ Upload the log files to Symantec Technical Support.

■ Set log levels.

■ Export or remove CIFS and NFS shares.

Refer to the SymantecNetBackup5200SeriesCommandReferenceGuide formoreinformation on how to use the Logs commands.

Table 2-3 NetBackup 5200-log file locations


<DIR> APPLIANCE

config_nb_factory.log

NetBackup52xx appliance configuration log

<DIR> APPLIANCE

app_change_control.log

NetBackup 52xx appliance command log


Table 2-3 NetBackup 5200-log file locations (continued)


<DIR> APPLIANCE

app_debug.log

NetBackup 52xx appliance debug log

<DIR> NBU

■ <DIR> netbackup

■ <DIR> openv

■ <DIR> volmgr

NetBackup logs, Volume Manager logs, andtheNetBackup logs that are contained in theopenv directory

<DIR> OS

boot.log

boot.msg

boot.omsg

messages

NetBackup52xx appliance operating system(OS) installation log

<DIR> WEBSERVER

■ <DIR> SYMCOpsCenterGUI

■ <DIR> SYMCOpsCenterServer

■ <DIR> SYMCOpsCenterWebServer

NetBackup Administrative Web userinterface log and theNetBackupWeb serverlog

You can view additional information about the NetBackup troubleshooting errorcodes from the appliance user interface. Click Monitor > Jobs and then click astatus code link in the jobs table to view the error code details.

To view NetBackup 52xx error information, download the following documentfrom the Symantec Support Web site:

http://www.symantec.com/docs/TECH136888

Note: At the time of the release of the NetBackup appliance, the error codeinformation was not complete. This Technote contains the most up-to-date errorcode information for the NetBackup appliance. At the next major release, theseerrors will be available as online Help from the administrative Web UI.

About troubleshooting decommissioning issuesWhen attempting to decommission amedia server appliance, youmay encounterone of the following issues:

■ A decommission attempt may fail is no policies are defined.


22

If aNetBackupappliancemaster serverhas amedia server appliance configuredand you attempt to decommission that media server, the attempt may fail. Ifa failure occurs, the following message is logged.If you attempt to decommission a media server appliance that is configuredto a NetBackup appliance master server in classic mode the attempt may fail.If a failure occurs, the following message is logged.

Server is a member of the following media sharing groups:

UNRESTRICTED_SHARING_GROUP unable to get list of policies.

This failure can occur because the NetBackup master server does not have apolicy defined. To avoid this issue,make sure that you define at least one policyon the master server.

■ The decommission processmaynot remove all of the require objects and causean error.After you have gone through the process decommissioning a media server,the disk pool and storage server objects of type PureDisk are not removed. Ifyou run FactoryReset on themedia server, configure it as amedia server andattempt to use it for backups and restores. You must use the NetBackupAdministration Console from the NetBackup master server to remove theseobjects.See the section, "Decommissioning amedia server", in theSymantecNetBackupAdministratorGuide ,Volume I formore information onhow to decommissiona NetBackup media server.You can also refer to the following Technote on the Symantec Support Website:http://www.symantec.com/docs/TECH31168

About troubleshooting FactoryReset problemsThe FactoryReset function is used to return an appliance to its default state. Thefollowing issues may occur when you use this function:

■ You may encounter one of the following issues when you perform aFactoryReset function on an appliance that has network issues:

■ The network may timeout.This situation is likely to occur if the appliance is amedia server applianceand it cannot communicate with the master server.

■ Any configured storage units on the master server may not get cleanedproperly.

If you encounter any of these situations, you must ensure that you clean upthe storage units properly, before you reconfigure the appliance.


■ For appliances with an attached Symantec Storage Shelf, you must removethe data from the Symantec Storage Shelfwhen youperform the factory reset.The factory reset must be done through the appliance shell menu.

Note: If you perform a factory reset operation and disconnect the SymantecStorage Shelf during this process, the factory reset operation fails. You mustleave the Symantec Storage Shelf attached during a factory reset.

See “Performing a factory reset on an appliancewith a Symantec Storage Shelfattached” on page 25.

Performing a factory reset on an appliance with a SymantecStorage Shelf attachedThe following procedure describes how to performa factory reset on an appliancewith a Symantec Storage Shelf attached.

Note: To avoid a factory reset failure, you must leave the Symantec Storage Shelfattached to the appliance when you perform this task.

To perform a factory reset on an appliancewith a Symantec Storage Shelf attached

1 Log in to the appliance shell menu as the administrator.

2 Stop the NetBackup processes by entering the following command:

Main_Menu > Support > Processes > NetBackup Stop

3 Type Return to go back to the Support view.

4 Enter the maintenance mode by entering the following command:

Main_Menu > Support > Maintenance

5 Enter the maintenance password.

The default password is P@ssw0rd.

6 At the maintenance prompt, enter the following command:

elevate

7 This step describes how to delete the data on /disk. This process may takeseveral minutes depending on the amount of data.

Perform the following tasks to delete the data on /disk:

■ Enter the following command to delete the data:rm -rf /disk/*


24

■ To see how much data remains on /disk, run the following command:

df -h /disk

On the screen that appears, check the Used size column to see how muchdata remains to be removed.

■ Wait a few minutes, and then run this command again to refresh thescreen and view the progress.

■ Repeat this command as often as necessary until the Used size columnshows about 1GB.

8 Exit from the maintenance mode by simultaneously pressing and releasingthe CTL and D keys two times.

9 Start the NetBackup processes by entering the following command:

Main_Menu > Support > Processes > NetBackup Start


11 Reset the appliance to the initial factory settings by entering the followingcommand:

Main_Menu > Support > FactoryReset

Whenever you perform a factory reset on an appliance, it must bereconfigured. For complete details, see the Symantec NetBackup ApplianceGetting Started Guide.

Performing a factory reset on an appliance with a Symantec StorageShelf attached

The following procedure describes how to performa factory reset on an appliancewith a Symantec Storage Shelf attached.

Note: To avoid a factory reset failure, you must leave the Symantec Storage Shelfattached to the appliance when you perform this task.

To perform a factory reset on an appliancewith a Symantec Storage Shelf attached

1 Log in to the appliance shell menu as the administrator.

2 Stop the NetBackup processes by entering the following command:

Main_Menu > Support > Processes > NetBackup Stop



4 Enter the maintenance mode by entering the following command:

Main_Menu > Support > Maintenance

5 Enter the maintenance password.

The default password is P@ssw0rd.

6 At the maintenance prompt, enter the following command:

elevate





df -h /disk




8 Exit from the maintenance mode by simultaneously pressing and releasingthe CTL and D keys two times.


Main_Menu > Support > Processes > NetBackup Start


11 Reset the appliance to the initial factory settings by entering the followingcommand:

Main_Menu > Support > FactoryReset

Whenever you perform a factory reset on an appliance, it must bereconfigured. For complete details, see the Symantec NetBackup ApplianceGetting Started Guide.


26

About controlling the NetBackup and administration console processesYou can use the NetBackup 5200 command-line interface show and controlNetBackup and administration console processes. You can stop the processes,start the processes, andmonitor them fromwithin the Support> Processes view.The following commands are available:

■ Support > Processes > AdminConsole > Show

Use this command to show the state of the administration console processes.

■ Support > Processes > AdminConsole > Start

Use this command to start all of the administration console processes.

■ Support > Processes > AdminConsole > Stop

Use this command to stop all of the administration console processes.

■ Support > Processes > NetBackup > Show

Use this command to show the state of the NetBackup processes.

■ Support > Processes > NetBackup > Start

Use this command to start all of the NetBackup processes.

■ Support > Processes > NetBackup > Stop

Use this command to stop all of the NetBackup processes.

TheNetBackup5200Administrator'sGuideprovides a complete list of the availablecommands.

http://www.symantec.com/docs/DOC2792

About changing VxAT propertiesWhen troubleshooting your appliance, you may need to configure the SymantecProduct Authentication Service (AT) Server parameters. ThechangeVxATProperties script is available for you to use tomake changes to theseparameters.

The changeVxATProperties script prompts you to enter the following details:

■ Authentication Service Host Name

■ Authentication Service Port Number

■ Authentication Service Domain Name

■ Authentication Service Domain Name

■ Authentication Service Password

Enter the following command to configure the AT parameters on UNIX:


/opt/SYMCOpsCenterServer/bin/changeVxATProperties.sh

About a login error message that does not go awayYou may encounter a login error message that states the following, "Please tryafter 5minutes". If you receive this errormessage, and it does not disappear afterfive minutes, then you can use the following procedure to diagnose the issue.

To determine why an error message does not go away after a period of time

1 Open the command-line shell on the appliance.

2 Use the following sequence of commands to navigate to the Processes view.

Main > Support > Processes

3 Enter the following command to review the status of the current processes.

AdminConsole Show

You can see that the Appliance Console Server service is down.

4 Check the following log to determine if a service authentication has failed.

/opt/SYMCOpsCenterServer/logs/OpsCenterServer_out.log

5 If the log shows, Service user authentication failed, perform the followingsteps:

■ Check the state of the VxAT service. If the service is down, bring it up.

■ You can check the state of the VxAT service from the root as,/opt/VRTSat/vxatd.

■ Enter AdminConsole Stop and then enter AdminConsole Start.

■ From the root, check for the VxAT domain, /opt/VRTSat/bin/vssatlistpd --pdrtype ab.

■ Check the host name in the domain and compare it with the host name ofthe system:

■ If they are not the same, then your Initial configuration might havebeen broken.

■ To resolve this issue, run the following command:sh /opt/NBUAppliance/scripts/chHost.sh hostname

About NetBackup support utilitiesThe NetBackup 5200, provides the following support utilities to help diagnoseNetBackup problems.


28

■ NetBackup Domain Network Analyzer (NBDNA)

Youcan run theNBDNAutility onaNetBackupprimaryor secondary appliance.This utility analyzes the following information:

■ The NetBackup domain and its configuration for network issues

■ Performance

■ Behavior in regards to host name lookup and connectivity betweenNetBackup hosts and their role within the NetBackup domain

■ Generates the reports that are meant for Symantec Technical Support.

This utility provides the following types of information in its output:

Running audit as Media Server.

Collection Version: x.x

Collection Time: Tuesday, October 7, 2010 at 19:17:11 PM

NBU Release: NetBackup-SuSE2.6.16 7.0.1.5

NBU Version: 7.0.1.5

NBU Major Version: 7.0

NBU Minor Version: 1

NBU Patch Type: GA

NBU GlobDB Host: "host name"

Is GlobDB HOST? No

UNAME:

Hostname: sample.name.symantec.com

Host's Platform: Linux

Perl Architecture: Linux

Initialization completed in 14.040101 seconds.

Brief Description of What It Does (for type 1):

----------------------------------------------------

1) Perform basic self checks.

2) Check connectivity to Master (and EMM) server.

3) If SSO configured, get list of media servers sharing devices.

4) Get list of all clients which could send data here for backup.

5) Test NBU ports for basic connectivity between media servers

sharing devices.

6) Test NBU ports for basic connectivity between media server and

clients it backs up.

7) Perform service level tests for phase 2

8) Capture data for reports; save reports.


9) Save data to report files.

----------------------------------------------------

Discovering and mapping the NetBackup domain network for analysis

by extracting data from current system's configuration.

(To see more details, consider using '-verbose' switch.)

Probing Completed in 4.695464 seconds.

Initiating tests...

COMPLETED. Thank you for your patience.

/log/dna/sample.name.symantec.com.NBDNA.20100907.191711.zip

Archive created successfully!

Return /log/dna/sample.name.symantec.com.NBDNA.20100907.191711.zip

to Symantec Support upon request.

■ NetBackup Support Utility (nbsu)

You canuse the nbsuutility to gather appropriate diagnostic information aboutNetBackup and the operating system. The NetBackup Troubleshooting Guidedescribes when you would use this utility, as well as how to run it.See, "NetBackup support utility (nbsu)" in the Symantec NetBackupTroubleshooting Guide.

About Fibre Channel HBA card configuration verificationAfter you install and configure a SAN Client Fibre Channel HBA card, you maydecide that you want to verify that it is configured properly. To do that, use theMain_Menu > Manage > FC Show command from the command line interface.When you run the Main_Menu > Manage > FC Show command and the HBA cardwas configured properly, you see an output that is similar to the following:

Testsys.FC> Show

FC HBA card(s) are configured correctly.

**** FC HBA Cards ****

02:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel

to PCI Express HBA (rev 03)





30








**** Drivers ****

qla2xxx is loaded

windrvr6 is loaded

**** Ports ****

Bus ID Slot Port WWN Status Mode Speed Remote Ports

2:00.0 Slot3 21:00:...:07 Linkdown Initiator 4 gb/s

2:00.1 Slot3 21:01:...:07 Linkdown Initiator 4 gb/s

3:00.0 Slot2 21:00:...:30 Disconnect Target 8 gb/s

3:00.1 Slot2 21:00:...:31 Online Initiator 2 gb/s 0x21000024...

6:00.0 Slot1 21:00:...:82 Fabric Target 8 gb/s

6:00.1 Slot1 21:00:...:83 Online Initiator 8 gb/s 0x21000024...

*** Devices ****

Device Vendor Host Type Remote Port

/dev/sg0 SYMANTEC 10.182.0.209 FCPIPE (NBU 50x0) 0x21000024ff232438

/dev/sg2 SYMANTEC 10.182.0.209 FCPIPE (NBU 50x0) 0x21000024ff3162be

*** Notes ****

(NOTE: Ports in mode "Initiator*" are configured for target mode

When SAN Client FT Media Server is active, however, are currently

running in initiator mode, i.e. SAN Client is disabled or inactive.)

About Fibre Transport media server verificationAfter you install and configure a Fibre Transport (FT) media server, you can usethe Settings > FibreTransport SANClient Show command to show the statusof the SAN Client feature. When you run the FibreTransport SANClient Show

command and the Fibre Transport (FT) media server is configured properly, yousee an output similar to the following:

Testsys.Settings> FibreTransport SANClient Show

Fibre Transport server installed and running.


You canalso use the Manage > FC Show command to verify and confirm the statusof the SAN Client feature. From the output that you receive after you have runthe Manage > C Show command, you can verify the following:

■ The windrv6 driver is installed.

■ The target ports are in Target mode and not initiator mode.

■ Under theStatus column, the targetmodeports shouldhave a status ofFabricif the port is physically connected to something such as a switch.Nothing ever appears under the Remote Ports column for target mode ports.To find more information about the target mode ports, you must look at theVxUL logs for the originator 199 (nbftsvr).

Using the Maintenance command line to assist with FTverificationWhen you need to learn more about the target mode ports it may be necessary tolog into the Maintenance command line to view the VxUL log files. Use the Main> Support > Maintenance command to log into the Maintenance command line.You enter Maintenance as the user name and then enter your password.

Once you are in the Maintenance command line, you can change directories tothe/logs/openv/nbftsvrdirectory.Next, youunzip the fileswithin that directory.An output similar to the following should appear:

Testsys:~ # gunzip /log/nbftsrvr/51216-199-*.gz

Testsys:~ # vxlogview -p nb -o 199

04/25/11 22:25:21.327 [hasSANClientLicense] Found SAN Client License

04/25/11 22:25:21.327 [FTServerMgrService::init] called with 0 arguments

04/25/11 22:25:21.337 [forkDriver] getshm(): shmget id 2719746 attached

at 0x2aaaaaab6000

04/25/11 22:25:22.338 [Info] V-199-7 [V-1-38 Unable to load localization

resource for the requested language. Locale=LC_CTYPE=en_US.UTF-8;

LC_NUMERIC=C;LC_TIME=C;LC_COLLATE=C;LC_MONETARY=C;LC_MESSAGES=C;

LC_PAPER=C;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=C;

LC_IDENTIFICATION=C; TextId=7; DefaultMsg=]

04/25/11 22:25:22.338 [ReloadJungo] nbfdrv64 running as root

04/25/11 22:25:22.338 [SCPAL_Initialize] Jungo WinDriver version 901

compatible with linked BSI driver version 1011

04/25/11 22:25:22.338 [ConfigureSDCs] Using multiple PCI buses.

04/25/11 22:25:22.857 [gethrtime] detected 2.40956 GHz CPU

04/25/11 22:25:22.857 [InitializeSDC24xx] ISP-2532 instance 0 PN 1

ssdid 0x815d (QLE2562) WWN:[21:00:00:24:ff:23:24:30] intlevel=58

PCI Express


32

04/25/11 22:25:22.857 [InitializeSDC24xx] ISP-2532 instance 2 PN

2 ssdid 0x815d (QLE2562) WWN:[21:00:00:24:ff:23:27:82] intlevel=58

PCI Express

04/25/11 22:25:22.857 [SetTargets] (Targets=2)

04/25/11 22:25:22.909 [InitSGRVPool] SGRVpool allocated with support

for 322 * 256K buffers in two pools

04/25/11 22:25:22.909 [InitSGRVPool] WARN: SGRVpool space for

20 pipes of 16 buffers or 32 pipes of 10 buffers

04/25/11 22:25:23.949 [ActivateInterface24xx] PN=1 FWstate 4 Lost

Rx sync, awaiting resynchronization

04/25/11 22:25:23.949 [SC_UpdateTgtAttrTable] ISP2532 at tgt 0

WWN[0x20:0:0:0x24:0xff:0x23:0x24:0x30] intlevel 58

04/25/11 22:25:24.989 [ActivateInterface24xx] PN=2 FWstate 4 Lost

Rx sync, awaiting resynchronization

04/25/11 22:25:24.989 [SC_UpdateTgtAttrTable] ISP2532 at tgt 1

WWN[0x20:0:0:0x24:0xff:0x23:0x27:0x82] intlevel 58

04/25/11 22:25:25.301 [ProcessManage] node IOTID[2,65535,255,65535]

0x846140 STMTMTYPE_AttachChange type 3 STMTMP2P_Up 2 gbit/s

04/25/11 22:25:28.306 [gethrtime] detected 2.40034 GHz CPU


0x846240 STMTMTYPE_PortDBChange type 0 STMTMLoggedIn


0x846340 STMTMTYPE_FabricChange type 1 STMTMFABRIC_Up

04/25/11 22:25:30.371 [FTServerMgr_i::listAllPseudoDevices] targets 2

luns 2


0x846440 STMTMTYPE_PortDBChange type 0 STMTMLoggedIn


0x846540 STMTMTYPE_PortDBChange type 1 STMTMLogOut

04/25/11 22:36:52.167 [ProcessManage] node IOTID[2,1,0,65535] 0x846640

STMTMTYPE_PortDBChange type 0 STMTMLoggedIn

From this log output you can see that only one of the two detected target modeports is upand running. In this example the following two lines aremost important:

■ The lines that contain STMTMP2P_Up along with the negotiated link speed

■ The last line of the output.If the last line states STMTMLogOut instead of STMTMLoggedIn, then that canindicate that the link to the switch was not negotiated properly. That maymean there is an issue with the zoning.


Troubleshooting target mode port from the clientOnce the target ports are operational, you may need to continue yourtroubleshooting efforts fromthe client. You canuse theNetBackupAdministrator'sconsole or use the nbftconfig command from the command-line interface. Fromtheadministrator's console select,DeviceManagement>Devices>SANClients.From the command line, run the following command, where appsanclient1 isthe name of the SAN Client:

nbftconfig -listclients -verbose -C appsanclient1

Testsys:~ # nbftconfig -listclients -verbose -C appsanclient1

SAN Client Name : appsanclient1

SAN Client Version : 7.0

SAN Client State : disabled

Master Server Name : nbuappliance1

FT Server Connections: 1

Client Ports/Server : 2

Usage Preference : always

SAN Client HBA Port : 16

SAN Client Device State: active

Media Server Name : Testsys

Media Sever State : active

Media Server HBA Port : 1

Media Server Port Mode : FABRIC

Media Server LUN : 1

SAN Client Device State: active

Media Server Name : Testsys

Media Sever State : active

Media Server HBA Port : 1

Media Server Port Mode : FABRIC

Media Server LUN : 0

If the output does not display correctly, you can use the nbftconfig-rescanclient client command.Youwould thenwait a fewminutes andattemptto list the clients again. If running this command fails, go to the client system andverify that the operating system can detect the SCSI devices. You can see a list ofdevices in the /proc/scsi/scsi directory.

appsanclient1:~ # cat /proc/scsi/scsi

Attached devices:

Host: scsi0 Channel: 00 Id: 08 Lun: 00

Vendor: DP Model: BACKPLANE Rev: 1.00

Type: Enclosure ANSI SCSI revision: 05


34


Vendor: DELL Model: PERC 5/i Rev: 1.00

Type: Direct-Access ANSI SCSI revision: 05





Vendor: Dell Model: Virtual CDROM Rev: 123

Type: CD-ROM ANSI SCSI revision: 02


Vendor: Dell Model: Virtual Floppy Rev: 123


Note: If the nbftconfig -rescanclient client command works, Symantecrecommends that you still go to the client system and verify that the operatingsystem can detect the SCSI devices. It's possible that the output that is receivedafter the nbftconfig -rescanclient client command is run can contain staleinformation.

From the example, the expected SCSI devices are missing. You should have seentwo devices from the vendor, ARCHIVE, and themodel Python. In this situation,you should look at the /sys/class/fc_remote_ports directory. This directoryshows you the remote FC ports that the operating system can see.

appsanclient1:~ # ls -la /sys/class/fc_remote_ports/

total 0

drwxr-xr-x 2 root root 0 Apr 25 11:06 .

drwxr-xr-x 24 root root 0 Apr 24 17:55 ..

Again, the expected entries aremissing. At this time, Symantec recommends thatyou reload the FC HBA driver, qla2xxx with rmmod qla2xxx and then modprobe

qla2xxx. Reloading the driver may correct your issue. In this example, reloadingthe driver corrected the issue because the driver had not been previously loaded.From the following output you can see the entries, and you can see that the remoteport_name matches the appliance Port WWN):

appsanclient1:~ # modprobe qla2xxx

appsanclient1:~ # ls -la /sys/class/fc_remote_ports/

total 0

drwxr-xr-x 3 root root 0 Apr 27 14:48 .

drwxr-xr-x 24 root root 0 Apr 24 17:55 ..

drwxr-xr-x 2 root root 0 Apr 27 14:48 rport-4:0-0

appsanclient1:~ # cat /sys/class/fc_remote_ports/rport-4\:0-0/port_name


0x21000024ff232782

appsanclient1:~ # cat /proc/scsi/scsi

Attached devices:


Vendor: DP Model: BACKPLANE Rev: 1.00

Type: Enclosure ANSI SCSI revision: 05








Vendor: Dell Model: Virtual CDROM Rev: 123

Type: CD-ROM ANSI SCSI revision: 02


Vendor: Dell Model: Virtual Floppy Rev: 123



Vendor: ARCHIVE Model: Python Rev: V000

Type: Sequential-Access ANSI SCSI revision: 02


Vendor: ARCHIVE Model: Python Rev: V000

Type: Sequential-Access ANSI SCSI revision: 02

If you still do not see anything in the /sys/class/fc_remote_ports directory oryou do not see any SCSI devices, then the root cause is most likely a zoning issue.You may need to contact Technical Support for assistance if you come to thisdetermination.

It's possible for you to see expected entries in the /proc/scsi/scsi directory andyet have the information be stale. To ensure that the information is not stale, usethe sg_inq command to query the current status.

appsanclient1:~ # sg_inq -i /dev/sg3

VPD INQUIRY: Device Identification page

Designation descriptor number 1, descriptor length: 55

id_type: T10 vendor identification, code_set: ASCII

associated with the addressed logical unit

vendor id: SYMANTEC

vendor specific: FATPIPE 1.0 Testsys.enx1.symantec.com

appsanclient1:~ # sg_inq -i /dev/sg4

VPD INQUIRY: Device Identification page

Designation descriptor number 1, descriptor length: 55


36

id_type: T10 vendor identification, code_set: ASCII

associated with the addressed logical unit

vendor id: SYMANTEC

vendor specific: FATPIPE 1.1 Testsys.enx1.symantec.com

If these commands fail then it is likely that there is stale information in theoperating system . You can run the rescan-scsi-bus util

(/bin/rescan-scsi-bus.sh) to force a refresh. Run it first with the -r option toremove the stale entries. If that does not work, try the following:

■ If you do not see the entries you expect after you run the rescan-scsi-busutil -r command, try it with the -i option. That forces the FC link to berenegotiated.

■ Restart the FTmedia server on the appliance (enable and thendisable it). Thenrescan with the rescan-scsi-bus util command.

If that command fails, the most likely root cause is a zoning issue. You shouldcontact Support to help identify the zoning on the appliance.

■ Finally, if you have had to rescan the SCSI devices or there are new SCSIdevices, restart the FT client daemon. Once it has been restarted, you can lookat the VxUL log for originator 200 to check that it sees the SCSI devicescorrectly:

appsanclient1:~ # /usr/openv/netbackup/bin/vxlogview -p nb -o 200

04/27/11 13:58:25.736 [FATClientMgrService::init] Client

successfully started (FATClientMgrService.cpp:277)

04/27/11 13:58:25.822 [DiscoveryTaskThread] DiscoveryTask Thread

1094740288 Startup

04/27/11 13:58:25.822 [AddDevice] /dev/sg3

Inquiry "SYMANTECFATPIPE 1.0 nbapp36.engba.symantec.com"

TargetHBA:LUN:InitiatorHBA = 1:0:0x40 State = 1 RefCount = 0

04/27/11 13:58:25.822 [AddDevice] /dev/sg4

Inquiry "SYMANTECFATPIPE 1.1 nbapp36.engba.symantec.com"

TargetHBA:LUN:InitiatorHBA = 1:1:0x40 State = 1 RefCount = 0

About troubleshooting storage expansion unitproblems

The following describes some problems that you may encounter with a storageexpansion unit (Symantec Storage Shelf) and related expansion tasks. Links totopics that can help you resolve these problems are also included.

37Troubleshooting proceduresAbout troubleshooting storage expansion unit problems

Table 2-4 Troubleshooting storage expansion unit problems

ActionProblem description

Stop the expansion and then try itagain.

See “Resolving an expansion hangproblem” on page 39.

When the message “Expansion operation is inprogress. Do not interrupt …” appears, theexpansion hangs (more than 15 minutes).

Stop the NetBackup processes, checkstorage availability, and try theexpansion again.

See “Resolving a problem that preventsexpansion” on page 40.

Cannot expand storagebecauseMain > Manage

> Storage > Listviewshows "Storage inuse"and Deduplication volume total storage is lessthan 4TB.

Stop theNetBackupprocesses and thenrestart them.

See “Resolving a problem that causesfailed backups when disk space isavailable” on page 41.

Backups fail even though the available space on“Deduplication Storage” is more than 24TB.

Stop the factory reset, delete data fromthe disk, and perform the factory resetagain.

See “Resolving a problem that causesa factory reset to migrate data from astorage expansion unit” on page 42.

Factory reset attempts to migrate data fromstorage expansion unit for several minutes.

Check to see if a move operation is inprogress, and stop it. Then resizestorage volumes, try the expansionagain, and restart the NetBackupservices.

See “Resolving an expansion problemcaused by pressing Ctrl C duringexpansion” on page 43.

User presses Crtl C during expansion.

About NetBackup 5220 and Symantec Storage Shelf matched pairsWhen you order a NetBackup 5220 and a Symantec Storage Shelf together, theseunits are initialized together at the factory to create a matched pair. Matchedpairs provide optimum performance and they should always be used together tohelp ensure successful installation and configuration.

Troubleshooting proceduresAbout troubleshooting storage expansion unit problems

38

Tohelp you identifymatchedpairs, check the labels on the back of eachSymantecStorage Shelf. Each unit has two labels that identify the serial numbers of thematched pair.

Resolving an expansion hang problemIf you attempt to expand the NetBackup disk storage pool to include the storageexpansion unit and the process hangs, the process must be stopped.

The following procedure describes how to clear the current hang condition andretry the expansion.

To clear the current expansion hang condition and retry the expansion

1 Press Ctrl C to stop the expansion operation.

If the system does not respond, stop the independent processes manually asfollows:

■ Open a new terminal and log in as root.

■ Run the following command:ps –ef | grep configure_storage

■ Run the following command:kill -9 <PID>

Where <PID> is obtained from the ps –ef output.

■ Run the following command:ps –ef | grep fsadm

Stop this process by entering the following command:kill -9 <PID>.


■ Run the following command:ps –ef | grep vxresize

Stop this process by entering the following command:kill -9 <PID>


2 Verify that the fsadm and the vxresize processes are stopped.

3 Stop the NetBackup services by entering the following command:

Main > Support > Processes > NetBackup Stop

4 Check to see if disk_2 has been added to the nbuapp disk group by enteringthe following command:


vxprint

If disk_2 appears in the nbuapp disk group, do the following:

■ Run the following command:vxdiskadm

■ When the menu appears, select 3 to remove the disk.

■ Identify the disk to be removed by entering disk_2.

■ To confirm the disk to be removed, press y.

■ When you are prompted for additional disks to remove, select n.

■ To exit from this menu, select q.

5 To expand the NetBackup disk storage pool to include the storage expansionunit, enter the following command:

Main > Manage > Storage > Expand


Main > Support > Processes > NetBackup Start

7 If the expansion hangs again, restart the appliance.

Resolving a problem that prevents expansionIf you cannot expand the NetBackup disk storage pool, the following conditionsmay exist:

■ The Main > Manage > Storage > List view shows the storage expansionunit as In use.

■ The Deduplication volume shows a total storage of less than 4TB.

The following procedure describes how to clear the current condition and retrythe expansion.

To clear the current condition and retry the expansion



2 Check to see if disk_2 has been added to the nbuapp disk group by enteringthe following command:

vxprint

3 Based on the vxprint output, perform the appropriate tasks as follows:


40

If the vxprint output shows thatsubdisks are part of disk_2 and alsopart of any volumes, do the following:

If the vxprint output shows thatsubdisks are part of disk_2 but arenot part of any volumes, do thefollowing:

Run the following commands:

■ cd /opt/NBUAppliance/scripts/

■ ./configure_storage.pl

--rollback_expansion

■ Resize the disk storage volumes byentering the following command:

Main > Manage > Storage >

Resize

■ Remove the subdisks from disk_2

with the following command:

/opt/VRTS/bin/vxedit –g

nbuapp rm <sub-disk>

■ Removedisk_2 fromthenbuappdiskgroup as follows:

■ Run the following command:

vxdiskadm

■ When the menu appears, select 3to remove the disk.

■ To confirm the disk to be removed,press y.

■ When you are prompted foradditional disks to remove, selectn.


■ Resize the disk storage volumes byentering the following command:

Main > Manage > Storage >

Resize

4 To expand the NetBackup disk storage pool to include the storage expansionunit, enter the following command:




Resolving a problem that causes failed backups when disk space isavailable

If the deduplication volume has available space and the system reports a backupfailure due to inadequate space,NetBackup servicesmust be stoppedand restarted.

The following procedure describes how to stop and then restart the NetBackupservices to clear the current condition.


To stop and then restart the NetBackup services so that backups are successful



2 Restart the NetBackup processes by entering the following command:


Resolving a problem that causes a factory reset to migrate data froma storage expansion unit

If a factory reset on an appliance is performed incorrectly, it may cause theappliance to migrate data from a connected storage expansion unit.

The following procedure describes how to stop the data migration and removemigrated data from the storage expansion unit.

To stop the data migration during an appliance factory reset and remove themigrated data from the storage expansion unit

1 Press Ctrl C to stop the factory reset process.







df -h




4 Restart the NetBackup processes by entering the following command:


5 Perform the factory reset procedure again.


42

Resolving an expansion problem caused by pressing Ctrl C duringexpansion

If a user presses Ctrl C while expansion is in progress, the expansion fails.

The following procedure describes how to clear the current condition and retrythe expansion.

To clear the current condition and retry the expansion

1 Check to see if a move operation is still in progress by entering the followingcommand:

vxtask list

2 Make sure that the NetBackup processes are stopped. If necessary, enter thefollowing command:


3 Based on the vxtask list output, perform the appropriate tasks as follows:


If the vxtask listoutput shows thata move operation is not in progress,do the following:

If the vxtask listoutput shows thata move operation is in progress, dothe following:

■ Verify that thepdvol or the advol diskvolumes have subdisks that are partof disk_2, by entering the followingcommand:

vxprint

■ Run the following commands:

■ cd

/opt/NBUAppliance/scripts/

■ ./configure_storage.pl

--rollback_expansion

■ Stop the move operation by enteringthe following command:

vxtask –g nbuapp abort <Job

ID>

Where <Job ID> is themove operationthat appears in the vxtask output.

■ Identify the name of the subdisk thatis part of disk_2, but is not part ofany volume by entering the followingcommand:

vxprint

■ Delete the subdisk by entering thefollowing command:

/opt/VRTS/bin/vxedit –g

nbuapp rm <sub-disk name>

Where<sub-diskname> is the subdiskname in the vxprint output.

■ Remove disk_2 from the nbuapp diskgroup as follows:

■ Run the following command:

vxdiskadm

■ When the menu appears, select 3to remove the disk.

■ To confirm the disk to be removed,press y.

■ When you are prompted foradditional disks to remove, selectn.


4 Resize the disk storage volumes by entering the following command:

Main > Manage > Storage > Resize

5 Expand the NetBackup disk storage pool to include the storage expansionunit, by entering the following command:





44

Resolving a boot order change problemThe following situations can cause the boot order to change, which can preventthe appliance from booting up.

■ A new Symantec Storage Shelf is connected to an appliance that is currentlyin use.

■ An appliance is restarted or turned on after the Symantec Storage Shelf isdisconnected.

■ Power outage causes a restart of both components and the appliance is turnedon before the Symantec Storage Shelf.If you are logged in to the appliance during any of these situations, you mayexperience either a blank screenwith a blinking cursor or a screen that displaysGRUB.The following procedure describes how to clear the current condition so thatthe appliance can boot successfully.

To resolve a boot order change problem

1 Connect a monitor to the VGA port on the appliance.

2 Connect a keyboard to one of the USB ports on the appliance.

3 Make sure that the Symantec Storage Shelf is connected to the appliance andis turned on.

4 Restart the appliance by turning off the power, then turn it on again.

5 When the following Version screen appears, immediately press F2 to entersetup.


6 On the setup screen, press the right arrow key until the Boot Options tab ishighlighted, then press Enter.

7 On theBootOptions screen, press the downarrowkeyuntilHardDiskOrderis highlighted, then press Enter.


46

8 On the following screen, press the up or down arrow key until Boot Option#1 is highlighted, then press Enter.

9 When theBootOption#1popup appears, select ICHx/ESB2EmbeddedRAIDand press Enter.


10 Return to the Boot Options tab by pressing ESC.

The correct boot order should now appear with the ICHx/ESB2 EmbeddedRAID set as Boot Option #1.

11 Press F10 to save this configuration and exit from the setup.

The appliance restarts automatically and should boot successfully.


48

SymantecNetBackup 5200 SeriesTroubleshootingGuide WBT/NBU 5220... · 10 9 8 7 6 5 4 3 2 1. ... See...

Documents

Transcript of SymantecNetBackup 5200 SeriesTroubleshootingGuide WBT/NBU 5220... · 10 9 8 7 6 5 4 3 2 1. ... See...