Slide 1

Troubleshooting

Slide 2

Tools

Slide 3

2012 Citrix | Confidential Do Not Distribute Troubleshooting Tools Inspecting HTTP Headers In many cases it is useful to view the HTTP headers when debugging various problems, including: Persistence failures when using cookie persistence Performing caching and tuning cache headers Verifying compression is occurring Isolating authentication issues Many tools that are available are very useful for this task and are easy to use: Live HTTP Headers for Mozilla/Firefox IE HTTP Headers for Internet Explorer IEHTTPAnalyzer Fiddler

Slide 4

2012 Citrix | Confidential Do Not Distribute Troubleshooting Tools Inspecting HTTP Headers: Live HTTP Headers Live HTTP Headers is available at http://livehttpheaders.mozdev.org free toolhttp://livehttpheaders.mozdev.org Once installed, go to Tools > livehttpheaders to start capturing headers. Once opened, all HTTP request and response headers will be displayed, as shown. If desired, you can also filter what requests will be captured.

Slide 5

2012 Citrix | Confidential Do Not Distribute Troubleshooting Tools Inspecting HTTP Headers: IE HTTP Headers IE HTTP Headers can be downloaded from http://www.bluck.info/iehttpheaders - free toolhttp://www.bluck.info/iehttpheaders After installation, go to View > Explorer Bar > ieHTTPHeaders to display the view pane.

Slide 6

2012 Citrix | Confidential Do Not Distribute Troubleshooting Inspecting HTTP Headers IEHTTPAnalyzer - must purchase (found on student Desktop) or download from http://www.ieinspector.com/httpanalyzer/http://www.ieinspector.com/httpanalyzer/

Slide 7

2012 Citrix | Confidential Do Not Distribute Troubleshooting Inspecting HTTP Headers Fiddler - Freeware Download from http://www.fiddler2.com/fiddler2/http://www.fiddler2.com/fiddler2/

Slide 8

Tracing

Slide 9

2012 Citrix | Confidential Do Not Distribute Troubleshooting Tools Packet captures can be done on the NetScaler via the new trace option on the diagnostics tab, as shown

Slide 10

2012 Citrix | Confidential Do Not Distribute Troubleshooting Tools The dialog to perform a trace has several options: Packet size: Specify 0 to capture the entire packet Duration of data per file: By default 1 hr. Format: Select tcpdump to capture in pcap format Filter expression: TCPDUMP capture filter to only capture certain packets being processed Capturing mode: Additional options for capturing packets

Slide 11

2012 Citrix | Confidential Do Not Distribute Troubleshooting Tools Packet Traces, continued Once a trace is performed, the trace will have to be downloaded

Slide 12

2012 Citrix | Confidential Do Not Distribute Troubleshooting Tools Packet Traces, continued Select where you want the trace saved and open with your favorite trace tool

Slide 13

2012 Citrix | Confidential Do Not Distribute Network Traffic Capture The nstrace utility: Is used for packet capture on the NetScaler system Has files stored in /var/nstrace Has names nstracexx.cap (Native) or nstracexx.pcap (tcpdump) Has syntax nstrace.sh sz 0 Prefers the native format for packet capture since it captures more information Can view the native NetScaler and pcap format with Wireshark

Slide 14

2012 Citrix | Confidential Do Not Distribute Sample nstrace.sh Output

Slide 15

SNMP

Slide 16

2012 Citrix | Confidential Do Not Distribute Troubleshooting Tools SNMP Overview SNMP is a standard means of receiving information from a device by either polling variables (OIDs) or receiving alerts (traps). The NetScaler system provides a list of over a thousand variables in a MIB file. In some devices, SNMP can both read data and write configurations, but with the NetScaler system it is read only. Common tools that are used to keep track and plot long-term SNMP data include NetScaler Command Center, HP OpenView and MRTG.

Slide 17

2012 Citrix | Confidential Do Not Distribute Troubleshooting Tools SNMP Community Configuration The first step in monitoring via SNMP is to configure a community and permission (select ALL). System -> SNMP -> Community

Slide 18

2012 Citrix | Confidential Do Not Distribute Troubleshooting Tools SNMP Manager Configuration To limit SNMP polling to a set of IPs or networks, configure managers as well.

Slide 19

2012 Citrix | Confidential Do Not Distribute Troubleshooting Tools Network Monitoring with SNMP Some of the most commonly polled SNMP OIDs HA State:.1.3.6.1.4.1.5951.4.1.1.6.0 Average CPU:.1.3.6.1.4.1.5951.4.1.1.41.1.0 Memory Utilization (%):.1.3.6.1.4.1.5951.4.1.1.41.2.0 Server Connections:.1.3.6.1.4.1.5951.4.1.1.46.1.0 Client Connections:.1.3.6.1.4.1.5951.4.1.1.46.2.0 HTTP Requests:.1.3.6.1.4.1.5951.4.1.1.48.67.0 HTTP Responses:.1.3.6.1.4.1.5951.4.1.1.48.53.0 Interface Stats (table):.1.3.6.1.4.1.5951.4.1.1.54 Since many OIDs are dynamically generated, tools such as the iReasoning MIB browser (http://www.iReasoning.com) are handy for locating them.http://www.iReasoning.com

Slide 20

2012 Citrix | Confidential Do Not Distribute Troubleshooting Most Common Issues Do not rule out basic problems. Is there a duplex mismatch? #1 cause of poor performance Is a feature enabled? Check routing, in particular when USIP is enabled. Do not assume coders follow HTTP? Does it work as TCP? When installing a NetScaler system, do not assume the rest of the network is correct it usually is broken in subtle ways. Divide and conquer. Try to break the problem down into pieces, and verify each. Always perform a trace when you have a problem. A trace tells you what IS happening and not what you think should be happening.

Slide 21

Logs

Slide 22

2012 Citrix | Confidential Do Not Distribute Logged Information The following components enable recording and extracting information for troubleshooting the NetScaler: nsconmsg newnslog

Slide 23

2012 Citrix | Confidential Do Not Distribute Nsconmsg Command Output

Slide 24

2012 Citrix | Confidential Do Not Distribute Accessing Logged Information The log file and time parameters should be replaced with the necessary values to obtain desired log information The log file and time parameters are available to: Uncompress an archived log file Discover the time period covered by the log View load-balancing statistics from the archived log Extract logging information for a shorter duration Start log process for newnslog

Slide 25

2012 Citrix | Confidential Do Not Distribute Displaying Console Messages

Slide 26

2012 Citrix | Confidential Do Not Distribute Calendar Dialog

Slide 27

2012 Citrix | Confidential Do Not Distribute Sample nsconmsg Output Using grep

Slide 28

2012 Citrix | Confidential Do Not Distribute Displaying Event Information

Slide 29

2012 Citrix | Confidential Do Not Distribute Displaying CPU Information

Slide 30

2012 Citrix | Confidential Do Not Distribute Displaying NIC Statistics

Slide 31

2012 Citrix | Confidential Do Not Distribute Load Balancing Display Output

Slide 32

Statistics

Slide 33

2012 Citrix | Confidential Do Not Distribute Statistics Monitoring statistics of the NetScaler system or a feature can be obtained by running the stat command Stat feature displays summary statistics for the feature Additional stat parameters are: detail: for detailed feature statistics fullValues: for number and strings displayed in full form ntimes: for number of times statistics to be displayed logFile: for reading stats from a log file names internal: for viewing internal counter names

Slide 34

2012 Citrix | Confidential Do Not Distribute Displaying Statistics in the CLI -stat command: Provides statistics on certain components of a NetScaler system -E.g.: - stat cpu - stat system - stat lb vserver

Slide 35

2012 Citrix | Confidential Do Not Distribute NetScaler System Statistical Utility

Slide 36

Work better. Live better.