Post on 03-Dec-2015
description
March 2003 Log Files 1
Log Files
March 2003 Log Files 2
ObjectivesWhen you have completed this module you will
be able to do the following:• Describe statistics collected by NetCache• Describe the contents of the logs• Compare log rotation and log pushing• Manage log files• Describe log file compression
March 2003 Log Files 3
NetCache Log Files Overview• Introduction to
NetCache Logs– Available Logs– Understanding Log
files– Messages Log
• Configuring Logs– Enabling and
formatting– Customizing logs
• Managing Logs– Routine admin– File switching
techniques– Log pushing– Log rotation
• Log Content
March 2003 Log Files 4
Available Log Files
Data > Log Files
March 2003 Log Files 5
Understanding Log Files
• All logs enabled by default• Predefined or custom formats• Extended Log File Format• Options for viewing, copying, deleting,
and forwarding log files• Log switching
– Rotation (default)– Pushing
March 2003 Log Files 6
Log Setup
Setup > Content Director > General > LoggingContent Director
Setup > GRM Server> Logging
GRM Server
Setup > ICAP> ICAP 1.0ICAP
Setup > HTTP> LoggingWeb Access
Setup > Streaming Media > LoggingStreaming Details
Setup > Streaming Media > LoggingStreaming Access
Setup > NNTP> LoggingNNTP Access
no configuration optionsMessages
March 2003 Log Files 7
Log Formats
• Two Formats– Predefined– Custom
• Logs (except Messages Log) adhere to W3C Extended Log File Format (www.w3c.org)
• File headers may be customized
W3C Extended Log File Format1
Format An extended log file contains a sequence of lines containing ASCII characters terminated by either the sequence line feed (LF) or carriage-return, line feed (CRLF). Log file generators should follow the line termination convention for the platform on which they are executed. Analyzers should accept either form. Each line may contain either a directive or an entry.
Entries consist of a sequence of fields relating to a single HTTP transaction. Fields are separated by white space, the use of tab characters for this purpose is encouraged. If a field is unused in a particular entry dash "-" marks the omitted field. Directives record information about the logging process itself.
1 Baker, Phillip; M, and Behlendorf, Brian, “Extended Log File Format:W3C Working Draft WD-logfile-960323,” available at http://www.w3.org/TR/WD-logfile
March 2003 Log Files 8
Customizing Log Files
• Log file headers may be customized• HTTP headers may be logged• Custom log files created by adding or
deleting headers in predefined log format
March 2003 Log Files 9
Managing Log Files
• Enable or disable logs• Maximum log size is 1,999 MB• Log Pushing• Log Rotation
March 2003 Log Files 10
Log Management Options
March 2003 Log Files 11
Log Switching• Log Pushing• Log Rotation (default)• Selected technique applies to all enabled logs• Max log file size = 1999 MBytes• Messages log
– Maximum of 10 files– Always starts at midnight Sunday– Files saved for nine weeks– Oldest file is deleted
March 2003 Log Files 12
Log Pushing• Web Access, NNTP and streaming logs can be
pushed to a FTP server at scheduled intervals• Naming
log_name.cache_hostname.timestamp (yyyy.mm.dd.hh.mm)
• Logs deleted after push• Messages log cannot be pushed• If the push fails:
– retry every 5 minutes– Autosupport message every hour– Once problem fixed – 10 files pushed every five
minutes until all are pushed
Log pushing Log pushing is one of two log management techniques that are available. With log pushing set, the Web access log, NNTP access log, and Streaming logs are pushed, using FTP, to one or two locations that you specify. (NetCache pushes only the log files that are enabled to the location you specify.) Log pushing enables you to save your logs.
Syntax Web access, NNTP access, and streaming log files pushed to an FTP server are named using the following convention:
log_name.hostname.timestamp
Requirements for log pushing The destination server must be running an FTP server and must allow the NetCache Appliance to rename files in the directory you specify as the log repository. To accomplish this, it is best to set up a user account for the NetCache Appliance logging on the FTP server and push the logs there.
March 2003 Log Files 13
What happens if a log push is not successful If a regularly scheduled log push is not successful, the NetCache Appliance attempts to push the logs to the FTP server every 5 minutes. The NetCache Appliance continues to attempt to push the logs every 5 minutes until it succeeds. If the log push continues to be unsuccessful, the NetCache Appliance sends an autosupport message once each hour saying that the log push failed.
March 2003 Log Files 14
Push Compressed Logs
• CLIconfig.system.logs.push_compressed [on|off]– Works only when logs.technique is push
Setup > System > Logging
March 2003 Log Files 15
Log Rotation
• Default technique
• Active log and 9 previous logs
• Rotated logs are stored on the NetCache until rotated out
• View with the NetCache Manager
• Naming - log_name.number
• Messages logs not rotated
Log rotation If you choose the log rotation mechanism for log handling, logs remain on the NetCache Appliance until they are rotated out. The NetCache Appliance keeps on disk the current log file and up to nine previous log files. Rotated logs remain in the NetCache Appliance logs directory and are viewable from the NetCache Manager utility.
When the NetCache Appliance rotates the logs, it renames the active log files by appending the host name and a timestamp to the name of the log file. The NetCache Appliance also deletes the oldest log file.
Changing from log rotation to log pushing When you change the log file handling mechanism from log rotation to log pushing, the NetCache Appliance schedules all the old logs to be pushed to the FTP server. If you do not want the old logs to be pushed to the FTP server, delete the logs before committing the configuration change.
March 2003 Log Files 16
Data > System Status > General
System Status These options enable the user to view the general status of this NetCache appliance. The information is current as of the time you open the page. However, you can set various refresh rates to update these statistics. Click “Refresh Now” to automatically update the statistics.
Refer to Online Help for specific configuration information.
March 2003 Log Files 17
Data > System Configuration
System Configuration This page describes the current status of the NetCache Appliance hardware configuration.
March 2003 Log Files 18
nclog Command
usage:nclog -d <filename> Delete the named lognclog -l List saved logsnclog -s Switch log files nownclog -t [-lines] <filename> View tail of the
named lognclog -ta [-lines] <protocol> View tail of the
active log of the named protocol
nclog -v <filename> View the named lognclog -va <protocol> View the active log
of the named protocol
nclog Command This command may be used to manage log files from the command line. Refer to the command line help for additional information.
March 2003 Log Files 19
Saved LogsNetCache> nclog -l
Saved logs:
/logs/messages 2000.08.31.21.23/logs/auditlog 1970.01.01.00.00/logs/http_log.0 2000.08.31.20.34/logs/nntp_log.0 2000.08.31.20.34/logs/streaming_log.0 2000.08.31.20.34/logs/streaming_details_log.0 2000.08.31.20.34
* /logs/nntp_log.1 2000.08.31.20.34* /logs/streaming_details_log.1 2000.08.31.20.34* /logs/streaming_log.1 2000.08.31.20.34
* /logs/http_log.1 2000.08.31.20.48
* Denotes current log
March 2003 Log Files 20
Setup > System > Logging
Logging The options on this page are used to specify how the NetCache appliance manages log files.
Refer to Online Help for specific configuration information.
March 2003 Log Files 21
Setup > System > Logging cont’d
March 2003 Log Files 22
Messages Log
• Monitors NetCache status• Managed differently• Always enabled• Not customizable• Not rotated or pushed• Naming
– Active log file = messages– Saved log file = messages.0 through messages.9
March 2003 Log Files 23
Messages Log
Messages log The Messages Log records all NetCache messages, including reboots of the NetCache Appliance, device failures, and errors. It also reports status messages about normal NetCache operation.
Refer to Online Help for specific configuration.
March 2003 Log Files 24
Messages Log ExampleThu Aug 10 16:51:43 GMT [rc:notice]: netcache licensed
Thu Aug 10 16:51:43 GMT [rc:notice]: nntp licensed
Thu Aug 10 16:51:43 GMT [rc:notice]: mms licensed
Thu Aug 10 16:51:43 GMT [rc:notice]: rtsp licensed
Thu Aug 10 16:51:45 GMT [rc:info]: Listening for HTTP requests on 0.0.0.0:3128
Thu Aug 10 16:51:45 GMT [rc:info]: Listening for ICP requests on 0.0.0.0:3130
Thu Aug 10 16:51:46 GMT [rc:info]: Listening for ADMIN requests on 0.0.0.0:3132
Thu Aug 10 16:51:47 GMT [init_jvm:warning]: Java disabled: Missing /etc/java/classes.zip
Thu Aug 10 16:51:47 GMT [dyn_dev_qual_admin:info]: Firmware is up-to-date on all disk drives
March 2003 Log Files 25
Setup > HTTP > Logging
Log Format The Log Format option is used to define a log format style for the Web access log.
Refer to Online Help for specific configuration information.
March 2003 Log Files 26
Data > Log Files > Web Access
Web access log The Web Access Log records all information about cache hits and misses. This log is only for web caching.
Refer to Online Help for specific configuration information.
March 2003 Log Files 27
Web Access Log Example903286.34 0.010 10.32.69.226 HIT/200 11659 GET
http://www.cnn.com/virtual/2000/style/main.css - - "text/css" -971903286.51 1.910 10.32.69.226 MISS_PRIVATE_CCTRL/200 945 GET
http://www.cnn.com/virtual/editions/europe/2000/roof/change.pop/frameset.exclude.html - DIRECT/207.25.71.30 "text/html" -
971903286.62 0.100 10.32.69.226 MISS_PRIVATE_CCTRL/200 790 GET http://www.cnn.com/virtual/editions/europe/2000/roof/change.pop/top.exclude.html - DIRECT/207.25.71.30 "text/html" -
971903286.63 0.120 10.32.69.226 MISS/200 1284 GET http://www.cnn.com/virtual/editions/europe/2000/roof/change.pop/ban.editions.gif - DIRECT/207.25.71.30 "image/gif" -
971903286.64 0.010 10.32.69.226 MISS/200 499 GET http://www.cnn.com/SPECIALS/multimedia/images/9901/dot.gif -DIRECT/207.25.71.30 "image/gif" -
971903286.79 0.170 10.32.69.226 HIT/200 11659 GET http://www.cnn.com/virtual/2000/style/main.css - - "text/css" -
971903287.11 0.590 10.32.69.226 MISS_PRIVATE_CCTRL/200 18818 GEThttp://www.cnn.com/virtual/editions/europe/2000/roof/change.pop/content.html - DIRECT/207.25.71.23 "text/html" -
March 2003 Log Files 28
Log File Exercises
• View and Download Log Files• Configure Log File Pushing
March 2003 Log Files 29
Log Files Exercises
• 15 minutes in length• Utilizing Breakout rooms• Instructor will visit all rooms• Broadcast announcement 5 minutes
prior to regroup• Stay focussed, start telnet, start GUI• Share microphones, or no one else can
be heard
March 2003 Log Files 30
NetCache Log Files Exercises
Objectives When you have completed this module you will be able to do the following:
• View and manage log files
• Push log files
Exercise Overview The purpose of this activity is for you to perform the procedures to manage NetCache log files. During these exercises, you will be able to go through each step in the process, and you will have an opportunity to verify that each step was successfully completed.
Time Estimate
Required Hardware, Software, and Tools
Hardware
• Workstation
• NetCache Appliance
Software
• NetCache 5.4 or later
• Netscape Navigator 4.7
• Internet Explorer 5.5
March 2003 Log Files 31
Viewing and Downloading Log Files You can view up to the last lines of any log file in the View Log Files page of the NetCache Manager utility. You can also view the entire log file in ASCII format by downloading it to a remote workstation.
Perform the following steps to view a log file:
1. Open the NetCache Manager.
2. Click Data > Log Files > Messages.
3. View the last 45 lines (default) of the Messages Log.
Perform the following steps to save a log file:
4. Click Data > Log Files > Messages.
5. Select “Copy the file to your computer” and submit.
6. You are prompted for the location where you want the file saved.
7. Select the path and enter a file name.
8. Click Save.
9. On your workstation, view the entire file with any ASCII editor
Practice viewing logs from the command line: 10. In order to have something to look for in the web access log, go to a browser
that is using your NetCache as proxy, type in a request and see the page return.
Example: http://pdc/acls/blue.htm
11. Use telnet to access the NetCache command line.
12. View the active web access log: netcache>nclog –ta http
13. Right click the blue title bar of the telnet command window and select Edit > Find
14 In the Find window, in the Find what text box, type: http://pdc/acls/blue.htm and click the Find Next button.
March 2003 Log Files 32
15. Observe the web access log entry for your request.
Pushing Log Files You can configure the NetCache to push each active log (Web access log, NNTP access log, Streaming log, and Streaming details log) to an FTP or Web server so you can save your logs.
Set up NetCache log pushing as follows:
1. Open NetCache Manager.
2. Select Setup > System > Logging.
3. You can specify a number between 1 and 1,999 megabytes in the Maximum Log File Size, in Megabytes field. For this exercise choose 1 megabyte.
4. Select Rotate the Logs.
5. Scroll to the bottom and select Switch Log Files Now.
Push the logs
6. Scroll to How to Switch Log File and select Push the log file to the following URL.
7. Enter URL format the location where you want the log files pushed:
ftp://user_name:password@ftp.server_name.domain
ftp://user_name:password@[ftpserver IP Address]
Example:
ftp://anonymous:x@pdc/class/logs/user[1 - 8]
8. Establish a schedule that will switch the files every minute.
9. Commit the changes.
10. Ensure that the logs were pushed to the directory you specified.
11. Restore your saved configuration.
End of Log Files Exercises