Managing Databases and the CSV Repository · OL-24187-04 5 Managing Databases and the Comma...

Cisco Application ViOL-24187-04

C H A P T E R 5
Managing Databases and the Comma Separated Value Repository
IntroductionThis chapter explains how to use utility scripts to manage the Collection Manager (CM) database and the comma separated value (CSV) repository.

Note The Cisco Collection Manager is able to operate with both the Cisco ASR 1000 Series and the Cisco SCE platform at the same time. A separate database is normally created for each platform. Refer to the Cisco Service Control Management Suite Collection Manager User Guide for information on the Cisco SCE platform databases.

Note For instructions on using utility scripts, see the “Using Utility Scripts” section on page 4-2.

Note By default, the Collection Manager handles database related transactions in uppercase table names. However, in a Windows based MySql server, table names are stored in lowercase and name comparisons are not case-sensitive; and, it automatically converts all table names to lowercase. If you are using a Windows based database server, you need to modify the lower_case_table_names server configuration entry.

• Common Database Management Tasks, page 5-2

• Managing the Bundled Sybase Database, page 5-8

• Managing the External MySQL Database, page 5-10

• Partitioning the MySQL Database, page 5-11

• Managing the Oracle Database, page 5-13

• Aggregating Database Data, page 5-14

• Managing the CSV Repository, page 5-18

5-1sibility and Control Collection Manager User Guide

Chapter 5 Managing Databases and the Comma Separated Value Repository Common Database Management Tasks

Common Database Management TasksThe following database management tasks apply to all the supported databases:

• Generating a list of the database tables

• Defining and applying the schedule for the periodic deletion of old records

• Deleting a table

• Manually deleting old records from a table

Use the following scripts to configure and maintain the database:

• ~scmscm/scripts/dbtables.sh

• ~scmscm/scripts/dbperiodic.sh

• ~scmscm/db_maint/create_periodic_del_procs.sh

• ~scmscm/scripts/droptable.sh

• ~scmscm/scripts/prunetable.sh

• ~scmscm/scripts/sybutfsupport.sh

• ~scmscm/scripts/partitions.sh

• ~scmscm/scripts/createaggevents.sh

Listing Database TablesTo display a list of all of the tables in the database, run the dbtables.sh script:

~scmscm/scripts/dbtables.sh --nf -c

Where applicable, the script displays the number of lines in the table and the earliest and latest timestamps.

Table 5-1 lists the dbtables.sh --nf -c script options.

Table 5-1 dbtables.sh --nf Options

Option Description-l Lists the existing table names only (without statistics).

-a Includes the nonreport tables in the listing.

-c Lists the existing table names with their record counts.

-f Enables fast line counting, using the client rather than JDBC.

-t { sec_num } The maximum waiting time, in seconds, for the response. The default is no timeout.

-h Prints this help message and exits.

5-2Cisco Application Visibility and Control Collection Manager User Guide

OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository Managing the CM:NFR Periodic Deletion of Old Records

The dbtables.sh script generates the following sample output:

>~scmscm/scripts/dbtables.sh --nfExecuting query ...

name| num_lines| min_time| max_time|----------------+--------+------------------------+------------------------+RPT_USAGE_NF| 4| 2011-03-24 10:07:40.0|| 2007-02-15 2011-03-24 13:44:13.0|RPT_TRANSACTION_NF| 110367| 2011-03-24 09:37:53.0| 2011-03-24 13:44:13.0

Managing the CM:NFR Periodic Deletion of Old RecordsTo manage the periodic deletion of old records, perform the following steps:

1. If they were not installed during the Collection Manager installation, install the periodic delete procedures:

Log in as the scmscm user, start the Collection Manager, wait 1 to 2 minutes for the creation of the database tables, and then run the script:

~scmscm/db_maint/create_periodic_del_procs.sh --nf

2. Edit the periodic delete configuration file. See the “Configuring Periodic Delete” section on page 5-4.

3. To apply the new configuration, run the dbperiodic.sh utility script. See the “Applying the Periodic Delete Configuration File” section on page 5-5.

Caution If you enable both aggregation and the periodic delete feature, be sure to increase their respective config values.

Periodic deletion of a table does not begin while a previous periodic deletion is still running. This prevents excessive load on the database, which would degrade insertion performance in the adapters.

When two or more tables are scheduled to be reduced at the same time, the tables are processed in the order in which they are listed in the periodic delete configuration file.

For ease of configuration, you can schedule periodic deletion for all tables consecutively on one schedule.

All periodic delete activity is recorded in the system log file. For Solaris, the system log file is located in the /var/adm/messages directory; for Linux, the system log file is located in the /var/log/messages directory. The following is an example of the system log file at the time when the periodic delete action occurs.

Feb 17 05:00:07 app-redhat79 logger: /opt/scmscm/db_maint/periodic_del.sh for hourly all - running single del for RPT_USAGE_NF to 14 daysFeb 17 05:00:09 app-redhat79 logger: Starting periodic delete for RPT_USAGE_NF keeping 14 daysFeb 17 05:00:09 app-redhat79 logger: Deleted rows: 0 from the table RPT_USAGE_NF

Caution Be sure to disable the MySQL partitioning process in the Collection Manager when you enable the periodic delete feature.

• Configuring Periodic Delete, page 5-4

• Applying the Periodic Delete Configuration File, page 5-5


OL-24187-04


Configuring Periodic Delete

By default, the periodic delete configuration file (nf-dbperiodic.conf) is located in the directory ~scmscm/db_maint/. The file has a structure similar to an INI file, where each section describes a particular data reduction operation for a specific set of tables, to be performed according to a specified schedule.

Note The name of each section of the file is not used when the file is parsed. Use any desired name.

Each section begins with the section name in brackets, and contains the parameters shown in the following table. (Not all parameters are required in each section of the configuration file.) Separate the parameters and their values by an equal sign (=). Table 5-2 contains examples of periodic delete configuration files.

Note Values for all parameters except active and daystokeep can be either a single value, a list of values separated by commas, a range of values (two values separated by a dash), or an asterisk (*) which signifies all possible values. A range is not possible for tablenames.

In the following example, all fields are set to their default values.

# This nf-dbperiodic.conf file emulates the legacy style for periodic# deletion. All tables are processed every hour on the hour, and# records are kept for 30 days.[4 hourly 30 days]active = truetablenames = RPT_USAGE_NF,RPT_TRANSACTION_NF,RPT_GLB_USAGE_NF daystokeep = 14minute = 0hour = 0,4,8,12,16, 20

Table 5-2 Parameters in the Periodic Delete Configuration File

Parameter Name Explanation Values Default Example

active Specifies whether or not to use this section of the configuration file

true/false true false

tablenames Names of the tables to which this section applies

Names of tables separated by commas, or * for all tables

* (all) RPT_USAGE_NF

daystokeep Number of days to keep records

Positive integers 14 30

minute

hour

day

month

The time at which to perform the deletion in this section of the configuration file

0 … 59, *

0 … 23, *

1 … 31, *

1 … 12, *

30

(all) *

(all) *

(all)

0

0,4,8,12,16,20

1

1,3,5,7,9,11


OL-24187-04


In this example, all tables are reduced at 4:30 A.M., leaving 10 days of data in each table. In addition, the real-time tables are reduced every hour, leaving three days of data in each table.

# This nf-dbperiodic.conf file reduces all tables once a day and# real-time tables once an hour.active = truetablenames = *daystokeep = 10minute = 30hour = 4[hourly real-time]active = truetablenames = RPT_USAGE_NF,RPT_TRANSACTION_NF,RPT_GLB_USAGE_NFdaystokeep = 3minute = 0hour = *

Applying the Periodic Delete Configuration File

To load and apply a new periodic delete configuration file or to view the current file, run the dbperiodic.sh script:

~scmscm/scripts/dbperiodic.sh --nf[ --dump ] [ --load | --loadfile=path_to_dbperiodic.conf ]

When the script is used to load a new configuration file, it parses the file, verifies its validity, and updates the scmscm user's crontab to reflect the changes. Table 5-3 lists the dbperiodic.sh options.

The following example shows how to print the current periodic delete configuration:

~scmscm/scripts/dbperiodic.sh --dump

Note This script prints the loaded periodic delete configuration. If the current periodic delete configuration file is not yet loaded, the configuration can vary from the script output.

The following example shows how to load the periodic delete configuration file from ~scmscm/db_maint/dbperiodic.conf:

~scmscm/scripts/dbperiodic.sh --nf --load

Table 5-3 dbperiodic.sh Options

Option Description

--load Load the periodic delete configuration from /export/home/scmscm/db_maint/ nf-dbperiodic.conf

--loadfile=path to periodic delete configuration file

Load the periodic delete configuration file from the specified directory

--dump Print the periodic delete configuration

--h Display these options


OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository Deleting a Table

Deleting a TableTo delete one table or all current tables from the database, run the droptable.sh script:

~scmscm/scripts/droptable.sh --nf [ -f ] tableParameter

Table 5-4 lists the droptable.sh options.

The following example shows how to drop a CM:NFR table named RPT_USAGE_NF from the database with no request for confirmation:

~scmscm/scripts/droptable.sh --nf -f RPT_USAGE_NF

The following example shows how to drop all tables from the database:

~scmscm/scripts/droptable.sh --nf ALLTABLES

Deleting Old RecordsTo remove records from a database table based on the timestamps of the records, run the prunetable.sh script:

~scmscm/scripts/prunetable.sh --nf [ -f ] num_days table_name

Table 5-5 lists the prunetable.sh options.

The following example shows how to delete all records that are more than seven days old from a table named RPT_USAGE_NF.

Table 5-4 droptable.sh Options

Option Description

table_name Drop table_name from the database

ALLTABLES Drop all tables from the database

-f Drop by force (no questions asked or errors reported)

-h Display these options

Table 5-5 prunetable.sh Options

Option Description

num_days The maximum age (in days) of records not to deleted.

table_name The table for which to delete records

-f Drop by force (no questions asked or errors reported).

-h Display these options.


OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository UNICODE Support for Sybase Database

Because the –f flag is not specified, there can be requests for confirmation and errors are reported.

>~scmscm/scripts/prunetable.sh --nf 7 RPT_USAGE_NF

UNICODE Support for Sybase DatabaseTo add UTF support for the Sybase database, run the following script:

~scmscm/scripts/sybutfsupport.sh


OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository Managing the Bundled Sybase Database

Managing the Bundled Sybase DatabaseManaging the bundled Sybase database includes:

• Backing up and restoring a database

• Updating Sybase with a changed IP address

• Updating the Sybase database statistics

The Cisco Collection Manager adds a timestamp to each NetFlow Record it receives and stores in the database. This timestamp is used when various maintenance operations are performed on the database tables.

Use the following scripts to maintain the bundled Sybase database only:

• ~scmscm/scripts/sybback.sh

• ~scmscm/scripts/sybrestore.sh

• ~scmscm/db_maint/update_statistics.sh

Note The Sybase back up and restore options require that the Sybase database resides on the same server as the Cisco Collection Manager.

Backing up the Sybase DatabaseTo create text file backups of all the tables in the database, run the sybback.sh --nf script:

~scmscm/scripts/sybback.sh --nf path_to_backup_directory

The script converts all tables to ASCII files and copies the files to a backup directory. Table 5-6 lists the sybback.sh --nf options.

Note Modify the path if you want to change the default location. The destination location path is defined as: ~scmscm/cm/config/dbpacks/sybase/ase12.5.1/sybase.properties file.

Restoring a Sybase Database To restore a database from the backup file created by the sybback.sh script, run the sybrestore.sh script:

~scmscm/scripts/sybrestore.sh --nf

Table 5-7 lists the sybrestore.sh options.

Table 5-6 sybback.sh Options

Option Description



OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository Updating Sybase with a Changed IP Address

Note The scripts sybback.sh and sybrestore.sh are not a viable backup mechanism for Sybase. They are designed for backing up and restoring small amounts of data, for example, transferring small tables between machines.

Note If you require a viable backup mechanism, consult the Sybase Backup Server product documentation.

Updating Sybase with a Changed IP AddressWhen you change the IP address, update the Sybase server. As the root user, issue the following command:

~scmscm/setup/syb_interfaces.sh

Updating Sybase Database StatisticsRun the update_statistics.sh script once a week at a time of low database load, for example, at night.

~scmscm/db_maint/update_statistics.sh

Table 5-7 sybrestore.sh Options

Option Description



OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository Managing the External MySQL Database

Managing the External MySQL DatabaseManaging the MySQL database includes:

• Backing up and restoring a database

The Cisco Collection Manager adds a timestamp to each NetFlow Record it receives and stores in the database. This timestamp is used when various maintenance operations are performed on the database tables.

Use the following scripts to maintain the MySQL database:

• ~scmscm/scripts/mysqlback.sh

• ~scmscm/scripts/mysqlrestore.sh

Note The MySQL back up and restore options require that the Sybase database resides on the same server as the Cisco Collection Manager.

Backing up the MySQL DatabaseTo create text file backups of all the tables in the database, use the mysqlback.sh script:

~scmscm/scripts/mysqlback.sh --nf

The backup files are stored in a SYBASE directory under the Cisco Collection Manager's home directory. The option to specify a directory path is not supported.

The script converts all tables to ASCII files and copies the files to a backup directory. Table 5-8 lists the mysqlback.sh options.

Restoring a MySQL Database To restore a database from the backup file created by the mysqlback.sh script, run the mysqlrestore.sh script:

~scmscm/scripts/mysqlrestore.sh --nf

The script converts all tables to ASCII files and copies the files to a backup directory. Table 5-9 lists the msqlrestore.sh options.

Table 5-8 mysqlback.sh Options

Option Description

--nf Write backup CM:NF MySQL files

--help Displays these options

Table 5-9 mysqylrestore.sh Options

Option Description

--nf Restore the CM:NF database using the sql files in the specified directory

--help Displays these options


OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository Partitioning the MySQL Database

Partitioning the MySQL DatabaseYou can store records in an external MySQL database. When you configure MySQL as the database, partitioning is supported.

• Prerequisites, page 5-11

• Configuring CM:NF Partitioning, page 5-11

PrerequisitesThe partitioning feature is supported only when the MySQL server meets the following conditions:

• MySQL version 5.1.06 or later

• thread_stack value in my.cnf file should be set to at least 256k under the [mysqld] section

Restart the MySQL server to enable the configuration changes to take effect.

Configuring CM:NF Partitioning

Caution If a partition is created on the existing data, all existing data is deleted . Back up existing data before running the partitioning script.

The default configuration file (nf-partitions.conf) is located in the ~scmscm/cm/config/dbpacks/mysql/4.0.20/partitions/netflow/directory.

The following is a sample of the configuration file:

[partitions_config1]table_names = RPT_TRANSACTION_NF,RPT_USAGE_NF,RPT_GLB_USAGE_NF no_of_partition = 30 period = 1 scheduler_time = 23:3

Each section begins with a unique section name starting with partitions_config in brackets and contains the parameters shown in Table 5-10. Separate the parameters and their values by an equal sign (=).

Table 5-10 Parameters in the Partitioning Configuration File


table_names Name of the table to partition.

Should start with RPT_ and end with_NF.. Should not be duplicated across sections.

— RPT_USAGE_NF

RPT_GLB_USAGE_NF

no_of_partition The number of partitions to create in the database.

1 - 65 — 30


OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository Configuring CM:NF Partitioning

To create and view partitions, run the partitions.sh script:

~scmscm/scripts/partitions.sh --nf [ --create ] [ --non_partitioned ] [ --partitioned ] [--partitions [table_name]] [events][ --help ]

Table 5-11 lists the partitions.sh options.

Perform the following steps to enable the partitioning feature:

1. Log in as scmscm user.

2. Change directory to ~scmscm/scripts/.

3. Customize the ~scmscm/cm/config/dbpacks/mysql/4.0.20/partitions//netflow/nf-partitions.conf file based on your requirements.

4. Run ~scmscm/scripts/loadsqlprocedures.sh --nf --par to load the required support files for partitioning.

5. Run ~scmscm/scripts/partitions.sh --nf --create to create the partitioning.

Note Ensure that the periodic deletion process is disabled in Collection Manager when you enable the partitioning feature.

period The number of days that the records are retained in each partition.

1 - 5 — 1

scheduler_time The time at which to trigger the daily partitioning event scheduler in the database.

Hour and minutes. Format: HH:MM

— 23:30

Table 5-10 Parameters in the Partitioning Configuration File (continued)


Table 5-11 partitions.sh Options

Option Description

--create Create partitions for database tables that are not partitioned

--non_partitioned Show details for database tables that are not partitioned

--partitioned Show details for database tables that are partitioned

--partitions [table_name] Show all table partitions details. [table_name] parameter shows mentioned table partitioned details.

--events Show configured partitioning events details

--help Display these options


OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository Managing the Oracle Database

During the creation of the partitions, the partitions.sh script performs the following actions:

• Enables the event scheduler on the MySQL server.

• If any of the tables (table_names) listed in the configuration file (partitions.conf) are not available in the database, partitioning is ignored for that particular table. The remaining tables are processed.

When the MySQL server is restarted after the partitioning, the event scheduler is disabled. To ensure the event scheduler remains enabled through a server restart, add “event_scheduler = 1” to the [mysqld] section of the my.cnf file.

Note If any missing table is created in the database later, run the partitions.sh script to create new partitioning. You can apply the same event scheduler for the new partitioning in each configuration section. This does not disturb the existing partitioning.

Note If any new tables are created in the database, define new configuration sections for these new tables and then run the partitions.sh script. This does not disturb the existing partitioning.

Although the partitioning event scheduler runs every day, it to creates or deletes a partition based on the configuration parameters no_of_partition and period. Even if the server is restarted, the partitioning event scheduler is not disturbed.

Managing the Oracle Database

Privileges The following are the mandatory Privileges that are to be granted to the CM-DB user: Create Session , Create Table, Create Procedure, Create Table Space, Unlimited Table Space, Create Index, Execute Any Procedure

Syntax:

grant CONNECT ,CREATE TABLE ,CREATE TABLESPACE ,CREATE PROCEDURE ,UNLIMITED TABLESPACE,CREATE ANY INDEX, EXECUTE ANY PROCEDURE to pqb_admin

Note Connect to the Oracle DB as SYSTEM/SYSDBA, log into the Oracle server with specific schema (apricot/avocado) and grant the above privileges:


OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository Aggregating Database Data

Aggregating Database DataYou can aggregate database data according to varying time periods and store data in the same database tables. Both raw and aggregated data are stored in order to maintain compatibility with reporting tools that pull this data to generate analysis reports. All reports that process raw data for display also can process the aggregated data because the data structure is the same.

During the aggregation process, table rows are aggregated per time period. These values are placed in a temporary table. The source records are then removed to allow insertion of the aggregated records.

The aggregation feature is supported in the following CM:NetFlow tables:

• RPT_USAGE_NF

• RPT_GLB_USAGE_NF

Configuring CM:NetFlow AggregationThe aggregation configuration file ~scmscm/cm/config/nf-aggregation.conf specifies the values used in the aggregation process.

The following is an example configuration file:

[agg_event1]procedure_names = P_RPT_USAGE_NF_AGGREGATE,P_RPT_GLB_USAGE_NF_AGGREGATEno_of_days = 2hourly_agg_days = 14scheduler_time = 23:30

Each section in the aggregation configuration file begins with a unique section name in brackets ([]) starting with agg_event . The file contains the parameters shown in Table 5-12. Separate the parameters and their values with an equal sign (=).


OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository Configuring CM:NetFlow Aggregation

Update the ~scmscm/cm/config/nf-aggregation.conf file to change the default configuration parameters and run the following script.

~scmscm/scripts/createaggevents.sh --nf [ --create ] [ --help ]

Table 5-13 lists the createaggevents.sh options.

Table 5-12 nf-aggregation.conf Parameters

Parameter Name Explanation Values Default Examples

procedure_names SQL procedure name to aggregate.

Any aggregation procedure name present in the respective database configuration location.

Sybase:

~scmscm/cm/config/dbpacks/

sybase/ase12.5.1/aggregation/netflow/

MySQL:


mysql/4.0.20/aggregation/netflow

Oracle:


oracle/9204e/aggregation/netflow

— P_RPT_USAGE_NF_AGGREGATE

and

P_RPT_GLB_USAGE_NF_AGGREGATE

no_of_days Number of days of raw data to retain in the table.

1 to 31 — 2

Hourly_agg_days Number of days of hourly aggregation to retain in the table.

14 — 14

scheduler_time The time at which to trigger the daily aggregation event scheduler.

Hour and minutes.

Format: HH:MM

— 23:30

Table 5-13 createaggevents.sh Options

Option Description--create Creates the corresponding aggregation events

--help Display these options


OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository Enabling CM:NF Aggregation in Sybase

Note Use loadsqlprocedure.sh --rdr --agg option to update the config changes into the database.

The Sybase, MySQL, and Oracle databases support aggregation. The following sections describe how to enable the aggregation feature:

• Configuring CM:NetFlow Aggregation, page 5-14

• Enabling CM:NF Aggregation in MySQL, page 5-17

• Enabling CM:NF Aggregation in Oracle, page 5-17

Enabling CM:NF Aggregation in SybaseSybase versions 12 and 15 support data aggregation.

Complete the following steps to enable the aggregation feature:

1. Login as scmscm user.

2. Run ~scmscm/scripts/loadsqlprocedures.sh --nf --agg to load the required support files for aggregation.

Note Data aggregation is a Cisco Collection Manager optional feature. Make sure the Oracle user has sufficient privileges to alter and execute the aggregation events. For example: grant select ,insert ,delete ,update on sys.job$ to pqb_admin;


OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository Enabling CM:NF Aggregation in MySQL

3. Create a cron job to run ~scmscm/cm/config/dbpacks/sybase/ase12.5.1/aggregation/netflow/cron/nf_run_aggregation.sh to schedule the aggregation process.

Cron job example:

crontab -e

Update the path and expected time to schedule the job:

30 23 * * * /<path>/nf_run_aggregation.sh

4. Refer to ~scmscm/cm/config/dbpacks/sybase/ase12.5.1/aggregation/cron/readme_run_aggregation_procedure.txt for more information.

Enabling CM:NF Aggregation in MySQLMySQL versions 5.1.06 and later support data aggregation.


1. Login as scmscm user.

2. Run ~scmscm/scripts/loadsqlprocedures.sh --nf --agg to load the required support files for aggregation.

Enabling CM:NF Aggregation in OracleOracle versions 9, 10, and 11 support data aggregation.


Step 1 Login as scmscm user.

Step 2 Run ~scmscm/scripts/loadsqlprocedures.sh --nf --agg to load the required support files for aggregation.

Note Ensure that the Oracle user has sufficient privileges to alter and execute the aggregation events (for example: GRANT SELECT, INSERT, DELETE ON sys.job$ TO pqb_admin).


OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository Managing the CSV Repository

Managing the CSV RepositoryYou can run a utility script to manage the repository of CSV files output by the Collection Manager. the CSV adapter writes these files to disk for use by a service provider’s operational support system (OSS) or by a third-party billing system. To prevent disk overflow, monitor the size of the CSV repository.

Note If the backup parameter is set to true, failure to delete CSV files can result in disk overflow. No CSV files are ever deleted.

Note The third-party application is responsible for managing the CSV files and deleting them as necessary.

To successfully invoke this script, the HTTP adaptor of the Collection Manager must be running. If the adapter is down, an error message is printed.

• CM:NF CSV Repository File Structure, page 5-18

• CM:NF Configuring the CSV File Repository, page 5-18

• CM:NF Configuring the Comma Escape, page 5-20

• CM:NF Configuring Escape of Nonprintable Characters, page 5-20

CM:NF CSV Repository File StructureCSV files are stored in several subdirectories. Each subdirectory is given the number of a NetFlow Record (NFR) tag. NFR tags define the type of the NFR. Each NFR is stored in the subdirectory whose name matches its NFR tag number.

The CSV files are numbered sequentially (automatically), with separate numbering in each directory. You can change the location of the parent directory by editing the ~scmscm/cm/config/nf.conf file.

CM:NF Configuring the CSV File RepositoryUse the ~scmscm/scripts/nf-csvconf.sh script to:

• List the number of NFRs currently stored in the repository.

• Configure the maximum number of CSV files and the maximum permissible number of reports (lines) in each file.

• Control whether a backup occurs when an old CSV file is about to be overwritten.

• Control whether each line in a CSV file contains an indication of the IP of the Cisco ASR 1000 that sent this NFR. (By default, this option is off.)


OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository CM:NF Configuring the CSV File Repository

Table 5-14 lists the nf-csvconf.sh options.

Note Instead of using this script, you can edit the ~scmscm/cm/config/nf-csvadapter.conf file. For changes in this file to take effect, you must restart the Collection Manager.

Note The same configuration is applied to all subdirectories in the CSV repository.

Note Setting these parameters does not change existing CSV files. Parameter value changes only affect files that are created subsequently.

The following example shows how to set the maximum number of CSV files per subdirectory to 1000.>~scmscm/scripts/nf-csvconf.sh --nf --maxfiles=1000

The following example shows how to set the maximum number records per CSV files to 10,000.

>~scmscm/scripts/nf-csvconf.sh --nf --maxlines=10000

The following example shows how to delete all files from the CSV repository:

~scmscm/scripts/nf-csvconf.sh --nf --clear

The following example shows how to disable backing up old CSV files in the repository:

~scmscm/scripts/nf-csvconf.sh –-backups=false

Table 5-14 nf-csvconf.sh Options

Option Description

--list Display the CSV repository contents (the number of NFRs currently stored in the repository).

--clear Delete all files from the CSV repository. (This option deletes all CSV files, but not the directories in which they are contained.)

--maxlines=N Set the maximum number of NFRs per CSV file to N (an integer between 1 and 20,000).

--maxfiles=M Set the maximum number of CSV files in each subdirectory to M (an integer between 10 and 10,000.)

--backups={true |false} Enable or disable backup of old CSV files.

--recordsource={true |false} Enable or disable the inclusion of the record source in CSV files.


OL-24187-04

Chapter 5 Managing Databases and the Comma Separated Value Repository CM:NF Configuring the Comma Escape

CM:NF Configuring the Comma EscapeWhen a comma is contained within a field in a CSV file, an escape sequence indicates that the comma does not mark the end of the field.

Three escape methods are supported:

• Single quotation marks—Single quotation marks surround any field that contains one or more commas. There is no special treatment of single quotation marks that currently exist in the NFRs.

• URL—%2C replaces each comma contained within a field. There is no special treatment of such sequences that currently exist in NFRs.

• Backslash—A backslash (\) precedes each comma contained within a field. There is no special treatment of backslashes that currently exist in NFRs.

The first two escape methods are compatible with Microsoft Excel. The backslash method is not compatible with Excel, but is retained for backward compatibility.

Single quotation marks are used by default. You can change the escape method by modifying the value of the escapeMethod attribute. This attribute is located in the ~/scmscm/cm/config/csvadapter.conf file. You must use one of the following values: backslash, quote, or url.

CM:NF Configuring Escape of Nonprintable CharactersThe CSV adapter can escape nonprintable characters. Enabling this option incurs a performance penalty for the adapter. By default, the option is disabled.

When the option is enabled, a backslash (\) precedes each character that cannot be printed, such as CR and LF, contained within a field.

Enable this option in the ~/scmscm/cm/config/nf-csvadapter.conf file. For changes to this configuration file to take effect, you must restart the Collection Manager.


OL-24187-04

Managing Databases and the CSV Repository · OL-24187-04 5 Managing Databases and the Comma...

Documents

Transcript of Managing Databases and the CSV Repository · OL-24187-04 5 Managing Databases and the Comma...