Home
About
News
Downloads
Plugins
Change Log
Screenshots
Installation Guide
wiki
FAQ
Forums
Mailing Lists

PerfParse facilitates the storage and analysis of binary performance data produced by Nagios and produces high-quality accurate graphs of live data from standard Nagios plugins. A permanent history of plugin results can then be viewed with advanced analysis tools.

This project is an Add-On for Nagios®. Many thanks to Ethan Galstad.

Ben Clewett © 2004 - 2005.
 
Wiki pageHere
Project pageHere
Latest ImageHere
  Ben Clewett   Creator - Coding
  Garry W. Cook   Web Site - Images - Docs
  Yves Mettier   Coding
  Flo Gleixner   Coding
  Tim Wuyts   Coding
  Friedrich Priewasser   Coding
Perfparse Installation guide - Thu Dec 2 22:00:00 GMT 2004

Perfparse Installation guide - Thu Dec 2 22:00:00 GMT 2004

Garry Cook

Dependancies

Ben Clewett

Yves Mettier

Flo Gleixner

This documentation is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

For more details see the file COPYING in the source distribution of Perfparse.



Chapter 1. Introduction

These documents should help you to collect Plugin Performance Data within the Nagios application, and use the PerfParse Add-On to analyze this data.

First, you should read the Performance Data page of the Nagios Documentation. This can be found in the Advanced Topics section: http://nagios.sourceforge.net/docs/1_0/perfdata.html

NoteWhile the information at the link above is sound, there may be perfdata examples which are incorrect. For details regarding the format/output of perfdata, please see the following: http://nagiosplug.sourceforge.net/developer-guidelines.html#AEN185

Once you've read these docs, follow the link to 'File-based method' and 'Default Method', as there are some differences between the two that you should be aware of.

The Nagios documentation mentioned above should not be skipped, as it provides further background and insight into Nagios Performance Data.


Chapter 2. Choose your installation


2.1. Periodic Nagios Log Parse


2.1.1. Description

Nagios-1.2, when compiled with --with-file-perfdata, writes performance data into a file. Perfparse-log2mysql can read that file and save the position of the last read line. This method is also possible with nagios-2.0.

When you don't delete the file with the performance data after you parsed it, be sure that you have defined the Service_Log_Save_Position variable in your Perfparse configuration file, otherwise you will have duplicate entries in your database.

Note If you enable Service_Log_Save_Position, check if you have devined the Service_Log_Position_Mark_File variable to a path writeable to Perfparse. If not, perfparse will detect an error and stop.


2.1.2. Pros

  • Does not slow Nagios because it is designed for that purpose in Nagios

  • Nagios and Perfparse run in an independant way


2.1.3. Cons

  • When you don't want to waste disk space with performance data file, you need to clear that file and reboot nagios every...

  • A delay between generation and parsing of data will be experienced.


2.2. Nagios Invokes Perfparse


2.2.1. Description

When you compile Nagios-1.2, don't specify --with-file-perfdata but --with-default-perfdata. Then create a Nagios command to run perfparse-log2mysql every time a log is generated.


2.2.2. Pros

  • No delay between detection of data and parsing of data

  • Smaller risk to loose performance data


2.2.3. Cons

  • Fork a new process for each line of performance data, which can slow Nagios

  • Nagios and Perfparse are very dependant one upon the other. If one is broken, the other will not work perfectly.


2.3. Periodic User Log Parse


2.3.1. Description

When you compile Nagios-1.2, don't specify --with-file-perfdata but with --with-default-perfdata. Then create a Nagios command to run a perl script that will append performance data to a file.


2.3.2. Pros

  • Nagios and Perfparse run in an independant way

  • No need to reboot Nagios for maintenance on the performance file : easy maintenance

  • When using the embedded Perl interpreter with the cache (options --enable-embedded-perl and --with-perlcache), Nagios will not fork every time performance data are generated


2.3.3. Cons

  • Need a little more work when configuring Nagios and Perfparse


2.4. Pipe to Perfparse


2.4.1. Description

This is possible only with perfparsed. Perfparsed creates a pipe and reads continuously what others write there.

When you compile Nagios-1.2, don't specify --with-file-perfdata but --with-default-perfdata. Then create a Nagios command to run perfparse-log2mysql every time a log is generated.


2.4.2. Pros

  • No delay between detection of data and parsing of data

  • No data stored in temporary files that need to be cleaned

  • No more need of cron

  • Nagios and Perfparse run in an independant way

  • No need to reboot Nagios for maintenance on the performance file : easy maintenance

  • When using the embedded Perl interpreter with the cache (options --enable-embedded-perl and --with-perlcache), Nagios will not fork every time performance data are generated


2.4.3. Cons

  • Need a little more work when configuring Nagios and Perfparse


2.5. Perfparse Daemon Server

Perfparsed can run a server to which a client can send performance data. Because the method Pipe to Perfparse is more efficient, we suggest that perfparsed does not read data in a socket on the same host as nagios sending data to perfparsed. Use the pipe instead.

Perfparse-log2socket can be used to send data through the network without the need of the networking abilities of third party tools like database servers. Perfparsed can do the same as a daemon.


2.5.1. Pros

  • Data can come from another machine

  • No more need of cron

  • Nagios and Perfparse run in an independant way


2.5.2. Cons

  • Slower than with a pipe if you are running perfparsed on the same machine as nagios


2.6. Perfparse Daemon Client

Perfparse-log2* can read data through a socket if the connect to a perfparsed server. They connect to the server and ask for the data between two dates.

Perfparsed must run as a server and have the output_file module enabled to store the performance data in a file. Perfparse-log2* will ask perfparsed to read the file, pass the data through a basic filter (host/metric/key) and send the results to the client who requested them.


2.6.1. Pros

  • Data can come from another machine

  • Can be centralized in a big crontab

  • Storage on the distant hosts that may have more disk space


2.6.2. Cons

  • Data are not up to date

  • Be careful to duplicates


Chapter 3. Recompiling Nagios

If you have not already compiled Nagios to take advantage of Performance Data, you'll first need to recompile Nagios using either the --with-file-perfdata or the --with-default-perfdata option (See the previous chapter). If you want to be sure to use all of the options that you originally compiled into Nagios, you should check the config.status script in your Nagios source directory.

NoteIf you have performed a make clean since compiling Nagios the last time, this config.status file will not exist. If this is the case, run ./configure --help to determine the options that you will need for your implementation.

In config.status, look for a line that begins with:

exec ${CONFIG_SHELL-/bin/sh} ./configure
		

What follows the above line are the options that you originally used when configuring Nagios. It should look similar to this:

exec ${CONFIG_SHELL-/bin/sh} ./configure --disable-statuswrl \
--enable-embedded-perl --with-perlcache
		

Therefore, you would want to use the following command:

./configure --disable-statuswrl --enable-embedded-perl \
--with-perlcache --with-file-perfdata

or

./configure --disable-statuswrl --enable-embedded-perl \
--with-perlcache --with-default-perfdata
		

and then:

make nagios
		

Your new Nagios binary will be created in the base/ subdirectory. Copy this to your Nagios bin/ directory (you may want to backup the original Nagios binary first).

NoteFirst copy path/to/nagios/bin/nagios to path/to/nagios/bin/nagios.orig, then copy the newly created Nagios binary to path/to/nagios/bin/nagios.pp. You can then copy either version to path/to/nagios/bin/nagios, in order to test the new binary and/or back out if there were errors.


Chapter 4. Compiling Perfparse

PerfParse takes the performance data from Nagios and files it as binary data inside a Relational Database, currently MySQL. This is completed by a periodic parse of the log file (serviceperf.log) containing this data, as mentioned above.

NotePlease use InnoDB tables and MySQL version = 4.0.11 upwards. This allows transactions so that bad data can be discarded, as well as referential integrity. Therefore the program can always be safely killed.

You will probably also need the mysql-devel packages installed on your system. You can download the MySQL packages directly from >. You will also need the gd graphic libs and all its dependencies. Download gd from >. And the glib-devel package is also required, probably available in your distribution, and downloadable from >.

NoteNagios does not have to be built with support for MySQL.

PerfParse (as of v.0.08) uses a standard configure mechanism. A typical user who has nagios installed in, say, /usr/local/nagios will want to use the following commands:

$ ./configure \
	--prefix=/usr/local/nagios \
	--with-imagedir=/usr/local/nagios/share/images/ \
	--with-cgidir=/usr/local/nagios/sbin \
	--with-http_image_path=/nagios/images

$ make

# make install -=or=- make install-strip
		

By using 'make install-strip', you remove the core information from the binary files, which makes them about half their normal size. This causes the PerfParse CGI to load much more quickly.

However, should PerfParse core-dump for some reason, you will not be able to retrieve any information from it, as it has no core to dump.

Whichever way you install, the necessary files will be placed in your Nagios path (default path shown):

perfparse-log2mysql - /usr/local/nagios/bin (extracts data from Nagios into DB)
perfparsed - /usr/local/nagios/bin (server that extracts data from Nagios into DB)
perfparse.sh.example - /usr/local/nagios/bin (script to use when run from cron)
perfparse-db-purge - /usr/local/nagios/bin (delete data from database)
perfparse-db-tool - /usr/local/nagios/bin (tool for working with the DB)
perfparse.cfg.example - /usr/local/nagios/etc (Perfparse configuration file)
perfparse.cgi - /usr/local/nagios/sbin (creates graphs of the perfdata)
		

If this is your first time using PerfParse, 'perfparse.sh.example' and 'perfparse.cfg.example' should be renamed, removing the '.example'. If you have upgraded PerfParse, please compare these new files with your existing files to see if there are any changes that may need to be added. This allows you to make changes and add options to the files without worrying about losing them during next upgrade.

You must also create a database within MySQL. Create a database (does not have to be the same one that Nagios uses) and a user in that database. Next, use the mysql_create script in the scripts directory to create the necessary tables within the database.

Eg, with user 'bob' and database 'nagios':

$ cat mysql_create.sql | mysql -u bob -p -D nagios
		

At some point in the future, creating the DB will be a function of perfparse-db-tool. Run 'perfparse-db-tool --help' to see if this feature is available. If not, use the method described above, and then run 'perfparse-db-tool --update' to be sure that any new DB changes are utilized.


Chapter 5. Enabling Performance Data


5.1. All methods

Next you'll need to enable the Performance Data options in your main Nagios config file.

In the 'PROCESS PERFORMANCE DATA OPTION' section, change the 'process_performance_data' option to '1'. The default is '0' or off.

Then do the specific operations to the chosen method.

Restart Nagios and Performance data should now be enabled. If you find that you don't get data after a few minutes, you may need to make a change from within the Nagios 'Process Info' cgi. To do this, log in to your Nagios site as the administrative user, go to the 'Process Info' page, and at the bottom of the Process Information box, ensure you have:

Performance Data Being Processed?    Yes
			

If Performance Data is not being processed, click the 'Enable performance data' link in the 'Process Commands' box.

If Performance Data is still not being added to the perfdata log files, you probably have not yet installed Nagios plugins that will generate Performance Data. The first version of plugins to support Performance Data is 1.4.0, which at the time of this writing is in an alpha state. Your best bet to ensure that you have a maximum number of Performance Data producing plugins is to use the latest version from CVS.


5.2. Periodic Nagios Log Parse

Then add the following four lines to the 'HOST AND SERVICE PERFORMANCE DATA PROCESSING COMMANDS' section:

xpdfile_host_perfdata_file=/usr/local/nagios/var/hostperf.log
xpdfile_host_perfdata_template=$TIMET$\t$HOSTNAME$\t$OUTPUT$\t$PERFDATA$

xpdfile_service_perfdata_file=/usr/local/nagios/var/serviceperf.log
xpdfile_service_perfdata_template=$TIMET$\t$HOSTNAME$\t$SERVICEDESC$\t$OUTPUT$\t$SERVICESTATE$\t$PERFDATA$
			

NoteThe last line above may wrap on your screen, please edit accordingly.


5.3. Nagios Invokes Perfparse

Edit misccommands.cfg and add the following :

define command{
        command_name    process-service-perfdata
        command_line    /usr/bin/printf "%b" "$TIMET$\t$HOSTNAME$\t$SERVICEDESC$\t$OUTPUT$\t$SERVICESTATE$\t$PERFDATA$\n" | $USER2$/bin/perfparse-log2mysql -c $USER2$/etc/perfparse.cfg
        }
			

Note$USER2$ is defined in resources.cfg and points to the Perfparse installation directory.

NoteIn perfparse.cfg, Service_Log must be defined as "-".


5.4. Periodic User Log Parse

Edit misccommands.cfg and add the following :

define command{
        command_name    process-service-perfdata
        command_line    $USER2$/bin/perfparse_nagios_command.pl $USER2$/var/perfdata-service.log "$TIMET$" "$HOSTNAME$" "$SERVICEDESC$" "$OUTPUT$" "$SERVICESTATE$" "$PERFDATA$"
        }
			

Note$USER2$ is defined in resources.cfg and points to the Perfparse installation directory.

Edit a script like this :

#!/bin/sh
$USER2=/perfparse/installation/directory

mv ${USER2}/var/perfdata-service.log ${USER2}/var/perfdata-service.log1
cat ${USER2}/var/perfdata-service.log1 | ${USER2}/bin/perfparse-log2mysql -c ${USER2}/etc/perfparse.cfg
rm ${USER2}/var/perfdata-service.log1
			

NoteIn perfparse.cfg, Service_Log must be defined as "-".


5.5. Pipe to Perfparse

With nagios-1.2, edit misccommands.cfg and comment out the following definitions for host and service performance data :

define command{
        command_name    process-host-perfdata
			...
        }

define command{
        command_name    process-service-perfdata
			...
        }
			

Note$USER2$ is defined in resources.cfg and points to the Perfparse installation directory.

Then in the nagios.cfg verify the following are in the file :

cfg_file=/usr/local/nagios/etc/nagios_perfparse.cfg
process_performance_data=1
host_perfdata_command=process-host-perfdata
service_perfdata_command=process-service-perfdata
			

In the perfparse.cfg, make sure the variable Service_Log is set to "|/path/to/perfparse/var/perfdata-service.log".

Note Permissions for "perfdata-service.log" need to be set as nagios daemon can write to the FIFO.

Finally, in the nagios_perfparse.cfg file :

define command{
        command_name    process-service-perfdata
        command_line    $USER2$/bin/perfparse_nagios_pipe_command.pl $USER2$/var/perfdata-service.log "$TIMET$" "$HOSTNAME$" "$SERVICEDESC$" "$OUTPUT$" "$SERVICESTATE$" "$PERFDATA$"
        }
			

or :

define command{
        command_name    process-service-perfdata
        command_line    $USER2$/bin/perfparse_nagios_pipe_command.pl /usr/local/nagios/var/rw/serviceperf.log "$TIMET$" "$HOSTNAME$" "$SERVICEDESC$" "$OUTPUT$" "$SERVICESTATE$" "$PERFDATA$"
        }
			

5.6. Perfparse Daemon Server


5.6.1. Client side

Configure Perfparse (either perfparsed or perfparse-log2socket) to output data to a socket. Set the following options in the configuration file :

Use_Storage_Socket_Output =         "yes"
Storage_Socket_Output_Host_Name =   "hostname"
Storage_Socket_Output_Port =        "1234"
				

Follow the other methods to configure the input of the client.


5.6.2. Server side

Configure perfparsed to run the server. Set the following options in the configuration file :

Service_Log =                ""
Server_Port =                "1234" 

				

Use any storage for the output.


5.7. Perfparse Daemon Client


5.7.1. Client side

Configure Perfparse (perfparse-log2*) to read from a socket. Set the following options in the configuration file :

Service_Log =                 ">host:port"
				

Follow the other methods to configure the output of the client with the module you want.


5.7.2. Server side

See Method Perfparse Daemon Server.


Chapter 6. Using Perfparse


6.1. Perfparse configuration file

To create a perfparsed.cfg config file using the defaults :

				./perfparsed --show_config   # to see the current config.
			

Note Run perfparsed, perfparse-log2any or any perfparse-log2* program with --show_config option to see the current config.

Because the output of a Perfparse program is compatible with its configuration file, you can run this to compare to the current perfparse.cfg file :

				./perfparsed --show_config > /var/tmp/perfparse.cfg
			

When /var/tmp/perfparse.cfg is generated, you can edit it and overwrite the old configuration file.


6.2. perfparse-log2mysql

First you should edit 'perfparse.cfg', adding the correct database settings (if using something other than 'nagios' as DB name, username, and password). The options are :

Use_Storage_Mysql (only in perfparsed ; considered as enabled in perfparse-log2mysql)
DB_User
DB_Pass
DB_Name
DB_Host
		

To parse the Performance Data log file using 'Perfparse', use the following command:

./perfparse-log2mysql -r
		

This will parse the data and display a report. Using these:

More options are available, see:

./perfparse-log2mysql --help
		

You can edit your crontab to run perfparse-log2mysql on a regular basis, taking the data from the performance log files and importing it into the DB. For instance, to run PP every 15 minutes, you might use something like the following:

# Run PerfParse every 15 minutes to update Nagios graphs.
15 * * * * /usr/local/nagios/bin/perfparse.sh
		

If you are using method Periodic Nagios Log Parse, PerfParse remembers its position, and will not commit the data until it's ready. Therefore it can be stopped, started, killed, without danger of missing or reprocessing data.

Only 'perfparse-log2mysql' accepts the options mentioned above. If you want to use options when running from cron, you'll need to edit the 'perfparse.sh' script, adding your options to the line that calls 'perfparse-log2mysql'.

NoteUpon upgrading PerfParse, this script is installed as perfparse.sh.example, so if you have edited the original 'perfparse.sh', you may need to add your options to the new file if changes have been made in the distribution version.

To view the your PerfParse graphs, use the same url path as Nagios (http://localhost/nagios/cgi-bin by default) and add /perfparse.cgi at the end.

You can also add a PerfParse link to the Nagios side menu by inserting the following lines of code in path/to/nagios/share/side.html:

tr
    td width=13
        img src="images/greendot.gif" width="13" height="14"
        name="perfparse-dot"
    /td
    td nowrap
        a href="/nagios/cgi-bin/perfparse.cgi?all_bin=1" target="main"
        onMouseOver="switchdot('perfparse-dot',1)"
        onMouseOut="switchdot('perfparse-dot',0)"
        class="NavBarItem"PerfData Graphs/a
    /td
/tr
			

6.3. perfparse-log2socket

Same as perfparse-log2mysql. Options are :

Use_Storage_Socket_Output
Storage_Socket_Output_Host_Name
Storage_Socket_Output_Port
			

6.4. perfparsed

Anyone to write this ?


Chapter 7. Database Maintenance

Deletion policies are defined at various places with default and user-defined policies available. A policy defines the time at which data is permanently deleted from the DB. The data is deleted by running the provided program 'perfparse-db-purge'. It is advised that this is run daily at some time when the server is at low load.

To purge the DB daily using 'perfparse-db-purge' from crontab, you might use something like this:

# Purge PerfParse DB daily at 3 AM.
00 3 * * * /usr/local/nagios/bin/perfparse-db-purge
		

You'll want to make sure that you've set up your deletion policies before you run perfparse-db-purge the first time. Otherwise you risk losing some data that you might want to keep around for a while.

If this is the first time you have installed PerfParse, then don't worry, you won't have enough data to risk deletion for about a month.

If upgrading from a version prior to 0.0.11, please do not run perfparse-db-purge until you have set your deletion policies in the PerfParse CGI or by using the perfparse-db-tool.

There are some default policies which may be edited but not deleted, these are as follows:

policy_name	    	    Description
-------------------------------------------------------
'Binary Default'        Default policy for Binary data.
'Raw Default'           Default policy for Raw data.
		

These are the defaults where the user has not specified any other value. If they don't exist, they are automatically created by the purge tool.

Another useful tool for maintaining your DB is 'perfparse-db-tool'. You can run 'perfparse-db-tool --help' for a list of available options. An important one to note is '--reset_host_delete_policy ', which will set the delete policy to host for all host data.

For the named host, this will update all binary and raw delete policies for the attached services and metrics, and set them to use the host default deletion policy. This will also update the host to use the respective default policy, raw and binary.

Another option of interest is '--update', which should be run anytime that you upgrade to a new version of Perfparse. New features requiring changes to the DB will then be available, as this will update the DB with new tables and/or fields.


Chapter 8. Conclusion

That's it, that's all. If you've got any suggestions for this or any other documentation, please feel free to drop me a line on the PerfParse Users mailing list. Instructions for joining the PerfParse Mailing lists can be found at http://perfparse.sourceforge.net/mailinglists.php.

Enjoy!


[ Back To Top ]

Monitored by Nagios   Powered by MySQL   Mailing lists and web space provided by SourceForge