Posts Tagged ‘cacti’

Cacti NetApp Ontap API data query

Sunday, June 23rd, 2013

I have been using the excellent template and scripts from this forum post, however, when the NetApp device is loaded with LUNs and volumes, the script will cause the Cacti to timeout, and during that time, consume CPU. The original cause of this problem was a workaround to some NetApp Perl API bug the original author found, which forced him to query the entire data set for each sub-query. This is nice for five, or even ten volumes, but when you’re around 400 volumes, things just look bad.

Due to that, I have taken upon myself to make this script more scalable, but forcing a single data query from the NetApp for each data type (volume, LUN, system, etc) and data query type (get, index, etc). A unique file was created with its name being the storage_device_name.data_type.query_type. Following queries to any subset of this data were just accessing this file, and not the remote NetApp device, killing network, CPU, and tending to time out on operation and leave huge blank parts in the graphs.

I will post my modified template in the forum as well, but I place it here, just so that it will be both available for me, and for any interested reader.

Get it here:NetApp_OnTap-SDK_cacti-20130623.tar.gz

 

Things to remember…

Monday, October 24th, 2011

As my work takes me to various places (where technology is concerned), I collect lots of browser tab of things I want to keep for later reference.
I have to admit, sadly, that I lack the time to sort them out, to make a real good and nice post about them. I do not want to lose them, however, so I am posting now those which I find or found in the past as more useful to me. I might expand either of them one day into a full post, or elaborate further on them. Either or none. For now – let’s clean up some tab space:
Reading IPMI sensors. Into Cacti, and into Nagios, with some minor modifications by myself (to be disclosed later, I believe):
Cacti
Nagios
This is somewhat info of the plugin check_ipmi_sensor
And its wiki (in German. Use Google for translation)
XenServer checks:
check_xen_pool
Checking XenServer using NRPE
But I did not care about Dom0 performance parameters, as they meant very little regarding the hypervisor’s behavior. So I have combined into it the following XenServer License Check. Unfortunately, I could run it only on the XenServer domain0, due to python version limitations on my Cacti /Nagios server.
You can obtain XenServer SDK
This plugin looks interesting for various XenServer checks, but I have never tried it myself.
Backing up (exporting) XenServer VMs as a scheduled task. I have had it modified extensively to match my requirements, but I am allowed to, it has some of its sources based on my blog 🙂
Installing Dell OpenManage on XenServer 5.6.1, and the nice thing is that it works fine on XenServer 6 as well.
Oracle ASM recovery tips . One day I will take it further, and investigate possible human errors and methods of fixing them. Experience, they say, has a value 🙂
A guide dealing with changing from raw to block devices in Oracle ASM . This is only a small part of it, but it’s the thing that interests me.
Understanding Steal Time in Linux Xen-based VMs.
Because I always forget, and I’m too lazy to search again and again (and reach the same page again and again): Upgrading PHP to 5.2 on Centos 5
And last – a very nice remote-control software fomr my Android phone. Don’t leave home without it. Seriously.

Reduced to only 23 tabs is excellent. This was a very nice job, and these links will be useful. To me, for sure. I hope that to you as well.

Work around ISP QoS limitations

Friday, May 29th, 2009

ISPs which enforce QoS limitations suddenly, without alerting the customer, are abusing their force. QoS limitation is not a bad thing, from the ISP’s point of view, but changing the customer deal without notifying him seems to me to be unfair.

This is a recipe for a QoS workaround.

Ingredients:

  • One fast Internet connection which is not used to its full capacity
  • Defined target service provider. I use Giganews as an NNTP, which is the fastest method of obtaining content today. You should have the service list of IPs. Luckily, Giganews use only two IP addresses
  • One “evil” ISP which enforces QoS for external targets
  • One server in the ISP’s hosting farm, which has no speed or transfer limitations, and is probably not bound by the ISP’s QoS
  • For a better looking dish – some graphing solution, such as Cacti or MRTG

Directions:

  • Setup OpenVPN Server on the hosted server
  • Setup OpenVPN Client on your NNTP/Other service client (your desktop, your Linux router, etc) – This can also be a Windows machine, but configuration varies a bit.
  • Define, in your OpenVPN client.conf line(s) which look like this:

route <SERVICE_IP>

route <SERVICE_IP2>

  • If this is a router machine, activate NAT on it. Of course – remember to set this rule to work after reboot too!

iptables -t nat -A POSTROUTING -o tun+ -j MASQUERADE

  • For your good feeling, try to pickup data from before and after, and compare.
  • Start the OpenVPN Service on the server, on the client, and restart your NNTP/Other service downloads.
  • Serve with a smile 🙂

The result dish is both tasty and good looking! see below:

QoS_override.png

A word of warning – OpenVPN is a VPN tool. As such, it uses encryption and varios methods which are very secure. This means that for high througput, such as mine (about 10Mb/s) you will see the impact on the router/workstation’s CPU. Under virtualization, I get about 2% additional system CPU utilization from a 2x3GHz Xeon CPU. For older router devices this could result in an overworked router. I am so glad I got rid of my old P2 350MHz router in favor of the virtualized one.

Centreon and batch-adding hosts

Monday, April 27th, 2009

Centreon is a nice GUI wrapper for Nagios. It is using MySQL as its configuration engine, and it functions quite well. One thing Cacti can do but Centreon can’t is mass automatic addition of servers. I have had a new site with an installed Centreon, and I wanted to add about 40 servers to be monitored. This is a tedious work, and I was searching for some semi-automatic method of doing it.
This is not perfect, but it worked for me.
In this case I do not replicate service-group relationship, but only add a mass of servers.

First – create a text file containing a list of servers and IPs. It should look like this:
serv1:1.2.3.4
serv2:10.2.1.3
new_srv:2.3.4.1

I have placed in in /tmp/machines

Second – find the last host entry. In my case the DB name is Centreon, so I run the following command:

mysql -u root -p centreon -e’select host_id from host’

This should return a colum with numbers. Find the largest one and increment it by one. In my example the last one was 19, so my initial host_id will be 20.

You should now find the host_template_model_html_id you are to use. There are few methods for that, but the easiest way is to find another host information which matches to some level your desired information. In my case it was called “DB1”, so this looks like this:

mysql -u root -p centreon -e”select host_template_model_htm_id from host where host_name=’DB1′”

Please note that my blog formatting might change the quote character. You might not want to copy/paste it, but type it yourselves.

The result of the above query should give us a template ID. In my case it was “2”, which is fine by me.

If you want a better reference for the values entered, you can do a whole select for a single host to verify your values match mine:

mysql -u root -p centreon -e”select * from host where host_name=’DB1’G”

This should give you long listing and information of the host, as a reference.

My script goes like this, based on the assumptions made above:

#!/bin/bash
HOST=20
for i in `cat /tmp/machines`
do
   NAME=`echo $i | cut -f1 -d:`
   IP=`echo $2 | cut -f2 -d:`
   echo "insert into host values ('$HOST',2,NULL,NULL,1,NULL,NULL,NULL,NULL,'$NAME','$NAME','$IP',NULL,NULL,'2','2','2','2','2',NULL,'2',NULL,NULL,'2','2','2','2',NULL,NULL,'2',NULL,NULL,'0',NULL,'1','1');" >> /tmp/insert_sql.sql
   echo "insert into extended_host_information values('',$HOST,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL);" >> /tmp/insert_sql.sql
   let HOST++
done

This should create a file called /tmp/insert_sql.sql which then should be first reviewed, and then inserted into your database.

Needless to say – back up your database first, just in case:

mysqldump -u root -p –opt -B centreon > /tmp/centreon_backup.sql

and then insert the newly created data:

mysql -u root -p centreon < /tmp/insert_sql.sql

Notice – at this point, no service relationship is created. I think it is quite a chore only to create the nodes. Adding the service relationships complicates things a bit, and I did not want to go there at this specific stage. However, for few tenths of monitored hosts, this is quite a lifesaver.

Notice that this is only Centreon configuration, and you will be required to apply it (through the GUI) to Nagios.

Xen VMs performance collection

Saturday, October 18th, 2008

Unlike VMware Server, Xen’s HyperVisor does not allow an easy collection of performance information. The management machine, called “Domain-0” is actually a privileged virtual machine, and thus – get its own small share of CPUs and RAM. Collecting performance information on it will lead to, well, collecting performance information for a single VM, and not the whole bunch.

Local tools, such as “xentop” allows collection of information, however, combining this with Cacti, or any other SNMP-based collection tool is a bit tricky.

A great solution is provided by Ian P. Christian in his blog post about Xen monitoring. He has created a Perl script to collect information. I have taken the liberty to fix several minor things with his permission. The modified scripts are presented below. Name the script (according to your version of Xen) “/usr/local/bin/xen_stats.pl” and set it to be executable:

For Xen 3.1

#!/usr/bin/perl -w

use strict;

# declare...
sub trim($);
#xen_cloud.tar.gz
# we need to run 2 iterations because CPU stats show 0% on the first, and I'm putting .1 second betwen them to speed it up
my @result = split(/n/, `xentop -b -i 2 -d.1`);

# remove the first line
shift(@result);

shift(@result) while @result && $result[0] !~ /^xentop - /;

# the next 3 lines are headings..
shift(@result);
shift(@result);
shift(@result);
shift(@result);

foreach my $line (@result)
{
  my @xenInfo = split(/[t ]+/, trim($line));
  printf("name: %s, cpu_sec: %d, cpu_percent: %.2f, vbd_rd: %d, vbd_wr: %dn",
    $xenInfo[0],
    $xenInfo[2],
    $xenInfo[3],
    $xenInfo[14],
    $xenInfo[15]
    );
}

# trims leading and trailing whitespace
sub trim($)
{
  my $string = shift;
  $string =~ s/^s+//;
  $string =~ s/s+$//;
  return $string;
}

For Xen 3.2 and Xen 3.3

#!/usr/bin/perl -w

use strict;

# declare…
sub trim($);

# we need to run 2 iterations because CPU stats show 0% on the first, and I’m putting .1 second between them to speed it up
my @result = split(/n/, `/usr/sbin/xentop -b -i 2 -d.1`);

# remove the first line
shift(@result);
shift(@result) while @result && $result[0] !~ /^[t ]+NAME/;
shift(@result);

foreach my $line (@result)
{
        my @xenInfo = split(/[t ]+/, trim($line));
        printf(“name: %s, cpu_sec: %d, cpu_percent: %.2f, vbd_rd: %d, vbd_wr: %dn“,
        $xenInfo[0],
        $xenInfo[2],
        $xenInfo[3],
        $xenInfo[14],
        $xenInfo[15]
        );
}
# trims leading and trailing whitespace
sub trim($)
{
        my $string = shift;
        $string =~ s/^s+//;
        $string =~ s/s+$//;
        return $string;
}

Cron settings for Domain-0

Create a file “/etc/cron.d/xenstat” with the following contents:

# This will run xen_stats.pl every minute
*/1 * * * * root /usr/local/bin/xen_stats.pl > /tmp/xen-stats.new && cat /tmp/xen-stats.new > /var/run/xen-stats

SNMP settings for Domain-0

Add the line below to “/etc/snmp/snmpd.conf” and then restart the snmpd service

extend xen-stats   /bin/cat /var/run/xen-stats

Cacti

I reduced Ian Cacti script to be based on a per-server setup, meaning this script gets the host (dom-0) name from Cacti, but cannot support live migrations. I will try to deal with combining live migrations with Cacti in the future.

Download and extract my modified xen_cloud.tar.gz file. Extract it, place the script and config in its relevant location, and import the template into Cacti. It should work like charm.

A note – the PHP script will work only on PHP5 and above. Works flawlessly on Centos5.2 for me.