| |

Xen VMs performance collection

Byetzion

Unlike VMware Server, Xen’s HyperVisor does not allow an easy collection of performance information. The management machine, called “Domain-0” is actually a privileged virtual machine, and thus – get its own small share of CPUs and RAM. Collecting performance information on it will lead to, well, collecting performance information for a single VM, and not the whole bunch.

Local tools, such as “xentop” allows collection of information, however, combining this with Cacti, or any other SNMP-based collection tool is a bit tricky.

A great solution is provided by Ian P. Christian in his blog post about Xen monitoring. He has created a Perl script to collect information. I have taken the liberty to fix several minor things with his permission. The modified scripts are presented below. Name the script (according to your version of Xen) “/usr/local/bin/xen_stats.pl” and set it to be executable:

For Xen 3.1

#!/usr/bin/perl -w

use strict;

# declare...
sub trim($);
#xen_cloud.tar.gz
# we need to run 2 iterations because CPU stats show 0% on the first, and I'm putting .1 second betwen them to speed it up
my @result = split(/n/, `xentop -b -i 2 -d.1`);

# remove the first line
shift(@result);

shift(@result) while @result && $result[0] !~ /^xentop - /;

# the next 3 lines are headings..
shift(@result);
shift(@result);
shift(@result);
shift(@result);

foreach my $line (@result)
{
  my @xenInfo = split(/[t ]+/, trim($line));
  printf("name: %s, cpu_sec: %d, cpu_percent: %.2f, vbd_rd: %d, vbd_wr: %dn",
    $xenInfo[0],
    $xenInfo[2],
    $xenInfo[3],
    $xenInfo[14],
    $xenInfo[15]
    );
}

# trims leading and trailing whitespace
sub trim($)
{
  my $string = shift;
  $string =~ s/^s+//;
  $string =~ s/s+$//;
  return $string;
}

For Xen 3.2 and Xen 3.3

#!/usr/bin/perl -w

use strict;

# declare…
sub trim($);

# we need to run 2 iterations because CPU stats show 0% on the first, and I’m putting .1 second between them to speed it up
my @result = split(/n/, `/usr/sbin/xentop -b -i 2 -d.1`);

# remove the first line
shift(@result);
shift(@result) while @result && $result[0] !~ /^[t ]+NAME/;
shift(@result);

foreach my $line (@result)
{
        my @xenInfo = split(/[t ]+/, trim($line));
        printf(“name: %s, cpu_sec: %d, cpu_percent: %.2f, vbd_rd: %d, vbd_wr: %dn“,
        $xenInfo[0],
        $xenInfo[2],
        $xenInfo[3],
        $xenInfo[14],
        $xenInfo[15]
        );
}
# trims leading and trailing whitespace
sub trim($)
{
        my $string = shift;
        $string =~ s/^s+//;
        $string =~ s/s+$//;
        return $string;
}

Cron settings for Domain-0

Create a file “/etc/cron.d/xenstat” with the following contents:

# This will run xen_stats.pl every minute
*/1 * * * * root /usr/local/bin/xen_stats.pl > /tmp/xen-stats.new && cat /tmp/xen-stats.new > /var/run/xen-stats

SNMP settings for Domain-0

Add the line below to “/etc/snmp/snmpd.conf” and then restart the snmpd service

extend xen-stats   /bin/cat /var/run/xen-stats

Cacti

I reduced Ian Cacti script to be based on a per-server setup, meaning this script gets the host (dom-0) name from Cacti, but cannot support live migrations. I will try to deal with combining live migrations with Cacti in the future.

Download and extract my modified xen_cloud.tar.gz file. Extract it, place the script and config in its relevant location, and import the template into Cacti. It should work like charm.

A note – the PHP script will work only on PHP5 and above. Works flawlessly on Centos5.2 for me.

Related posts:

  1. XenServer 6.0 with DRBD
  2. New version of Cacti, and using spine
  3. RHEL5 disk performance collection (sysstat)

Powered by YARPP.

Similar Posts

Service account

Byetzion

Assume you have a single purpose account. Maybe some service account, user which should run a single task ever, or even a case of a limited menu interface. You want your user(s) to reach there using SSH because that is the method to do it right and secured. You want it to be easy for…

Linux – Burn DL DVD+R media with custom layer break

Byetzion

To burn Dual-Layer (or Double-Layer, of you stick to the official name for DVD+R) medias, I use growisofs. The syntax is as follow: growisofs -dvd-compat -use-the-force-luke=break:1913760 -Z /dev/scd0=file.iso Change the break blocks to match your own values, and replace the file.iso with the actual name of your ISO file. If you do not set layer…

27 Comments

  1. Pingback: Monitoring Xen via SNMP - update - Ian P. Christian's personal blog

  2. With Xen 3.1 ‘/usr/sbin/xentop -b -i 2 -d.1’ – always returns Domain-0 CPU usage about 100%.
    And it should be full path to xentop since not all crontab jobs have right environment set.

    P.S. Thanks for the tool!

    1. Thanks for your comment.
      With Cacti, I tend to sample dom0 from other aspects, and not through this specific command. However, thanks for the info!

      Ez

  3. Hi,

    Nice job, but sadly the link to xen_cloud.tar.gz seems broken : “Error establishing a database connection”

    Cheers,

    1. Thanks for letting me know.
      There was a minor problem with the server which caused its backend DB to fail. It was fixed, and should not operate correctly. I have tested it to work.
      Let me know if it doesn’t, please.

      Thanks
      Ez

  4. I’m getting this error:
    Error: XML: Hash version does not exist.
    I think this is because you exported template with different version of cacti. Could you let me know which version you used?

  5. Hi,

    im having trouble getting things to work together.
    I could import the scripts into cacti and copied the 2 scripts into the cactio folders,
    than i used your script on the dom0, and can snmpwalk onto that machine:

    # snmpwalk -v2c -c public 172.17.4.101 NET-SNMP-EXTEND-MIB::nsExtendOutputFull.”xen-stats”
    NET-SNMP-EXTEND-MIB::nsExtendOutputFull.”xen-stats” = STRING: name: Domain-0, cpu_sec: 1563511, cpu_percent: 97.10, vbd_rd: 0, vbd_wr: 0
    name: monitoring-2, cpu_sec: 3833949, cpu_percent: 194.00, vbd_rd: 14746, vbd_wr: 20419
    name: uat-vm2, cpu_sec: 154919, cpu_percent: 0.00, vbd_rd: 25191, vbd_wr: 88203
    name: uat-vm4, cpu_sec: 94536, cpu_percent: 0.00, vbd_rd: 16, vbd_wr: 4
    name: vpn-vm2, cpu_sec: 110325, cpu_percent: 0.00, vbd_rd: 21, vbd_wr: 1

    But within cacti no graphs will be selectable when I try to create some,
    this is the output of the verbose query:

    + Running data query [19].
    + Found type = ‘4 ‘[script query].
    + Found data query XML file at ‘/var/www/cacti/resource/script_queries/xen_cloud_stats.xml’
    + XML file parsed ok.
    + Executing script for list of indexes ‘/usr/bin/php /var/www/cacti/scripts/xen_cloud_stats.php index’
    + Executing script query ‘/usr/bin/php /var/www/cacti/scripts/xen_cloud_stats.php query xen_name’
    + Found item [xen_name=”] index:
    + Found data query XML file at ‘/var/www/cacti/resource/script_queries/xen_cloud_stats.xml’
    + Found data query XML file at ‘/var/www/cacti/resource/script_queries/xen_cloud_stats.xml’
    + Found data query XML file at ‘/var/www/cacti/resource/script_queries/xen_cloud_stats.xml’

    Do i miss anything?

    greetings

  6. If you get errors like:

    NET-SNMP-EXTEND-MIB::nsExtendOutputFull.”xen-stats” = STRING: /bin/cat: /var/run/xen-stats: Permission denied

    it might be that SELinux is blocking access. I created this policy using audit2allow:

    module snmpd 1.0;

    require {
    type var_run_t;
    type snmpd_t;
    class file { read getattr };
    }

    #============= snmpd_t ==============
    allow snmpd_t var_run_t:file { read getattr };

    compile & load it, and snmpd is able to read the xen-stats file.

  7. Hi, i try this script on xen express 5.5 but i’ve this error

    Operator or semicolon missing before &amp at ./xen-stat.pl line 14.
    Ambiguous use of & resolved as operator & at ./xen-stat.pl line 14.
    Useless use of not in void context at ./xen-stat.pl line 14.
    Bareword “amp” not allowed while “strict subs” in use at ./xen-stat.pl line 14.
    Execution of ./xen-stat.pl aborted due to compilation errors.

    how can i fix it?

    1. This script is meant for Xen Community, and was tested on Centos5 versions. I did not try to adjust it to run on XS just yet.
      Sorry.

  8. > With Xen 3.1 ‘/usr/sbin/xentop -b -i 2 -d.1′ – always returns Domain-0 CPU usage about 100%.
    > And it should be full path to xentop since not all crontab jobs have right environment set.

    Change “-d.1” to “-d 1”. Apparently, some xentop implementations only accept seconds, not tenths of seconds.

  9. Hi,

    strangely I also see this error about &amp when trying it on OracleVM – which is VERY close to CentOS5 (just using a current Xen version – god bless them)

    [root@waxh0004 tmp]# perl xen_stats.pl
    Operator or semicolon missing before &amp at xen_stats.pl line 14.
    Ambiguous use of & resolved as operator & at xen_stats.pl line 14.
    Useless use of not in void context at xen_stats.pl line 14.
    Bareword “amp” not allowed while “strict subs” in use at xen_stats.pl line 14.
    Execution of xen_stats.pl aborted due to compilation errors.

    I’ll ask my perlspeaking friend to look into it, maybe she can fix it.
    (RHEL/CentOS are just not desirable for me any more. I think they spend too much energy on hiding all features / benefits of Xen with their old kernels and versions and libvirt, so they can promote the KVM better. It says enough about XENs superioty if they have to compare 2006’s Xen with 2010’s KVM :))

    1. Hi.
      It seems the script had an error due to WordPress formatting aids, and two ampersand signs together got translated to HTML marks of &

      I have fixed this issue on both scripts (you will probably use the 3.3 one) so if you download it now, you will get the correct (and working) version.

      Thanks for letting me know!

  11. hi darkfader,

    check the printf line
    quotation marks after ( and after n

    printf(“name: %s, cpu_sec: %d, cpu_percent: %.2f, vbd_rd: %d, vbd_wr: %dn”,

    georg doern

  12. Pingback: Xen Stats Revised - Shlrm.org Blag

  13. The Script does not run with SLES10SP3 with xen 3.2.3
    The output from xentop differs. I modified your script to work for me

    Here it is:
    #!/usr/bin/perl -w

    use strict;

    # declare…
    sub trim($);
    #xen_cloud.tar.gz
    # we need to run 2 iterations because CPU stats show 0% on the first, and I’m putting .1 second betwen them to speed it up
    my @result = split(/n/, `/usr/sbin/xentop -b -i 2 -d.1`);

    # remove the first line
    shift(@result);
    # remove the rest line till the Next “Name”-Header
    shift(@result) while @result && $result[0] !~ /^[t ]*NAME/;

    foreach my $line (@result)
    {
    my @xenInfo = split(/[t ]+/, trim($line));
    if ( $xenInfo[0] ne “NAME” ) {
    printf(“name: %s, cpu_sec: %d, cpu_percent: %.2f, vbd_rd: %d, vbd_wr: %dn”,
    $xenInfo[0],
    $xenInfo[2],
    $xenInfo[3],
    $xenInfo[14],
    $xenInfo[15]
    );
    }
    }

    # trims leading and trailing whitespace
    sub trim($)
    {
    my $string = shift;
    $string =~ s/^s+//;
    $string =~ s/s+$//;
    return $string;
    }

  14. Php script yields this error. Any thoughts as to why?

    $ /usr/bin/php /var/www/html/cacti/scripts/xen_cloud_stats.php public query xen_name
    PHP Notice: Undefined offset: 1 in /var/www/html/cacti/scripts/xen_cloud_stats.php on line 90
    PHP Notice: Undefined index: name in /var/www/html/cacti/scripts/xen_cloud_stats.php on line 93
    PHP Notice: Undefined index: name in /var/www/html/cacti/scripts/xen_cloud_stats.php on line 93
    PHP Notice: Undefined index: name in /var/www/html/cacti/scripts/xen_cloud_stats.php on line 97
    PHP Notice: Undefined index: name in /var/www/html/cacti/scripts/xen_cloud_stats.php on line 60
    PHP Notice: Undefined index: name in /var/www/html/cacti/scripts/xen_cloud_stats.php on line 60

    #php -v
    PHP 5.2.10 (cli) (built: Nov 13 2009 11:44:05)
    Copyright (c) 1997-2009 The PHP Group
    Zend Engine v2.2.0, Copyright (c) 1998-2009 Zend Technologies

  15. John try this :

    sudo snmpwalk -v2c -c public 10.10.1.19 NET-SNMP-EXTEND-MIB::nsExtendOutputFull.”xen-stats”

    For have a answer at this request, you need to have :

    * SNMPD.conf of the XenServer OK (extend xen-stats /bin/cat /var/run/xen-stats)
    Restart the snmp daemon after the modification
    * A good xen_stats.pl, you can download the script retouched by Bjoern (be careful when you copy from the web – ” = . after a copy/paste into your ssh connection)
    For test your xen_stats.pl, you must have information in the /var/run/xen-stats (result of the cron : */1 * * * * root /srv/scripts/xen_stats.pl > /tmp/xen-stats.new && cat /tmp/xen-stats.new > /var/run/xen-stats)
    * Import file in the cloud.tar.gz and verbose query the data queries xen cloud and you should see information of your XenServer

    It’s works on a XenServer 5.6.1 (CentOS) with a Cacti 0.8.7e on a Ubuntu Server

  16. if anybody have this problem ” /usr/bin/perl xen_stats.pl
    Unrecognized character xE2 in column 16 at xen_stats.pl line 19.”
    is because of the quotes, simply change ” by ” in line 19,
    are 2 different quote types, almost undetectable

  17. If anyone has a problem, “can not load the module. SNMP-FRAMEWORK-MIB
    The solution is to install the Management Information Base files
    aptitude install snmp-mibs-downloader

  19. Went threw all of the steps however the disk access graphs do not populate. I also have get this output returned from snmpwalk

    snmpwalk -v2c -c public hfcl05-03.vm NET-SNMP-EXTEND-MIB::nsExtendOutputFull..xen-stats.
    NET-SNMP-EXTEND-MIB::nsExtendOutputFull..xen-stats.: Unknown Object Identifier (Sub-id not found: nsExtendToken -> xen-stats.)

  20. Went threw all of the steps however the disk access graphs do not populate. I also have get this output returned from snmpwalk

    snmpwalk -v2c -c public host.vm NET-SNMP-EXTEND-MIB::nsExtendOutputFull..xen-stats.
    NET-SNMP-EXTEND-MIB::nsExtendOutputFull..xen-stats.: Unknown Object Identifier (Sub-id not found: nsExtendToken -> xen-stats.)

  21. Pingback: Monitoring Xen via SNMP – update
  22. Pingback: Xen performance monitoring - Cloudibee

Comments are closed.