Running Systems

As an administrator, I am responsible for many setups and configurations, sometimes hand tailored to supply an answer to a set of given demands.

As a human, I err, and the common method of verifying that you have avoided error is by answering this simple rule: “Does it work after these changes?”

In the world of computers there is hardly ever simple true or false. We would have expected it to be boolean world – either it works or it doesn’t, but we are not there. The world of computers is filled with “works better” and “works worse”, and sometimes we forget that.

This long prologue was meant to bring up the subject of monitoring and evaluating your actions. While the simplest method of evaluation remains “Does it work?”, there are some additional, more subtle methods of verifying that things work according to your specifications.

One of the tools which helps me see, in the mirror of time, the effect of changes I have done is a graphical tool called Cacti. This tool graphs a set of predefined parameters which were chosen by me. It has no special AI, it cannot guess anything, and I am quite happy with it, as I can understand for myself the course of events better.

This post is about a mis configured Amavisd daemon. Amavis is a wrapper which scans using both Spamassassin and a selected Antivirus (ClamAV, in my case, as it has proven itself to me as a good AV) mail supplied by the local MTA.

I had a directive looking like this in it:

[‘ClamAV-clamscan’, ‘clamscan’,
“–stdout –disable-summary -r –tempdir=$TEMPBASE {}”, [0], [1],
qr/^.*?: (?!Infected Archive)(.*) FOUND$/ ],

It worked, however, this server, as it appears, was heavily loaded for a while now. Since it’s a rather strong server, it was not really visible unless you take a look at the server’s Cacti. On about 80%+ of the time the CPUs were on 100% with the process ‘clamscan‘. I have decided yesterday to solve the heavy load, and for that modified the file ‘/etc/amavisd.conf‘ to include the primary ClamAV section as follows:

[‘ClamAV-clamd’,
&ask_daemon, [“CONTSCAN {}n”, “/tmp/clamd”],
qr/bOK$/, qr/bFOUND$/,
qr/^.*?: (?!Infected Archive)(.*) FOUND$/ ],

This uses clamd instead of clamscan. The results were a drastic decrease on the CPU consumption and system average load, as can be seen in the Cacti graph (around 4 AM):

Cacti load average graph

The point is that while both configuration worked, I had the tools to understand that the earlier configuration was not good enough. Through tracking parameters on the system for a while, I could monitor my configuration modifications using a wider perspective, and reach better conclusions.