Long while and not much to talk about
I’ve been to our own little “August Penguin” just few days ago, and it wasn’t too good. I avoided the lectures (who needs secure rsync?!?), but sat and talked with friends about all kinda stuff. One of the advantages of such an event is that you get to meet lots of your linux-related friends, and you get to talk technicalities with whoever you feel like doing it. It’s the place for it. You speak of funny geekish things you had, and people around you laugh. You tell your saddest stories (how the storage went berserk, for example), and people understand what you’re talking about. That’s the fun part. Other than that, it wasn’t that good. Wasn’t worth the long drive, and the day off (well, the day off was worth anyhow, for what it is).
I didn’t do much since. I’ve refused a job offer, as the salary was too low, and the driving was too long, and I will be interviewed for AIX clusters soon, which I don’t like anyhow…
Our client, the one with the hosting server, got hacked. It was a minor thing, and some defacement, but he detected it a day too late, and we could not dig it out of the logs. Not only, we found out the hacker run sendmail, on some arbitrary port, and we just removed it, and tried to locate the hole in though he got in. It’s something with PHP. It’s always something with PHP. I have no idea as to what, especially when there are around 400 hosted sites on this server. We’ll know better in the future, when it happens again.
A server of a friend of mine was highly loaded in the last few days. It appears Apache was killing his machine. His Apache is supposed, in 90% of the cases, to respond once, and close the connection. I’ve helped him, and removed the keepalive, and although it decreased the number of running servers dramatically, there were few instances of apache still consuming high CPU. Using extended server status, and top, I was able to notice the high load was due to some specific PHP script, which got to wait for 10 minutes, each time. I was able to point out at that script, and he checked it, and found a bug. Small PHP bugs get to increase load on servers dramatically. One should be aware of it. Anyone can write PHP scripts, but only some can write it with performance in mind. Mind you.