I know, it's very frustrating. The server that it's on is pretty fast (dual 2.4Ghz P4 Xeons) but it's getting beaten down by robots. I'm all for people indexing the site, but there have been 76 requests for robots.txt in the past 24 hours and some of the bots are hitting the site as hard as they can. Right now that's causing the wiki to go nuts trying to keep statistics and it winds up locking pages so that it can increment page counts. I'm trying to keep them out but every once in a while they overwhelm the server
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2004-08-01 16:03
, have you ever "deactived" wiki, making it static or something like that?
or install another cheap webserver for wiki.gallery.menalto.com handling all wiki stuff and keep the dual xeon for the rest.
bharat
Joined: 2002-05-21
Posts: 7994
Posted: Tue, 2004-08-03 04:54
All of that is way more effort than its worth. I just made my watchdog scripts slightly smarter about it so that should take care of this line of attack for a while. Usually every couple of months something new crops up and performance takes a hit for a little bit until I can give them the proper beatdown they deserve
Posts: 7994
I know, it's very frustrating. The server that it's on is pretty fast (dual 2.4Ghz P4 Xeons) but it's getting beaten down by robots. I'm all for people indexing the site, but there have been 76 requests for robots.txt in the past 24 hours and some of the bots are hitting the site as hard as they can. Right now that's causing the wiki to go nuts trying to keep statistics and it winds up locking pages so that it can increment page counts. I'm trying to keep them out but every once in a while they overwhelm the server
Posts: 32509
, have you ever "deactived" wiki, making it static or something like that?
or install another cheap webserver for wiki.gallery.menalto.com handling all wiki stuff and keep the dual xeon for the rest.
Posts: 7994
All of that is way more effort than its worth. I just made my watchdog scripts slightly smarter about it so that should take care of this line of attack for a while. Usually every couple of months something new crops up and performance takes a hit for a little bit until I can give them the proper beatdown they deserve