Necromancer
New Member
What the heck is a Yahoo Slurp Spider?
I just gotta ask
- Thread starter Necromancer
- Start date
- Status
It's the Yahoo bot program that searches and archives web pages so you can search for it on the Yahoo search engine.
The most prevalent one, too. Yahoo! Slurp is so prevalent, that a few Webmasters have banned them from their site, clamming them to be bandwidth hogs. Of course, these are usually forums with much larger memberships then ours, and the search spiders are usually proportionate to the membership. So you can imagine our forum, with an active membership usually not exceeding ten at any given moment (we have far more active members then that, but not all at the same time. It's spread out throughout the day), and we sometimes have up to seventy to over a hundred spiders on some days. Multiply that for a forum that has around 2-900 members on at the exact same moment, and you can imagine the hits the spider gives it.
Truthfully, though, I figure if your forum is set up to handle that many members, then the spiders should be a drop in the bucket. Besides, the more properly your site is indexed, the more accurately it will show up in searches.
Truthfully, though, I figure if your forum is set up to handle that many members, then the spiders should be a drop in the bucket. Besides, the more properly your site is indexed, the more accurately it will show up in searches.
- Status