What is a "Slurp Spider"?

[charlese]

When viewing "Current Active Users" (I wanted to see what the most popular posts were) , couldn't help but notice more than 12 of the viewers were identified as "Yahoo! Slurp Spiders" and there was one "Google Slurp Spider". Since it would be difficult, if not impossible to view 12 posts at the same time - this "Slurp Spider" must be something (or someone) that's quite unique! Have we attracted interest from these sources? Is this a Good thing?

[reible]

Hi,

Those are not people but rather bots that check in, you may want to do a search on line to get more information.

Ed

[paul heller]

The good news is that content here is indexed into the world wide web. That means that someone searching for woodworking advice in google or yahoo or msn or any of the other search sites just might find a path to something we are discussing here on this site (and might even be a future Shopsmith owner).

Paul

[admin]

They are search engines spiders that index the site, they were orignally not shown but I changed that to see how many we're indexing the site.

[charlese]

The one thing that was bothering me was the several spiders that were viewing personal public profiles! I can't think of a noble purpose for this slurping. Maybe advertising - maybe something else. As the old saying goes - "Just because I'm paranoid, doesn't mean no-one's after me." I googled Slurp spiders and it seems they are after information so they can sell it. - - Just business...

[dusty]

what do you really want?

http://www.google.com/search?sourceid=n ... urp+spider

If you read this, you'll find that vbulletin seems to support the right of the spider to slurp. This is an extract from one of the vbulletin team members posts - " all guests are guests, and are users online, its not really a lie that you had 900+ something or others browsing your site ".

[dusty]

Admin:

Now that I know they are there, I'd like to know just exactly what harm can they do?

I would also like to know what sort of information they typically seek that can be extracted from information in the forum archives?

Google, Yahoo and MSN seem to support their existence (the slurp spider, that is). Is this because they want and use the information that slurp spider collects?

Does the existence of the slurp spider explain why my connection seems so much slower or am I paranoid?

Right now, there are sixteen guests and fifteen are spiders. Six are in profiles. Only one is Google, the rest are Yahoo and I don't even use Yahoo.

Why are there NONE on sawmillcreek.org?

[admin]

Spiders are used by all search engines and are what makes a search engine work. Search engines use whats called a cache which is a snapshot of a web page, this snapshot is stored in the search engines index for use when someone searches for something. Yahoo has 10,600 web pages from shopsmith.net in its index, each of which have been cached/retrieved by the Slurp Spider.

Spiders can only get to what is "public" to other users of the web, so they cannot see bank account info etc. they are not capable of accessing pages that require a login, thats why they are listed as guests instead of users on this forum.

The "Slurp Spider" is what caches Yahoo's pages
Google uses the "Googlebot"
MSN uses "MSNBot"

[ryanbp01]

This brings a whole new meaning to "world wide web". That reminds me: time to watch "Arachnophobia".

See ya,

BPR

[dusty]

Admin:

Thank you for your response. I guess I can stop worrying about their existence since you don't.

But I still have a couple unanswered questions.

Admin:

Does the existence of the slurp spider explain why my connection seems so much slower .....

Why are there NONE on sawmillcreek.org? and others that I frequent?