View Single Post
Old 07-07-2009, 04:58 AM   #7 (permalink)
RyanJ
 
RyanJ's Avatar
 
Status: SAS Member
Join Date: Mar 2009
Location: Pacific NW (USA)
Gender: Male
Age: 24
Posts: 132



Default

This may or may not be a solution, but it's an idea:

Blocking Googlebot and other Google robots with robots.txt

Manually creating a robots.txt file

----

It's not necessarily easy stuff however, and blocking all bots (especially those not as well behaved) might be difficult because of the sheer number of them crawling the web.
RyanJ is offline   Reply With Quote