Subj : Last couple of days with my binkp mailer..
To   : Janis Kracht
From : mark lewis
Date : Wed Sep 29 2010 08:02 pm


JK> Wow Baidubot is surely hitting on BBBS' webserver over here :(  Oh
JK> well.. off to inet.bbb to get rid of them haha

ml> you could just deny them access to those links in BBBS that lead to
ml> places like the messages, files and games... they are robots.txt
ml> friendly...

just to follow up and clarify this... an example is my gallery site... i may
not want the spiders trapsing thru the exhibits and looking at the actual
photos available... only the microthumbs and thumbs, sure so i just set up a
disallow for the exhibits area and let them wander over the others... then, if
i see them in the exhibits areas, then i know that they're not following
robots.txt and i can then contact their admins with my complaint or just block
them at the perimeter and deny them access to everything... depending on my
mood, of course ;)

)\/(ark


* Origin:  (1:3634/12)