Blocking sitecrawler/bots/scanner
$8-15 USD / hour
need a script or a piece of code for htaccess or robots, that blocks all spiders, crawler, bot and scanner. that for example google or another search enignes cant index the site or content of a site, so that only a specific user can enter the page, example if he gets the correct url link.
Project ID: #6733247
About the project
Awarded to:
i can do this for you. there is specific config file to add to block bot crawlers ... its to add in the root folder ...
2 freelancers are bidding on average $11/hour for this job
You mean you want to block web spiders and crawler( like google , bing , yahoo) from indexing your website . am i right?