Block Bad Bots and Spiders using .htaccess

Below is a useful code block for blocking a lot of the known bad bots and site rippers currently out there.

You might also check out the following .htaccess rules to Harden your website’s Security even further.

Simply add the code to your /public_html/.htaccess file:

# Bad bot
SetEnvIfNoCase User-Agent "^abot" bad_bot
SetEnvIfNoCase User-Agent "^aipbot" bad_bot
SetEnvIfNoCase User-Agent "^asterias" bad_bot
SetEnvIfNoCase User-Agent "^EI" bad_bot
SetEnvIfNoCase User-Agent "^libwww-perl" bad_bot
SetEnvIfNoCase User-Agent "^LWP" bad_bot
SetEnvIfNoCase User-Agent "^lwp" bad_bot
SetEnvIfNoCase User-Agent "^MSIECrawler" bad_bot
SetEnvIfNoCase User-Agent "^nameprotect" bad_bot
SetEnvIfNoCase User-Agent "^PlantyNet_WebRobot" bad_bot
SetEnvIfNoCase User-Agent "^UCmore" bad_bot
SetEnvIfNoCase User-Agent "Alligator" bad_bot
SetEnvIfNoCase User-Agent "AllSubmitter" bad_bot
SetEnvIfNoCase User-Agent "Anonymous" bad_bot
SetEnvIfNoCase User-Agent "Asterias" bad_bot
SetEnvIfNoCase User-Agent "autoemailspider" bad_bot
SetEnvIfNoCase User-Agent "Badass" bad_bot
SetEnvIfNoCase User-Agent "Baiduspider" bad_bot
SetEnvIfNoCase User-Agent "BecomeBot" bad_bot
SetEnvIfNoCase User-Agent "Bitacle" bad_bot
SetEnvIfNoCase User-Agent "bladder\ fusion" bad_bot
SetEnvIfNoCase User-Agent "Blogshares\ Spiders" bad_bot
SetEnvIfNoCase User-Agent "Board\ Bot" bad_bot
SetEnvIfNoCase User-Agent "Board\ Bot" bad_bot
SetEnvIfNoCase User-Agent "Convera" bad_bot
SetEnvIfNoCase User-Agent "ConveraMultiMediaCrawler" bad_bot
SetEnvIfNoCase User-Agent "c-spider" bad_bot
SetEnvIfNoCase User-Agent "DA" bad_bot
SetEnvIfNoCase User-Agent "DnloadMage" bad_bot
SetEnvIfNoCase User-Agent "Download\ Demon" bad_bot
SetEnvIfNoCase User-Agent "Download\ Express" bad_bot
SetEnvIfNoCase User-Agent "Download\ Wonder" bad_bot
SetEnvIfNoCase User-Agent "dragonfly" bad_bot
SetEnvIfNoCase User-Agent "DreamPassport" bad_bot
SetEnvIfNoCase User-Agent "DSurf" bad_bot
SetEnvIfNoCase User-Agent "DTS Agent" bad_bot
SetEnvIfNoCase User-Agent "EBrowse" bad_bot
SetEnvIfNoCase User-Agent "eCatch" bad_bot
SetEnvIfNoCase User-Agent "edgeio" bad_bot
SetEnvIfNoCase User-Agent "Email\ Extractor" bad_bot
SetEnvIfNoCase User-Agent "EmailSiphon" bad_bot
SetEnvIfNoCase User-Agent "EmailWolf" bad_bot
SetEnvIfNoCase User-Agent "EmeraldShield" bad_bot
SetEnvIfNoCase User-Agent "ESurf" bad_bot
SetEnvIfNoCase User-Agent "Exabot" bad_bot
SetEnvIfNoCase User-Agent "ExtractorPro" bad_bot
SetEnvIfNoCase User-Agent "FileHeap!\ file downloader" bad_bot
SetEnvIfNoCase User-Agent "FileHound" bad_bot
SetEnvIfNoCase User-Agent "Forex" bad_bot
SetEnvIfNoCase User-Agent "Franklin\ Locator" bad_bot
SetEnvIfNoCase User-Agent "FreshDownload" bad_bot
SetEnvIfNoCase User-Agent "FrontPage" bad_bot
SetEnvIfNoCase User-Agent "FSurf" bad_bot
SetEnvIfNoCase User-Agent "Gaisbot" bad_bot
SetEnvIfNoCase User-Agent "Gamespy_Arcade" bad_bot
SetEnvIfNoCase User-Agent "genieBot" bad_bot
SetEnvIfNoCase User-Agent "GetBot" bad_bot
SetEnvIfNoCase User-Agent "GetRight" bad_bot
SetEnvIfNoCase User-Agent "Gigabot" bad_bot
SetEnvIfNoCase User-Agent "Go!Zilla" bad_bot
SetEnvIfNoCase User-Agent "Go-Ahead-Got-It" bad_bot
SetEnvIfNoCase User-Agent "GOFORITBOT" bad_bot
SetEnvIfNoCase User-Agent "heritrix" bad_bot
SetEnvIfNoCase User-Agent "HLoader" bad_bot
SetEnvIfNoCase User-Agent "HooWWWer" bad_bot
SetEnvIfNoCase User-Agent "HTTrack" bad_bot
SetEnvIfNoCase User-Agent "iCCrawler" bad_bot
SetEnvIfNoCase User-Agent "ichiro" bad_bot
SetEnvIfNoCase User-Agent "iGetter" bad_bot
SetEnvIfNoCase User-Agent "imds_monitor" bad_bot
SetEnvIfNoCase User-Agent "Industry\ Program" bad_bot
SetEnvIfNoCase User-Agent "Indy\ Library" bad_bot
SetEnvIfNoCase User-Agent "InetURL" bad_bot
SetEnvIfNoCase User-Agent "InstallShield\ DigitalWizard" bad_bot
SetEnvIfNoCase User-Agent "IRLbot" bad_bot
SetEnvIfNoCase User-Agent "IUPUI\ Research\ Bot" bad_bot
SetEnvIfNoCase User-Agent "Java" bad_bot
SetEnvIfNoCase User-Agent "jeteye" bad_bot
SetEnvIfNoCase User-Agent "jeteyebot" bad_bot
SetEnvIfNoCase User-Agent "JoBo" bad_bot
SetEnvIfNoCase User-Agent "JOC\ Web\ Spider" bad_bot
SetEnvIfNoCase User-Agent "Kapere" bad_bot
SetEnvIfNoCase User-Agent "Larbin" bad_bot
SetEnvIfNoCase User-Agent "LeechGet" bad_bot
SetEnvIfNoCase User-Agent "LightningDownload" bad_bot
SetEnvIfNoCase User-Agent "Linkie" bad_bot
SetEnvIfNoCase User-Agent "Mac\ Finder" bad_bot
SetEnvIfNoCase User-Agent "Mail\ Sweeper" bad_bot
SetEnvIfNoCase User-Agent "Mass\ Downloader" bad_bot
SetEnvIfNoCase User-Agent "MetaProducts\ Download\ Express" bad_bot
SetEnvIfNoCase User-Agent "Microsoft\ Data\ Access" bad_bot
SetEnvIfNoCase User-Agent "Microsoft\ URL\ Control" bad_bot
SetEnvIfNoCase User-Agent "Missauga\ Locate" bad_bot
SetEnvIfNoCase User-Agent "Missauga\ Locator" bad_bot
SetEnvIfNoCase User-Agent "Missigua Locator" bad_bot
SetEnvIfNoCase User-Agent "Missouri\ College\ Browse" bad_bot
SetEnvIfNoCase User-Agent "Mister\ PiX" bad_bot
SetEnvIfNoCase User-Agent "MovableType" bad_bot
SetEnvIfNoCase User-Agent "Mozi!" bad_bot
SetEnvIfNoCase User-Agent "Mozilla/3.0 (compatible)" bad_bot
SetEnvIfNoCase User-Agent "Mozilla/5.0 (compatible; MSIE 5.0)" bad_bot
SetEnvIfNoCase User-Agent "MSIE_6.0" bad_bot
SetEnvIfNoCase User-Agent "MSIECrawler" badbot
SetEnvIfNoCase User-Agent "MVAClient" bad_bot
SetEnvIfNoCase User-Agent "MyFamilyBot" bad_bot
SetEnvIfNoCase User-Agent "MyGetRight" bad_bot
SetEnvIfNoCase User-Agent "NASA\ Search" bad_bot
SetEnvIfNoCase User-Agent "Naver" bad_bot
SetEnvIfNoCase User-Agent "NaverBot" bad_bot
SetEnvIfNoCase User-Agent "NetAnts" bad_bot
SetEnvIfNoCase User-Agent "NetResearchServer" bad_bot
SetEnvIfNoCase User-Agent "NEWT\ ActiveX" bad_bot
SetEnvIfNoCase User-Agent "Nextopia" bad_bot
SetEnvIfNoCase User-Agent "NICErsPRO" bad_bot
SetEnvIfNoCase User-Agent "NimbleCrawler" bad_bot
SetEnvIfNoCase User-Agent "Nitro\ Downloader" bad_bot
SetEnvIfNoCase User-Agent "Nutch" bad_bot
SetEnvIfNoCase User-Agent "Offline\ Explorer" bad_bot
SetEnvIfNoCase User-Agent "OmniExplorer" bad_bot
SetEnvIfNoCase User-Agent "OutfoxBot" bad_bot
SetEnvIfNoCase User-Agent "P3P" bad_bot
SetEnvIfNoCase User-Agent "PagmIEDownload" bad_bot
SetEnvIfNoCase User-Agent "pavuk" bad_bot
SetEnvIfNoCase User-Agent "PHP\ version" bad_bot
SetEnvIfNoCase User-Agent "playstarmusic" bad_bot
SetEnvIfNoCase User-Agent "Program\ Shareware" bad_bot
SetEnvIfNoCase User-Agent "Progressive Download" bad_bot
SetEnvIfNoCase User-Agent "psycheclone" bad_bot
SetEnvIfNoCase User-Agent "puf" bad_bot
SetEnvIfNoCase User-Agent "PussyCat" bad_bot
SetEnvIfNoCase User-Agent "PuxaRapido" bad_bot
SetEnvIfNoCase User-Agent "Python-urllib" bad_bot
SetEnvIfNoCase User-Agent "RealDownload" bad_bot
SetEnvIfNoCase User-Agent "RedKernel" bad_bot
SetEnvIfNoCase User-Agent "relevantnoise" bad_bot
SetEnvIfNoCase User-Agent "RepoMonkey\ Bait\ &\ Tackle" bad_bot
SetEnvIfNoCase User-Agent "RTG30" bad_bot
SetEnvIfNoCase User-Agent "SBIder" bad_bot
SetEnvIfNoCase User-Agent "script" bad_bot
SetEnvIfNoCase User-Agent "Seekbot" bad_bot
SetEnvIfNoCase User-Agent "SiteSnagger" bad_bot
SetEnvIfNoCase User-Agent "SmartDownload" bad_bot
SetEnvIfNoCase User-Agent "sna-" bad_bot
SetEnvIfNoCase User-Agent "Snap\ bot" bad_bot
SetEnvIfNoCase User-Agent "SpeedDownload" bad_bot
SetEnvIfNoCase User-Agent "Sphere" bad_bot
SetEnvIfNoCase User-Agent "sproose" bad_bot
SetEnvIfNoCase User-Agent "SQ\ Webscanner" bad_bot
SetEnvIfNoCase User-Agent "Stamina" bad_bot
SetEnvIfNoCase User-Agent "Star\ Downloader" bad_bot
SetEnvIfNoCase User-Agent "Teleport" bad_bot
SetEnvIfNoCase User-Agent "TurnitinBot" bad_bot
SetEnvIfNoCase User-Agent "UdmSearch" bad_bot
SetEnvIfNoCase User-Agent "URLGetFile" bad_bot
SetEnvIfNoCase User-Agent "User-Agent" bad_bot
SetEnvIfNoCase User-Agent "UtilMind\ HTTPGet" bad_bot
SetEnvIfNoCase User-Agent "WebAuto" bad_bot
SetEnvIfNoCase User-Agent "WebCapture" bad_bot
SetEnvIfNoCase User-Agent "webcollage" bad_bot
SetEnvIfNoCase User-Agent "WebCopier" bad_bot
SetEnvIfNoCase User-Agent "WebFilter" bad_bot
SetEnvIfNoCase User-Agent "WebReaper" bad_bot
SetEnvIfNoCase User-Agent "Website\ eXtractor" bad_bot
SetEnvIfNoCase User-Agent "WebStripper" bad_bot
SetEnvIfNoCase User-Agent "WebZIP" bad_bot
SetEnvIfNoCase User-Agent "Wells\ Search" bad_bot
SetEnvIfNoCase User-Agent "WEP\ Search\ 00" bad_bot
SetEnvIfNoCase User-Agent "Wget" bad_bot
SetEnvIfNoCase User-Agent "Wildsoft\ Surfer" bad_bot
SetEnvIfNoCase User-Agent "WinHttpRequest" bad_bot
SetEnvIfNoCase User-Agent "WWWOFFLE" bad_bot
SetEnvIfNoCase User-Agent "Xaldon\ WebSpider" bad_bot
SetEnvIfNoCase User-Agent "Y!TunnelPro" bad_bot
SetEnvIfNoCase User-Agent "YahooYSMcm" bad_bot
SetEnvIfNoCase User-Agent "Zade" bad_bot
SetEnvIfNoCase User-Agent "ZBot" bad_bot
SetEnvIfNoCase User-Agent "zerxbot" bad_bot

# Apache < 2.4
<IfModule !mod_authz_core.c>
 <Limit GET POST>
  Order allow,deny
  Allow from all
  Deny from env=bad_bot
 </Limit>
</IfModule>
# Apache >= 2.4
<IfModule mod_authz_core.c>
 <Limit GET POST>
  <RequireAll>
   Require all granted
   Require not env bad_bot
  </RequireAll>
 </Limit>
</IfModule>

Was this answer helpful?

Also Read

How can I prevent my site from being hacked?

Typically, most sites are hacked because of poor passwords, older and exploitable software, or...

Redirect non-www to www over http and https

Use the following .htaccess code to redirect non-www URLs to www over http and https. Create, or...

How to Remove Multiple Slashes from URL

You can use this .htaccess rule for removing multiple trailing slashes anywhere in URL:...

Protect your WordPress from Brute Force Attacks

Recently, there was a worldwide, highly-distributed Brute Force Attack on WordPress sites....

How to execute php code as .htm or .html files

In your .htaccess file insert this line of code. addhandler application/x-httpd-php .htm .html