setenv - How to write a SetEnvIf directive for Nginx to block bad bots and spiders - Stack Overflow

admin管理员组

文章数量:1384114

I have some code I use for my apache driven websites, but recently have inherited a site that is using Nginx, so I can't use an htaccess directive there. Can anyone point me in the proper direction to put this in Nginx speak:

SetEnvIfNoCase User-agent (yandex|baidu|spider|mj12|blex|webmeup
|ichiro|moget|naverbot|sogou|youdao|majestic|copyscape|sitesnagger
|teleportpro|copier|reaper|stripper|webzip|peek|scrape
|grub|linkdex|zyb|zibber|bullseye|blowfish|backdoor|spyder
|download|demon|linklooker|emailcollector|emailsiphon|emailwolf
|creep|getright|suck|ninja|ahrefsBot|semrush|petalbot
|dataforseobot|semrush|dotbot|meta) badbot=1

Order Allow,Deny
Allow from ALL
Deny from env=badbot

I have some code I use for my apache driven websites, but recently have inherited a site that is using Nginx, so I can't use an htaccess directive there. Can anyone point me in the proper direction to put this in Nginx speak:

SetEnvIfNoCase User-agent (yandex|baidu|spider|mj12|blex|webmeup
|ichiro|moget|naverbot|sogou|youdao|majestic|copyscape|sitesnagger
|teleportpro|copier|reaper|stripper|webzip|peek|scrape
|grub|linkdex|zyb|zibber|bullseye|blowfish|backdoor|spyder
|download|demon|linklooker|emailcollector|emailsiphon|emailwolf
|creep|getright|suck|ninja|ahrefsBot|semrush|petalbot
|dataforseobot|semrush|dotbot|meta) badbot=1

Order Allow,Deny
Allow from ALL
Deny from env=badbot

• nginx
• setenv
• setenvif

Share Improve this question Follow asked Mar 17 at 20:52 pathfinderpathfinder 1,79611 gold badge1919 silver badges2222 bronze badges

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

if ($http_user_agent ~ "yandex|baidu|spider|mj12|blex|webmeup|ichiro|moget|
naverbot|sogou|youdao|majestic|copyscape|sitesnagger|teleportpro|copier|
reaper|stripper|webzip|peek|scrape|grub|linkdex|zyb|zibber|bullseye|
blowfish|backdoor|spyder|download|demon|linklooker|emailcollector|emailsiphon|
emailwolf|creep|getright|suck|ninja|ahrefsBot|semrush|petalbot|dataforseobot|
semrush|dotbot|meta|OAI-SearchBot|ChatGPT|GPTBot|Claude|anthropic|CCBot|
cohere-ai|ImagesiftBot|Perplexity|Omigili|CrowdTanglebot|LinkisBot|
TweetmemeBot|LivelapBot|Openbot|Snooper|aiHitBot|Huaweisymantecspider") 
{ return 444; }

This works great for me, and additionally if you return 444 it only closes connection on your end without returning anything and hopefully leave the bot at the other end eating it's own resources for a while.

If you want to be proper you should probably send back a 403.

本文标签： setenvHow to write a SetEnvIf directive for Nginx to block bad bots and spidersStack Overflow