官网宣称DataForSeoBot爬虫支持robots.txt协议,但其实这货不怎么守规矩,robots.txt禁止以后,还爬的飞起。
所以建议需要屏蔽这个蜘蛛的,两手抓:
1、robots.txt协议禁止爬取
User-agent: DataForSeoBot Disallow: /
2、通过修改Nginx或其它WEB服务器规则禁止抓取
if ($http_user_agent ~* "DataForSeoBot/1.0" ) { return 403; }
3、通过禁止蜘蛛IP方式禁止访问
这个办法不推荐,蜘蛛IP有不少,还会新增改变,很难全部禁止掉。
总结:禁止DataForSeoBot爬虫最佳办法就是直接在服务器规则里面禁止,效果最好。
yisouspider蜘蛛爬虫的IP清单(1086个IP)
以下YisouSpider的IP是我通过网站日志统计的,并不是全部的YisouSpider的IP,后续会更新这个IP清单。
总共1086个IP!!!!!!! 全部是阿里云的IP,YisouSpider全部使用的是阿里云的服务器....
42.156.254.9942.156.254.9842.156.254.9742.156.254.9642.156.254.9542.156.254.9442.156.254.9342.156.254.9242.156.254.9142.156.254.7842.156.254.7742.156.254.7642.156.254.7542.156.254.7442.156.254.7342.156.254.7242.156.254.7142.156.254.7042.156.254.6942.156.254.6842.156.254.6742.156.254.6642.156.254.6542.156.254.6442.156.254.6342.156.254.6242.156.254.6142.156.254.6042.156.254.5942.156.254.5742.156.254.5642.156.254.5542.156.254.5442.156.254.5342.156.254.5242.156.254.5142.156.254.5042.156.254.4942.156.254.4842.156.254.4742.156.254.4642.156.254.4542.156.254.4442.156.254.1842.156.254.1642.156.254.12042.156.254.11942.156.254.11842.156.254.11742.156.254.11642.156.254.11542.156.254.11442.156.254.11342.156.254.11242.156.254.11142.156.254.11042.156.254.10942.156.254.10842.156.254.10742.156.254.10642.156.254.10542.156.254.10442.156.254.10342.156.254.10242.156.254.10142.156.254.10042.156.139.9842.156.139.9742.156.139.9542.156.139.9442.156.139.942.156.139.8342.156.139.8242.156.139.8142.156.139.8042.156.139.7942.156.139.7842.156.139.7742.156.139.7642.156.139.7442.156.139.7342.156.139.7242.156.139.7142.156.139.7042.156.139.742.156.139.6942.156.139.6742.156.139.6642.156.139.6542.156.139.6442.156.139.6342.156.139.6242.156.139.6042.156.139.642.156.139.5642.156.139.5542.156.139.5442.156.139.5342.156.139.5242.156.139.5142.156.139.5042.156.139.4942.156.139.4742.156.139.4642.156.139.4542.156.139.4442.156.139.4342.156.139.4242.156.139.4142.156.139.4042.156.139.442.156.139.442.156.139.3842.156.139.3742.156.139.3442.156.139.3342.156.139.3242.156.139.3142.156.139.3042.156.139.342.156.139.2942.156.139.2842.156.139.2542.156.139.2442.156.139.2342.156.139.2242.156.139.2142.156.139.2042.156.139.1942.156.139.1742.156.139.1642.156.139.1542.156.139.1442.156.139.1342.156.139.12542.156.139.12442.156.139.12342.156.139.12242.156.139.12142.156.139.11942.156.139.11842.156.139.11742.156.139.11642.156.139.11542.156.139.11442.156.139.11242.156.139.11142.156.139.11042.156.139.10942.156.139.10842.156.139.10742.156.139.10542.156.139.10442.156.139.10342.156.138.9842.156.138.9742.156.138.9542.156.138.9442.156.138.942.156.138.8342.156.138.8242.156.138.8142.156.138.8042.156.138.7942.156.138.7842.156.138.7742.156.138.7642.156.138.7442.156.138.7342.156.138.7242.156.138.7142.156.138.7042.156.138.742.156.138.742.156.138.6942.156.138.6742.156.138.6642.156.138.6542.156.138.6442.156.138.6342.156.138.6242.156.138.6042.156.138.642.156.138.5642.156.138.5542.156.138.5442.156.138.5342.156.138.5242.156.138.5142.156.138.5042.156.138.4942.156.138.4742.156.138.4642.156.138.4542.156.138.4442.156.138.4342.156.138.4242.156.138.4142.156.138.4042.156.138.442.156.138.3842.156.138.3742.156.138.3442.156.138.3342.156.138.3242.156.138.3142.156.138.3042.156.138.342.156.138.342.156.138.2942.156.138.2842.156.138.2542.156.138.2442.156.138.2342.156.138.2242.156.138.2142.156.138.2042.156.138.1942.156.138.1742.156.138.1642.156.138.1542.156.138.1442.156.138.1342.156.138.12542.156.138.12442.156.138.12342.156.138.12242.156.138.12142.156.138.12042.156.138.11942.156.138.11842.156.138.11742.156.138.11642.156.138.11542.156.138.11442.156.138.11342.156.138.11242.156.138.11142.156.138.11042.156.138.10942.156.138.10842.156.138.10742.156.138.10642.156.138.10542.156.138.10442.156.138.10342.156.138.10242.156.137.9842.156.137.9742.156.137.9542.156.137.9442.156.137.942.156.137.8342.156.137.8242.156.137.8142.156.137.8042.156.137.7942.156.137.7842.156.137.7742.156.137.7642.156.137.7442.156.137.7342.156.137.7242.156.137.7142.156.137.7042.156.137.742.156.137.6942.156.137.6742.156.137.6642.156.137.6542.156.137.6442.156.137.6342.156.137.6242.156.137.6042.156.137.642.156.137.5542.156.137.5442.156.137.5342.156.137.5242.156.137.5142.156.137.5042.156.137.4942.156.137.4742.156.137.4642.156.137.4542.156.137.4442.156.137.4342.156.137.4242.156.137.4142.156.137.4042.156.137.442.156.137.442.156.137.3842.156.137.3742.156.137.3442.156.137.3342.156.137.3242.156.137.3142.156.137.3042.156.137.342.156.137.2942.156.137.2842.156.137.2542.156.137.2442.156.137.2342.156.137.2242.156.137.2142.156.137.2042.156.137.1942.156.137.1742.156.137.1642.156.137.1542.156.137.1442.156.137.1342.156.137.12542.156.137.12442.156.137.12342.156.137.12242.156.137.12042.156.137.11942.156.137.11842.156.137.11742.156.137.11642.156.137.11542.156.137.11442.156.137.11342.156.137.11242.156.137.11142.156.137.11042.156.137.1142.156.137.10942.156.137.10842.156.137.10742.156.137.10642.156.137.10542.156.137.10442.156.137.10342.156.137.10242.156.137.1042.156.136.9842.156.136.9742.156.136.9542.156.136.9442.156.136.942.156.136.8342.156.136.8242.156.136.8142.156.136.8042.156.136.7942.156.136.7842.156.136.7742.156.136.7642.156.136.7442.156.136.7342.156.136.7242.156.136.7142.156.136.7042.156.136.742.156.136.742.156.136.6942.156.136.6742.156.136.6642.156.136.6542.156.136.6442.156.136.6342.156.136.6242.156.136.6042.156.136.642.156.136.642.156.136.5642.156.136.5542.156.136.5442.156.136.5342.156.136.5242.156.136.5142.156.136.5042.156.136.4942.156.136.4742.156.136.4642.156.136.4542.156.136.4442.156.136.4342.156.136.4242.156.136.4142.156.136.4042.156.136.442.156.136.442.156.136.3842.156.136.3742.156.136.3442.156.136.3342.156.136.3242.156.136.3142.156.136.342.156.136.342.156.136.2942.156.136.2842.156.136.2542.156.136.2442.156.136.2342.156.136.2242.156.136.2142.156.136.2042.156.136.1942.156.136.1742.156.136.1642.156.136.1542.156.136.1442.156.136.1342.156.136.12542.156.136.12442.156.136.12342.156.136.12242.156.136.12142.156.136.12042.156.136.11942.156.136.11842.156.136.11742.156.136.11642.156.136.11542.156.136.11442.156.136.11242.156.136.11142.156.136.11042.156.136.1142.156.136.10942.156.136.10742.156.136.10642.156.136.10542.156.136.10442.156.136.10342.156.136.10242.120.161.9842.120.161.9742.120.161.9542.120.161.9442.120.161.942.120.161.8342.120.161.8242.120.161.8142.120.161.8042.120.161.7942.120.161.7842.120.161.7742.120.161.7642.120.161.7442.120.161.7342.120.161.7242.120.161.7142.120.161.7042.120.161.742.120.161.742.120.161.6942.120.161.6742.120.161.6642.120.161.6542.120.161.6442.120.161.6342.120.161.6242.120.161.6042.120.161.642.120.161.5642.120.161.5542.120.161.5442.120.161.5342.120.161.5242.120.161.5142.120.161.5042.120.161.4942.120.161.4742.120.161.4642.120.161.4542.120.161.4442.120.161.4342.120.161.4242.120.161.4142.120.161.4042.120.161.442.120.161.442.120.161.3842.120.161.3742.120.161.3442.120.161.3342.120.161.3242.120.161.3142.120.161.3042.120.161.342.120.161.342.120.161.2942.120.161.2842.120.161.2542.120.161.2442.120.161.2342.120.161.2242.120.161.2142.120.161.2042.120.161.1942.120.161.1742.120.161.1642.120.161.1542.120.161.1442.120.161.1342.120.161.12542.120.161.12442.120.161.12342.120.161.12242.120.161.12142.120.161.12042.120.161.11942.120.161.11842.120.161.11742.120.161.11642.120.161.11542.120.161.11442.120.161.11242.120.161.11142.120.161.11042.120.161.1142.120.161.10942.120.161.10842.120.161.10742.120.161.10642.120.161.10542.120.161.10442.120.161.10342.120.161.10242.120.161.1042.120.160.9842.120.160.9742.120.160.9442.120.160.942.120.160.8342.120.160.8242.120.160.8142.120.160.8042.120.160.7942.120.160.7842.120.160.7742.120.160.7642.120.160.7442.120.160.7342.120.160.7242.120.160.7142.120.160.7042.120.160.742.120.160.742.120.160.6942.120.160.6742.120.160.6642.120.160.6542.120.160.6442.120.160.6342.120.160.6242.120.160.6042.120.160.642.120.160.5642.120.160.5542.120.160.5342.120.160.5242.120.160.5142.120.160.5042.120.160.4942.120.160.4742.120.160.4642.120.160.4542.120.160.4442.120.160.4342.120.160.4242.120.160.4142.120.160.4042.120.160.442.120.160.442.120.160.3842.120.160.3742.120.160.3442.120.160.3342.120.160.3242.120.160.3142.120.160.3042.120.160.342.120.160.342.120.160.2942.120.160.2842.120.160.2442.120.160.2342.120.160.2242.120.160.2142.120.160.2042.120.160.1942.120.160.1742.120.160.1642.120.160.1542.120.160.1442.120.160.1342.120.160.12542.120.160.12442.120.160.12342.120.160.12242.120.160.12142.120.160.12042.120.160.11942.120.160.11842.120.160.11742.120.160.11642.120.160.11542.120.160.11442.120.160.11342.120.160.11242.120.160.11142.120.160.11042.120.160.1142.120.160.10942.120.160.10842.120.160.10742.120.160.10642.120.160.10542.120.160.10442.120.160.10342.120.160.102106.11.159.98106.11.159.84106.11.159.83106.11.159.82106.11.159.81106.11.159.80106.11.159.77106.11.159.76106.11.159.75106.11.159.74106.11.159.73106.11.159.72106.11.159.71106.11.159.70106.11.159.61106.11.159.60106.11.159.59106.11.159.58106.11.159.53106.11.159.52106.11.159.51106.11.159.50106.11.159.49106.11.159.48106.11.159.47106.11.159.46106.11.159.45106.11.159.44106.11.159.43106.11.159.42106.11.159.41106.11.159.40106.11.159.38106.11.159.37106.11.159.36106.11.159.35106.11.159.34106.11.159.226106.11.159.203106.11.159.156106.11.159.117106.11.159.116106.11.159.114106.11.159.113106.11.159.112106.11.159.111106.11.159.110106.11.159.109106.11.159.108106.11.159.107106.11.159.106106.11.159.105106.11.159.104106.11.159.103106.11.159.102106.11.159.100106.11.158.99106.11.158.98106.11.158.97106.11.158.94106.11.158.93106.11.158.92106.11.158.91106.11.158.90106.11.158.77106.11.158.75106.11.158.74106.11.158.73106.11.158.72106.11.158.71106.11.158.69106.11.158.68106.11.158.65106.11.158.59106.11.158.58106.11.158.57106.11.158.56106.11.158.54106.11.158.53106.11.158.51106.11.158.50106.11.158.49106.11.158.48106.11.158.46106.11.158.45106.11.158.44106.11.158.43106.11.158.42106.11.158.41106.11.158.40106.11.158.39106.11.158.38106.11.158.37106.11.158.36106.11.158.35106.11.158.34106.11.158.33106.11.158.248106.11.158.222106.11.158.201106.11.158.141106.11.158.123106.11.158.122106.11.158.120106.11.158.114106.11.158.109106.11.158.108106.11.158.107106.11.158.106106.11.158.105106.11.158.104106.11.158.103106.11.158.102106.11.158.101106.11.158.100106.11.157.99106.11.157.98106.11.157.93106.11.157.92106.11.157.91106.11.157.90106.11.157.85106.11.157.83106.11.157.77106.11.157.76106.11.157.75106.11.157.73106.11.157.72106.11.157.71106.11.157.70106.11.157.68106.11.157.62106.11.157.61106.11.157.60106.11.157.59106.11.157.58106.11.157.57106.11.157.56106.11.157.54106.11.157.49106.11.157.48106.11.157.47106.11.157.46106.11.157.45106.11.157.44106.11.157.43106.11.157.42106.11.157.41106.11.157.40106.11.157.39106.11.157.38106.11.157.37106.11.157.35106.11.157.34106.11.157.33106.11.157.250106.11.157.248106.11.157.197106.11.157.137106.11.157.126106.11.157.125106.11.157.124106.11.157.121106.11.157.120106.11.157.109106.11.157.108106.11.157.107106.11.157.106106.11.157.105106.11.157.104106.11.157.103106.11.157.102106.11.157.101106.11.157.100106.11.156.94106.11.156.93106.11.156.92106.11.156.86106.11.156.85106.11.156.84106.11.156.81106.11.156.80106.11.156.69106.11.156.67106.11.156.66106.11.156.65106.11.156.59106.11.156.58106.11.156.57106.11.156.56106.11.156.54106.11.156.53106.11.156.52106.11.156.51106.11.156.50106.11.156.49106.11.156.48106.11.156.47106.11.156.45106.11.156.44106.11.156.43106.11.156.42106.11.156.41106.11.156.39106.11.156.38106.11.156.37106.11.156.36106.11.156.35106.11.156.34106.11.156.33106.11.156.232106.11.156.231106.11.156.200106.11.156.155106.11.156.154106.11.156.125106.11.156.124106.11.156.115106.11.156.114106.11.156.113106.11.156.112106.11.156.111106.11.156.110106.11.156.109106.11.156.108106.11.156.107106.11.156.106106.11.156.105106.11.156.104106.11.156.103106.11.156.101106.11.156.100106.11.155.99106.11.155.98106.11.155.97106.11.155.94106.11.155.93106.11.155.92106.11.155.91106.11.155.90106.11.155.77106.11.155.76106.11.155.75106.11.155.74106.11.155.72106.11.155.71106.11.155.70106.11.155.69106.11.155.68106.11.155.59106.11.155.57106.11.155.56106.11.155.53106.11.155.52106.11.155.51106.11.155.50106.11.155.49106.11.155.48106.11.155.47106.11.155.46106.11.155.45106.11.155.44106.11.155.43106.11.155.42106.11.155.41106.11.155.40106.11.155.38106.11.155.37106.11.155.36106.11.155.35106.11.155.34106.11.155.33106.11.155.226106.11.155.133106.11.155.132106.11.155.117106.11.155.116106.11.155.115106.11.155.114106.11.155.109106.11.155.108106.11.155.107106.11.155.106106.11.155.104106.11.155.103106.11.155.102106.11.155.101106.11.155.100106.11.154.99106.11.154.98106.11.154.97106.11.154.89106.11.154.88106.11.154.85106.11.154.84106.11.154.83106.11.154.82106.11.154.81106.11.154.80106.11.154.77106.11.154.76106.11.154.75106.11.154.74106.11.154.65106.11.154.59106.11.154.58106.11.154.57106.11.154.56106.11.154.54106.11.154.53106.11.154.52106.11.154.51106.11.154.50106.11.154.49106.11.154.48106.11.154.47106.11.154.46106.11.154.45106.11.154.44106.11.154.43106.11.154.42106.11.154.41106.11.154.40106.11.154.38106.11.154.37106.11.154.35106.11.154.34106.11.154.33106.11.154.237106.11.154.195106.11.154.140106.11.154.136106.11.154.126106.11.154.118106.11.154.117106.11.154.116106.11.154.115106.11.154.114106.11.154.113106.11.154.112106.11.154.111106.11.154.110106.11.154.109106.11.154.108106.11.154.107106.11.154.106106.11.153.99106.11.153.98106.11.153.97106.11.153.89106.11.153.88106.11.153.85106.11.153.83106.11.153.82106.11.153.81106.11.153.80106.11.153.79106.11.153.78106.11.153.74106.11.153.69106.11.153.68106.11.153.65106.11.153.62106.11.153.61106.11.153.59106.11.153.58106.11.153.57106.11.153.56106.11.153.54106.11.153.53106.11.153.52106.11.153.51106.11.153.50106.11.153.49106.11.153.43106.11.153.42106.11.153.41106.11.153.40106.11.153.39106.11.153.38106.11.153.37106.11.153.36106.11.153.35106.11.153.34106.11.153.33106.11.153.253106.11.153.252106.11.153.214106.11.153.150106.11.153.138106.11.153.126106.11.153.125106.11.153.124106.11.153.123106.11.153.122106.11.153.121106.11.153.120106.11.153.118106.11.153.115106.11.153.114106.11.153.111106.11.153.110106.11.153.106106.11.153.105106.11.153.104106.11.152.94106.11.152.93106.11.152.92106.11.152.90106.11.152.79106.11.152.78106.11.152.75106.11.152.74106.11.152.73106.11.152.72106.11.152.71106.11.152.70106.11.152.66106.11.152.65106.11.152.62106.11.152.61106.11.152.60106.11.152.57106.11.152.56106.11.152.53106.11.152.52106.11.152.49106.11.152.48106.11.152.47106.11.152.46106.11.152.45106.11.152.44106.11.152.43106.11.152.42106.11.152.41106.11.152.40106.11.152.39106.11.152.38106.11.152.37106.11.152.36106.11.152.35106.11.152.34106.11.152.252106.11.152.228106.11.152.217106.11.152.186106.11.152.168106.11.152.146106.11.152.141106.11.152.126106.11.152.125106.11.152.124106.11.152.121106.11.152.120106.11.152.118106.11.152.117106.11.152.116106.11.152.113106.11.152.112106.11.152.111106.11.152.110106.11.152.105106.11.152.104106.11.152.103106.11.152.102106.11.152.100
下一篇:上隆自动化离线3D模型库软件