Poradnik Jak ukryć zaplecze przed robotami majestic, ahrefs, konkurencją itp.

5 Czerwca 2017

Cześć

To moja pierwsza próba przygotowania poradnika, który ma być przydatny zwłaszcza dla tych osób, które zaczynają swoją przygodę z SEO. Już na starcie musicie wiedzieć, że to parszywa branża ;), a działając zwłaszcza na zagranicznym rynku natkniecie się na wielu konkurentów, którzy będą chcieli wyłapać Wasze stronki, Wasze zaplecza, a później im zaszkodzić. Dlatego ważne jest to, aby jak najbardziej im to utrudnić. Podstawą jest więc tworzenie takiego zaplecza, które trudno będzie zidentyfikować. Pierwszym krokiem powinno być tutaj wykorzystanie odpowiednich reguł w pliku .htaccess. Dzięki temu analizując stronę w narzędziach typu majesticseo.com czy ahrefs.com nie będzie się dało odnaleźć linków prowadzących do niej z zaplecza. Oto gotowa formuła, którą dokleić należy w swoim pliku:

RewriteCond %{HTTP_USER_AGENT} .*AhrefsBot.* [NC] 
RewriteCond %{HTTP_USER_AGENT} .*ajSitemap.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*ask.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*BLEXBot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*boitho.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*DBLBot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*dotbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Exabot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*exabot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*gigabot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Java.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*libw.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*MJ1.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*MJ12bot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*psbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Rankivabot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Rankivabot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*rogerbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*SemrushBot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Sogou.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Twice.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Voil.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*xenu.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Yahoo.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Yand.* [NC,OR]
RewriteRule ^.* - [F,L]

Order Allow,Deny
Allow from all
Deny from 110.0.0.0/8
Deny from 222.0.0.0/8 
Deny from 221.0.0.0/8
Deny from 220.0.0.0/8
Deny from 219.0.0.0/8
Deny from 218.0.0.0/8
Deny from 211.0.0.0/8
Deny from 210.0.0.0/8
Deny from 203.0.0.0/8
Deny from 202.0.0.0/8
Deny from 183.0.0.0/8
Deny from 182.0.0.0/8
Deny from 180.0.0.0/8
Deny from 175.0.0.0/8
Deny from 169.208.0.0
Deny from 147.0.0.0/8
Deny from 125.0.0.0/8
Deny from 124.0.0.0/8
Deny from 120.0.0.0/6
Deny from 112.0.0.0/5
Deny from 111.0.0.0/8

5 Czerwca 2017

Działa to? Myślę, że nie jesteś w stanie wyciąć wszystkich botów i adresów ip.

5 Czerwca 2017

w sumie zawsze chcialem to zrobic, ale listy nie mialem, plusik.

poza tym jak mamy odpowiedni plugin to po zastosowaniu tej listy nadal mozemy blokowac kolejne boty bo sie nam wyswietla ip uzytkownika na stronie + jego user agent

5 Czerwca 2017

Brattanek, wszystkiego nigdy nie ukryjesz, ale to co najpopularniejsze zablokować można.

Albo trzeba

5 Czerwca 2017

dobra robota:)

2,1 tys. · 6 Czerwca 2017

Podobny temat jest już na forum, przypięty w spisach.@brattanek. Grunt to ukryć majestyka seopowesuite i ahrefsa.

Wysłane z mojego SM-G920F przy użyciu Tapatalka

6 Czerwca 2017

Możesz podać linka?

7,7 tys. · 9 Czerwca 2017

Sposób na blokowanie dobrych botów

User-agent: *
Disallow: /

Pod względem optymalizacji twój plik htaccess jest troszkę zły Między botem lepiej dodać " | ". Zmniejszy się waga pliku tym bardziej, że co niektórzy mają zdecydowanie więcej wpisów w htaccess.

12 Czerwca 2017

Blokowanie botów:



==========================================================================


RewriteEngine ON
RewriteCond %{HTTP_USER_AGENT} .*ahref.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*AhrefsBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*MJ12bot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*rogerbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*spbot.* [NC,OR]
RewriteRule .* - [F,L]


RewriteEngine ON
RewriteCond %{HTTP_USER_AGENT} .*ahref.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*AhrefsBot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*adsacomponent.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*aqua_products.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*asterias.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*b2w.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*baidu.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*BLEXBot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*blowfish.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*bullseye.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*bunnyslippers.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*collect.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*copernic.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*copyrightcheck.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*cosmos.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*crawl.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*crescent.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*cuil.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*dotbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*emailwolf.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*exabot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*extractorpro.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Ezooms.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*freefind.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*getright.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*gigabot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*grub.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*harvest.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*hloader.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*httplib.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*humanlinks.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*ia_archiver.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*iron33.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*jakarta.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*larbin.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*libweb.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*linkextractorpro.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*linkwalker.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*lwp-trivial.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*majestic-12.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Majestic-SEO.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*miixpc.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*MJ12bot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*moget.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*naver.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*NCBot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*netants.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*netmechanic.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*nicerspro.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*nutch.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*openfind.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*perman.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*prowebwalker.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*pycurl.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*python-urllib.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*repomonkey.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*rma.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*rogerbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*scan.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*search.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*searchmetrics.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*SearchmetricsBot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*SemrushBot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*seokicks.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*SEOkicks-Robot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*siphon.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*sistrix.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*sitesnagger.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*sogou.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*sootle.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*soso.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*spanner.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*spbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*spider.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*spyder.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*stanford.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*suzuran.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*szukacz.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*teleport.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*teleportpro.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*telesoft.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*thenomad.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*tocrawl.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*turingos.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*url_spider_pro.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*vci.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*webauto.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*webbandit.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*webcopier.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*webenhancer.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*websauger.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*webstripper.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*webvac.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*webzip.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*wget.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*whowhere.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*www-collector-e.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*xenu.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*xovi.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*yandex.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*zeus.* [NC]
RewriteRule .* - [F,L]

RewriteCond %{HTTP_HOST} ^olddomain.com [NC]
RewriteRule ^(.*)$ http://newdomain.com/$1 [L,R=301]

RewriteCond %{HTTP_HOST} ^www.olddomain.com [NC]
RewriteRule ^(.*)$ http://newdomain.com/$1 [L,R=301]


================================================================================
BLOCK IP RANGES ALSO......

Sure thing, here it is. Just paste it into the field that appears once you click "cut/paste" in the "Blocked IP Addresses/Ranges" section.
Probably some overlap in the IP ranges as again a combination of Spyder Spanker support info and Terry Kyle (as of mid-2014), but seems to be working fine.

72.44.32.0/19
67.202.0.0/18
75.101.128.0/17
174.129.0.0/16
204.236.192.0/18
184.73.0.0/16
184.72.128.0/17
184.72.64.0/18
50.16.0.0/15
50.19.0.0/16
107.20.0.0/14
23.20.0.0/14
54.242.0.0/15
54.234.0.0/15
54.236.0.0/15
54.224.0.0/15
54.226.0.0/15
54.208.0.0/15
54.210.0.0/15
54.221.0.0/16
54.204.0.0/15
54.196.0.0/15
54.198.0.0/16
54.80.0.0/13
50.112.0.0/16
54.245.0.0/16
54.244.0.0/16
54.214.0.0/16
54.212.0.0/15
54.218.0.0/16
54.200.0.0/15
54.202.0.0/15
54.184.0.0/13
204.236.128.0/18
184.72.0.0/18
50.18.0.0/16
184.169.128.0/17
54.241.0.0/16
54.215.0.0/16
54.219.0.0/16
54.193.0.0/16
79.125.0.0/17
46.51.128.0/18
46.51.192.0/20
46.137.0.0/17
46.137.128.0/18
176.34.128.0/17
176.34.64.0/18
54.247.0.0/16
54.246.0.0/16
54.228.0.0/16
54.216.0.0/15
54.229.0.0/16
54.220.0.0/16
54.194.0.0/15
54.72.0.0/14
175.41.128.0/18
122.248.192.0/18
46.137.192.0/18
46.51.216.0/21
54.251.0.0/16
54.254.0.0/16
54.255.0.0/16
54.252.0.0/16
54.253.0.0/16
54.206.0.0/16
54.79.0.0/16
175.41.192.0/18
46.51.224.0/19
176.32.64.0/19
103.4.8.0/21
176.34.0.0/18
54.248.0.0/15
54.250.0.0/16
54.238.0.0/16
54.199.0.0/16
177.71.128.0/17
54.232.0.0/16
54.233.0.0/18
54.207.0.0/16
96.127.0.0/16
96.128.0.0/9
97.0.0.0/10
97.64.0.0/11
97.96.0.0/12
97.112.0.0/13
97.120.0.0/14
97.124.0.0/15
97.126.0.0/16
97.127.0.0/18
216.145.16.0/24
54.243.130.0/24
216.145.16.0/24
212.74.41.128/27
54.243.130.0/24
103.4.8.0/21
107.20.0.0/14
122.248.192.0/18
174.129.0.0/16
175.41.128.0/18
175.41.192.0/18
176.32.64.0/19
176.34.0.0/18
176.34.128.0/17
176.34.64.0/18
177.71.128.0/17
184.169.128.0/17
184.72.0.0/18
184.72.128.0/17
184.72.64.0/18
184.73.0.0/16
204.236.128.0/18
204.236.192.0/18
23.20.0.0/14
46.137.0.0/17
46.137.128.0/18
46.137.192.0/18
46.51.128.0/18
46.51.192.0/20
46.51.216.0/21
46.51.224.0/19
50.112.0.0/16
50.16.0.0/15
50.18.0.0/16
50.19.0.0/16
54.184.0.0/13
54.193.0.0/16
54.194.0.0/15
54.196.0.0/15
54.198.0.0/16
54.199.0.0/16
54.200.0.0/15
54.202.0.0/15
54.204.0.0/15
54.206.0.0/16
54.207.0.0/16
54.208.0.0/15
54.210.0.0/15
54.212.0.0/15
54.214.0.0/16
54.215.0.0/16
54.216.0.0/15
54.218.0.0/16
54.219.0.0/16
54.220.0.0/16
54.221.0.0/16
54.224.0.0/15
54.226.0.0/15
54.228.0.0/16
54.229.0.0/16
54.232.0.0/16
54.233.0.0/18
54.234.0.0/15
54.236.0.0/15
54.238.0.0/16
54.241.0.0/16
54.242.0.0/15
54.244.0.0/16
54.245.0.0/16
54.246.0.0/16
54.247.0.0/16
54.248.0.0/15
54.250.0.0/16
54.251.0.0/16
54.252.0.0/16
54.253.0.0/16
54.254.0.0/16
54.255.0.0/16
54.72.0.0/14
216.145.16.0/24
54.79.0.0/16
54.80.0.0/13
67.202.0.0/18
72.44.32.0/20
72.44.48.0/21
72.44.56.0/22
72.44.60.0/23
72.44.62.0/24
72.44.63.0/25
72.44.63.128/26
72.44.63.192/27
72.44.63.224/28
72.44.63.240/29
72.44.63.248/30
72.44.63.252/31
72.44.63.254/32
75.101.128.0/17
79.125.0.0/17
96.127.0.0/18 




72.44.32.0 - 72.44.63.255
67.202.0.0 - 67.202.63.255
75.101.128.0 - 75.101.255.255
174.129.0.0 - 174.129.255.255
204.236.192.0 - 204.236.255.255
184.73.0.0 - 184.73.255.255
184.72.128.0 - 184.72.255.255
184.72.64.0 - 184.72.127.255
50.16.0.0 - 50.17.255.255
50.19.0.0 - 50.19.255.255
107.20.0.0 - 107.23.255.255
23.20.0.0 - 23.23.255.255
54.242.0.0 - 54.243.255.255
54.234.0.0 - 54.235.255.255
54.236.0.0 - 54.237.255.255
54.224.0.0 - 54.225.255.255
54.226.0.0 - 54.227.255.255
54.208.0.0 - 54.209.255.255
54.210.0.0 - 54.211.255.255
54.221.0.0 - 54.221.255.255
54.204.0.0 - 54.205.255.255
54.196.0.0 - 54.197.255.255
54.198.0.0 - 54.198.255.255
54.80.0.0 - 54.87.255.255
50.112.0.0 - 50.112.255.255
54.245.0.0 - 54.245.255.255
54.244.0.0 - 54.244.255.255
54.214.0.0 - 54.214.255.255
54.212.0.0 - 54.213.255.255
54.218.0.0 - 54.218.255.255
54.200.0.0 - 54.201.255.255
54.202.0.0 - 54.203.255.255
54.184.0.0 - 54.191.255.255
204.236.128.0 - 204.236.191.255
184.72.0.0 - 184.72.63.255
50.18.0.0 - 50.18.255.255
184.169.128.0 - 184.169.255.255
54.241.0.0 - 54.241.255.255
54.215.0.0 - 54.215.255.255
54.219.0.0 - 54.219.255.255
54.193.0.0 - 54.193.255.255
79.125.0.0 - 79.125.127.255
46.51.128.0 - 46.51.191.255
46.51.192.0 - 46.51.207.255
46.137.0.0 - 46.137.127.255
46.137.128.0 - 46.137.191.255
176.34.128.0 - 176.34.255.255
176.34.64.0 - 176.34.127.255
54.247.0.0 - 54.247.255.255
54.246.0.0 - 54.246.255.255
54.228.0.0 - 54.228.255.255
54.216.0.0 - 54.217.255.255
54.229.0.0 - 54.229.255.255
54.220.0.0 - 54.220.255.255
54.194.0.0 - 54.195.255.255
54.72.0.0 - 54.75.255.255
175.41.128.0 - 175.41.191.255
122.248.192.0 - 122.248.255.255
46.137.192.0 - 46.137.255.255
46.51.216.0 - 46.51.223.255
54.251.0.0 - 54.251.255.255
54.254.0.0 - 54.254.255.255
54.255.0.0 - 54.255.255.255
54.252.0.0 - 54.252.255.255
54.253.0.0 - 54.253.255.255
54.206.0.0 - 54.206.255.255
54.79.0.0 - 54.79.255.255
175.41.192.0 - 175.41.255.255
46.51.224.0 - 46.51.255.255
176.32.64.0 - 176.32.95.255
103.4.8.0 - 103.4.15.255
176.34.0.0 - 176.34.63.255
54.248.0.0 - 54.249.255.255
54.250.0.0 - 54.250.255.255
54.238.0.0 - 54.238.255.255
54.199.0.0 - 54.199.255.255
177.71.128.0 - 177.71.255.255
54.232.0.0 - 54.232.255.255
54.233.0.0 - 54.233.63.255
54.207.0.0 - 54.207.255.255
96.127.0.0 - 97.127.63.255
216.145.16.0 - 216.145.16.255
54.243.130.0-54.243.130.255
216.145.16.0-216.145.16.255
212.74.41.128-212.74.41.159
54.243.130.0-54.243.130.255
103.4.8.0-103.4.15.255
107.20.0.0-107.23.255.255
122.248.192.0-122.248.255.255
174.129.0.0-174.129.255.255
175.41.128.0-175.41.191.255
175.41.192.0-175.41.255.255
176.32.64.0-176.32.95.255
176.34.0.0-176.34.63.255
176.34.128.0-176.34.255.255
176.34.64.0-176.34.127.255
177.71.128.0-177.71.255.255
184.169.128.0-184.169.255.255
184.72.0.0-184.72.63.255
184.72.128.0-184.72.255.255
184.72.64.0-184.72.127.255
184.73.0.0-184.73.255.255
204.236.128.0-204.236.191.255
204.236.192.0-204.236.255.255
23.20.0.0-23.23.255.255
46.137.0.0-46.137.127.255
46.137.128.0-46.137.191.255
46.137.192.0-46.137.255.255
46.51.128.0-46.51.191.255
46.51.192.0-46.51.207.255
46.51.216.0-46.51.223.255
46.51.224.0-46.51.255.255
50.112.0.0-50.112.255.255
50.16.0.0-50.17.255.255
50.18.0.0-50.18.255.255
50.19.0.0-50.19.255.255
54.184.0.0-54.191.255.255
54.193.0.0-54.193.255.255
54.194.0.0-54.195.255.255
54.196.0.0-54.197.255.255
54.198.0.0-54.198.255.255
54.199.0.0-54.199.255.255
54.200.0.0-54.201.255.255
54.202.0.0-54.203.255.255
54.204.0.0-54.205.255.255
54.206.0.0-54.206.255.255
54.207.0.0-54.207.255.255
54.208.0.0-54.209.255.255
54.210.0.0-54.211.255.255
54.212.0.0-54.213.255.255
54.214.0.0-54.214.255.255
54.215.0.0-54.215.255.255
54.216.0.0-54.217.255.255
54.218.0.0-54.218.255.255
54.219.0.0-54.219.255.255
54.220.0.0-54.220.255.255
54.221.0.0-54.221.255.255
54.224.0.0-54.225.255.255
54.226.0.0-54.227.255.255
54.228.0.0-54.228.255.255
54.229.0.0-54.229.255.255
54.232.0.0-54.232.255.255
54.233.0.0-54.233.63.255
54.234.0.0-54.235.255.255
54.236.0.0-54.237.255.255
54.238.0.0-54.238.255.255
54.241.0.0-54.241.255.255
54.242.0.0-54.243.255.255
54.244.0.0-54.244.255.255
54.245.0.0-54.245.255.255
54.246.0.0-54.246.255.255
54.247.0.0-54.247.255.255
54.248.0.0-54.249.255.255
54.250.0.0-54.250.255.255
54.251.0.0-54.251.255.255
54.252.0.0-54.252.255.255
54.253.0.0-54.253.255.255
54.254.0.0-54.254.255.255
54.255.0.0-54.255.255.255
54.72.0.0-54.75.255.255
216.145.16.0 - 216.145.16.255
54.79.0.0-54.79.255.255
54.80.0.0-54.87.255.255
67.202.0.0-67.202.63.255
72.44.32.0-72.44.63.254
75.101.128.0-75.101.255.255
79.125.0.0-79.125.127.255
96.127.0.0-96.127.63.255 



=============================================================================================

Options +FollowSymlinks 
RewriteEngine On
RewriteCond %{HTTP_HOST} ^domainA.de [NC]
RewriteRule ^(.*)$ http://domainB.de/$1 [L,R=301]
RewriteCond %{HTTP_HOST} ^www.domainA.de [NC]
RewriteRule ^(.*)$ http://domainB.de/$1 [L,R=301]

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /

RewriteCond %{REQUEST_URI} !/robots.txt$
RewriteCond %{HTTP_USER_AGENT} ^.*BLEXBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BlackWidow.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Nutch.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Jetbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebVac.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Stanford.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*scooter.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*naver.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*dumbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Hatena\ Antenna.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*grub.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*looksmart.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebZip.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*b2w/0.1.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Copernic.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*psbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Python-urllib.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetMechanic.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*URL_Spider_Pro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CherryPicker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailCollector.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Email.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CopyRightCheck.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Crescent.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ProWebWalker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CheeseBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LNSpiderguy.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ia_archiver.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Alexibot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Telesoft.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Website\ Quester.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*moget.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mister\ PiX.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*TheNomad.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WWW-Collector-E.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*RMA.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*libWeb/clsHTTP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*asterias.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*httplib.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*turingos.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*spanner.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Harvest.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*InfoNaviRobot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bullseye.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NICErsPRO.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Microsoft\ URL\ Control.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*DittoSpyder.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Foobot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorldForumBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SpankBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BotALot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*lwp-trivial.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorld.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BunnySlippers.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*URLy\ Warning.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkWalker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*cosmos.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*hloader.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*humanlinks.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkextractorPro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Explorer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mata\ Hari.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LexiBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Image\ Collector.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*The\ Intraformant.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*True_Robot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BlowFish.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SearchEngineWorld.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JennyBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BuiltBotTough.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ProPowerBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BackDoorBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*toCrawl/UrlDispatcher.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebEnhancer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*suzuran.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebViewer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*VCI.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Szukacz.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*QueryN.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Openfind.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Openbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Webster.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EroCrawler.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkScan.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Keyword.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Kenjin.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Iron33.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bookmark\ search\ tool.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FairAd\ Client.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Gaisbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Aqua_Products.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Radiation\ Retriever\ 1.1.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Flaming\ AttackBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Oracle\ Ultra\ Search.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MSIECrawler.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PerMan.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*searchpreview.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*sootle.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Enterprise_Search.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bot\ mailto:craftbot@yahoo.com.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ChinaClaw.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Custo.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*DISCo.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Download\ Demon.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*eCatch.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EirGrabber.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Express\ WebPictures.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EyeNetIE.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FlashGet.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetWeb!.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go!Zilla.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go-Ahead-Got-It.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Grafula.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Image\ Stripper.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Image\ Sucker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Indy\ Library.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*InterGET.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Internet\ Ninja.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JetCar.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JOC\ Web\ Spider.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LeechFTP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mass\ Downloader.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MIDown\ tool.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mister\ PiX.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NearSite.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetSpider.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Net\ Vampire.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetZIP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Octopus.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Explorer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Navigator.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PageGrabber.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Papa\ Foto.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*pavuk.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*RealDownload.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ReGet.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SmartDownload.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SuperHTTP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*tAkeOut.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport\ Pro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*VoidEYE.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Image\ Collector.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Sucker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebFetch.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebGo\ IS.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebLeacher.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebReaper.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Website\ eXtractor.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Website\ Quester.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebWhacker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WWWOFFLE.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Xaldon\ WebSpider.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Zeus.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Semrush.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BecomeBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*AhrefsBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MJ12bot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*rogerbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*exabot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Xenu.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*dotbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*gigabot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BlekkoBot.*$ [NC]
RewriteRule ^(.*)$ http://webmd.com/ [L,R=301]

Order Allow,Deny
Allow from all
Deny from 212.113.35.162
Deny from 212.113.37.105
Deny from 212.113.37.106
Deny from 213.186.119.134
Deny from 213.186.120.196
Deny from 213.186.122.2
Deny from 213.186.122.3
Deny from 213.186.127.3
Deny from 173.199.115.115
Deny from 5.10.83

RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

# END WordPress


========================================================================================================


# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /

RewriteCond %{HTTP_USER_AGENT} .*Twice.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Yand.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Yahoo.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Voil.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*libw.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Java.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Sogou.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*psbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Exabot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*boitho.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*ajSitemap.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Rankivabot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*DBLBot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*MJ1.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Rankivabot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*ask.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*rogerbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*exabot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*MJ12bot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*dotbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*gigabot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*SemrushBot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*BLEXBot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*xenu.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*AhrefsBot.* [NC]
RewriteRule ^.* - [F,L]

Order Allow,Deny
Allow from all
Deny from 110.0.0.0/8
Deny from 111.0.0.0/8
Deny from 112.0.0.0/5
Deny from 120.0.0.0/6
Deny from 124.0.0.0/8
Deny from 125.0.0.0/8
Deny from 147.0.0.0/8
Deny from 169.208.0.0
Deny from 175.0.0.0/8
Deny from 180.0.0.0/8
Deny from 182.0.0.0/8
Deny from 183.0.0.0/8
Deny from 202.0.0.0/8
Deny from 203.0.0.0/8
Deny from 210.0.0.0/8
Deny from 211.0.0.0/8
Deny from 218.0.0.0/8
Deny from 219.0.0.0/8
Deny from 220.0.0.0/8
Deny from 221.0.0.0/8
Deny from 222.0.0.0/8

RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

# END WordPress


==========================================================================================


RewriteEngine On
RewriteCond %{REQUEST_URI} !/robots.txt$
RewriteCond %{HTTP_USER_AGENT} ^.*BLEXBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BlackWidow.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Nutch.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Jetbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebVac.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Stanford.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*scooter.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*naver.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*dumbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Hatena\ Antenna.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*grub.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*looksmart.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebZip.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*b2w/0.1.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Copernic.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*psbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Python-urllib.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetMechanic.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*URL_Spider_Pro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CherryPicker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailCollector.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Email.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CopyRightCheck.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Crescent.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ProWebWalker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CheeseBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LNSpiderguy.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ia_archiver.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Alexibot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Telesoft.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Website\ Quester.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*moget.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mister\ PiX.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*TheNomad.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WWW-Collector-E.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*RMA.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*libWeb/clsHTTP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*asterias.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*httplib.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*turingos.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*spanner.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Harvest.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*InfoNaviRobot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bullseye.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NICErsPRO.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Microsoft\ URL\ Control.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*DittoSpyder.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Foobot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorldForumBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SpankBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BotALot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*lwp-trivial.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorld.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BunnySlippers.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*URLy\ Warning.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkWalker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*cosmos.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*hloader.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*humanlinks.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkextractorPro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Explorer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mata\ Hari.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LexiBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Image\ Collector.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*The\ Intraformant.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*True_Robot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BlowFish.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SearchEngineWorld.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JennyBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BuiltBotTough.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ProPowerBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BackDoorBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*toCrawl/UrlDispatcher.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebEnhancer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*suzuran.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebViewer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*VCI.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Szukacz.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*QueryN.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Openfind.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Openbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Webster.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EroCrawler.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkScan.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Keyword.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Kenjin.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Iron33.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bookmark\ search\ tool.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FairAd\ Client.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Gaisbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Aqua_Products.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Radiation\ Retriever\ 1.1.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Flaming\ AttackBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Oracle\ Ultra\ Search.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MSIECrawler.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PerMan.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*searchpreview.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*sootle.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Enterprise_Search.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bot\ mailto:craftbot@yahoo.com.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*ChinaClaw.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Custo.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*DISCo.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Download\ Demon.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*eCatch.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*EirGrabber.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Express\ WebPictures.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*EyeNetIE.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*FlashGet.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*GetWeb!.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Go!Zilla.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Go-Ahead-Got-It.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Grafula.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Image\ Stripper.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Image\ Sucker.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Indy\ Library.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*InterGET.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Internet\ Ninja.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*JetCar.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*JOC\ Web\ Spider.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*LeechFTP.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Mass\ Downloader.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*MIDown\ tool.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Mister\ PiX.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*NearSite.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*NetSpider.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Net\ Vampire.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*NetZIP.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Octopus.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Explorer.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Navigator.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*PageGrabber.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Papa\ Foto.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*pavuk.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*RealDownload.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*ReGet.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*SmartDownload.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*SuperHTTP.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*tAkeOut.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport\ Pro.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*VoidEYE.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Image\ Collector.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Sucker.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*WebFetch.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*WebGo\ IS.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*WebLeacher.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*WebReaper.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Website\ eXtractor.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Website\ Quester.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*WebWhacker.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*WWWOFFLE.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Xaldon\ WebSpider.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Zeus.*$ [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^.*Semrush.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BecomeBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*AhrefsBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MJ12bot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*rogerbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*exabot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Xenu.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*dotbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*gigabot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BlekkoBot.*$ [NC]
RewriteRule ^.*.* http://www.webmd.com/ [L]








==========================================================================================

Hiding htaccess / hiding PBN via htaccess

htacess.... vs robots code..

- So what I will do here is redirect the link crawlers e.g. Moz/Ahrefs/Majestic to another URL then add my redirect to my new site

- for this to work you will need to also block these bots in robots.txt

- You could use the robots.txt as well, but don't expect it to block anything, most link crawlers don't obey it anyway...so you still have to block them on .htaccess 


For hiding links behind a 301 you could use an aged domain, yes but I use new domains mostly. 

Let me try to simplify. You have the money site you want to rank and hide any links to, let's call it domain A.

Then you register domain B as your 301 domain, you redirect it while blocking bots and build your links to domain B.

Now when a competitor checks the links to domain A, if it has been blocked correctly they will not see the 301 (domain B) and therefore not be able to see the links pointing at it.


==========================================================================================

https://wordpress.org/plugins/redirection/
plugin


REDIRECT CODE


//Redirect from non-WWW to WWW

- for any domain .htaccess takes effect on:

         RewriteEngine On
         RewriteCond %{HTTP_HOST} !^www\.
         RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]

- for a certain domain, example.com:

         RewriteEngine On    
         RewriteCond %{HTTP_HOST} ^example\.com$ [NC]
         RewriteRule ^(.*)$ http://www.example.com/$1 [R=301,L]


Simply taking this snippet and pasting it AFTER the default WordPress stuff.
Now that should work. And you can use the expanded list of bots.



=============================================================================================

7,7 tys. · 12 Czerwca 2017

@A-K fajna wrzutka ale dlaczego ona jest trak tragicznie rozpisana?

12 Czerwca 2017

.

Edytowane 9 Września 2021 przez A-K

25 Października 2017

Jak to jest z tym, jesli nie wrzuce teraz tej blokady i zaczne klepac linki to potem ahrefs (i inne) mimo blokowania botow i tak beda trzymac stare raporty? W sensie ze ktos zauwazy jakie linki robilem zanim zablokowalem boty?

2,1 tys. · 25 Października 2017

@foreignseo tak, po pewnym czasie zostaną uznane za utracone.

25 Października 2017

10 godzin temu, Moriarty napisał:

@foreignseo tak, po pewnym czasie zostaną uznane za utracone.

Tb tez to tak spowalnia strone? Nawet kokpit admina laduje sie niemilosiernie wolno (jak sie przechodzi np z pisania postu do ustawien/pluginow). No chyba ze to nie wina pliku htaccess

@edit

aa dobra znalazlem przyczyne Xd cos innego wadzilo

Edytowane 25 Października 2017 przez foreignseo

25 Października 2017

I jak tam u was z blokowaniem, zawsze od razu blokujecie dostęp robotą, czy czasami nie zwracacie na to uwagi ?

26 Października 2017

Używam podobnego pliku (nie wiem czy identycznego, bo nie chce mi się sprawdzać każdej linijki i porównywać). U mnie działa perfekcyjnie, największe serwisy typu Majestic, Ahrefs, Moz itp. nic nie pokazują. Często jak stawiam zaplecze to nie wrzucam tego .htaccess, dopiero po zrobieniu pewnej partii stronek podmieniam plik hurtowo. Ale to też dlatego, że nie linkuje z zaplecza od pierwszego dnia, tylko daje mu się odleżeć, więc w tym czasie gdy nie ma .htaccess to nie a tam linków wychodzących.

27 Października 2017

Moja propozycja,

robots.txt:

User-agent: Rogerbot 
User-agent: Exabot 
User-agent: MJ12bot 
User-agent: Dotbot 
User-agent: Gigabot 
User-agent: AhrefsBot 
User-agent: BlackWidow 
User-agent: Bot\ [EMAIL="craftbot@yahoo.com"]mailto:craftbot@yahoo.com[/EMAIL] 
User-agent: ChinaClaw 
User-agent: Custo 
User-agent: DISCo 
User-agent: Download\ Demon 
User-agent: eCatch 
User-agent: EirGrabber 
User-agent: EmailSiphon 
User-agent: EmailWolf 
User-agent: Express\ WebPictures 
User-agent: ExtractorPro 
User-agent: EyeNetIE 
User-agent: FlashGet 
User-agent: GetRight 
User-agent: GetWeb! 
User-agent: Go!Zilla 
User-agent: Go-Ahead-Got-It 
User-agent: GrabNet 
User-agent: Grafula 
User-agent: HMView 
User-agent: HTTrack 
User-agent: Image\ Stripper 
User-agent: Image\ Sucker 
User-agent: Indy\ Library
User-agent: InterGET 
User-agent: Internet\ Ninja 
User-agent: JetCar 
User-agent: JOC\ Web\ Spider 
User-agent: larbin 
User-agent: LeechFTP 
User-agent: Mass\ Downloader 
User-agent: MIDown\ tool 
User-agent: Mister\ PiX 
User-agent: Navroad 
User-agent: NearSite 
User-agent: NetAnts 
User-agent: NetSpider 
User-agent: Net\ Vampire 
User-agent: NetZIP 
User-agent: Octopus 
User-agent: Offline\ Explorer 
User-agent: Offline\ Navigator 
User-agent: PageGrabber 
User-agent: Papa\ Foto 
User-agent: pavuk 
User-agent: pcBrowser 
User-agent: RealDownload 
User-agent: ReGet 
User-agent: SiteSnagger 
User-agent: SmartDownload 
User-agent: SuperBot 
User-agent: SuperHTTP 
User-agent: Surfbot 
User-agent: tAkeOut 
User-agent: Teleport\ Pro 
User-agent: VoidEYE 
User-agent: Web\ Image\ Collector 
User-agent: Web\ Sucker 
User-agent: WebAuto 
User-agent: WebCopier 
User-agent: WebFetch 
User-agent: WebGo\ IS 
User-agent: WebLeacher 
User-agent: WebReaper 
User-agent: WebSauger 
User-agent: Website\ eXtractor 
User-agent: Website\ Quester 
User-agent: WebStripper 
User-agent: WebWhacker 
User-agent: WebZIP 
User-agent: Wget 
User-agent: Widow 
User-agent: WWWOFFLE 
User-agent: Xaldon\ WebSpider 
User-agent: Zeus
Disallow: /

Oraz .htaccess

SetEnvIfNoCase User-Agent .*Twice.* bad_bot
SetEnvIfNoCase User-Agent .*Yand.* bad_bot
SetEnvIfNoCase User-Agent .*Yahoo.* bad_bot
SetEnvIfNoCase User-Agent .*Voil.* bad_bot
SetEnvIfNoCase User-Agent .*libw.* bad_bot
SetEnvIfNoCase User-Agent .*Java.* bad_bot
SetEnvIfNoCase User-Agent .*Sogou.* bad_bot
SetEnvIfNoCase User-Agent .*psbot.* bad_bot
SetEnvIfNoCase User-Agent .*Exabot.* bad_bot
SetEnvIfNoCase User-Agent .*boitho.* bad_bot
SetEnvIfNoCase User-Agent .*ajSitemap.* bad_bot
SetEnvIfNoCase User-Agent .*Rankivabot.* bad_bot
SetEnvIfNoCase User-Agent .*DBLBot.* bad_bot
SetEnvIfNoCase User-Agent .*MJ1.* bad_bot
SetEnvIfNoCase User-Agent .*Rankivabot.* bad_bot
SetEnvIfNoCase User-Agent .*ask.* bad_bot
SetEnvIfNoCase User-Agent .*rogerbot.* bad_bot
SetEnvIfNoCase User-Agent .*exabot.* bad_bot
SetEnvIfNoCase User-Agent .*MJ12bot.* bad_bot
SetEnvIfNoCase User-Agent .*dotbot.* bad_bot
SetEnvIfNoCase User-Agent .*gigabot.* bad_bot
SetEnvIfNoCase User-Agent .*SemrushBot.* bad_bot
SetEnvIfNoCase User-Agent .*BLEXBot.* bad_bot
SetEnvIfNoCase User-Agent .*xenu.* bad_bot
SetEnvIfNoCase User-Agent .*AhrefsBot.* [NC]

<Limit GET POST HEAD>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Limit>

Zaloguj się

Poradnik Jak ukryć zaplecze przed robotami majestic, ahrefs, konkurencją itp.

Rekomendowane odpowiedzi

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Odnośnik do komentarza

Jeśli chcesz dodać odpowiedź, zaloguj się lub zarejestruj nowe konto

Zarejestruj nowe konto

Zaloguj się

Ostatnio aktywne tematy

Powiadomienie o plikach cookie