25. November 2017, 06:48:27
SMFPortal.de

Autor Thema: Fragen zu robots.txt und .htaccess  (Gelesen 9738 mal)

0 Mitglieder und 1 Gast betrachten dieses Thema.

02. Juli 2013, 22:00:44
Gelesen 9738 mal

Skippy


Hallo,

bisher hatte ich es noch nicht gemacht, aber nun will ich meine Foren per robots.txt und .htaccess gegen übermäßige Bot-/Crawlerbesuche schützen.
In den letzten Tagen geben sich die Bots von Yandex und ahref die Klinke in meinen Foren in die Hand, die sind teils stundenlang Dauergäste.

Mit robots.txt und .htaccess kenne ich mich aber kaum aus. Ich habe gestern eine Weile gegoogelt und auch hier im Forum geschaut, und habe mir dann aus gefundenen Beispielen meine eigenen Dateien zusammengestellt.
Unten habe ich sie eingefügt, so wie sie momentan ausschauen. In der robots.txt werden sicherlich noch ein paar Einträge dazu kommen.

In der .htaccess sind zu Beginn eine Menge IP's gelistet, für die Zugriffe gesperrt sind. Da wo die drei Punkte ... eingefügt sind, folgen normalerweise noch mehr IP's, die habe ich hier jetzt nur um Platz zu sparen weggelassen.
Ganz unten, wo bei der URL und Sub-URL jetzt "meine-homepage" eingetragen ist, stehen in den echten Dateien stattdessen natürlich die richtigen URLs.

Beide Dateien habe ich gestern Mittag dann in die root-Verzeichnisse der Foren hochgeladen und mich dann gefreut, dass ich nun einen halbwegs guten Schutz habe.
Doch wie ich sehen musste, war ahref danach weiterhin oft in beiden Foren unterwegs.
Auch Yandex hat sich nicht an die eingetragene Zeitsperre von 60 Sekunden gehalten, die ich mal zum Testen per "Crawl-delay: 60" in der robots.txt eingetragen habe.

Darum vermute ich, dass in den Dateien etwas falsch ist. Würde da bitte mal jemand drüber gucken und mir schreiben, ob und was da falsch ist? Oder fehlt vielleicht auch noch was Wichtiges?

Danke und Grüße,
Skippy / Martina


Die robots.txt:
Code
User-agent: *
Disallow: /?PHPSESSID
Disallow: /*?*
Disallow: /*msg*
Disallow: /search2/
Disallow: /stats/
Disallow: /trackip/
Disallow: /*sort=*
Disallow: /index.php?action=activate
Disallow: /index.php?action=admin
Disallow: /index.php?action=calendar
Disallow: /index.php?action=emailuser
Disallow: /index.php?action=findmember
Disallow: /index.php?action=help
Disallow: /index.php?action=helpadmin
Disallow: /index.php?action=login
Disallow: /index.php?action=logout
Disallow: /index.php?action=mlist
Disallow: /index.php?action=modifykarma
Disallow: /index.php?action=pm
Disallow: /index.php?action=post
Disallow: /index.php?action=printpage
Disallow: /index.php?action=profile
Disallow: /index.php?action=recent
Disallow: /index.php?action=register
Disallow: /index.php?action=reminder
Disallow: /index.php?action=search
Disallow: /index.php?action=theme
Disallow: /index.php?action=unread
Disallow: /index.php?action=unreadreplies
Disallow: /index.php?action=verificationcode
Disallow: /index.php?action=who
Disallow: /index.php?action=stats
Disallow: /index.php?theme
Disallow: /attachments/
Disallow: /avatars/
Disallow: /Packages/
Disallow: /Smileys/
Disallow: /Sources/
Disallow: /Themes/
Disallow: /index.php?*;wap
Disallow: /index.php?*;wap2
Disallow: /index.php?*;imode

user-agent: AhrefsBot
disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: rogerbot
Disallow: /

User-agent: exabot
Disallow: /

User-agent: dotbot
Disallow: /

User-agent: gigabot
Disallow: /

User-agent: BacklinkCrawler
Disallow: /

User-agent: Yandex
Crawl-delay: 60


Die .htaccess:
Code
AuthType	Basic	
Order deny,allow
# Stand 04.01.2013
#BY
deny from 212.95.58.
#CZ Tschechien Masaryk-Uni
deny    from    147.251.48.4
#KR Korea
deny from 119.64.0.0/13
#GE Georgia
deny from 213.131.35.0/24
deny from 213.131.32.0/19
#JP Japan
deny from 150.26.0.0/15
deny from 150.28.0.0/14
deny from 150.32.0.0/11
deny from 150.64.0.0/11
deny from 150.96.0.0/14
deny from 150.100.0.0/16
.....

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^libwww-perl
RewriteCond %{HTTP_USER_AGENT} ^BacklinkCrawler
RewriteCond %{HTTP_USER_AGENT} ^ahrefsbot
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow
RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot@yahoo.com
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw
RewriteCond %{HTTP_USER_AGENT} ^Custo
RewriteCond %{HTTP_USER_AGENT} ^DISCo
RewriteCond %{HTTP_USER_AGENT} ^dotbot
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon
RewriteCond %{HTTP_USER_AGENT} ^eCatch
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf
RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro
RewriteCond %{HTTP_USER_AGENT} ^exabot
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE
RewriteCond %{HTTP_USER_AGENT} ^FlashGet
RewriteCond %{HTTP_USER_AGENT} ^GetRight
RewriteCond %{HTTP_USER_AGENT} ^GetWeb!
RewriteCond %{HTTP_USER_AGENT} ^gigabot
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It
RewriteCond %{HTTP_USER_AGENT} ^GrabNet
RewriteCond %{HTTP_USER_AGENT} ^Grafula
RewriteCond %{HTTP_USER_AGENT} ^HMView
RewriteCond %{HTTP_USER_AGENT} ^HTTrack
RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper
RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker
RewriteCond %{HTTP_USER_AGENT} ^Indy\ Library
RewriteCond %{HTTP_USER_AGENT} ^InterGET
RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja
RewriteCond %{HTTP_USER_AGENT} ^JetCar
RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider
RewriteCond %{HTTP_USER_AGENT} ^larbin
RewriteCond %{HTTP_USER_AGENT} ^LeechFTP
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader
RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX
RewriteCond %{HTTP_USER_AGENT} ^mj12bot
RewriteCond %{HTTP_USER_AGENT} ^Navroad
RewriteCond %{HTTP_USER_AGENT} ^NearSite
RewriteCond %{HTTP_USER_AGENT} ^NetAnts
RewriteCond %{HTTP_USER_AGENT} ^NetSpider
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire
RewriteCond %{HTTP_USER_AGENT} ^NetZIP
RewriteCond %{HTTP_USER_AGENT} ^Octopus
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber
RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto
RewriteCond %{HTTP_USER_AGENT} ^pavuk
RewriteCond %{HTTP_USER_AGENT} ^pcBrowser
RewriteCond %{HTTP_USER_AGENT} ^pixray
RewriteCond %{HTTP_USER_AGENT} ^Python-urllib
RewriteCond %{HTTP_USER_AGENT} ^RealDownload
RewriteCond %{HTTP_USER_AGENT} ^ReGet
RewriteCond %{HTTP_USER_AGENT} ^rogerbot
RewriteCond %{HTTP_USER_AGENT} ^Screaming\ Frog\ SEO
RewriteCond %{HTTP_USER_AGENT} ^SearchmetricsBot
RewriteCond %{HTTP_USER_AGENT} ^seokicks
RewriteCond %{HTTP_USER_AGENT} ^sistrix
RewriteCond %{HTTP_USER_AGENT} ^sitebot
RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger
RewriteCond %{HTTP_USER_AGENT} ^SmartDownload
RewriteCond %{HTTP_USER_AGENT} ^SuperBot
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP
RewriteCond %{HTTP_USER_AGENT} ^Surfbot
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro
RewriteCond %{HTTP_USER_AGENT} ^twenga2
RewriteCond %{HTTP_USER_AGENT} ^unister
RewriteCond %{HTTP_USER_AGENT} ^VoidEYE
RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector
RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker
RewriteCond %{HTTP_USER_AGENT} ^WebAuto
RewriteCond %{HTTP_USER_AGENT} ^WebCopier
RewriteCond %{HTTP_USER_AGENT} ^WebFetch
RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS
RewriteCond %{HTTP_USER_AGENT} ^WebLeacher
RewriteCond %{HTTP_USER_AGENT} ^WebReaper
RewriteCond %{HTTP_USER_AGENT} ^WebSauger
RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor
RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester
RewriteCond %{HTTP_USER_AGENT} ^WebStripper
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker
RewriteCond %{HTTP_USER_AGENT} ^WebZIP
RewriteCond %{HTTP_USER_AGENT} ^Wget
RewriteCond %{HTTP_USER_AGENT} ^Widow
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider
RewriteCond %{HTTP_USER_AGENT} ^Xenu\ Link\ Sleuth
RewriteCond %{HTTP_USER_AGENT} ^xovi
RewriteCond %{HTTP_USER_AGENT} ^Zeus
RewriteRule ^.*$ - [F]

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?meine-homepage\.de(/.*)?$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(forum\.)?meine-homepage\.de(/.*)?$ [NC]
RewriteRule \.(jpg|gif|png|JPG|GIF|PNG)$ - [F]

03. Juli 2013, 17:06:57
Antwort #1

Hans 33


Ich habe vor einiger Zeit mit dem robots.txt experimentiert, bin aber nicht glücklich geworden.

Ein Forum ist mir bei google abgeschmiert. Ich konnte leider nicht feststellen, ob es am robots.txt gelegen ist oder ob die Korrelation nur zufällig war.

Jetzt habe ich das robots.txt wieder entfernt. Mittlerweile rückt das Forum mit dem wichtigsten Suchbegriff bei google wieder langsam nach vorne, aber auch hier könnte es zufällig sein.

04. Juli 2013, 10:23:51
Antwort #2

Skippy


Dass ein Forum durch das Einfügen einer robots.txt abschmieren kann, hab ich bis jetzt noch nirgends gelesen.
Aber wer weiß, vielleicht kann das wirklich passieren.

Gestern war der ahref-Bot wieder ein häufiger Besucher bei mir im Forum.
Soweit ich weiß gehört der zu den Bots, die sich nicht an Einträge in der robots.txt halten, darum habe ich ihn ebenfalls noch in die .htaccess eingetragen.
Der Name "ahrefsbot" für den User-Agenten müsste dort doch eigentlich auch noch richtig sein. Hmm...

06. Juli 2013, 11:10:42
Antwort #3

Amarok


Moinsens!

Also, ich habe meine Foren mit Hilfe von .htaccess-Dateien komplett abgedichtet. Und ich bin damit sehr zufrieden. Es bedingt immer eine ordentliche Recherche-Arbeit bei den fünf großen Internet-Registries (ARIN, APNIC, AFRINIC, LACNIC und RIPE) um die Daten aktuell zu halten.

Hier mal die .htaccess aus meinem Kurzwellenforum:
Code
order allow,deny
# .htaccess-Datei von UTDX.DE, die nur bekannte unseriöse crawler enthält.
# Ausländische DSL-Anbieter bzw. ISP bleiben geöffnet!!!
# 2COM-RUS
deny from 46.188.0.0/17
# 2DAY-TELECOM-KAZ
deny from 37.99.0.0/17
# 3NT-UK/USA
deny from 37.1.192.0/19
# 4D-DATACENTERS-UK
deny from 94.228.32.0/20
# 91.er-Kleinstbereiche!
deny from 91.192.0.0/12
deny from 91.208.0.0/13
deny from 91.216.0.0/14
deny from 91.220.0.0/16
deny from 91.221.0.0/19
deny from 91.221.32.0/20
deny from 91.221.48.0/21
deny from 91.221.56.0/23
# Allow from 91.221.58.0/23 BWI-BUNDESWEHR-DE
deny from 91.221.60.0/22
deny from 91.221.64.0/18
deny from 91.221.128.0/17
deny from 91.222.0.0/15
deny from 91.224.0.0/12
# ADCNET-ARM
deny from 95.140.192.0/21
# ADTECHNOLOGY-LTV
deny from 188.92.72.0/21
# ADVANCEDHOSTERS-USA
deny from 46.229.160.0/20
# AHBR-UK
deny from 5.144.176.0/21
# AIRNET-RUS
deny from 93.92.200.0/24
# AKADO-RUS
deny from 188.123.224.0/19
# ALBA-UKR
deny from 193.151.12.0/22
# ALKAR-UKR
deny from 77.239.167.0/24
deny from 93.127.12.0/22
deny from 217.24.160.0/20
# ALTUSHOST-GLOBAL
deny from 185.2.28.0/22
deny from 213.5.64.0/21
# ALLHOSTSHOP24-USA
deny from 216.18.224.0/20
# AMAZON-Worldwide
deny from 23.20.0.0/14
deny from 46.137.0.0/16
deny from 50.16.0.0/14
deny from 50.112.0.0/16
deny from 54.234.0.0/15
deny from 54.240.0.0/12
deny from 67.202.0.0/18
deny from 72.44.32.0/19
deny from 75.101.128.0/17
deny from 107.20.0.0/14
deny from 174.129.0.0/16
deny from 176.34.0.0/16
deny from 184.72.0.0/15
deny from 184.169.128.0/17
deny from 204.236.128.0/17
deny from 216.182.224.0/20
# ATT-USA
deny from 12.216.0.0/16
# AVANET-UKR
deny from 194.187.148.0/22
# AVANTE-USA
deny from 66.248.192.0/19
# AWD-DE (Internetzugang des AWD)
deny from 62.154.152.0/24
# BALTICOM-LTV
deny from 77.93.0.0/19
# BANKINFOIRM-UKR (TRIOLAN)
deny from 178.150.0.0/15
# BSI-DE
deny from 77.87.228.0/22
# BSKYB-UK -> siehe SKY-UK!!!!!
# BUNDESARBEITSAGENTUR-DE (Arbeitsamt)
deny from 195.88.116.0/23
# BUNDESINNENMINISTERIUM-DE
deny from 81.160.0.0/17
# BEELINE-RUS
# siehe CORBINA-RUS
# BELTELECOM-BLR & andere weissrussische Adressen
deny from 37.44.64.0/18
deny from 80.94.160.0/21
deny from 82.209.192.0/18
deny from 86.57.128.0/17
deny from 87.252.224.0/19
deny from 91.149.128.0/18
deny from 91.187.0.0/19
deny from 93.84.0.0/15
deny from 93.125.0.0/17
deny from 93.125.0.0/17
deny from 109.232.112.0/21
deny from 178.120.0.0/13
deny from 178.159.240.0/21
deny from 178.172.128.0/17
deny from 193.232.248.0/22
deny from 194.85.254.0/23
deny from 194.158.192.0/19
deny from 194.226.120.0/22
deny from 194.226.124.0/23
deny from 195.50.16.0/20
deny from 195.137.180.0/24
deny from 213.184.224.0/19
deny from 217.21.32.0/19
# BELWUE-DE
deny from 193.196.0.0/15
# BERMOSCOM-DE/RUS
deny from 91.218.114.0/23
# BIS-RUS
deny from 46.191.240.0/20
# BLUECOATSYSTEMS-USA
deny from 199.19.248.0/21
# BLUEWIN-SUI
deny from 85.1.55.0/24
# BRITELINE-DE
deny from 82.198.192.0/19
# BRIZ-UKR
deny from 46.250.0.0/19
# BUSINESSDIALOG-SEY
deny from 91.217.178.0/24
# CANTV-VEN
deny from 186.88.0.0/13
deny from 201.208.0.0/14
# Charif, Fakir & Co, Damascus-SYR
deny from 109.238.144.0/20
# CHINESE TELECOM-CHN/CHINANET-CHN
deny from 1.80.0.0/14
deny from 1.192.0.0/13
deny from 1.202.0.0/15
deny from 14.16.0.0/12
deny from 27.152.0.0/13
deny from 36.248.0.0/14
deny from 42.62.0.0/17
deny from 42.120.0.0/15
deny from 58.22.0.0/15
deny from 58.48.0.0/13
deny from 58.60.0.0/14
deny from 58.68.128.0/17
deny from 58.83.128.0/16
deny from 58.208.0.0/12
deny from 58.240.0.0/15
deny from 58.242.0.0/15
deny from 58.248.0.0/13
deny from 59.32.0.0/12
deny from 59.56.0.0/14
deny from 59.60.0.0/15
deny from 59.148.0.0/15
deny from 59.172.0.0/15
deny from 60.0.0.0/13
deny from 60.8.0.0/14
deny from 60.12.0.0/16
deny from 60.28.0.0/15
deny from 60.208.0.0/12
deny from 60.162.0.0/15
deny from 60.166.0.0/15
deny from 60.168.0.0/13
deny from 60.176.0.0/12
deny from 60.194.0.0/15
deny from 60.247.0.0/16
deny from 61.52.0.0/15
deny from 61.55.0.0/16
deny from 61.128.0.0/10
deny from 61.144.0.0/14
deny from 101.64.0.0/13
deny from 101.224.0.0/13
deny from 103.6.76.0/22
deny from 103.21.208.0/20
deny from 106.2.0.0/15
deny from 110.75.160.0/19
deny from 110.80.0.0/13
deny from 110.88.0.0/14
deny from 110.184.0.0/13
deny from 111.0.0.0/10
deny from 111.72.0.0/13
deny from 111.128.0.0/11
deny from 111.160.0.0/13
deny from 111.172.0.0/14
deny from 111.176.0.0/13
deny from 111.192.0.0/12
deny from 112.0.0.0/10
deny from 112.100.0.0/14
deny from 112.111.0.0/16
deny from 112.124.0.0/14
deny from 113.56.0.0/15
deny from 113.64.0.0/11
deny from 113.96.0.0/12
deny from 113.112.0.0/13
deny from 113.140.0.0/14
deny from 114.32.0.0/12
deny from 114.80.0.0/12
deny from 114.112.0.0/14
deny from 114.216.0.0/13
deny from 114.224.0.0/12
deny from 114.240.0.0/12
deny from 115.24.0.0/15
deny from 115.100.0.0/14
deny from 115.224.0.0/12
deny from 116.16.0.0/12
deny from 116.76.0.0/15
deny from 116.90.80.0/20
deny from 116.112.0.0/14
deny from 116.224.0.0/12
deny from 117.24.0.0/13
deny from 117.32.0.0/14
deny from 117.40.0.0/14
deny from 117.128.0.0/10
deny from 118.72.0.0/13
deny from 118.120.0.0/14
deny from 118.140.0.0/14
deny from 118.144.0.0/15
deny from 118.186.0.0/16
deny from 118.192.0.0/16
deny from 119.4.0.0/14
deny from 119.39.0.0/16
deny from 119.62.0.0/16
deny from 119.75.208.0/20
deny from 119.84.0.0/14
deny from 119.96.0.0/13
deny from 119.144.0.0/14
deny from 119.164.0.0/14
deny from 119.176.0.0/12
deny from 119.254.0.0/15
deny from 120.32.0.0/12
deny from 120.80.0.0/13
deny from 120.128.0.0/14
deny from 120.132.0.0/16
deny from 120.192.0.0/11
deny from 121.8.0.0/13
deny from 121.250.0.0/15
deny from 121.204.0.0/14
deny from 122.136.0.0/13
deny from 122.0.64.0/18
deny from 122.64.0.0/11
deny from 122.96.0.0/15
deny from 122.192.0.0/14
deny from 122.224.0.0/12
deny from 123.62.0.0/15
deny from 123.112.0.0/12
deny from 123.128.0.0/13
deny from 123.138.0.0/15
deny from 123.149.0.0/16
deny from 123.150.0.0/15
deny from 123.164.0.0/14
deny from 124.16.0.0/15
deny from 124.72.0.0/16
deny from 124.89.0.0/17
deny from 124.92.0.0/14
deny from 124.128.0.0/13
deny from 124.160.0.0/16
deny from 124.200.0.0/13
deny from 124.226.0.0/15
deny from 124.228.0.0/14
deny from 124.232.0.0/15
deny from 124.236.0.0/14
deny from 124.240.128.0/18
deny from 125.36.0.0/14
deny from 125.64.0.0/13
deny from 125.88.0.0/13
deny from 125.127.0.0/16
deny from 125.210.0.0/14
deny from 125.216.0.0/15
deny from 125.218.0.0/16
deny from 140.224.0.0/16
deny from 159.226.0.0/16
deny from 171.208.0.0/12
deny from 175.42.0.0/15
deny from 175.44.0.0/16
deny from 180.76.0.0/16
deny from 180.95.128.0/17
deny from 180.96.0.0/11
deny from 180.152.0.0/13
deny from 180.160.0.0/12
deny from 182.50.0.0/20
deny from 182.88.0.0/14
deny from 182.112.0.0/12
deny from 182.240.0.0/13
deny from 183.0.0.0/10
deny from 183.92.0.0/14
deny from 183.192.0.0/10
deny from 202.75.192.0/19
deny from 202.85.208.0/20
deny from 202.98.0.0/16
deny from 202.102.0.0/17
deny from 202.105.0.0/16
deny from 202.106.0.0/15
deny from 202.108.0.0/16
deny from 202.112.0.0/13
deny from 202.120.0.0/15
deny from 202.136.48.0/20
deny from 202.173.224.0/19
deny from 203.93.0.0/16
deny from 203.156.192.0/18
deny from 203.186.96.0/19
deny from 203.198.0.0/16
deny from 210.14.64.0/19
deny from 210.22.0.0/16
deny from 210.51.0.0/16
deny from 210.177.0.0/16
deny from 211.70.0.0/15
deny from 211.88.0.0/13
deny from 211.96.0.0/13
deny from 211.128.0.0/11
deny from 211.160.0.0/13
deny from 221.192.0.0/14
deny from 218.0.0.0/12
deny from 218.16.0.0/14
deny from 218.20.0.0/16
deny from 218.24.0.0/15
deny from 218.30.0.0/15
deny from 218.56.0.0/13
deny from 218.64.0.0/11
deny from 218.96.0.0/14
deny from 218.104.0.0/14
deny from 218.108.0.0/15
deny from 218.188.0.0/15
deny from 218.194.0.0/16
deny from 218.200.0.0/13
deny from 218.213.0.0/16
deny from 218.240.0.0/13
deny from 219.76.0.0/14
deny from 219.128.0.0/11
deny from 219.154.0.0/15
deny from 219.156.0.0/15
deny from 219.222.0.0/15
deny from 219.232.0.0/14
deny from 220.154.0.0/15
deny from 220.160.0.0/11
deny from 220.192.0.0/12
deny from 220.248.0.0/14
deny from 220.252.0.0/16
deny from 221.0.0.0/13
deny from 221.10.0.0/16
deny from 221.12.0.0/17
deny from 221.12.128.0/18
deny from 221.122.0.0/15
deny from 221.176.0.0/13
deny from 221.207.0.0/16
deny from 221.216.0.0/13
deny from 221.232.0.0/14
deny from 221.238.0.0/15
deny from 222.32.0.0/11
deny from 222.64.0.0/11
deny from 222.176.0.0/13
deny from 222.184.0.0/13
deny from 222.204.0.0/15
deny from 222.240.0.0/13
deny from 223.4.0.0/14
deny from 223.202.0.0/15
# CHOOPA-USA (AHREFS)
deny from 173.199.64.0/18
# CISCO-USA/GLOBAL
deny from 173.36.0.0/14
deny from 208.90.56.0/21
# CITYLINK-RUS
deny from 178.19.240.0/20
# CLOUDATA-MOL
deny from 176.123.0.0/19
# CMC-VTN
deny from 101.99.0.0/18
# CNT-RUS
deny from 213.85.0.0/17
# CODERO-USA
deny from 66.226.72.0/21
deny from 69.64.64.0/19
deny from 206.225.80.0/21
deny from 206.225.92.0/22
deny from 216.55.168.0/22
deny from 216.55.160.0/21
# COLO4-USA/CAN
deny from 173.237.128.0/18
# COLOA-USA/CAN
deny from 74.119.144.0/22
deny from 184.170.240.0/20
deny from 192.111.128.0/20
deny from 198.8.80.0/20
deny from 199.58.184.0/22
deny from 199.116.112.0/21
deny from 199.187.208.0/22
deny from 199.229.248.0/21
# COLOCALL-UKR
deny from 62.149.0.0/19
# COMCAST-BUSINESS-USA
deny from 50.76.0.0/14
deny from 75.144.64.0/19
# COMPASS-USA
deny from 64.246.160.0/19
# CONTABO-DE
deny from 79.143.176.0/20
deny from 193.200.241.0/24
# CORBINA-RUS
deny from 37.144.0.0/14
deny from 89.178.0.0/15
deny from 93.80.0.0/15
deny from 95.24.0.0/13
deny from 128.69.119.0/24
# CSSGROUP-LTV (CHRISBARTEO)
deny from 94.142.128.0/21
# D-HOSTING-DE
deny from 212.42.224.0/19
# DANISNET-MOL
deny from 79.140.160.0/20
# DASSAULT-FR
deny from 178.255.208.0/21
# DATABASEBYDESIGNLLC-USA
deny from 76.72.160.0/20
# DATACENTER3000-RUS
deny from 81.17.144.0/20
# DATAPORT-DE
deny from 141.91.0.0/16
# DELTA-GEO
deny from 93.188.8.0/21
# DEUTSCHE POST-DE
deny from 192.109.50.0/24
# DIGI-HNG
deny from 78.131.0.0/17
# DMEHOSTING-USA
deny from 74.221.208.0/20
# DMS-USA
deny from 23.29.48.0/20
# DNSSLAVE-USA
deny from 50.115.160.0/20
deny from 173.0.48.0/20
deny from 198.175.124.0/22
deny from 199.119.224.0/22
# DOD-USA
deny from 11.0.0.0/8
# DOMAINFACTORY-DE
deny from 46.252.16.0/20
# DOMAINTOOLS-USA
deny from 199.30.228.0/22
# DONEKOSERV-RUS
deny from 91.201.64.0/24
# DSCONTACT-UK (dedicatedservercontact.com)
# siehe REDSTATION-UK
# DREAMHOST-USA
deny from 64.90.32.0/19
# DTAG-BUSINESS (Muss definitiv nicht!!!!)
deny from 194.25.0.0/16
# EGIHOSTING-CAN
deny from 23.27.0.0/16
deny from 50.117.0.0/17
deny from 50.118.128.0/17
deny from 64.145.64.0/19
deny from 68.68.96.0/20
deny from 69.46.64.0/19
deny from 72.13.80.0/20
deny from 136.0.0.0/16
deny from 142.111.0.0/16
deny from 142.252.0.0/16
deny from 166.88.0.0/16
deny from 172.252.0.0/16
deny from 173.245.64.0/19
deny from 173.245.194.0/23
deny from 173.245.220.0/22
deny from 205.164.0.0/18
deny from 209.73.128.0/19
deny from 209.107.192.0/19
deny from 216.151.176.0/20
deny from 216.172.128.0/19
# EIS North-West, State Medical Insurance Company, St. Petersburg, RUS
deny from 82.115.208.0/20
# ELAN-POL
deny from 91.236.74.0/23
# ELIGHTTELECOM-RUS
deny from 46.180.0.0/15
deny from 176.196.0.0/15
# EMERGENCY24-USA
deny from 12.171.94.0/24
# EMIRNET-UAE
deny from 86.96.0.0/14
deny from 195.229.240.0/22
# ENET-CAN
deny from 64.79.64.0/19
# ENZU-USA
deny from 23.88.0.0/15
deny from 67.202.104.0/22
deny from 69.162.132.0/22
deny from 172.246.0.0/16
deny from 184.105.219.0/24
deny from 192.80.128.0/18
deny from 192.157.192.0/18
deny from 198.56.128.0/17
deny from 198.71.80.0/20
deny from 198.98.96.0/19
deny from 199.48.68.0/22
deny from 199.188.72.0/22
deny from 199.193.248.0/21
deny from 199.229.232.0/22
deny from 199.231.208.0/21
# EONIX-USA
deny from 50.2.0.0/15
deny from 75.75.224.0/19
deny from 173.44.128.0/17
deny from 173.213.64.0/18
deny from 173.232.0.0/16
deny from 206.214.64.0/19
deny from 208.89.216.0/21
# ER-TELECOM-RUS
deny from 5.164.176.0/22
deny from 46.0.0.0/16
deny from 94.180.0.0/17
deny from 109.194.0.0/15
deny from 176.212.0.0/14
deny from 188.232.0.0/14
# ETOP-POL
deny from 80.72.37.0/24
# EVEREST-UKR
deny from 109.108.224.0/19
# FARLEB-INVEST-UKR
deny from 89.105.224.0/19
deny from 93.127.0.0/17
deny from 93.178.192.0/18
# FASTIT-DE / MYLOC-DE
deny from 62.141.32.0/20
deny from 81.30.144.0/20
deny from 85.114.128.0/19
deny from 93.186.192.0/20
# FDCSERVERS-USA
deny from 67.159.0.0/18
# FIBERNET-DNK
deny from 37.75.160.0/19
# FLAGMAN-UKR
deny from 46.46.64.0/18
# FOBOSS-UKR
deny from 93.170.144.0/20
# FREENET-UKR
deny from 109.251.0.0/16
# FRONTIERNET-USA
deny from 50.32.0.0/12
deny from 50.48.0.0/13
# FUTURENET-RUS
deny from 195.43.128.0/24
# G-COM-POL
deny from 193.151.100.0/22
# G-PORTAL-DE
deny from 176.57.128.0/18
# GALAXYVISIONS-CAN/USA
deny from 209.104.192.0/19
# GHOSTNET-DE
deny from 94.249.128.0/17
# GNET-UKR
deny from 91.231.40.0/22
# GODADDY-GLOBAL
deny from 97.74.0.0/16
deny from 184.168.0.0/16
deny from 188.121.32.0/19
deny from 208.109.0.0/16
# GOLDEN TELECOM-UKR
deny from 46.118.0.0/15
deny from 94.27.0.0/17
# GYRON-UK
deny from 89.145.64.0/18
# HALDEX-UK
deny from 213.174.128.0/19
# HAWKHOST-USA
deny from 72.29.127.0/24
# HETZNER-DE
deny from 5.9.0.0/16
deny from 46.4.0.0/16
deny from 78.46.0.0/15
deny from 80.83.176.0/20
deny from 85.10.192.0/18
deny from 88.198.0.0/16
deny from 89.144.56.0/21
deny from 91.220.49.0/24
deny from 144.76.0.0/16
deny from 176.9.0.0/16
deny from 178.63.0.0/16
deny from 188.40.0.0/16
deny from 193.28.90.0/24
deny from 193.47.99.0/24
deny from 193.110.6.0/23
deny from 193.223.77.0/24
deny from 195.13.45.0/24
deny from 213.133.96.0/19
deny from 213.239.192.0/18
# HOPONE-USA
deny from 209.160.64.0/20
deny from 209.160.0.0/18
# HOSTEUROPE-DE
deny from 83.169.0.0/18
deny from 176.28.0.0/18
deny from 178.77.64.0/18
# HOSTINGPANAMA-PAN
deny from 200.106.144.0/21
# HOSTNOC-USA
deny from 96.9.128.0/18
deny from 173.212.192.0/18
deny from 184.22.0.0/16
deny from 184.82.0.0/16
# HOSTWINDS-USA
deny from 199.59.56.0/21
# HURRICANE-USA
deny from 64.62.128.0/17
deny from 64.71.128.0/18
deny from 65.19.128.0/18
deny from 65.49.0.0/17
deny from 66.160.128.0/18
deny from 66.160.192.0/20
deny from 66.220.0.0/19
deny from 72.52.64.0/18
deny from 74.82.0.0/18
deny from 184.104.0.0/15
deny from 209.51.160.0/19
deny from 216.66.0.0/18
deny from 216.66.64.0/19
deny from 216.218.128.0/17
# FINANZINFORMATIK-DE
deny from 195.140.0.0/17
# HOSTINGSOLUTIONS-USA
deny from 50.30.32.0/20
deny from 69.64.32.0/19
deny from 173.224.112.0/20
deny from 199.189.84.0/22
deny from 199.217.112.0/21
deny from 209.239.112.0/20
# IDEAL-SOLUTIONS-SEY
deny from 193.107.16.0/22
# IFX-CHL
deny from 200.73.16.0/23
# IMPERIAL COLLEGE LONDON - UK
deny from 155.198.0.0/16
# INFERNO-NET British Virgin Islands
deny from 188.143.232.0/23
# INFINITE-USA
deny from 173.213.109.0/24
# INFIUM-UKR
deny from 188.190.96.0/19
# INFOLINE-RUS
deny from 5.35.0.0/18
# INNENMINISTERIUM-DE
deny from 195.126.85.0/24
# INNENMINISTERIUM-IT
deny from 212.14.128.0/19
# INSTITUTE OF MACHINERY-RUS
deny from 194.85.14.0/23
# INTERGENIA-DE
deny from 62.75.128.0/17
deny from 85.25.0.0/16
deny from 188.138.0.0/17
# INTERNAP-USA
deny from 8.10.58.0/23
deny from 63.175.174.0/23
deny from 63.251.0.0/16
deny from 64.7.192.0/19
deny from 64.74.0.0/16
deny from 64.86.0.0/16
deny from 64.94.0.0/15
deny from 66.79.128.0/19
deny from 66.150.0.0/15
deny from 66.171.192.0/20
deny from 69.25.0.0/16
deny from 69.88.128.0/19
deny from 70.42.0.0/16
deny from 72.5.0.0/16
deny from 74.201.0.0/16
deny from 74.217.0.0/16
deny from 75.98.0.0/18
deny from 75.98.64.0/19
deny from 199.19.124.0/22
deny from 199.21.76.0/22
deny from 199.21.80.0/22
deny from 199.244.252.0/22
deny from 205.162.12.0/22
deny from 205.247.40.0/24
deny from 206.64.105.0/24
deny from 206.191.128.0/18
deny from 206.253.192.0/19
deny from 208.8.174.0/24
deny from 208.33.216.0/22
deny from 208.146.32.0/20
deny from 209.191.128.0/18
deny from 216.52.0.0/16
deny from 216.133.224.0/19
deny from 216.223.0.0/18
# INTERNETMEMORYRESEARCH-FR
deny from 37.16.72.0/21
# INTERNETX-DE
deny from 62.116.160.0/19
# INTERSERVER-CAN
deny from 173.214.160.0/20
# INTERSVYAZ2-RUS
deny from 78.29.16.0/20
deny from 109.191.128.0/17
# IPNET-UKR
deny from 82.193.96.0/19
# IPSTRADA-USA
deny from 199.15.232.0/21
# IPONLY-SE / DPNOC-SE / PREMIUMREGISTRATIONS-SE
deny from 82.99.0.0/18
deny from 193.28.43.0/24
deny from 193.43.151.0/24
deny from 193.138.139.0/24
deny from 193.180.62.0/24
deny from 193.183.180.0/24
deny from 193.192.54.0/23
deny from 193.235.207.0/24
deny from 194.14.170.0/24
deny from 195.90.104.0/23
deny from 195.149.123.0/24
deny from 212.16.160.0/19
deny from 212.37.0.0/19
deny from 212.112.160.0/19
deny from 213.80.0.0/18
deny from 213.212.0.0/18
# IPTELLIGENT-USA
deny from 96.47.224.0/20
# IQOM-BUSINESS-DE
deny from 178.210.96.0/19
# IRAN-INR
deny from 2.144.0.0/14
deny from 2.176.0.0/12
deny from 5.22.0.0/17
deny from 5.22.192.0/19
deny from 5.23.112.0/21
deny from 5.28.32.0/21
deny from 5.34.192.0/19
deny from 5.52.0.0/16
deny from 5.53.32.0/19
deny from 5.57.32.0/21
deny from 5.61.24.0/21
deny from 5.61.72.0/21
deny from 5.63.8.0/21
deny from 5.102.32.0/20
deny from 5.112.0.0/12
deny from 5.134.128.0/18
deny from 5.144.128.0/21
deny from 5.145.112.0/21
deny from 5.159.48.0/21
deny from 5.160.0.0/15
deny from 5.190.0.0/16
deny from 31.7.64.0/18
deny from 31.7.128.0/20
deny from 31.24.200.0/21
deny from 31.24.232.0/21
deny from 31.25.88.0/21
deny from 31.25.104.0/21
deny from 31.25.128.0/21
deny from 31.25.232.0/21
deny from 31.29.32.0/19
deny from 31.40.0.0/21
deny from 31.47.32.0/19
deny from 31.56.0.0/14
deny from 31.130.176.0/20
deny from 31.170.48.0/20
deny from 31.171.216.0/21
deny from 31.184.128.0/18
deny from 31.193.221.0/21
deny from 31.193.144.0/20
deny from 31.217.208.0/21
deny from 37.9.248.0/21
deny from 37.19.80.0/20
deny from 37.32.0.0/19
deny from 37.32.32.0/20
deny from 37.32.112.0/20
deny from 37.44.56.0/21
deny from 37.49.144.0/21
deny from 37.63.128.0/17
deny from 37.75.240.0/21
deny from 37.98.0.0/17
deny from 37.98.200.0/21
deny from 37.114.192.0/18
deny from 37.123.192.0/21
deny from 46.18.248.0/21
deny from 46.21.80.0/20
deny from 46.28.72.0/21
deny from 46.32.0.0/19
deny from 46.34.96.0/19
deny from 46.34.160.0/19
deny from 46.36.96.0/19
deny from 46.38.128.0/19
deny from 46.100.0.0/16
deny from 46.143.0.0/16
deny from 46.148.32.0/20
deny from 46.164.64.0/18
deny from 46.183.184.0/21
deny from 46.209.0.0/16
deny from 46.224.0.0/15
deny from 46.245.0.0/17
deny from 46.248.32.0/19
deny from 46.249.96.0/19
deny from 46.251.160.0/19
deny from 46.255.216.0/21
deny from 62.60.128.0/17
deny from 62.102.128.0/20
deny from 62.193.0.0/19
deny from 77.36.128.0/17
deny from 77.77.64.0/18
deny from 77.104.64.0/18
deny from 77.237.64.0/19
deny from 77.237.160.0/19
deny from 77.245.224.0/20
deny from 78.31.224.0/21
deny from 78.38.0.0/15
deny from 78.109.192.0/20
deny from 78.110.112.0/20
deny from 78.111.0.0/20
deny from 78.154.32.0/19
deny from 78.157.32.0/19
deny from 78.158.160.0/19
deny from 79.127.0.0/17
deny from 79.132.192.0/19
deny from 79.170.144.0/21
deny from 79.175.128.0/18
deny from 80.66.176.0/20
deny from 80.69.240.0/20
deny from 80.71.112.0/20
deny from 80.75.0.0/20
deny from 80.191.0.0/16
deny from 80.242.0.0/20
deny from 80.253.128.0/19
deny from 81.28.32.0/19
deny from 81.29.240.0/20
deny from 81.31.160.0/19
deny from 81.31.224.0/19
deny from 81.90.144.0/20
deny from 81.91.128.0/19
deny from 81.163.0.0/21
deny from 82.99.192.0/18
deny from 82.115.0.0/19
deny from 83.147.192.0/18
deny from 85.9.64.0/18
deny from 85.15.0.0/18
deny from 85.133.128.0/17
deny from 85.185.0.0/16
deny from 85.198.0.0/18
deny from 86.57.0.0/17
deny from 86.109.32.0/19
deny from 87.107.0.0/16
deny from 87.247.160.0/19
deny from 87.248.128.0/19
deny from 88.135.32.0/20
deny from 89.144.128.0/18
deny from 89.165.0.0/17
deny from 89.184.192.0/19
deny from 89.221.80.0/20
deny from 89.235.64.0/18
deny from 91.98.0.0/15
deny from 91.106.64.0/19
deny from 91.108.128.0/19
deny from 91.184.64.0/19
deny from 91.186.192.0/19
deny from 91.242.44.0/23
deny from 91.243.126.0/23
deny from 91.243.160.0/20
deny from 93.126.0.0/18
deny from 109.72.192.0/20
deny from 109.74.224.0/20
deny from 109.95.56.0/21
deny from 109.95.64.0/21
deny from 109.109.32.0/19
deny from 109.110.160.0/19
deny from 109.111.32.0/19
deny from 109.112.192.0/18
deny from 109.125.128.0/18
deny from 109.162.128.0/17
deny from 109.201.0.0/19
deny from 109.203.160.0/19
deny from 109.225.128.0/18
deny from 109.230.64.0/18
deny from 109.238.176.0/20
deny from 109.239.0.0/20
deny from 128.65.160.0/19
deny from 128.140.0.0/17
deny from 130.185.72.0/21
deny from 130.255.192.0/18
deny from 178.157.0.0/18
deny from 178.252.168.0/18
deny from 188.75.64.0/18
deny from 188.245.0.0/16
deny from 213.217.32.0/19
deny from 217.66.212.0/23
deny from 217.218.160.0/19
deny from 217.219.160.0/20
# IRONPATH-USA
deny from 66.112.208.0/20
deny from 67.209.176.0/20
# ISKRATELECOM-RUS
deny from 82.199.96.0/19
deny from 79.172.64.0/18
deny from 213.193.0.0/19
# ISPHOST-UKR
deny from 192.162.19.0/24
deny from 193.105.210.0/24
# ISPONE-BUSINESS-DE
deny from 84.38.64.0/20
deny from 109.73.48.0/20
# ISPPRO-DE
deny from 81.7.0.0/18
deny from 91.143.80.0/20
# IWACOM-POL
deny from 194.28.48.0/22
# IWEB-USA
deny from 108.163.128.0/18
# JENOPTIK-DE
deny from 213.248.64.0/18
# Jijikov Sergei Aleksandrovich
deny from 31.184.238.0/24
# JITESI-USA
deny from 142.4.117.96/29
# KDDI-JP
deny from 27.34.128.0/19
# KEYWEB-DE
deny from 84.19.160.0/19
deny from 87.118.64.0/18
# KIEVSTAR-UKR
deny from 5.248.0.0/16
deny from 37.229.0.0/16
deny from 46.185.0.0/18
deny from 94.153.0.0/17
deny from 176.8.0.0/16
deny from 178.137.0.0/16
# KOMLAN-RUS
deny from 31.130.32.0/19
# KRISTALL-RUS
deny from 46.175.96.0/21
# KRYPT-USA
deny from 67.229.0.0/16
deny from 98.126.0.0/16
# LAMBDANET-DE
deny from 83.133.0.0/16
# LANCOM-UKR
deny from 78.30.192.0/18
# LANDESAMT FÜR STATISTIK UND DATENERHEBUNG (BAYERN)-DE
deny from 195.200.70.0/23
# LANDESNETZWERK SACHSEN-ANHALT (inkl. Polizei)-DE
deny from 164.133.0.0/16
# LEASEWEB-global (HOL, DE, USA)
deny from 37.58.48.0/20
deny from 46.165.192.0/18
deny from 82.192.64.0/19
deny from 83.149.64.0/18
deny from 85.17.0.0/16
deny from 94.75.220.0/24
deny from 95.211.0.0/16
deny from 108.59.0.0/20
deny from 199.115.112.0/21
deny from 217.73.56.0/21
# LIMESTONE-USA
deny from 64.31.0.0/18
deny from 69.162.64.0/18
deny from 216.144.240.0/20
# LINKUP-UKR
deny from 176.119.0.0/20
# M9COM-RUS
deny from 195.135.239.0/22
# MAROCTELECOM-MRC
deny from 41.140.0.0/14
deny from 41.248.0.0/14
# Markus Bach Betriebs Gesellschaft mbH-DE
deny from 195.137.212.0/23
# MAXNET-UKR
deny from 178.165.0.0/18
# MESH-DE
deny from 213.203.192.0/19
# METROTELECOM-UKR
deny from 193.29.16.0/24
# MICROSOFT-USA
deny from 65.52.0.0/14
deny from 131.253.32.0/20
deny from 131.253.21.0/24
deny from 131.253.22.0/23
deny from 131.253.24.0/21
deny from 137.117.0.0/16
deny from 137.135.0.0/16
deny from 157.56.0.0/14
deny from 157.54.0.0/15
deny from 157.60.0.0/16
# MICT-THA
deny from 123.242.160.0/19
# MIR-TELEMATIKI-RUS
deny from 91.218.244.0/22
deny from 141.105.64.0/21
# MOLDTELECOM-MLD
deny from 46.55.64.0/18
deny from 188.237.0.0/16
# MTSNET-RUS
deny from 213.87.128.0/21
deny from 213.87.240.0/22
# MULTISCAN-RUS
deny from 83.217.192.0/19
# N9UF-FR
deny from 93.0.0.0/11
deny from 109.0.0.0/11
# NASCHNET-UKR
deny from 188.239.0.0/18
# NAVIGATOR-UKR
deny from 213.169.64.0/19
# NCN-RUS
deny from 37.204.0.0/16
# NETDIREKT-DE
deny from 78.159.96.0/19
deny from 84.16.224.0/19
deny from 89.149.192.0/18
deny from 95.168.160.0/19
deny from 178.162.128.0/17
deny from 188.72.192.0/18
deny from 212.95.32.0/18
deny from 217.20.112.0/20
# NETQUADRANT-AUS
deny from 202.146.208.0/21
# NEUE MEDIEN MÜNNICH-DE
deny from 85.13.128.0/19
# NFORCE-HOL
deny from 77.247.176.0/21
# NOBIS-USA
deny from 23.19.0.0/16
deny from 64.120.0.0/17
deny from 69.147.224.0/19
deny from 108.62.0.0/16
deny from 108.177.128.0/17
deny from 142.91.0.0/16
deny from 142.234.0.0/16
deny from 173.234.0.0/16
# NOVOTELECOM-RUS
deny from 178.49.0.0/17
# OBENETWORK-SE
deny from 46.227.64.0/21
# OC3-USA
deny from 96.44.128.0/18
# OLIMPSTROY-RUS
deny from 91.241.14.0/23
# ONLINEDIENST-NORDBAYERN-DE
deny from 86.109.224.0/19
# OPTIMATE-DE
deny from 5.62.0.0/17
deny from 37.10.0.0/17
deny from 95.156.192.0/18
deny from 109.230.192.0/18
# ORENBURGSKAYA-RUS
deny from 95.105.96.0/19
# OSYPENKO-UKR
deny from 193.107.224.0/22
# OVH-GLOBAL
deny from 5.39.0.0/17
deny from 5.135.0.0/16
deny from 8.0.0.0/8
deny from 37.59.0.0/16
deny from 46.105.0.0/16
deny from 87.98.128.0/17
deny from 91.121.0.0/16
deny from 94.23.0.0/16
deny from 142.4.192.0/19
deny from 176.31.0.0/16
deny from 178.32.0.0/15
deny from 188.165.0.0/16
deny from 192.95.0.0/18
deny from 198.27.64.0/18
deny from 198.50.128.0/17
deny from 198.100.144.0/20
deny from 198.245.48.0/20
deny from 213.186.32.0/19
deny from 213.251.128.0/18
# PACIFICSTAR-USA
deny from 205.209.128.0/18
# PCEXTREME-HOL
deny from 93.180.64.0/21
# PEGTECH-USA
deny from 137.175.0.0/17
deny from 142.0.128.0/20
deny from 142.4.96.0/19
deny from 192.74.224.0/19
deny from 198.2.192.0/18
deny from 198.200.32.0/19
deny from 199.180.100.0/22
deny from 199.188.104.0/21
# PETERSTAR-RUS
deny from 46.47.192.0/18
# PLATH GMBH - DE
deny from 212.202.229.62
# PODOL-UKR
deny from 193.239.254.0/23
# PORTLANE-SE
deny from 46.246.0.0/17
# PROXAD-FR
deny from 88.160.0.0/11
# PSYCHZ-USA
deny from 108.171.240.0/20
deny from 192.210.48.0/20
deny from 199.15.112.0/21
deny from 199.83.88.0/21
deny from 216.99.144.0/20
# QWEST-USA
deny from 65.125.184.240/29
# REDE EXITUS-B
deny from 186.227.240.0/20
# REDSTATION-UK
deny from 5.152.192.0/19
deny from 31.3.224.0/19
deny from 37.220.0.0/19
deny from 80.84.48.0/20
deny from 109.73.64.0/20
deny from 109.200.0.0/19
deny from 149.3.136.0/22
# RENOME-UKR
deny from 78.26.128.0/17
# RESILANS-SE
deny from 194.71.0.0/16
# RITC-RUS
deny from 109.105.64.0/19
# ROOT-LUX
deny from 94.242.192.0/18
# ROSTELECOM - MACROREGIONAL-ROSTELECOM-RUS
deny from 31.23.128.0/17
deny from 95.71.0.0/17
deny from 109.62.128.0/17
deny from 212.3.128.0/19
# RUBIN-RUS
deny from 77.232.128.0/19
# SAKURA-JP
deny from 49.212.0.0/16
# SANKT-PETERBURGSKOE KABELNOE TELEVIDENIE JSC-RUS
deny from 109.205.248.0/21
# SAUDIA-CYBERIA-ARS
deny from 212.107.96.0/19
# SCARTEL-RUS
deny from 188.162.0.0/16
# SCHLUND-DE
deny from 87.106.0.0/16
deny from 212.227.0.0/16
# SECUREDSERVERS-USA
deny from 184.95.32.0/19
deny from 198.15.64.0/18
# SECUREPRIVATENETOWRK-USA
deny from 66.212.16.0/20
deny from 67.215.224.0/19
deny from 216.144.224.0/20
# SEECH-INFOCOM-UKR
deny from 77.93.32.0/19
# SERPRO-BR
deny from 161.148.0.0/16
# SERVERGURUS-USA
deny from 64.250.114.0/24
# SERVERHUBCLOUD-USA
deny from 173.213.111.0/24
# SEVEREL-USA
deny from 109.206.160.0/19
# SEZNAM-CZR
deny from 77.75.76.0/23
# SINGLEHOP-USA
deny from 65.60.0.0/18
deny from 67.212.160.0/19
deny from 69.175.0.0/17
deny from 96.127.128.0/18
deny from 99.198.96.0/19
deny from 108.163.192.0/18
deny from 108.178.0.0/18
deny from 173.236.0.0/17
deny from 184.154.0.0/16
deny from 198.20.64.0/18
deny from 198.143.128.0/18
# SKY-UK
deny from 5.64.0.0/13
deny from 81.128.0.0/11
deny from 82.108.0.0/14
deny from 87.82.0.0/15
deny from 87.84.0.0/14
deny from 90.192.0.0/11
deny from 193.195.0.0/16
deny from 212.240.0.0/16
deny from 217.204.0.0/14
# SKYNET-RUS
deny from 94.19.0.0/16
deny from 188.242.0.0/15
# SOFTLAYER-USA
deny from 50.22.0.0/15
deny from 173.192.0.0/15
deny from 208.43.0.0/16
# SPBNIT-RUS (St. Petersburg Internet)
deny from 95.52.0.0/14
# SPEEDOFLIGHT-USA
deny from 68.235.36.0/22
# SPRINT-USA
deny from 206.160.0.0/16
# STARLINK-RUS
deny from 77.50.0.0/16
# STEADFAST-USA
deny from 23.29.128.0/19
deny from 50.31.0.0/17
deny from 67.202.64.0/18
deny from 69.162.128.0/18
deny from 208.66.168.0/21
deny from 208.100.0.0/18
deny from 208.117.0.0/18
deny from 216.86.144.0/20
# STEEPHOST-UKR
deny from 91.207.4.0/22
deny from 91.207.8.0/23
deny from 195.190.13.0/24
# STRATO-RZ (Einzeladressen)
deny from 85.214.21.33
deny from 85.214.130.217
deny from 85.214.254.220
# SVYAZINFORM-RUS
deny from 31.148.0.0/17
# SWIFTWAY-POL/FR/HOL
deny from 46.21.144.0/20
# TALKTALK-UK
deny from 62.24.128.0/17
deny from 92.0.0.0/11
# TANGRAM-CAN
deny from 109.72.148.0/22
# TATTELECOM-RUS
deny from 178.204.0.0/15
# TECHSUPPORT-RUS
deny from 91.237.249.0/24
# TELCOM-UKR
deny from 178.210.128.0/19
# TELEHOUSE-RUS
deny from 82.146.56.0/21
deny from 94.127.144.0/21
# TELEMOST-UKR
deny from 178.215.160.0/20
# TELESET-RUS
deny from 87.224.128.0/17
# TENET-UKR
deny from 85.238.96.0/20
# TEO-LTH
deny from 212.59.0.0/19
# TERONET-LTV
deny from 91.224.246.0/23
# THALES-DE
deny from 62.225.145.224/28
# THEPLANET-GLOBAL / Planet Technologies Worldwide
deny from 74.52.0.0/14
deny from 174.120.0.0/14
deny from 184.172.0.0/15
# TOT-THA
deny from 118.172.0.0/19
# TRENDMICRO-WELTWEIT
deny from 150.70.0.0/16
# TRIOLAN-UKR
deny from 37.46.224.0/19
deny from 37.57.0.0/16
deny from 80.73.0.0/20
deny from 109.86.0.0/15
deny from 159.224.0.0/16
deny from 178.150.156.0/24
# TTKC-RUS
deny from 31.40.8.0/21
# TU-BRAUNSCHWEIG-DE
deny from 134.169.0.0/16
# TURNKEY-USA
deny from 64.128.116.0/23
deny from 64.128.190.0/23
deny from 66.206.32.0/20
deny from 67.231.240.0/20
deny from 173.198.192.0/18
deny from 173.233.64.0/19
deny from 198.48.48.0/20
deny from 199.223.252.0/22
deny from 208.85.0.0/21
deny from 209.240.96.0/20
# TVCOM-UKR
deny from 89.185.0.0/19
# UAB-LTH
deny from 78.158.0.0/19
# UARNET-UKR
deny from 194.44.0.0/16
# UBIQUITY-USA
deny from 108.62.152.0/21
# UFANET-RUS
deny from 77.79.128.0/18
deny from 79.140.16.0/20
deny from 94.41.0.0/16
deny from 95.105.0.0/18
# UK2NET-UK
deny from 83.170.64.0/19
# UKRTELECOM-UKR (u.a. AHREFSBOT!)
deny from 37.52.0.0/14
deny from 93.190.40.0/21
deny from 94.45.96.0/19
deny from 94.178.0.0/15
deny from 95.132.0.0/14
deny from 178.92.0.0/14
deny from 212.113.32.0/19
deny from 213.186.96.0/19
# UNIFIED-LAYER-USA
deny from 142.4.0.0/19
# UNIVERSITÄT LEIPZIG-DE (Informatik-Fakultät - ewiger Client-Sniffer)
deny from 139.18.0.0/15
# UNIVERSITÄT DER BUNDESWEHR MÜNCHEN - DE
deny from 137.193.0.0/16
# UNIVERSITÄT SHEFFIELD-UK
deny from 143.167.0.0/16
# UPC-GLOBAL
deny from 87.206.0.0/16
deny from 89.76.0.0/16
# VERISIGN-USA
deny from 69.58.176.0/20
# VERSAWEB-USA / -RUS
deny from 72.46.128.0/19
deny from 208.66.72.0/21
# VERSIYA-RUS
deny from 89.107.120.0/21
# VIAEUROPA-SE
deny from 93.182.128.0/18
# VIMPELCOM-RUS
deny from 128.68.0.0/14
deny from 128.72.0.0/14
deny from 176.14.0.0/15
deny from 217.118.64.0/19
# VIRGIN-UK
deny from 82.37.0.0/17
# VIVANET-UKR
deny from 46.150.64.0/19
# VLADIVOSTOKTELECOM-RUS
deny from 176.100.128.0/19
# VOLIA-UKR
deny from 77.120.0.0/14
deny from 77.123.160.0/19
deny from 93.72.0.0/13
# VOLUMEDRIVE-USA
deny from 74.118.192.0/22
deny from 142.0.32.0/20
deny from 173.242.112.0/20
deny from 192.69.88.0/21
deny from 199.19.104.0/21
deny from 199.115.228.0/22
deny from 199.168.136.0/21
deny from 199.180.112.0/21
deny from 204.124.180.0/22
# VOXILITY-ROU
deny from 109.163.224.0/20
# VPNTUNNEL-SE
deny from 5.254.128.0/19
# VPS21-CAN
deny from 199.193.64.0/21
# VPS-HOSTING-LTV
deny from 91.226.32.0/23
# WANADOO-FR
deny from 193.252.118.0/24
# WEBAIR-USA
deny from 69.42.64.0/19
# WEBEXXPURTS-SE
deny from 5.34.240.0/21
deny from 5.153.232.0/21
deny from 5.157.0.0/18
deny from 37.72.184.0/21
deny from 37.203.208.0/21
deny from 46.29.248.0/21
deny from 91.108.176.0/21
deny from 130.185.152.0/21
deny from 151.237.176.0/20
deny from 176.61.136.0/21
deny from 178.216.48.0/21
deny from 185.3.132.0/22
# WEBSENSE-UK
deny from 85.115.32.0/19
# WEISSRUSSLAND-BLR
deny from 5.100.192.0/19
deny from 31.24.88.0/21
deny from 31.130.200.0/21
deny from 37.17.0.0/17
deny from 37.44.64.0/18
deny from 37.45.0.0/16
deny from 46.28.96.0/21
deny from 46.53.128.0/17
deny from 46.56.0.0/16
deny from 46.175.168.0/21
deny from 46.182.48.0/21
deny from 46.191.0.0/17
deny from 46.216.0.0/16
deny from 77.74.32.0/21
deny from 79.98.48.0/21
deny from 79.170.104.0/21
deny from 80.94.160.0/20
deny from 80.94.224.0/20
deny from 80.249.80.0/20
deny from 81.30.80.0/20
deny from 82.209.192.0/18
deny from 87.252.224.0/19
deny from 86.57.128.0/17
deny from 87.252.224.0/19
deny from 91.149.128.0/18
deny from 91.149.157.0/24
deny from 91.149.175.0/24
deny from 91.149.189.0/24
deny from 93.84.0.0/15
deny from 93.125.0.0/17
deny from 109.126.128.0/18
deny from 128.65.0.0/18
deny from 128.140.240.0/20
deny from 178.120.0.0/13
deny from 178.159.240.0/21
deny from 178.172.128.0/17
deny from 193.232.248.0/22
deny from 194.85.254.0/23
deny from 194.158.192.0/19
deny from 194.226.120.0/22
deny from 194.226.124.0/23
deny from 195.137.180.0/24
deny from 195.222.88.0/23
deny from 213.184.224.0/19
deny from 217.21.32.0/19
# WHOLESALEINTERNET-USA
deny from 69.30.192.0/18
deny from 69.197.128.0/18
deny from 173.208.128.0/17
deny from 204.12.192.0/18
# WILDPARK-UKR
deny from 80.252.240.0/20
deny from 212.92.224.0/19
# WIRECARD-DE
deny from 195.93.244.0/23
# WOLFF-IT-GMBH-AUT
deny from 62.218.122.0/24
# WORLDSTREAM-HOL
deny from 217.23.0.0/20
# WOWRACK-USA
deny from 208.115.96.0/19
deny from 216.244.64.0/19
# WZCOM-USA
deny from 74.117.176.0/21
deny from 199.80.52.0/22
deny from 199.101.132.0/22
deny from 208.88.224.0/22
deny from 208.94.232.0/22
# XCITY-UKR
deny from 46.63.0.0/17
# XEEX-GLOBAL
deny from 69.26.160.0/19
deny from 69.26.172.0/22
deny from 113.212.64.0/19
deny from 209.159.140.0/23
deny from 216.151.128.0/19
deny from 216.152.240.0/20
# XIRRA-DE
deny from 91.232.96.0/23
# XOXO-USA
deny from 208.176.0.0/15
# XSERVER-UKR
deny from 193.169.86.0/23
# YANDEX-RUS
deny from 37.140.128.0/18
deny from 77.88.0.0/18
deny from 87.250.224.0/19
deny from 93.158.128.0/18
deny from 95.108.128.0/17
deny from 130.193.32.0/19
deny from 141.8.128.0/18
deny from 178.154.128.0/17
deny from 213.180.192.0/19
# YANDEX-USA (!!!)
deny from 100.43.64.0/19
deny from 199.21.96.0/22
# YESUP-USA
deny from 166.48.0.0/16
# ZAT-UKR
deny from 46.30.160.0/21
# ZTELECOM-RUS
deny from 188.134.0.0/18
# TEMPORÄRE SCHNÜFFLER
# Irgendwer Telefonica-O2 (Registrant in EINSATZTREFF und UTDX gleichzeitig)
# T-SYSTEMS-BUSINESS /DTAG-BUSINESS
deny from 80.154.76.144/29
deny from 195.145.38.0/24
# RESOURCES-DE
deny from 213.61.241.80/29
# ECATEL-HOL
deny from 80.82.65.25
# ZHOU PIZHONG - USA
deny from 63.141.224.0/19
# ======================================================================
allow from all
# Umleitung Fehlerseiten
AuthGroupFile /dev/null
AuthName Intern
AuthType Basic
ErrorDocument 401 /error401.htm
ErrorDocument 402 /error402.htm
ErrorDocument 403 /error403.htm
ErrorDocument 404 /error404.htm
ErrorDocument 500 /error500.htm
ErrorDocument 503 /error503.htm

Darin sind IP-Blöcke eingetragen die
a) Hosting-, Housing-, Collocation- oder reine Business-Anbieter sind,
b) ISP-Adressen und Blöcke, die immer wieder diverse Funktionen aufrufen (protokolliert über die interne Bannliste, in die ich die IP-Blöcke zuerst eingetragen habe), z. B. Registrierung ohne diese zu beenden.
c) Iran und China bleiben, soweit schon erfasst, komplett aussen vor.

Wenn Du weitere Fragen haben möchtest stehe ich Dir gerne zur Verfügung. Z. B. kann ich Dir einige Recherche-Dokumente zur Verfügung stellen, anhand derer Du Deine .htaccess weiter bearbeiten kanns.

Herzliche Grüße

Thomas

Ich bin nicht die Signatur - ich rocke hier nur....!

08. Juli 2013, 10:51:05
Antwort #4

Skippy


Hallo Thomas,

erst mal sorry für meine verspätete Antwort!
Danke für das Zeigen deiner .htaccess, sie ist ja sehr umfangreich. :!.!:
Ich werde sie nun weitestgehend für mich so übernehmen.

Ja, es wäre super, wenn du mir solche Recherche-Dokumente zur Verfügung stellen würdest.
Ich hab nämlich keine Ahnung davon, wo ich Informationen über neue Bots bzw. über neue oder geänderte IP's herbekommen kann.

Ach ja, noch was.
Kann ich die .htaccess und die robots.txt  ebenfalls auch so für meine Websites nehmen, oder muss dazu  etwas im Code geändert werden? Gibt es Unterschiede dabei, ob man die Dateien in Foren oder Websites einsetzt?

LG,
Martina

08. Juli 2013, 12:44:14
Antwort #5

Amarok


Servus, Martina.

Kein Thema mit der späten Atnwort. Wir sind ja hier nicht auf der Flucht... ^^

Ich lade Dir die Tabellen mal als PDF hoch und schicke Dir dann den Link. Allerdings muss ich dazu sagen, dass die darin enthaltenen "Empfehlungen" (allow bzw. deny) sich auf eine andere Seite beziehen, bei der ich sehr, sehr hohe Maßstäbe angelegt habe, was das ausschließen angeht.

Bei der von mir angegebenen .htaccess gibt es nur eines zu beachten:

Am Ende gibt es einen Hinweis
Code
# Umleitung Fehlerseiten
.

Danach leite ich auf eigene Fehlerseiten um:
Code
ErrorDocument 401 /error401.htm
ErrorDocument 402 /error402.htm
ErrorDocument 403 /error403.htm
ErrorDocument 404 /error404.htm
ErrorDocument 500 /error500.htm
ErrorDocument 503 /error503.htm

Wenn Du eigene Fehlerseiten hast musst die natürlich entsprechend einbauen. Willst Du auf die servereigenen Fehlerseiten weiterhin verwenden nimmst Du alles ab
Code
# Umleitung Fehlerseiten
raus.

Für die Recherche nehme ich erstmal die fünf Hauptregistrare für IP-Adressen.
Für ASIEN inkl. Australien -> http://wq.apnic.net/apnic-bin/whois.pl
Für Nordamerika und Karibik -> http://whois.arin.net/ui
Für Lateinamerika -> http://lacnic.net/cgi-bin/lacnic/whois?lg=ES
Für Afrika (mit einigen Ausnahmen) -> http://www.afrinic.net
und
für Europa und teile Nordafrikas, Vorderasien und Russland -> http://apps.db.ripe.net/search/query.html

Ich schaue dann immer, ob es ein ISP ist, der Privatkunden Internetzugängen anbietet (z. B. Deutsche Telekom, O2/Alice etc) oder ein Hostinganbieter (z. B. HOSTEUROPE, ODN, HETZNER usw.). Letztere kommen bei mir nicht rein. Warum auch? Was will eine Serveranwendung in meinen Foren?

Im weiteren schaue ich halt nach dem Land. Draußen bleiben bei mir grundsätzlich China und Iran. Ohne wenn und aber. Da ist meine .htaccess aber noch nicht komplett!

Bei allen anderen trage ich die IP-Adressen bzw. -Bereiche erstmal in die foreneigene Bannliste ein. Vorteil: Über das Protokoll kann ich dann feststellen, ob der immer wieder versucht zuzugreifen. Dann geht er nach einer gewissen Zeit bzw. einer gewissen Anzahl an Zugriffsversuchen direkt in die .htaccess. Wichtiger Indikator für einen Bot: Die Schlagzahl mit der auf einzelne Bereiche des Forums zugegriffen wird. Steht im Protokoll jede Sekunde ein Eintrag ist es definitiv ein Bot. Ab auf die .htaccess.

Klar, das ist viel Arbeit. Aber mit der Zeit rechnet sich das.

Einen Nachteil kann das ganze allerdings haben: Da IP-Adressen nicht für ewig und drei Tage zugewiesen sind sondern irgendwann frei und dann von einem anderen Unternehmen genutzt werden würde ich Dir empfehlen, eigene Fehlerseiten (s. o.) zu coden und darin zumindest die Möglichkeit der Kontaktaufnahme anzubieten. So konnte ich zum Beispiel einen für 1&1 neu zugewiesenen Block für Privatkunden schnell aus der .htaccess entfernen, nachdem ich bei der RIPE nachgeforscht habe. Dazu würde ich dann auch ein kleines Tool einbauen, das dem Besucher seine eigene IP anzeigt. Beispiel:

http://www.einsatztreff.de/error403.htm

Huch, jetzt habe ich extrem viel getippt....  :???:

Noch Fragen? ;)

Grüße

Thomas

Ich bin nicht die Signatur - ich rocke hier nur....!

08. Juli 2013, 21:37:56
Antwort #6

Skippy


Danke für diese sehr ausführlichen Erklärungen!
Ich werde mich da nun erst mal durcharbeiten.
Falls mir dabei noch Fragen einfallen sollte, melde ich mich noch mal.
Danke auch schon mal im Voraus für den Link zur PDF.

09. Juli 2013, 02:43:51
Antwort #7

Mikado


Hallo,
das bedeutet aber auch, dass man ausgesperrt wird, wenn man sich zufällig nicht in Deutschland aufhält.  ;)
Ich bin zur Zeit in den Staaten und bin, wie ich eben festgestellt habe, auch ausgesperrt, weil ich zufällig in New York in einem Hotel sitze und die falsche IP-Nummer nutze.
Viele Grüße
Petra

09. Juli 2013, 08:13:32
Antwort #8

Amarok


Moin in die Runde.

Ja, die Gefahr besteht leider. Darum ja mein Tipp mit den eigenen Fehlerseiten, speziell der Fehlerseite 403 (auf die läuft man nämlich, wenn man durch die .htaccess geblockt wird), in die man die Kontaktmöglichkeit mit einbaut.

Gott sei Dank ist man da als Webmaster ja so flexibel, dass man die .htaccess den entsprechenden Veränderungen anpassen kann. Denn der Markt um die IP-Adressen der einzelnen Registrare ist in ständiger Bewegung und ich muss mich auch immer wieder stichprobenartig bei der RIPE umschauen. Irgendwas ist ja immer....
 ;D

@ Petra: Auf welche Seite hast Du denn versucht zuzugreifen?

Grüße

Thomas

Ich bin nicht die Signatur - ich rocke hier nur....!

09. Juli 2013, 16:25:54
Antwort #9

Mikado


Hallo Thomas,
es war nur reine Neugierde. Ich habe die Seite http://www.einsatztreff.de/ aufmachen wollen.
Übrigens wir sind gerade auf dem Weg nach Baltimore und haben im Zug (Amtrak) WIFI. Auch hier kann ich die Seite nicht lesen.  ;D

Viele Grüße
Petra

09. Juli 2013, 18:09:38
Antwort #10

Amarok


Hallo, Petra.

Yo, falsches Land....  ;D

An das Forum kommen tatsächlich nur Deutsche Provider ran. Wurde Dir denn Deine IP-Adresse über meine eigene Fehlerseite angezeigt?

Grüße

Amarok

Ich bin nicht die Signatur - ich rocke hier nur....!

09. Juli 2013, 19:02:38
Antwort #11

TE

Gast
An das Forum kommen tatsächlich nur Deutsche Provider ran.
Nicht wirklich, ich nutze einen deutschen Provider und kriege ebenfalls deine Fehlerseite zu sehen ;) Ich hab'n Business-Anschluß von UnityMedia..
« Letzte Änderung: 09. Juli 2013, 19:04:10 von TE »

09. Juli 2013, 19:06:19
Antwort #12

Amarok


Joah,

Business- (Teil-)Anbieter bleiben auch draußen!

Konntest Du denn meine eigene  Fehlerseite mit Deiner IP-Adresse sehen?

73 & 55

Ich bin nicht die Signatur - ich rocke hier nur....!

10. Juli 2013, 21:40:13
Antwort #13

Mikado


Hallo Thomas,
bin zwar im Moment in Gaithersburg im Hotel, kann natürlich die Seite nicht sehen, aber meine IP auf der 403-Seite.
Sind denn Deine User niemals im Ausland, so dass Du sicher gehen kannst, dass immer alle User online gehen können? Bei mir wäre das echt ein Problem.  ;D
Viele Grüße
Petra

10. Juli 2013, 22:42:27
Antwort #14

Amarok


Yep!

Es gab nur eine Reise eines Mitglieds im Ausland, aber den IP-Bereich hatte ich damals noch nicht in der .htaccess drin. Erst nach der Rückkehr hab ich den dann eingepflegt.

Ist antürlich schwierig, zugegeben. Aber die wenigsten sind im AUsland und wenn dann nur zum Urlaubmachen. Und wer will im Urlaub schon ins Internet?
;D

Schönen Abend noch und beste Grüße!

Thomas

Ich bin nicht die Signatur - ich rocke hier nur....!

 

Internes

Nutzungsbedingungen Impressum

Wissenswertes

Hilfe Knowledge Base

Nützliches

Downloads Kleinanzeigen Socialmedia