Bot Protection
Keitaro automatically detects bots by the following attributes:
- IP in the bot database of Keitaro
- IP in the user's list
- UserAgent in the bot database of Keitaro
- Part of UserAgent in the user's list
- Spammers myip.ms in the black list (disabled by default)
- Empty UserAgent
You can change the settings at the page “Maintenance > Settings > Bots”.
How to protect a campaign from bots
You need to create a special stream in the campaign.
- Create a stream with the type “Forced” and add the filter “Bots” in the “IS” mode.
- Сhoose the necessary action. For example, “Show 404 not found”.
How to cloak a website with Keitaro
Read page cloaking a website.
What types of bots are there in the bot database?
- Bots of search engines: Google, Bing and others;
- Bots of advertising services: Google Adwords, Facebook;
- Bots of ad monitoring systems and disclosing of links.
How to update, and how often are the bot databases updated?
The bot databases are updated on the page “Maintenance > Geo DBs”. Updates are issued 2 times a month.
How to add custom bot list?
At the page “Maintenance > Settings > Bots”, press “Edit list” . You can write IPs and CIDR. Other formats are not supported.
Example:
78.12.31.1 92.12.1.0/24
Keitaro optimizes the list. It's okay when uploading 100.000, then you see 50.000 after saving.
How to add a large number of IPs to bot list?
1. Upload the file to the server. For example as file /var/www/keitaro/file.txt. 2. Run these commands
cd /var/www/keitaro php bin/cli.php bots:import file.txt
What is the signature of bots?
It is a part of UserAgent. For example, for Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
the signature Googlebot
.
How to add signature of bots
Open the page Settings > Bots. You can set the signatures separated by commas:
signature1, signature2, ...
How to detect bots through a provider
The provider database is not supplied with Keitaro due to limitations imposed by the IP2Location license agreement.
You can connect the database in two ways:
- Buy the license. Enter the license key at the page “Maintenance > Settings > Integration”.
- Download the GeoISP.dat file manually to the directory
/var/geoip/GeoISP/
.
After integrating the database, you can use the “ISP” filter in the streams.