Crowdsec: secure your web server network – installation and configuration

In this tutorial, I will explain to you how to secure the network part of your web servers with Crowdsec.

The tutorial is made on Ubuntu 20.04

Crowdsec overview

Crowdsec is an open source tool that allows you to “secure” or rather add a layer of security to your Linux servers (soon Windows too) by detecting network attacks (port scanning, SSH brute force, web content scanning … ) and ban IP addresses.

If you know fail2ban, Crowdsec does the same thing better:

• Consumes very few resources
• Pooling of IP addresses with a database shared between all Crowndsec users
• Easy to use

To facilitate the exchange of configuration entered by users, there is a hub (a kind of marketplace) where you will find configurations to secure your server.

Prerequisites in order to be able to use Crowdsec

One of the prerequisites for using Crowdsec is to have an active firewall (IPTABLE) on your server.

If there is no firewall, Crowdsec will not be able to add a rule to block the traffic.

Before activating your firewall, you must ensure that incoming traffic 80, 443 and 22 are authorized.

Start by checking if the firewall is active with the command below:

sudo ufw status

If this is disabled, we will start by adding the basic rules for incoming traffic: 22, 80 and 443:

sudo ufw allow 22
sudo ufw allow 80
sudo ufw allow 443

For each rule (command), a confirmation message is displayed indicating that the rule is added.

Now that the rules are created, activate ufw:

sudo ufw enable

The command requires confirmation

Now that ufw is enabled, enter the command below to display the status and list of rules:

sudo ufw status verbose

One last thing to do before installing Crowdsec is to activate the firewall logs, because they will be analyzed in order to detect attacks.

Enter the command below to activate the logs:

sudo ufw logging on

The command returns : Logging enabled.

The logs are saved in the file /var/log/syslog.

To ensure that the logs are correctly recorded, from another server, we can launch a telnet on an unopened port:

telnet ip 514

Our server is ready, we are going to install Crowdsec.

Crowdsec installation

The installation of Crowdsec is very simple, during the installation, it will also detect the different components (Nginx, Apache2, MySQL …) that it can monitor and configure itself “automatically”.

1. The installation of Crowdsec is very simple, during the installation, it will also detect the different components (Nginx, Apache2, MySQL …) that it can monitor and configure itself “automatically”.

curl -s https://packagecloud.io/install/repositories/crowdsec/crowdsec/script.deb.sh | sudo bash

2. Update the list of packages:

sudo apt update

3. Install Crowdsec:

sudo apt install crowdsec

On the screenshot below, the installation process where we can see the discovery and installation and configuration automatically for sshd and nginx.

Crowdsec is now installed and “functional”.

Before we move on to configuring and installing a Bouncer, I’ll show you how to interact with the cscli client.

Using the cscli client – Crowdsec

A command line client is available for Crowdsec in order to be able to configure and supervise it.

To display the list of available commands use the command:

cscli

The metric command displays various statistics related to the crowdsec service:

cscli metrics

As we can see on the capture, we can see the source files used and their statistics and also the different parsers used which make it possible to detect

To display the logs that have matched a rule:

cscli alerts list

To get the details of an alert:

cscli alerts inspect ID

To display the list of bans (blocking):

cscli decisions list

At the moment we still have no Bouncer installed, if use the command yet you will get the following message: No active decisions.

When we have installed a bouncer, I will explain how to ban a manual ip address and how to remove it.

Display the list of parsers:

cscli parsers list

Reminder: the parsers are the configurations that will allow you to read and interpret the logs.

Display the list of scenarios:

cscli scenarios list

Reminder: the scenarios are the configurations which will detect the behaviors and the actions to be taken.

The last element that we can have is also the postoverflows

cscli postoverflows list

postoverflows are configuration files which contain whitelists to avoid blocking of certain services like CloudFlare, indexing bots …

Install a bouncer

Now, we will see how to install a Bouncer, which will allow us to block attacks.

The list of Bouncers is available on the Hub Crowdsec. There are several blockers available that you can put:

• Firewall / iptable, which allows you to block the server’s firewall
• WordPress using a plugin will prevent site accessNginx
• Cloudflare
• …

In the tutorial, I will explain how to set up the bouncer : cs-firewall-bouncer.

Go to the page at the Crowdsec or Github hub:

• CrowdSec Hub
• crowdsecurity/cs-firewall-bouncer: Crowdsec bouncer written in golang for firewalls (github.com)

The following instructions are valid at the time of writing this tutorial, the bouncer may have changed.

1. Download the sources:

wget https://github.com/crowdsecurity/cs-firewall-bouncer/releases/download/v0.0.12/cs-firewall-bouncer.tgz

2. Unzip the archive:

tar xzvf cs-firewall-bouncer.tgz

3. Go to the compressed folder (adapt to the version of the archive):

cd cs-firewall-bouncer-vX.Y.ZZ/

4. Start the installation script:

sudo ./install.sh

During installation, you will be prompted to install ipset if it is not present.

The configuration of the Bouncer can be found in the file : /etc/crowdsec/cs-firewall-bouncer/cs-firewall-bouncer.yaml.

View the file and verify that it is in iptables mode:

cat /etc/crowdsec/cs-firewall-bouncer/cs-firewall-bouncer.yaml

Use the command below to see the Bouncer:

sudo cscli bouncers list

Check the status of the Bouncer:

sudo systemctl status cs-firewall-bouncer

From now on, attacks will be blocked by the Linux firewall depending on the scenarios.

Install a component with cscli and the hub

In this part, I will explain to you how to install a component of the hub using the client (cscli).

As I installed Crowdsec on a Web server, we will install the white list for the crawlers.

On the configuration page CrowdSec Hub we find the line to execute.

sudo cscli postoverflows install crowdsecurity/seo-bots-whitelist

If you do not use the root account, you must add sudo in front of

Reload the configuration to take it into account:

sudo systemctl reload crowdsec

You now know how to set up a configuration with the client.

There are other commands that allow updating and uninstalling.

Update :

cscli hub update

Uninstall:

cscli {composant} remove {nom_du/composant}

Act on decisions

It is possible to act on the decisions manually to ban an IP address or to disarm.

Ban an IP address :

sudo cscli decisions add --ip 172.69.32.87

The IP address 172.69.32.87 will be blocked for a period of 4 hours.

Ban a 24-hour IP address:

sudo cscli decisions add --ip 172.69.32.87 --duration 24h

Ban an IP address range:

sudo cscli decesions add --range 172.69.0.0/16

To delete a ban, replace add by delete:

sudo cscli decisions delete --ip 172.69.32.87
sudo cscli decesions delete --range 172.69.0.0/16

To conclude

Through this tutorial, you now have all the information you need to install, configure and administer Crowdsec in order to secure your Linux server.

Personally, since I discovered this software, I install it instead of fail2ban, because I really like its community aspect and its ease of use.

More Crowdsec tutorials will follow where I will explain how to add file sources and how to create a scenario.

Finally, know that there is a dashboard available based on a Docker image and that it is possible to use a MySQL / PostgreSQL database to pool decisions to enter several of your servers.