What is the length of the password? e.g. in “mypassword” the answer is 10. The longer a password, the better it is because it would take longer to break a password of length 15 than a password of length 5.

How many different characters did they use? e.g. in “mypassword” the answer is 9. The more characters you can throw into your password, the bigger the domain of possibilities for someone to break your password.

How many times did they use the same character in sequence? e.g. in “mypassword” the answer is 1 because only ‘ss’ was used. If you use the characters in sequence many times, it makes your password weaker. So the less sequences there are in your password, the stronger it is.

What characters did they use in sequence? e.g. in “mypassword” the answer is ‘s’. This question could be helpful to choose a better character to use in sequence (if you really need to use a sequence).

How many characters did they use in sequence? e.g. in “mypassword” the answer is 1 because only ‘s’ was used. So you have sequences in your password. But if the same character is used in all sequences, it makes your password weaker. Similarly, if there are sequences but each sequence has a different character, it’s still better than having the same character used in all sequences.

What’s the ratio of non-sequence characters to sequence characters? This ratio should be high for a more secure password.

What percentage of the password were the sequence characters? The higher the percentage, the weaker the password.

How many times did you alternate between alphabetical characters, numerical characters, and punctuation/other characters? e.g. in “mypassword” the answer is zero because only alphabetical characters were used. e.g. in “myp@ssw0rd” the answer is 4 because “myp@” is one alternation, “@ssw” is second, “ssw0” is third, “0rd” is fourth.

How many times did you use upper and lower case characters (if applicable)? The more variety in characters of upper- and lower-case the more secure the password.

What’s the distribution of upper and lower case characters? e.g. in “myPassWOrd”, there are 7 lower case and 3 upper case characters. Should the distribution be closer to 50% for each?

How many different languages were used for the alphabetical characters? e.g. in “mypassword” only one language (English) was used. If your application allows Unicode characters, use a variety of characters from different languages.

These were a few questions that came to my mind. Do you have any to add?

## We are under attack

The heading may be alarmist but there’s nothing to be alarmed about. I have been collecting IPs and user names for SSH attacks over the past few weeks. Initially I did not use throttling so I got a whole bunch of names. Recently I have implemented iptables rules so that if a user enters wrong password three times in one minute when attempting to connect via SSH, further connections are blocked for a few minutes. This change means that usually I only see three attempts in logs and then nothing more. Unless, of course, the attacker is smart and re-tries after waiting a few minutes. There have been a handful of such smart attackers. Mostly I see just three attempts from an IP and then nothing more.

My interest lies in two statistics: (1) which user names are tried and how many times; and (2) which geographic locations are used for these attacks. I have compiled a list of unique user names with the counts of attempts. The top ten names and their counts are as below.

347 root
39 user
30 nagios
23 test
18 oracle
17 webmaster
14 teste
13 stud
12 web

Now comes the geographic locations of attackers. The top ten countries, after doing whois \$IP, were as below.

73 CN
10 US
8 TW
6 IN
6 BR
4 DE
4 CL
3 RU
3 PK
3 PH

In these numbers, China is the location from which most attacks originated. The first lesson learned here is if you don’t live in China, you can try to block all SSH traffic coming from there. Similarly, if you don’t live in Taiwan, India, Brazil, and so on, you can cut down on the number of attacks by blocking SSH access from those countries. A helpful resource is Country IP Blocks if you want to pursue this route. Of course, you could do the opposite and only allow traffic from countries from where you expect to access your machine. This means that you can still be targeted but by machines with IPs of these countries only.

In conclusion, this has been a fun exercise for me. There have been lessons learned which can help secure my machine against malicious would-be users. But more than that I have learned how to use my Linux machine to gather this data. You should give it a try sometime.

So you have your SSH server on Linux up and running and you are able to login easily. But you may still be vulnerable to attacks on your SSH server, primary being brute force dictionary attacks. This post will point out some areas of vulnerabilities and also advise some solutions.

### Only Allow Certain Users to Login via SSH

Your Linux machine has many users built into it. But only some of them need to have access to the server via SSH. If we can lock down a list of such users, we can reduce the number of users an attacker can pretend to be. For example, there are only three users on my machine who need to login. The good thing is that SSH allows you to setup either a list of users or a group of users who are allowed access.

I prefer the group method. The idea is to create a new Linux group and add users to it. If a user is a member of this group then he/she can login via SSH. If they are not a member they cannot. Then in the sshd config file we can specify this group as being the only one with access. Here’s how we do it.

Create a group

Add an existing user to this group. Thanks to Howto: Linux Add User To Group.
sudo usermod -a -G mysshgroup codeghar

Allow access only to this group in /etc/ssh/sshd_config file by making sure the following line exists in it.
AllowGroups mysshgroup

Everyone who knows Linux knows there’s always a root user. So all the attackers out there know they just need to break its password to gain (complete) access to the system. What if we disallow root from logging in directly? This cuts down on a huge threat to your system.

Disable root login in /etc/ssh/sshd_config file by making sure the following line exists in it.

Now that we have disallowed all users except those in the mysshgroup group to login, we have to make sure these users do not use empty passwords. If we don’t restrict this, a user, say codeghar, might have an empty password and an attacker can login very easily. We don’t want that.

Disable empty passwords in /etc/ssh/sshd_config file by making sure the following line exists in it.

### Allow Public Key Authentication Only

This means that we disable password-based authentication and only use public key authentication. This completely eliminates dictionary attacks because we just don’t use passwords. This requires a few extra steps, as explained below. Thanks to Setting up public key authentication over SSH for this part of the post.

On your client machine, you need to generate a new public and private key pair using RSA as encryption type and key strength as 4096 bits. You will be asked for a password. You can leave it blank but I recommend you put something in there. The reason is that if your private key gets into the wrong hands, they still need a password to be able to use it.
ssh-keygen -t rsa -b 4096

Now that a public/private key pair has been generated, you can use the keys. You first need to copy the public key from your client machine to your ssh server machine. You copy it to the user’s home directory on the server in the .ssh directory. For example, on the server you would need to copy it to /home/codeghar/.ssh/ directory. If it doesn’t exist, create it.
scp /home/codeghar/.ssh/id_rsa.pub codeghar@example.com:/home/codeghar/.ssh/myclient.pub

On the remote server, do the following so that the public key is appended to the authorized keys file
cat /home/codeghar/.ssh/myclient.pub >> /home/codeghar/.ssh/authorized_keys

Now you can remove the original file because it’s no longer needed.
rm /home/codeghar/.ssh/myclient.pub

This has to be done for each client and also for each allowed user on the server. For example, if user codeghar uses two client machines, he/she needs to generate these pairs on each machine and then copy over the public key to the server. And if the server has two allowed users, both need to have these public keys in their own authorized keys files for them to be able to login.

You have copied over the public key but you still need to tell ssh to only use this key for authentication. Disable password authentication and enable public key authentication in /etc/ssh/sshd_config file by making sure the following lines exist in it.
RSAAuthentication yes
PubkeyAuthentication yes

You are now ready to activate these changes. Be aware that if you activate these changes and you made a mistake somewhere, you may lose all access. For this reason, establish an ssh connection first for the ‘just in case’ scenario and do not logout unless you are satisfied all things are working as they should.

To activate the changes, run the following command:
sudo /etc/init.d/ssh restart

Test the changes you have made so far by trying to login using a user who is not in the mysshgroup group, by logging in as root, and by using password authentication. All should work well.

If you are still asked for a password, or you get a Permission denied (publickey) error, then the ssh daemon/process thinks the settings of .ssh directory and/or the authorized_keys file on the server are insecure. To remedy the situation, you should change the permissions as below (hat tip: Public and Private Keys):

chmod 700 /home/codeghar/.ssh
chmod 600 /home/codeghar/.ssh/authorized_keys

### Throttle ssh Connections

Although you have taken the aforementioned steps to make your server more secure, attackers are still going to attempt. With the above changes you should be secure but I prefer to add another layer: iptables firewalling. The idea is to restrict three attempts per minute for a new ssh connection from a single IP. This way you still have the ability to try three times a minute in case you provide wrong credentials. But it also means an attacker is restricted to three attempts per minute per IP. It doesn’t completely eliminate brute force attacks but it ought to slow these down.

You need the following three rules. Thanks to Using iptables to rate-limit incoming connections for helping with this section.
sudo iptables --append INPUT --protocol tcp --match tcp --destination-port 22 --in-interface eth0 --match state --state NEW --match recent --set
sudo iptables --append INPUT --protocol tcp --match tcp --destination-port 22 --in-interface eth0 --match state --state NEW --match recent --update --seconds 60 --hitcount 4 -j DROP
sudo iptables --append INPUT --protocol tcp --match tcp --destination-port 22 --jump ACCEPT

The first rule adds the IP of the client machine which initiates a new ssh connection. The second rule checks if this IP has attempted four or more connections in the past 60 seconds. If it has, this attempt is dropped. If it hasn’t, then it is allowed to ssh using the third rule. This way you get to try three times per minute but the fourth and subsequent attempts are blocked for a whole minute.

## Sample Rules for iptables

I have been trying to find a perfect beginner’s script for iptables. You know, something to get you started as you customize rules for your own system. I settled on something as below. What I have tried to do is use the longer version of flags to make things more understandable for beginners. I have also tried to provide comments to try to explain the “why” since the rules themselves provide the “how”. I assume that we only have one interface, say eth0, so I omit mentioning the interface, except in a few places.

# Since these rules are for the Filter queue type, we add the following
*filter
# I assume (correct me if I am wrong) that the following means
# by default accept everything in the Input, Forward, and Output chains,
# then match the rules that follow
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
# We accept everything from the local interface, lo, because
# we want it to work at all times without restrictions
--append INPUT --in-interface lo --jump ACCEPT
# By default, if a connection is already established,
# or a new but similar to an established connection is attempted, we allow it.
# Again, I might have misunderstood the concept of Related state.
--append INPUT --match state --state RELATED,ESTABLISHED --jump ACCEPT
# No source and destination ports may be 0 (I assume). If it is so, we drop it
--append INPUT --protocol tcp --match tcp --destination-port 0 --jump DROP
--append INPUT --protocol udp --match udp --destination-port 0 --jump DROP
--append INPUT --protocol tcp --match tcp --source-port 0 --jump DROP
--append INPUT --protocol udp --match udp --source-port 0 --jump DROP
#
# This is where you might want to put your customized rules
# <Start Customization>
#
# We want to be able to accept SSH connections from any IP.
# To secure SSH, we can do it within the sshd_config file.
# Of course, if you expect SSH connections from particular IPs only,
# you can restrict here as the first line of defense.
--append INPUT --protocol tcp --match tcp --destination-port 22 --jump ACCEPT
# Personally I don't like everyone to be allowed to ping my servers.
# But certain situations, such as you being on the road a lot of the time,
# may warrant allowing ping from anywhere, as we are doing below
--append INPUT --protocol icmp --match icmp --icmp-type 8 --jump ACCEPT
# If your server runs a SIP application, you may want to allow traffic on port 5060
--append INPUT --protocol udp --match udp --destination-port 5060 --jump ACCEPT
#
#
# <End Customization>
#
#
# All traffic which does not match the rules above should be dropped by default
--append INPUT --jump DROP
--append FORWARD --jump DROP
# Allow all traffic going to the outside world
# because we do not want to block anything in that direction
--append OUTPUT --jump ACCEPT
# We save the rules
COMMIT

I could not have shared these rules with you without help from Linux Firewalls Using iptables, Netfilter IPTables Mini Howto, Sample iptables ruleset, and Debian wiki on iptables.

## Choosing a Linux Distribution

Recently I have had more time to work with Linux. I had been using Ubuntu in some way for two years when I needed to set up Linux on a few years old server. Since I was comfortable with Ubuntu, I thought I might as well go ahead and use it. But then I found out that there were other alternatives as well. This caused a headache which still isn’t resolved to this day. Which distribution is the best to get hands-on, real world experience with?

### Comfort

You have to look at your comfort level when choosing a distribution. If you are familiar with something, even in passing, it would be an easier path to go with what you know. On the other hand, all distributions may be different but they have more in common than there are differences. So learning another distribution style is not as difficult as one might expect.

### Hardware Support

If the distribution you choose is not able to function on the hardware you have available, you should not choose it. If you can get it to work, with or without a lot of effort, all the power to you. If, however, you can’t get it to work, you might as well look for another option. I went ahead with Ubuntu on the server because it supported all its hardware out of the box. I did not have to tweak anything or waste a lot of time. On the same server I was unable to install CentOS because Red Hat had dropped support for server’s RAID card in its current distribution.

### Purpose

For what purpose are you using a distribution? Is it going to be for starting out, testing, development, or deployment? For all these scenarios, there are many distributions fitting them just fine. For starting out, a friendly distribution like Ubuntu could work. If you are testing Linux for its feasibility in your environment, just about any distribution would work. A distribution for doing development work should be fast moving with new technology so that you can use it to its fullest extent. If it’s for production deployment, being conservative in your selection is recommended.

### Cutting Edge Technology

Some distributions strive to be on the cutting-edge. I count Fedora, openSUSE, and Ubuntu in this category. They release new stuff every few months. So you get to work with what’s new. For example, on Ubuntu, I found Django packages ready to install and work. Since I wanted a package and I found it, I was able to start working. I did not have to jump through hoops just to get to the point where I would be able to work.

### Enterprise

Yes, an enterprise version would be more stable and maybe more secure. But it is also less likely to include new technology in an easily accessible format. Taking the example of Django, I have not found any tutorial on the web to install it on CentOS using an RPM package. All tutorials I have read ask you to download and install from source. Yes, it’s the traditional way to do things but if package management is the future, we should look for packages first and source code later. Now if I am developing and deploying an application developed with Django, I want to have the peace of mind that I installed a package that had been tested to work well with the whole operating system, and not something I installed without knowing how it would turn out.

To me this is the most important point after hardware support. I am willing to learn a whole another distribution if it is enterprise level with great hardware support but also keeps up with new technology. Since not one distribution will always fill these requirements, we have to look at the best tool for the job at hand.

### Security

I was shocked to learn a few days ago that Ubuntu server’s default firewall policy was to accept all traffic. CentOS, on the other hand, has a pretty aggressive firewall policy. Combined with recent scandal of Debian and OpenSSL, it has dented my confidence in Ubuntu. It’s not that Ubuntu is insecure, it’s just the appearance of security in the ecosystem is absent (to me, at least). It’s also not that these things cannot be rectified by me, it’s that why would I need to take an extra step when a prudent decision could do it for me in the first place.

Another aspect I look to is being root. Does one have to actually be root or would sudo do? I like the sudo model better since it forces you to actually type your permission when doing critical work. Yes, if you are careful su and su - would work as well as sudo. But I like the added carefulness of sudo. So the first thing I do after installing a distribution is to see if it has sudo and then enable it for at least one user.

### Support

Support is a very important part of decision-making process. Support may be of three kinds: distribution creator, third-party professional, and community and friends. Support includes help as well as software updates. One can get help from many sources, and community is an essential part of this support ecosystem. It can get you started and get you out of trouble. Almost all (ok, maybe all) distributions provide software updates. Then there is an extra level of support which we know as enterprise or corporate support (think Red Hat). It is provided by either the creators and maintainer of the distribution or from third-party entities.

For a home user, software updates and community support should be sufficient. For a business, however, ‘corporate’ support is essential on production systems. Businesses like to pay someone to get extra insurance in case it is needed. If a server is essential to business operations, it is very important that the team running the server knows what it is doing, has community support for minor issues, and corporate support when things go really bad.

Red Hat, Novell, and Canonical provide this kind of support as they create their distributions. Of course, if you have a good team running your servers, you may not need to get corporate support. But if your manager is a non-technical person, she will most probably require it. And if it’s not your money being spent, why argue?

### Conclusion

This was meant to be a discussion of factors I would look into when choosing a distribution. Nothing more, nothing less.

Disclaimer: I have edited, and will edit, this post as new arguments come up.

## iptables Introduction

I have always wanted to learn how to write iptables rules for Linux. In my quest, I have used these resources to teach me what little I have learned so far: Hardening Linux; Iptables Tutorial; Ubuntu Setup;. This is an introduction to iptables.

First, we need to learn how to write an iptables rule. The general format would be

iptables table command chain match target

There are three main tables: filter, nat, and mangle. filter is considered to be default if you do not specify a table. So your first rule would start to look like

iptables -t filter

Commands include, but are not limited to, append, insert, delete, and replace. Let’s say we are adding a new rule so our command now looks like

iptables -t filter -A

There are three main chains: input, output, and forward. Input deals with all traffic incoming to the server, output is traffic generated by server, and forward is traffic not for the server but for some other machine. Let’s say we need to deal with incoming traffic. Now our command looks like

iptables -t filter -A INPUT

Matching is the heart and soul of the rule. The most common things in matching are interface, source IP address, source port, destination IP address, destination port, and protocol. Let’s say our example deals with incoming interface eth0, for HTTP from any computer. Our command may look like

iptables -t filter -A INPUT -i eth0 -dport 80 -p tcp

Since any computer may connect, we have left out source IP and source ports.

Last part is target. Most common targets are accept, reject, and drop. Since we are looking to accept HTTP traffic in our source example, we will use accept. Now our command looks like

iptables -t filter -A INPUT -i eth0 -dport 80 -p tcp -j ACCEPT

We have created our first iptables rule. It will accept all incoming web traffic on port 80. See, it isn’t too hard to get started with iptables.

### Location of iptables Rules in CentOS

CentOS stores its rules in /etc/sysconfig/iptables

### Location of iptables Rules in Ubuntu

By default, Ubuntu has a policy of accepting all incoming traffic. Therefore, there are no default iptable rules. However, if you want to create your own, then put them in a file and modify /etc/network/interfaces by adding the following line:

pre-up iptables-restore < /etc/iptables.up.rules

where iptables.up.rules is the file where all rules were stored.