Ansible tutorial: setting up a dedicated server - feature image

Ansible tutorial: setup of a dedicated server with Ansible

Hello, since this is my first article on HappyCoders.eu, I want to introduce myself briefly: My name is Sven Woltmann, I have been programming for 35 years (20 years professionally) and my main focus is on backend and microservice development with Java and the Spring framework. I have founded several startups and have been CTO at Fonpit AG, which operates the world’s largest multilingual Android community, AndroidPIT, for nine years. You can find out more about me and my main areas of expertise by clicking on my name.

Before I write about my main topics, in this first series of articles, I’ll show you how I set up the server on which my blog will run – starting with the installation of the operating system image, the SSH and firewall configuration, the installation of Docker, MySQL and WordPress, the setup of HAProxy and the setup of a free HTTPS certificate from Let’s Encrypt via Certbot.

I will do the whole thing completely with Ansible – infrastructure as code – after all, the blog is called “Happy Coders”! You can find the complete code in my GitLab repository: https://gitlab.com/SvenWoltmann/happycoders-tutorial-server-setup.

In addition to the article, there is a (German-only) video tutorial: Setup of a dedicated server with Ansible

I will divide the series into the following four parts:

  1. Setup of a dedicated server with Ansible
  2. Setup of user accounts, SSH and firewall with Ansible
  3. Setup of Docker, MySQL, and WordPress with Ansible
  4. Setup of HAProxy and an HTTPS certificate from Let’s Encrypt with Ansible

I plan – as soon as the blog is up and running – to write a second tutorial series about the development of microservices with Java, Spring Boot, Spring Data and logging via the ELK stack, which will be built using Jenkins and deployed in a Kubernetes cluster, where I will write the Jenkins job as code and, of course, install Kubernetes via Ansible again.

One more thing: Maybe you’re wondering why I use Windows. The answer is: I grew up with Windows. I also think Mac is chic, but for the same money, I get a much more powerful Windows PC/laptop. And I like to use Linux on the server, but I personally don’t like the desktop environments. And since there is the Linux subsystem for Windows, the detour via setting up a VM to be able to use, e.g. Ansible, is no longer necessary.

Let’s start with the first part, setting up the server.

My server

I use a dedicated server with a Xeon E3 third-generation CPU, 32 GB RAM and two enterprise HDDs from the Hetzner “Serverbörse” for 26 Euros a month. A VM at Amazon with four virtual cores, 32 GB RAM and 2 TB HDD memory starts at 250 Euros. At the German provider ProftBricks, you get four Intel cores, 32 GB RAM and 2 TB HDDs for 298.88 Euros. The CPU speed can’t be compared exactly without tests, but this quick check shows that for the dedicated server, I only pay a fraction of what a VM would cost. By the way, both cloud providers add traffic costs, while Hetzner includes unlimited traffic with 1 GBit/s bandwidth.

I start with the installation of the operating system image. The Hetzner server is currently running in the rescue system. For authentication, I deposited my public key with the order, so that – after adding the server’s IP address to my local hosts file – I can log in with ssh:

ssh [email protected]
Hetzner rescue system
Hetzner rescue system

Manually, you would now start the installation with installimage. However, the script also has an automatic mode – you can find out the parameters by entering installimage -h:

Hetzner installimage parameters
Hetzner installimage parameters

The command itself is located at root/.oldroot/nfs/install and the available images are at /root/.oldroot/nfs/images/. From there, I select the most recent Debian image, Debian-95-stretch-64-minimal.tar.gz. I want to create a RAID 1, make the first 512 MB an ext3 boot partition, and format the remaining space as an ext4 file system. The command would now look like this:

root/.oldroot/nfs/install/installimage \
-a \
-n happy1.happycoders.eu \
-b grub \
-r yes \
-l 1 \
-i /root/.oldroot/nfs/images/Debian-95-stretch-64-minimal.tar.gz \
-p /boot:ext3:512M,/:ext4:all \
-d sda,sdb

However, I don’t want to do this manually, but automatically from my computer via Ansible, so that I can also run it on other servers at any time with one click.

What is Ansible?

Ansible is an open-source tool for automating the configuration and administration of the IT infrastructure. In contrast to other tools such as Chef or Puppet, Ansible requires neither a server component nor a repository – that is costly and/or requires extensive maintenance – from which the server retrieves the configuration data. Ansible only needs an SSH connection to the server.

Ansible executes so-called “tasks”, where each task refers to a “module” that performs a specific task, such as installing a package, copying a file, or creating a user. You do not define exactly what the module should do, but what the desired target state is. The module then decides what it has to do to reach the desired target state.

Whenever I use an Ansible module for the first time, I will explain it briefly and link to the corresponding documentation on the Ansible website.

Installation of Ansible

The installation is described in detail in the Ansible Installation Guide.

Windows is not directly supported, but this is not a problem. The Windows Subsystem for Linux Installation Guide shows you how to run full-fledged Linux under Windows 10 – parallel to Windows and without the overhead of a VM. Depending on the Linux distribution you choose, follow the appropriate section in the Ansible Installation Guide. Since I personally use Debian on my servers, I also installed it on my Windows laptop:

Ansible under Windows 10
Ansible under Windows 10

Installing the image with Ansible

First, I create a new Ansible project. This is nothing more than an empty directory within my Git monorepo: /happycoders/git/main/ansible/.

First, Ansible needs a hosts file that lists the servers it should control. In my case, this is just one server: happy1.happycoders.eu. In the hosts file, hosts can also be grouped together to control multiple servers at the same time. For example, I could add a second server, happy2.happycoders.eu and combine both servers to a group, “production” or add test servers to a group, “test”. This would look like this:

[production]
happy1.happycoders.eu
happy2.happycoders.eu

[test]
test1.happycoders.eu
test2.happycoders.eu

Since I currently have only one server, my hosts file looks like this for now:

happy1.happycoders.eu

Ansible expects this file in /etc/ansible by default. To change this, a file ansible.cfg must be created in the project directory and the path of the hosts file must be specified in it. The ansible.cfg file looks like this:

[defaults]
inventory = hosts

Creating the Ansible playbook

Ansible actions (“tasks”) are implemented in so-called “roles”. Which of these roles should be executed on which servers is defined in “playbooks”. I’ll start with the playbook: I create a playbook file, install_image.yml in the project directory /happycoders/git/main/ansible/, with the following content:

---
- hosts: "{{ target }}"
gather_facts: false
remote_user: root
roles:
- install_image

Explanation of the parameters:

  • The hosts parameter specifies on which server the playbook should run. Since I want to be able to run this playbook on all future servers too, I use a variable, target. When executing the ansible-playbook command, I specify this variable on the command line (how exactly, you will see shortly).
  • With gather_facts: false, I tell Ansible not to collect data about the server. At the moment, it couldn’t anyway because to do this, Python would have to be installed on the server.
  • With remote_user I specify that Ansible should connect to the server as root user. There is no other user at this time.
  • Finally, I use the array roles to define which roles should be executed on the server. In this case, the role has the same name as the playbook.

Creating the Ansible role

In the second step, I create a role of the same name. Since roles can consist of more than one file, each role is located in a directory roles/<role name>/. So I create the directory roles/install_image/ under my project directory.

In addition to the aforementioned tasks, a role can contain files, templates, and variables, each of which is located in its own subdirectory. But I’ll show you later – for the install_image role, I only need tasks and for those, I create the directory roles/install_image/tasks/. In that directory, Ansible expects to find the actual task definitions in the file main.yml. In this file, I define the following five tasks, which are executed by Ansible in exactly this order:

  1. In the first step, I have to install Python to be able to perform the following steps at all.
  2. Then I check if the installimage script I want to use to install the operating system image exists at the expected location.
  3. If this is not the case, Ansible should cancel the playbook.
  4. Otherwise, I run the installimage script now.
  5. Finally, the server should be rebooted.

In Ansible code, it looks as follows:

Step 1 – Install Python:

- name: Install Python on the rescue image
raw: test -e /usr/bin/python || (apt -y update && apt install -y python-minimal)

In the first line, I define the name of the task. This name is displayed when the role is executed. In the second line, I say that the Ansible module raw should be executed, which executes the specified command on the server. There is actually a more convenient way to install a package, but this is only available once Python is installed. I will show you that method later.

Step 2 – Check if the script exists:

- name: Check if "installimage" script exists
stat:
path: /root/.oldroot/nfs/install/installimage
register: stat_result

To check if a file exists, you can use the Ansible module stat: the parameter path defines which file should be checked and the parameter register defines the name of a variable in which the result of the check is stored. You can see how this result is evaluated in the next step.

Step 3 – Cancel if the script does not exist:

- block:
- name: Abort when "installimage" script doesn't exist
debug:
msg: "installimage script not found; you have either already installed the image or Hetzner renamed the script. Aborting."
- meta: end_play
when: stat_result.stat.exists == false

I’ll explain this step from the bottom up: With the Ansible Conditional when, I define that I only want to execute something in a certain case, namely if the file does not exist (stat_result.stat.exists == false). If this is the case, I want to perform two actions: display an error message via the Ansible module debug and abort the playbook via the Ansible module meta. To make this possible, I combine these two actions in a “block”. The second action shows that a task does not always have to have a name. I could also define a name here, e.g. name: End the Playbook, but I don’t think that’s necessary here.

Step 4 – Run the script:

- name: Execute installimage
shell: "/root/.oldroot/nfs/install/installimage -a -n {{ inventory_hostname }} -b grub -r yes -l 1 -i /root/.oldroot/nfs/images/Debian-94-stretch-64-minimal.tar.gz -p /boot:ext3:512M,/:ext4:all -d sda,sdb -f yes"

The Ansible module shell allows me to execute the installimage script. Since I want to replace the variable inventory_hostname (hostname of the server), which is defined by the Ansible playbook, I have to enclose it in double curly brackets and put the complete command in quotation marks.

Step 5 – Reboot:

- name: Rebooting...
shell: reboot

This step is self-explanatory.

Extract variables

What I don’t like about the script is that a lot of it is hardcoded. Maybe Hetzner will change the path of the installimage script in the future; maybe I want to install another image in the future or create a RAID 5 on more than two disks.

To make this possible, I extract values that I might want to change in the future into a so-called host variable file. This must be in the directory host_vars and gets the name of the server. So I create the file host_vars/happy1.happycoders.eu and fill it as follows (the comment in the line “raid …” is not part of the variable value):

---
install_image:
path: /root/.oldroot/nfs/install/installimage
image: /root/.oldroot/nfs/images/Debian-95-stretch-64-minimal.tar.gz
raid: yes -l 1 # yes, level: 1
drives: sda,sdb

I now replace the values in the task definition with the corresponding variable names, which I enclose in double curly braces – as you have already seen with inventory_hostname. The file roles/install_image/tasks/main.yml is now complete and looks like this:

---
- name: Install Python on the rescue image
raw: test -e /usr/bin/python || (apt -y update && apt install -y python-minimal)

- name: Check if "installimage" script exists
stat:
path: "{{ install_image.path }}"
register: stat_result

- block:
- name: Abort when "installimage" script doesn't exist
debug:
msg: "installimage script not found; you have either already installed the image or Hetzner renamed the script. Aborting."
- meta: end_play
when: stat_result.stat.exists == false

- name: Execute installimage
shell: "{{ install_image.path }} -a -n {{ inventory_hostname }} -b grub -r {{ install_image.raid }} -i {{ install_image.image }} -p /boot:ext3:512M,/:ext4:all -d {{ install_image.drives }} -f yes"

- name: Rebooting...
shell: reboot

Execute the playbook

I am now ready to run the playbook. To do this, I change back to the project directory and execute the following command:

ansible-playbook --extra-vars "target=happy1.happycoders.eu" install_image.yml

With the parameter --extra-vars, I pass the variable target to the playbook, which I used in its first line.

Attention: In the Linux Subsystem for Windows 10, the drives are mounted by default in a way that every user has full read and write access to all directories and files. This causes Ansible to abort with the following warning:

Warning about write permissions when running Ansible under Windows
Warning about write permissions when running Ansible under Windows

To change this, the drive containing the Ansible directory must first be unmounted and then remounted with the metadata option enabled. For example, if it is the C: drive, you would do this as follows:

umount /mnt/c
mount -t drvfs C: /mnt/c -o noatime,uid=1000,gid=1000,metadata

More information can be found in the Microsoft blog article Chmod/Chown WSL Improvements. Next, you can remove the global write permissions from the project directory as follows:

chmod 755 /happycoders/git/main/ansible

Back to the ansible-playbook command. If everything works fine, you can now see the following output in the console:

Executing the install_image Playbook
Executing the install_image Playbook

During the reboot, the connection breaks up, that’s perfectly OK. After a short time, the server should be reachable again. Since the new image has changed the fingerprint of the server key, I have to remove the old key from my known_hosts file:

ssh-keygen -f "/home/sven/.ssh/known_hosts" -R happy1.happycoders.eu
ssh-keygen -f "/home/sven/.ssh/known_hosts" -R 46.4.99.9

Then I can log back in to the server and see that the new image has been successfully installed:

ssh [email protected]
cat /etc/debian_version
New image has successfully been installed
New image has successfully been installed

Python again

The installed Debian image does not have Python installed by default.

In my playbook for the further configuration of the server, I will not be able to do without the “facts” that Ansible collects from the server when starting a playbook (IP address, operating system version, etc.). However, Python must be installed for this to work.

To collect the “facts”, gather_facts must be set to true – for the installation of Python, however, it must be set to false. Consequently, the Python installation must be moved to a separate playbook, which must be executed before all future playbooks.

Therefore, I create the playbook file install_python.yml with the following content in my project directory:

---
- hosts: "{{ target }}"
gather_facts: false
remote_user: root
roles:
- install_python

The content is almost the same as in the install_image.yml playbook with the only difference being that the install_python role is referenced here. To explain the playbook, I refer back to the description of the install_image.yml playbook.

For the new role, I create the file roles/install_python/tasks/main.yml:

---
- name: Install Python
raw: test -e /usr/bin/python || (apt -y update && apt install -y python-minimal)

This is equal to the first task of the install_image role. Since we software developers don’t like to see duplicated code, I can now remove the first task from the install_image role and put the install_python role in front of the install_image role in the install_image.yml playbook instead:

  roles:
- install_python
- install_image

I am running the playbook as follows:

ansible-playbook --extra-vars "target=happy1.happycoders.eu" install_python.yml

And I get this output:

Installation of Python with Ansible
Installation of Python with Ansible

Summary and outlook

In this article, I explained what Ansible is and I showed you how to use Ansible to automatically install the operating system image on a dedicated server.

Next, I want to install all available patches, set the hostname and create a hosts file, create another user so I don’t have to log in as root, optimize the SSH configuration, create Bash aliases, install some helpful command-line tools, and configure the firewall.

How this works is shown in the second part of the series: Setup of user accounts, SSH and firewall with Ansible.

I hope you enjoyed this first part of the series and I thank you for staying with me until the end. I would be happy if you’d leave a comment and let me know how you liked the tutorial, what I could have done better and also what topics you are generally interested in. Thanks a lot and happy coding!

If you liked the article and you know someone for whom it could be interesting, then I would be happy if you share it via one of the following share buttons.

Leave a Comment

Your email address will not be published. Required fields are marked *