Configuring systemd services

Introduction

On modern Linux OS distributions, systemd is the default init system, replacing SysV. The following tutorial aims to show some methods to manage configuration of systemd-based services. Newer packages supplied by Varnish Software have moved away from external files containing startup parameters to the systemd best practice of keeping the parameters in the system .service file. The guide shows how to manage the Varnish systemd service including how to configure startup parameters. Even though the Varnish service is the focus of this guide, the methods are applicable to any other similar service.

Prerequisites

You’ll need:

• A Linux-based server that uses systemd as its init system with either a privileged account or with sudo capabilities
• Varnish Enterprise or Varnish Cache installed (you can use your distribution-supplied packages or head over to the official site for installation instructions)
• Basic understanding of command line usage

Verify systemd status of your distro

See the table below to verify that your distribution is systemd based:

Basic terminology

Some useful terms to know when coming fresh to systemd from a SysV background:

• A systemd unit is any system resource systemd can manage, including, but not limited to service, socket, device and target.
• A unit file is a configuration file that encodes information about the unit required for systemd to manage that resource.
• A systemd target is the concept systemd introduces to handle boot ordering and event synchronization. Where SysV used runlevels, systemd has the more flexible targets that roughly describe various states and events. systemd provides a number of predefined such targets that are useful when working with service type units.
• service in systemd is a unit that takes care of running and maintaining a process or a group of processes. A service unit file is a highly standardized and structured configuration file in contrast to SysV init scripts that are (shell-)scripts with some standard headers bolted on top. In addition to starting and stopping services, systemd can also be asked to take action if a service fails.

Managing service status

systemd provides the systemctl command, which allows us to manipulate services and service status. Before going on to describe configuration of the service, you will learn how to check and manipulate the status of the service.

Check service status#

To see the current status of the varnish.service, type the following command on your command line:

sudo systemctl status varnish

Provided your service is properly installed, you should see something like this:

● varnish.service - Varnish Enterprise, a high-performance HTTP accelerator
   Loaded: loaded (/usr/lib/systemd/system/varnish.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2018-02-27 12:27:05 UTC; 1s ago
  Process: 12672 ExecStart=/usr/sbin/varnishd -P /var/run/varnish.pid -f $VARNISH_VCL_CONF -a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} -t $VARNISH_TTL -S $VARNISH_SECRET_FILE -s $VARNISH_STORAGE $DAEMON_OPTS (code=exited, status=0/SUCCESS)
 Main PID: 12673 (varnishd)
   CGroup: /system.slice/varnish.service
           ├─12673 /usr/sbin/varnishd -P /var/run/varnish.pid -f /etc/varnish/default.vcl -a :6081 -T 127.0.0.1:6082 -t 120 -S /etc/varnish/secret -s ma...
           └─12683 /usr/sbin/varnishd -P /var/run/varnish.pid -f /etc/varnish/default.vcl -a :6081 -T 127.0.0.1:6082 -t 120 -S /etc/varnish/secret -s ma...

Feb 27 12:27:04 centos-7-amd64.local systemd[1]: Starting Varnish Enterprise, a high-performance HTTP accelerator...
Feb 27 12:27:05 centos-7-amd64.local varnishd[12673]: Platform: Linux,3.10.0-693.11.1.el7.x86_64,x86_64,-junix,-smalloc,-smalloc,-hcritbit
Feb 27 12:27:05 centos-7-amd64.local varnishd[12672]: Debug: Platform: Linux,3.10.0-693.11.1.el7.x86_64,x86_64,-junix,-smalloc,-smalloc,-hcritbit
Feb 27 12:27:05 centos-7-amd64.local varnishd[12673]: Child (12683) Started
Feb 27 12:27:05 centos-7-amd64.local varnishd[12672]: Debug: Child (12683) Started
Feb 27 12:27:05 centos-7-amd64.local varnishd[12673]: Child (12683) said Child starts
Feb 27 12:27:05 centos-7-amd64.local systemd[1]: Started Varnish Enterprise, a high-performance HTTP accelerator.

The output will vary between versions of systemd and versions of the Varnish package you are using.

Enable service#

If the service is not yet enabled, you might get output like this when running systemctl status:

● varnish.service - Varnish Enterprise, a high-performance HTTP accelerator
   Loaded: loaded (/usr/lib/systemd/system/varnish.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

Note the varnish.service; disabled part, which says that this service is not enabled, and will thus not automatically start on system startup. In order to properly enable the service, and ensure automatic startup, run the following command:

sudo systemctl enable varnish

This will set up the required symlinks that systemd uses to manage enablement, and reload systemd configuration. It will not start the service, as enabling and starting services is orthogonal: services may be enabled without being started, and started without being enabled. After you have successfully enabled the service, you can now check the status again, and it should look like this:

sudo systemctl status varnish

● varnish.service - Varnish Enterprise, a high-performance HTTP accelerator
   Loaded: loaded (/usr/lib/systemd/system/varnish.service; enabled; vendor preset: disabled)
   Active: inactive (dead)

If you wanted to enable and start the service in one operation, use the --now option to systemctl enable:

sudo systemctl enable varnish --now

This will enable and start the Varnish service right away.

Disable service#

To disable a service, use the disable command with systemctl like so:

sudo systemctl disable varnish

This will remove the symlinks, preventing the service from automated startup. If the service is running, it will not be stopped.

If you want to disable and stop the service in one operation, use the --now option to systemctl disable:

sudo systemctl disable varnish --now

This will disable and stop the Varnish service right away.

Unit file configuration

Unit (configuration) files in systemd are the files that encode information about the various units (including services, sockets, devices and more) that systemd can manage. This guide focuses on services, and in this context, the unit file we operate on is the .service file. The varnish.service unit configuration file contains information about how systemd should execute, monitor and manage the Varnish daemon.

Unit file sections#

There are some predefined sections in the unit file, specifically [Unit], which has metadata about the service, including a description, and possibly ordering information, instructing systemd to start the unit after or before other units.

The [Install] section contains information on how systemd should set up the service when performing the enable action. By default, the varnish.service has WantedBy=multi.user.target, which tells systemd that it should start varnish.service when the multi.user.target starts. This is roughly equivalent to enabling the service for runlevel 3 in SysV init terms.

Last but not least, the [Service] section contains the information on how the process (in our case the varnishd daemon) should be started and managed during its lifetime. The most important property in this section is ExecStart. This is the command line that will be executed on service startup, and any arguments that are to be passed to the daemon process need to be specified here.

Configure startup parameters#

You can edit the options passed to varnishd by editing ExecStart in the varnish.service file.

There are three methods of editing service files, of which two are recommended, and the third is discouraged. Editing unit files directly is possible, but is not recommended, and could lead to problems later. Packages typically ship unit files that install in the system-wide default location. Editing the files in that location directly could cause loss of changes, or alternatively cause important upstream changes to not be applied.

In place of this, systemd has two alternate editing options. One is via override files, which simply override the required parts of the unit configuration. The other is a replacement unit file, which overrides the entire unit definition.

Editing unit configuration via systemctl edit#

Note that for some older versions of the systemd utilities, systemctl does not provide the edit command. If this is the case for your system, please refer to the next section.

To create an override file, use the edit command in systemctl like so:

sudo systemctl edit varnish.service

This will open a blank file allowing you to enter overrides for the service. Below is an example, adding -p first_byte_timeout=600 to the defaults:

[Service]
ExecStart=
ExecStart=/usr/sbin/varnishd -a :6081 -f /etc/varnish/default.vcl -s malloc,256m -p first_byte_timeout=600

Note that in order to override an already existing setting (in our case ExecStart) we need to unset it first. If you try to override without unsetting first, systemd will complain.

After you write the changes and exit the editor, systemctl status should show something like this:

● varnish.service - Varnish Cache, a high-performance HTTP accelerator
   Loaded: loaded (/lib/systemd/system/varnish.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/varnish.service.d
           └─override.conf

To create a full replacement unit file, use the edit --full command:

sudo systemctl edit --full varnish.service

This will open your editor with the contents of the original unit file, allowing you to change anything. When you write the changes and exit, systemd will now consider the resulting file (/etc/systemd/system/varnish.service on CentOS7 and Ubuntu Xenial) to be the active unit file, and will not evaluate the original. This also means that any settings can be configured without unsetting them explicitly first.

The output of systemctl status changes accordingly to reflect the active unit file:

● varnish.service - Varnish Cache, a high-performance HTTP accelerator
   Loaded: loaded (/etc/systemd/system/varnish.service; enabled; vendor preset: enabled)

Manual changes to unit configuration#

Editing of the systemd units does not require using interactive command line tools. Identical results can be achieved with dropping override files and or replacement unit files into the relevant systemd directories on your system.

The override directory is (for Debian, Ubuntu and CentOS/RHEL) located at /etc/systemd/system. In order to configure only a limited change to the package-supplied unit-file, create the directory /etc/systemd/system/varnish.service.d, and then create a file /etc/systemd/system/varnish.service.d/override.conf with the required changes (see previous section for an example).

If you want to replace the packaged service definition, create the file /etc/systemd/system/varnish.service with the required unit configuration. Note that this way to configure services will prevent any upstream changes from altering the service, which in a configuration managed environment is often what you want.

systemctl daemon-reload#

Note that when you add replacement or override configurations manually, you need to tell systemd to reload the configuration with systemctl daemon-reload. When you trigger a reload, systemd will read the new unit configuration from disk, but not before. This needs to be done every time changes are made to unit files.

Inspect local overrides with systemd-delta#

To get an overview of local unit overrides, you can use the command systemd-delta, it provides output such as what follows below:

sudo systemd-delta
[EXTENDED]   /lib/systemd/system/varnish.service → /etc/systemd/system/varnish.service.d/override.conf
[OVERRIDDEN] /etc/systemd/system/hitch.service → /lib/systemd/system/hitch.service

--- /lib/systemd/system/hitch.service   2017-06-06 15:27:52.000000000 +0200
+++ /etc/systemd/system/hitch.service   2018-03-19 13:31:52.407432199 +0100
@@ -16,3 +16,4 @@

 [Install]
 WantedBy=multi-user.target
+#Small change

The [EXTENDED] service shows the location of the separate override file, whereas the [OVERRIDDEN] service displays the difference between the original and the currently used replacement unit file.

References

• Systemd’s Manuals and Documentation for Users and Administrators
• Red Hat Enterprise Linux 7 System Administrators Guide
• Digital Ocean Tutorial How To Use Systemctl to Manage Systemd Services and Units
• Digital Ocean Tutorial Understanding Systemd Units and Unit Files