System D and /etc/fstab

Once again Lennart Poettering and Kay Sievers have managed to complicate things and broken the simple model that served the Unix and Linux communities well for over 30 years, i.e. that details of filesystem mounts that should persist across a reboot are listed in /etc/(v)fstab.

With systemd, mount units may either be configured via unit files, or via /etc/fstab. Mounts listed in /etc/fstab are converted into native units dynamically using systemd-fstab-generator at boot time or when the system manager configuration is reloaded. In general, configuring mount points through /etc/fstab is the preferred approach.

A systemd unit configuration file whose name ends in “.mount” encodes information about a file system mount point controlled and supervised by systemd. If a mount point is configured in a unit file in both /etc/fstab and a unit file that is stored below /usr, the former will take precedence. If the unit file is stored below /etc, it will take precedence. This means that native unit files take precedence over traditional configuration files, but this is superseded by the rule that configuration files in /etc always take precedence over configuration files in /usr.

Here is an example of the mount file for /tmp on Fedora 20:

$ cat /usr/lib/systemd/system/tmp.mount

#  This file is part of systemd.
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

Description=Temporary Directory


Here is an example of the output of systemd-fstab-generator for the /home/fpm line in /etc/fstab:

# grep FPMWORK /etc/fstab
LABEL=FPMWORK /home/fpm    ext4   defaults        0 3 

# Automatically generated by systemd-fstab-generator



Now you have more work to do to understand where and how pseudo files and pseudo and regular filesystems are mounted. It is no longer enough to just look at /etc/fstab.

SSH2 Subsystems

SSH2 subsystems are a useful convenience feature to predefine remote commands for SSH clients to invoke easily. Subsystems provide a layer of abstraction for defining and invoking the remote commands. A subsystem need not be a separate program; it can invoke a function built into the SSH server itself.

SFTP is the most common SSH subsystem that you are going to encounter. For example on Linux distributions, the default /etc/ssh/sshd_config file defines one subsystem, This is the configuration line on Fedora 20:

# override default of no subsystems
Subsystem       sftp    /usr/libexec/openssh/sftp-server

Do not remove or comment out the above line. It is required for sftp to work.

Note that the subsystem syntax is slightly different between OpenSSH and SSH2 implementations.

# SSH version 2
subsystem-sftp       /usr/libexec/openssh/sftp-server

# OpenSSH version 2
subsystem sftp       /usr/libexec/openssh/sftp-server

Subsystems can be defined in the SSH v2 server configuration file using the following syntax.

subsystem-<name>      argument

The argument is the command which will be executed when the subsystem is requested.

$ ssh user@remote -s <name>

The argument can be a list of commands separated with a semicolon, or it could be the path to a shell script.

Alternately, you can use the syntax internal-<name<> to invoke an in-process server.

internal-<name>      argument

This may simplify configurations using ChrootDirectory to force a different filesystem root on clients. This should be used for example when the user is chrooted and does not have access to the server binary.

Mask V Disable a Systemd Service Unit

In the systemd world, you should be aware of the difference between disabling and masking a service unit.

To prevent a service unit that corresponds to a system service from being automatically started at boot time:

# systemctl disable name.service

When invoked systemd reads the [Install] section of the selected service unit and removes the appropriate symbolic link. In RHEL7, for example, the symbolic link would be to the /usr/lib/systemd/system/name.service file from the /usr/lib/systemd/system/ directory.

Every service unit that is known to systemd may be started if it is needed – even if it is disabled. To explicitly tell systemd that a service should never run, you have to use the mask command:

# systemctl mask name.service

This command replaces the /etc/systemd/system/name.service unit file with a symbolic link to /dev/null, rendering the actual unit file inaccessible to systemd.

To revert this action and unmask a service unit:

# systemctl unmask name.service

The Syslog Protocol

I have used syslog for over 30 years now but other than knowing that it uses UDP and port 514, I have never looked at the underlying protocol in any detail.

Syslog is standardized by the IETF in RFC 5424

This document describes the syslog protocol, which is used to convey
event notification messages. This protocol utilizes a layered
architecture, which allows the use of any number of transport
protocols for transmission of syslog messages. It also provides a
message format that allows vendor-specific extensions to be provided
in a structured way.

This RFC does not define any transports. They are defined in other documents. One such transport is defined in RFC 5426 and is consistent with the traditional UDP transport.

Here is the ABNF (Augmented Backus–Naur Form) definition for a syslog message:


                        SP APP-NAME SP PROCID SP MSGID
      PRI             = "<" PRIVAL ">"
      PRIVAL          = 1*3DIGIT ; range 0 .. 191

      PROCID          = NILVALUE / 1*128PRINTUSASCII
      MSGID           = NILVALUE / 1*32PRINTUSASCII

      DATE-MONTH      = 2DIGIT  ; 01-12
      DATE-MDAY       = 2DIGIT  ; 01-28, 01-29, 01-30, 01-31 based on
                                ; month/year
      TIME-HOUR       = 2DIGIT  ; 00-23
      TIME-MINUTE     = 2DIGIT  ; 00-59
      TIME-SECOND     = 2DIGIT  ; 00-59
      TIME-SECFRAC    = "." 1*6DIGIT

      SD-ELEMENT      = "[" SD-ID *(SP SD-PARAM) "]"
      SD-PARAM        = PARAM-NAME "=" %d34 PARAM-VALUE %d34
      SD-ID           = SD-NAME
      PARAM-NAME      = SD-NAME
      PARAM-VALUE     = UTF-8-STRING ; characters '"', '\' and
                                     ; ']' MUST be escaped.
      SD-NAME         = 1*32PRINTUSASCII
                        ; except '=', SP, ']', %d34 (")

      MSG             = MSG-ANY / MSG-UTF8
      MSG-ANY         = *OCTET ; not starting with BOM
      MSG-UTF8        = BOM UTF-8-STRING
      BOM             = %xEF.BB.BF
      UTF-8-STRING    = *OCTET ; UTF-8 string as specified
                        ; in RFC 3629

      OCTET           = %d00-255
      SP              = %d32
      PRINTUSASCII    = %d33-126
      NONZERO-DIGIT   = %d49-57
      DIGIT           = %d48 / NONZERO-DIGIT
      NILVALUE        = "-"

Syslog message size limits are dictated by the syslog transport in use. There is no upper limit per se. Any transport receiver must be able to accept messages of up to and including 480 octets in length, should be able to accept messages of up to and including 2048 octets in length and may accept messages larger than 2048 octets in length. If a transport receiver receives a message with a length larger than it supports, it should truncate the message or it may discard the message.

I was not aware that facility and severity values are not normative. They are described in the RFC purely for informational purposes. Facility values must be in the range of 0 to 23 inclusive. Severity values must be in the range of 0 to 7 inclusive.

The RFC also defines an optional but useful set of structured data elements. For example the SD-ID timeQuality may be used by an originator to describe its notion of system time and should be written if the originator is not properly synchronized with a reliable external time source or if it does not know whether its time zone information is correct.

Interestingly, TCP port 512 is reserved not for syslog but for remote shells such as rsh and remsh.


OpenVZ is OS containerization like Solaris Zones or FreeBSD Jails. It is virtualization at the OS level whereby all containers share the same architecture and kernel version. It uses a single patched kernel called the Linux Containers (LXC) kernel.

Its one real advantage is that containers can be faster and more efficient compared with true virtualization because it does not have the overhead of a true hypervisor such as VMware or KVM.

Older versions of OpenVZ uses a common file system so each virtual environment is just a directory of files that is isolated using chroot as also occurs in Solaris zones. Newer versions can have different file systems.

Each container performs and executes exactly like a stand-alone server; a container can be rebooted independently and have root access, users, IP addresses, memory, processes, files, applications, system libraries and configuration files. However, if the common Linux kernel crashes, all containers crash.

OpenVZ comes with command line tools only. It forms the basis for the Parallels Cloud Server, a commercial virtualization solution offered by Parallels.