This is a guest blogpost by Martin Langer, Ph.D. student for “Secured Time Synchronization Using Packet-Based Time Protocols” at Ostfalia University of Applied Sciences, Germany.
In the previous posts, I already introduced the Network Time Security (NTS) protocol and described the most important features. Although the specification process has not been completed, there are already some independent NTS implementations and public time servers (IETF106). NTPsec is one of the important representatives of this series and already offers an advanced NTS solution. In this post, I’ll give you a short guide to setting up an NTS-secured NTP client/server with NTPsec.
Overview
NTPsec is a fork of the NTP reference implementation NTPD that has removed inherited burdens to provide a smaller codebase. In addition, the relevance of NTPsec in the Linux environment is increasing and it is one of the first NTS-capable NTP implementations.
Due to the missing Windows support, this guide is only intended for Linux systems. Therefore I’m using a Raspberry Pi 3B with the recent Debian-based operating system Raspbian. The procedure described here also works with other Linux distributions in the same or similar form.
Software Components Used
In order to use NTS, we need the following software components. You can find detailed instructions for installation and setup below:
- Raspbian Buster Lite (2019-09-26)
- NTPsec 1.1.8
- OpenSSL 1.1.1d
NTS Properties of NTPsec
Currently (Q4/2019), NTPsec has implemented the NTS draft 17. The following adjustments of the specification up to the latest draft version 20 do not contain any significant changes so that NTPsec is representative. The following NTS features are included in the implementation:
- AEAD algorithms
- Default NTS-KE port
- 123 (TCP)
- Default NTP port
- 123 (UDP)
- Error handling (server-side)
- Discard request packets without response*
*The server only discards invalid request packets and does not send an NTSN (NTS NAK) back to the client. This is okay, but a client then doesn’t know whether a request has been lost or if an error occurred during verification.
Preparation
Okay, now we can finally get started. First, we log into our Raspberry Pi (RPi) and make sure that we have an Internet connection. This device can be accessed directly with a connected screen and keyboard or headless via SSH. Then we update the system and the installed packages:
# update system sudo apt update sudo apt upgrade
Next, we check the OpenSSL version, which must be 1.1.1b or higher:
# check openssl version openssl version
To ensure the correct functioning of NTPsec, we should also disable the local time synchronization service. This also applies to other services (such as OpenNTPD, Chrony, …).
# disable the local time service sudo systemctl stop systemd-timesyncd.service sudo systemctl disable systemd-timesyncd.service
If an NTP service (e.g. NTPD) is used instead, we can remove it as follows:
# remove NTP sudo apt remove --auto-remove ntp
This is necessary since NTPD and NTPsec use the same files and names for the application.
Installation of NTPsec
The installation of NTPsec can be done via the package manager as well as by manually building the source code.
Method 1: Using the Package Manager
The easiest way to install NTPsec is through the use of a package manager. For NTS support, we need the version 1.1.8 or higher. To check which package version is available, we use the command apt show:
apt show ntpsec | grep Version
If version 1.1.8 is available, we can simply install it as follows:
sudo apt install ntpsec
That’s it! You can skip the following steps and go directly to the configuration (see below).
Method 2: Build the Source Code Manually
The manual setup consists of building and installing NTPsec as well as its registering as a system service (systemd).
Part 1: Building and Installing
This installation guide is based on the NTPsec instructions and can be carried out without much effort. We begin with the installation of some basic tools and compilers. Then we create a temporary working directory in which we copy and build the NTPsec source code.
# install basic tools and compilers (g++, gcc, make, ...) sudo apt install build-essential # create a temporary directory cd /home/pi mkdir ntpsec_source cd ntpsec_source
Next, there are two ways to build the code. Either we clone the current git repository (variant 1) or we download the recent NTPsec version (variant 2).
Variant 1: Using The Git Repository
# install git sudo apt install git # clone the NTPsec git repository (recent version) git clone https://gitlab.com/NTPsec/ntpsec.git cd ntpsec
Variant 2: Downloading Binaries
# download the current NTPsec version as tarball wget https://gitlab.com/NTPsec/ntpsec/-/archive/NTPsec_1_1_8/ntpsec-NTPsec_1_1_8.tar.gz # extract the source code tar xfz ntpsec-NTPsec_1_1_8.tar.gz cd ntpsec-NTPsec_1_1_8
The following script now installs all tools or packages (bison, libssl-dev, libcap-dev, libseccomp-dev) that are needed for building. These can also be installed manually. But in our case we use the script:
# install necessary packages sudo ./buildprep
Now we can make initial configurations. We can display the available options with
./waf configure --help. If no parameters are specified, the NTPsec installation is done in the /usr/local/ directory by default. The application (ntsd) is then located in /usr/local/sbin. With the parameter –prefix=<dest/path> we can change the path arbitrarily. If NTPsec is to be configured as a server, the parameter –refclock=all is necessary, since it allows the use of any local reference clock (GPS, PTP, SHM, etc.).
# configure NTPsec: default path (/usr/local) and reference clock support ./waf configure --refclock=all
Finally, we build the source code and install the binaries:
# build NTPsec ./waf build # install NTPsec sudo ./waf install # delete the temporary working directory (NTPsec source code) cd ../.. rm -r -f ntpsec_source
Let’s check the NTPsec version number:
# check NTPsec version sudo /usr/local/sbin/ntpd -V
The output should look like
ntpd ntpsec-1.1.8 2019-11-20T04:44:12Zand must be 1.1.8 or higher.
Part 2: Setting up the NTP Service (Systemd)
Perfect. Next, we set up NTPsec as a system service. For this, we create an ‘ntp’ user and an ‘ntp’ group with restricted rights. We also create folders for log files and certificates.
# create ntp directories sudo mkdir -p /var/lib/ntp/certs sudo mkdir -p /var/log/ntpstats # add system user 'ntp' sudo adduser --system --no-create-home --disabled-login --gecos '' ntp # add group 'ntp' sudo addgroup --system ntp # add user 'ntp' to group 'ntp' sudo addgroup ntp ntp # set folder permissions (recursive) sudo chown -R ntp:ntp /var/lib/ntp /var/log/ntpstats # enable the NTPsec service sudo systemctl enable ntpd
Now NTPsec is executed automatically when rebooting the system. At the moment the service is not running yet, because we have to configure it first.
Configuration of NTPsec
After the installation of NTPsec the configuration has to be created. The NTP service uses ntp.conf, which is located in the /etc/ directory by default. If this file is not yet available, it must be created:
# create NTP config file sudo touch /etc/ntp.conf sudo nano /etc/ntp.conf
The following examples are based on the NTPsec instructions (General, NTS-QuickStart) and differ between client and server.
Client Configuration
Let’s have a look to the client configuration. In this ntp.conf I have listed 4 examples for a possible client configuration. These can be changed or commented out if necessary:
# Example 1: unsecured NTP server ntp1.glypnod.com iburst minpoll 3 maxpoll 6 # Example 2: NTS-secured NTP (default NTS-KE port (123); using certificate pool of the operating system) server ntp1.glypnod.com iburst minpoll 3 maxpoll 6 nts # Example 3: NTS-secured NTP (custom certificate and NTS-KE port) server nts3-e.ostfalia.de:443 iburst minpoll 3 maxpoll 6 nts ca /var/lib/ntp/certs/rootCaBundle.pem # Example 4: NTS-secured NTP (skip DNS check) server nts3-e.ostfalia.de:443 iburst minpoll 3 maxpoll 6 nts ca /var/lib/ntp/certs/rootCaBundle.pem noval # optional: allows a fast frequency error correction on startup driftfile /var/lib/ntp/ntp.drift # optional: collect statistics statsdir /var/log/ntpstats statistics loopstats peerstats clockstats rawstats filegen loopstats file loopstats type day enable filegen peerstats file peerstats type day enable filegen clockstats file clockstats type day enable filegen rawstats file rawstats type day enable # optional: logging logfile /var/log/ntp.log logconfig =syncall +clockall +peerall +sysall
In the first example, the Raspberry Pi synchronizes its clock with a public NTPsec time server using a classical unsecured NTP connection.
In the second example, the client communicates with the same time server via an NTS-secured NTP connection. The initial channel to the NTS-KE server uses the default port 123 TCP (currently implementation-specific). Since the NTPsec time server uses certificates issued by Let’s Encrypt, we do not need to set any additional parameters. To check the certificates, the client uses the local root CA pool (/etc/ssl/certs/ca-certificates.crt), which also allows the verification of certificates issued by Let’s Encrypt.
In the third example, we connect to the time server of the Ostfalia University, which is also NTS-capable. This uses TCP port 443 for the NTS-KE connection and uses self-signed test certificates. To check the server certificate we have to specify the root CA manually. The corresponding certificate can be downloaded by the following command:
sudo wget http://nts3-e.ostfalia.de/homePi/CLIENT/rootCaBundle.pem -P /var/lib/ntp/certs/
The fourth example differs from the third one only from the deactivated domain validation. This can be useful when we running a local NTS server with certificates without a registered domain.
The other entries in the configuration file are optional and are used to record statistics and log files. The descriptions as well as the complete parameter list (incl. NTS) can be found in ntp_conf.adoc. Here only very briefly described:
server <name>: The destination NTP time server (DNS name or IP address) iburst: Sends 8 NTP requests directly after startup minpoll <val>: Minimal request interval (power of two) maxpoll <val>: Maximal request interval (e.g: '6' means 2^6 = 64 sec) nts: [NTS] Enables NTS support ca <file>: [NTS] The trusted root CA certificate for the server noval: [NTS] Skips the DNS verification
Server Configuration
The configuration of the server is a little easier:
server 127.127.1.0 prefer fudge 127.127.1.0 stratum 10 nts enable cert /var/lib/ntp/certs/serverCert.pem key /var/lib/ntp/certs/serverKey.pem
In this example, we use the local clock as the reference time source, which we distribute to all connected clients. When activating NTS, we have to specify a server certificate and the private key. This may have been created manually or issued by an external CA instance (e.g. Let’s Encrypt again).
By the way, the client and server configurations can be combined to run both simultaneously. For example, a client could fetch the time NTS-secured from external time servers and distribute it as a server in the local network without NTS.
Debug and Advanced Information
Starting the Daemon:
Now we can start our NTPsec service:
sudo systemctl start ntpd
After this, the service should run, which we can easily check:
# check NTPsec service systemctl status ntpd
The whole thing should look like this:
● ntpd.service - Network Time Service
Loaded: loaded (/lib/systemd/system/ntpd.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2019-11-25 09:48:25 CET; 27min ago
Docs: man:ntpd(8)
Process: 498 ExecStart=/usr/local/sbin/ntpd -g -N -u ntp:ntp (code=exited, status=0/SUCCESS)
Main PID: 504 (ntpd)
Tasks: 3 (limit: 2200)
Memory: 26.9M
CGroup: /system.slice/ntpd.service
└─504 /usr/local/sbin/ntpd -g -N -u ntp:ntp
Manual Start of NTPsec:
Of course, you can also start NTPsec manually:
sudo /usr/local/sbin/ntpd
If you also want to see the log output live, then you need to specify some parameters:
sudo /usr/local/sbin/ntpd -n -d
For an NTS-secured NTP client (here from example 3 of the configuration) we see the following:
2019-11-20T05:22:15 ntpd[6847]: INIT: ntpd ntpsec-1.1.8+ 2019-11-20T03:55:12Z (git rev 905721870): Starting 2019-11-20T05:22:15 ntpd[6847]: INIT: Command line: /usr/local/sbin/ntpd -n -d 2019-11-20T05:22:15 ntpd[6847]: INIT: precision = 2.031 usec (-19) 2019-11-20T05:22:15 ntpd[6847]: INIT: successfully locked into RAM 2019-11-20T05:22:15 ntpd[6847]: CONFIG: readconfig: parsing file: /etc/ntp.conf 2019-11-20T05:22:15 ntpd[6847]: LOG: switching logging to file /home/pi/ntpsec/ntp.log 2019-11-20T05:22:15 ntpd[6847]: INIT: Using SO_TIMESTAMPNS 2019-11-20T05:22:15 ntpd[6847]: IO: Listen and drop on 0 v6wildcard [::]:123 2019-11-20T05:22:15 ntpd[6847]: IO: Listen and drop on 1 v4wildcard 0.0.0.0:123 2019-11-20T05:22:15 ntpd[6847]: IO: Listen normally on 2 lo 127.0.0.1:123 2019-11-20T05:22:15 ntpd[6847]: IO: Listen normally on 3 wlan0 192.168.2.102:123 2019-11-20T05:22:15 ntpd[6847]: IO: Listen normally on 4 lo [::1]:123 2019-11-20T05:22:15 ntpd[6847]: IO: Listen normally on 5 wlan0 [fe80::63dc:3600:6bad:c768%3]:123 2019-11-20T05:22:15 ntpd[6847]: IO: Listening on routing socket on fd #22 for interface updates 2019-11-20T05:22:15 ntpd[6847]: PROTO: 0.0.0.0 c011 81 mobilize assoc 59451 2019-11-20T05:22:15 ntpd[6847]: INIT: This system has a 32-bit time_t. 2019-11-20T05:22:15 ntpd[6847]: INIT: This ntpd will fail on 2038-01-19T03:14:07Z. 2019-11-20T05:22:15 ntpd[6847]: PROTO: 0.0.0.0 c01d 0d kern kernel time sync enabled 2019-11-20T05:22:15 ntpd[6847]: PROTO: 0.0.0.0 c012 02 freq_set kernel 0.000000 PPM 2019-11-20T05:22:15 ntpd[6847]: PROTO: 0.0.0.0 c011 01 freq_not_set 2019-11-20T05:22:15 ntpd[6847]: PROTO: 0.0.0.0 c016 06 restart 2019-11-20T05:22:15 ntpd[6847]: INIT: OpenSSL 1.1.1d 10 Sep 2019, 1010104f 2019-11-20T05:22:15 ntpd[6847]: NTSc: Using system default root certificates. 2019-11-20T05:22:16 ntpd[6847]: DNS: dns_probe: nts3-e.ostfalia.de:443, cast_flags:1, flags:21901 2019-11-20T05:22:16 ntpd[6847]: NTSc: DNS lookup of nts3-e.ostfalia.de:443 took 0.009 sec 2019-11-20T05:22:16 ntpd[6847]: NTSc: nts_probe connecting to nts3-e.ostfalia.de:443 => 141.41.241.70:123 2019-11-20T05:22:17 ntpd[6847]: NTSc: Using file /home/pi/CLIENT/rootCaBundle.pem for root certificates. 2019-11-20T05:22:17 ntpd[6847]: NTSc: set cert host: nts3-e.ostfalia.de 2019-11-20T05:22:17 ntpd[6847]: NTSc: Using TLSv1.3, TLS_AES_256_GCM_SHA384 (256) 2019-11-20T05:22:17 ntpd[6847]: NTSc: certificate subject name: /C=DE/ST=NDS/L=Wolfenbuettel/O=Ostfalia/CN=nts3-e.ostfalia.de 2019-11-20T05:22:17 ntpd[6847]: NTSc: certificate issuer name: /C=DE/ST=NDS/L=Wolfenbuettel/O=Ostfalia/CN=OstfaliaRootCA 2019-11-20T05:22:17 ntpd[6847]: NTSc: certificate is valid. 2019-11-20T05:22:17 ntpd[6847]: NTSc: Good ALPN from: nts3-e.ostfalia.de:443 2019-11-20T05:22:17 ntpd[6847]: NTSc: read 848 bytes 2019-11-20T05:22:17 ntpd[6847]: NTSc: Got 8 cookies, length 100, aead=15. 2019-11-20T05:22:17 ntpd[6847]: NTSc: NTS-KE req to nts3-e.ostfalia.de:443 took 0.056 sec, OK 2019-11-20T05:22:17 ntpd[6847]: DNS: dns_check: processing nts3-e.ostfalia.de:443, 1, 21901 2019-11-20T05:22:17 ntpd[6847]: DNS: Server taking: 141.41.241.70 2019-11-20T05:22:17 ntpd[6847]: DNS: dns_take_status: nts3-e.ostfalia.de:443=>good, 0 2019-11-20T05:22:17 ntpd[6847]: PROTO: 141.41.241.70 e014 84 reachable 2019-11-20T05:22:22 ntpd[6847]: PROTO: 141.41.241.70 f01a 8a sys_peer 2019-11-20T05:22:22 ntpd[6847]: PROTO: 0.0.0.0 c014 04 freq_mode
Everything works well. The certificate is valid and the client has received 8 cookies. The AEAD algorithm 15 (AEAD_AES_SIV_CMAC_256) is used to secure packets.
Peer Status:
To check the current status of the connection(s), we must enter the following command:
/usr/local/bin/ntpq -p
The NTS-Secured NTP Client:
remote refid st t when poll reach delay offset jitter =============================================================================== *nts3-e.ostfalia 192.53.103.104 2 8 3 8 377 6.0908 0.9518 1.3586
As you can see, the synchronization works fine. No packet loss and a small time offset.
The NTS-Secured NTP Server:
remote refid st t when poll reach delay offset jitter =============================================================================== *LOCAL(0) .LOCL. 10 l 31 64 77 0.0000 0.0000 0.0000
The server side also works, but gives less information.
NTS Status:
The following command allows us to view the NTS statistics:
/usr/local/bin/ntpq -c nts
If we use both NTS client and NTS server at the same time, then we get these values:
NTS client sends: 63 NTS client recvs good: 63 NTS client recvs w error: 0 NTS server recvs good: 24 NTS server recvs w error: 7 NTS server sends: 24 NTS make cookies: 48 NTS decode cookies: 31 NTS decode cookies old: 0 NTS decode cookies too old: 0 NTS decode cookies error: 0 NTS KE probes good: 1 NTS KE probes_bad: 0 NTS KE serves good: 3 NTS KE serves_bad: 0
Manipulated or invalid packets are detected and discarded. The number is listed accordingly (NTS server recvs w error).
Wireshark Examples and Pcap Files
At this point, I would like to show a few examples, how the whole thing looks like in Wireshark. Since NTS is not yet supported in Wireshark, there are still incorrect interpretations of the binary data. But that shouldn’t bother us at all.
Example 1: NTS with TLS 1.2 and 512-Bit AEAD Algorithm:
In this figure we see an NTS-secured NTP connection. This starts with the NTS-KE over TLS 1.2. Due to the strong AEAD algorithm and the resulting large cookies, the NTP packets have a size of 296 bytes (or 338 bytes including all headers). The yellow markings in Wireshark are due to misinterpretations of the content.
Example 2: Possible NTS Behavior with Packet Loss or Manipulation:
Here we see the behavior of manipulated NTP packets. The server simply discards them. The client then sends another packet and wants an additional cookie because one is missing. Therefore, the size of the following request increases by the size of a cookie. This is repeated until the client has no cookies left and repeats the NTS-KE. This behavior also occurs if the client does not receive the response due to packet loss.
Exampe 3: NTPsec with Default Settings and NTS
Like example 1, but with TLS 1.3 and a 256 bit AEAD algorithm. The NTP messages are therefore smaller.
Example 4: Multiple Unsecured NTP and NTS-Secured NTP Connections
In this example, we see unsecured and NTS-secured NTP connections of the Ostfalia time server. The NTS connections use different configurations (AEAD algorithms), so that the packet sizes vary.
The current connections of the Ostfalia time server are recorded and can be freely downloaded. This allows better diagnosis and live tracking when testing with NTS. The current pcap files can be found here and here. The pcap files of the Wireshark examples above are here.:
Thanks for reading ;)
Featured image “Nightlight” by Stig Nygaard is licensed under CC BY 2.0.