Syncthing

From ArchWiki
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Syncthing is an open-source file synchronization client/server application, written in Go, implementing its own, equally free Block Exchange Protocol. All transit communications between syncthing nodes are encrypted using TLS, and all nodes are uniquely identified with cryptographic certificates.

Installation

Install the syncthing package.

Syncthing provides a #Web-GUI for control and monitoring. GUI wrappers like #Syncthing-GTK and #Syncthing Tray (provided in separate packages) also exist.

Running Syncthing

Starting Syncthing

Run the syncthing binary manually from a terminal. The multiple optional parameters are described in syncthing(1).

Note: You can run multiple copies of syncthing, but only one instance per user as syncthing locks the database to it. Check logs for errors related to locked database.

Autostarting Syncthing

Syncthing can either be installed as a systemd system-wide service or as a systemd user service to run automatically at startup.

System service

Running Syncthing as a system service ensures that it is running at startup even if the user has no active session, it is intended to be used on a server. Enable and start the syncthing@myusername.service where myusername is the actual name of the Syncthing user.

Note: If a service account was created explicitly for syncthing (e.g. via useradd -r) then ensure that the user has a valid home directory otherwise the service will immediately fail. Syncthing attempts to put configuration files into $HOME/.config/syncthing

User service

Running Syncthing as a systemd user service ensures that Syncthing only starts after the user has logged into the system (e.g., via the graphical login screen, or ssh). Thus, the user service is intended to be used on a (multiuser) desktop computer. To use the user service, start/enable the user unit syncthing.service (i.e. with the --user flag).

Syncthing systemd user service can be started after a specific (optionally encrypted) device has been mounted and stopped when device has been unmounted. To create a user service dependent on a mount point, after device has been mounted, systemd mount name has to be identified by running systemctl list-units -t mount command. Afterwards, create a new service similar to the one below:

/home/$USER/.config/systemd/user/syncthing.service
[Unit]
Description=Syncthing
BindsTo=run-media-user-and-hash.mount

[Service]
ExecStart=/usr/bin/syncthing

[Install]
WantedBy=run-media-user-and-hash.mount

Above service needs to be enabled by running systemctl enable --user syncthing.service.

Tip: It is also possible to run the systemd-user service at boot (i.e. without logging in) using Systemd/User#Automatic start-up of systemd user instances.

Syncthing-GTK

syncthing-gtk-python3AUR provides a GTK graphical user interface, desktop notifications and integration with the file managers Nautilus, Nemo and Caja. Syncthing can be launched by Syncthing-GTK: use the interface settings to run syncthing-gtk at startup, and to state whether to launch the syncthing daemon.

Note: When launching the syncthing daemon using both systemd and syncthing-gtk, it might happen that two syncthing instances run concurrently leading to high CPU consumption: one launched by syncthing-gtk, and the other (slightly later) by systemd. To solve this, either avoid launching syncthing using systemd, or configure syncthing-gtk to wait for the syncthing daemon.

Web-GUI

Syncthing provides a web interface accessible by default on http://localhost:8384.

Tip: To access the GUI remotely, see the FAQ.

Syncthing Tray

syncthingtrayAUR complements the Web-GUI by providing a Qt-based system tray icon and desktop notifications. There exists a desktop environment neutral version and a plasmoid for Plasma 5. It also provides integration with systemd and the Dolphin file manager.

The packages also comes with the syncthingctl utility which allows to interact with Syncthing from the command line.

Configuration

After installation, Syncthing already has a proper start-up configuration. New servers and/or folders can be added by visiting the web interface. For detailed instructions on how to setup a simple network, read Syncthing's getting started.

After a successful first start, a default repository at ~/Sync is created. You can see this in the web admin interface. On the right is the list of nodes you have added. On the left is the list of repositories, which are folders you can choose to share with other nodes.

To add another node, click "Add Node" underneath the list of nodes. You will be prompted for their Node ID (which can be found on the other machine by clicking Edit > Show ID) as well as a short name and the address. If you specify "dynamic" for the address, the syncthing announce server will be used to automatically exchange addresses between nodes. If you want to know more about Node IDs, including the cryptographic implications, you can read the appropriate Syncthing documentation page.

After saving the configuration, the syncthing server restarts automatically. Next, you can either change the configuration of the default node (click its name and then Edit), or create a new one to share data with. Simply tick the node you wish to share the data with, and they will have permission to access it.

Local network setup

In the typical case several machines share a LAN (Local Area Network) behind a NAT (Network Address Translation) router, it is advised for a versatile configuration to:

  • Activate both local and global discovery on each node. This will allow discovery in all situations, including if some of the nodes are mobile devices like laptops or Android phones, and leave the LAN and connect to the internet from the outside. This way they will still be found with global discovery.
  • Use a different listen address port for each machine, like tcp://:22010, tcp://:22011, tcp://:22012 and so forth. This will differentiate the nodes on the global discovery servers and avoid the "Connected to myself - should not happen" message on the other local devices whenever they leave the LAN.
  • If running multiple instances for different users on the same machine, set a different port for each user's localAnnouncePort (IPv4 broadcasts) as to avoid Syncthing complaints and choose the same localAnnounceMCAddr (IPv6 multicasts) as to find other devices on the LAN without global discovery (see Options Element).
  • If two instances on the same machine should find each other without global discovery, add tcp://127.0.0.1:xxxxx as device's second address, e.g., tcp://127.0.0.1:22001 and tcp://127.0.0.1:22002 (see Device Element).
  • Enable if possible UPnP port forwarding or manually forward each port to the right machine on the LAN. When a new node is discovered, Syncthing tries to use its configured listening port, 22000 by default. If this port happens to be closed, it will seek another port locally: whenever NAT traversal is enabled in Syncthing, it will attempt to use UPnP to map a random external port to the internal listening port chosen, for example 22000. If UPnP is not supported or if this is not desirable, each port should be manually forwarded to the right machine on the LAN. Eventually, if no open port can be found on both sides, relaying will be used.

Using inotify

inotify (inode notify) is a Linux kernel subsystem that acts to extend filesystems to notice changes to the filesystem, and report those changes to applications. Syncthing supports inotify and the functionality can be enabled in the configuration menu for individual folders.

Participate in the infrastructure

One can participate in the Syncthing infrastructure by running a global discovery server or a relay server.

Running a relay

Syncthing has the ability to connect two devices via a relay when it is not possible to establish a direct connection between them. Relayed connections are end-to-end encrypted in the usual manner, so the relay has no insight into the connection other than the knowledge of the IP addresses and device IDs.

Anyone can run a relay server and it will automatically join the Syncthing relay pool and be available to all Syncthing's users. To run your own relay, install syncthing-relaysrv and Start/Enable syncthing-relaysrv.service. Rate limiting and other options can be configured via the command line. These options can be set in the ExecStart directive of the service drop-in file as follows:

/etc/systemd/system/syncthing-relaysrv.service.d/override.conf
[Service]
ExecStart=
ExecStart=/usr/bin/syncthing-relaysrv -global-rate 500000 -provided-by relayprovidername
Note: The relay listens by default to port 22067 for data and 22070 for service status (used for public statistics), they should therefore be open for TCP connections. The default ports can be respectively overridden with the -listen and -status-srv options if necessary.
Tip: The traffic statistics of a particular relay are accessible by default on port 22070, e.g. http://example.com:22070/status

Running a discovery server

Global discovery is used by Syncthing to find peers on the internet. Any device announces itself at startup to the discovery server which stores the device ID, IP address, port and current time. Then on request, for a given device ID, it returns the information stored in JSON format, for instance.

As an example, the request https://discovery.syncthing.net/?device=ITZRNXE-YNROGBZ-HXTH5P7-VK5NYE5-QHRQGE2-7JQ6VNJ-KZUEDIU-5PPR5AM returns {"seen":"2020-02-29T14:56:08.34589801Z","addresses":["quic://212.121.228.172:22000","tcp://212.121.228.172:22000"]} .

A list of public of global discovery server is provided. In addition, anyone can run a discovery server, to run your own, install the syncthing-discosrvAUR package.

The discovery server requires certificates to run, which should ideally be placed in /var/discosrv. The user/group syncthing needs permissions to be able to read the certificate files. You need to edit the systemd unit file to correctly point to the certificates (and to undertake any other configuration change you may want, see list).

/usr/lib/systemd/system/syncthing-discosrv.service
[Unit]
Description=Syncthing discovery server
After=network.target

[Service]
User=syncthing
Group=syncthing
ExecStart=/usr/bin/syncthing-discosrv -db-dsn /var/discosrv/discosrv.db -cert /var/discosrv/cert.pem -key /var/discosrv/key.pem
Restart=on-failure
SuccessExitStatus=2

PrivateDevices=true
ProtectSystem=full
ProtectHome=true
NoNewPrivileges=true

[Install]
WantedBy=multi-user.target

To point the client to your discovery server, change the Global Discovery Servers variable under Settings to https://yourserver:8443/ (default port) or whatever port you have reconfigured to. The variable takes a comma-separated list of discovery servers. It is possible to include multiple ones, including the default one.

If you are using self-signed certificates, the client refuses to connect unless you append the discovery server ID to its domain. The ID is printed to stdout upon launching the discovery server. Amend the Global Discovery Servers entry to add the ID: https://yourserver.com:8443/?id=AAAAAAA-BBBBBBB-CCCCCCC-DDDDDDD-EEEEEEE-FFFFFFF-GGGGGGG-HHHHHHH.

Tips and tricks

Stop journal spam

Syncthing can be quite noisy even while it is not doing anything. The service ExecStart can be overridden to filter output directly without an extra script (adjust "grep" as needed):

/etc/systemd/system/syncthing@.service.d/nospam.conf
[Service]
ExecStart=
ExecStart=/bin/bash -c 'set -o pipefail; /usr/bin/syncthing -no-browser -no-restart -logflags=0 | grep -v "INFO: "'

Run in VirtualBox

It is possible to have Syncthing connect both locally and globally within a VirtualBox virtual machine (VM) while keeping its network adapter in the standard NAT mode (as opposed to bridged networking attached to the host computer's adapter).

To enable this mode, Syncthing should listen to a port in the VM different from the listening port already used by the host. For example, if the default 22000 port is used by the host, one could use 22001 in the VM. The listening port in the VM can be changed through Syncthing's Sync Protocol Listen Addresses to tcp://:22001 in the GUI Settings.

The 22001/TCP port of the host must be forwarded to the guest in this configuration. This can be done with the following command:

$ VBoxManage modifyvm myvmname --natpf1 "syncthing,tcp,,22001,,22001"

In this setup, relaying should not be necessary: local devices can connect to the VM on port 22001 while global devices are accessible as long as they have themselves an open port.

Note: local discovery in this setup is limited because the discovery listening port 21027 is already used by the host. The guest is therefore not able to build a table of local announcements though it can still broadcast to the local network via the VM NAT and announce itself. The steps described above allow to run a functioning server in the default NAT configuration but bridged networking is recommended for an optimal setup.

Running through a proxy

Syncthing can be run through a proxy to enable use behind a corporate firewall or tunneling via SSH. According to the using proxies documentation it is necessary to set the all_proxy environment variable, and it must indicate a socks5 proxy type.

  • If the service is run from a script or from the command line, you must set the variables beforehand as follows:
export all_proxy="socks5://proxy_address:proxy_port"
export no_proxy="127.0.0.1"
  • If it is run as a service, you must define the variables in the service configuration as follows:
/etc/systemd/system/syncthing@myusername.service.d/override.conf
[Service]
Environment="all_proxy=socks5://proxy_address:proxy_port"
Environment="no_proxy=127.0.0.1"

You must then reload systemd daemons configurations:

# systemctl daemon-reload

and restart the syncthing@myusername.service.

This file can be edited using systemd facility systemctl edit --full syncthing@myusername.service according to the systemd#Editing provided units section.

Syncthing FUSE

SyncthingFUSE is FUSE driver which allows to mount syncthing share without actualy syncing it to local storage.

When you open a file, the contents are served from a local cache, if possible. If the contents are not in the cache, then SyncthingFUSE asks peers for the contents and adds them to the cache. The local cache will not grow larger than a fixed size, though. If no peers are currently available for the file, then opening the file will fail.

Troubleshooting

Database issue

One may encounter database issue at some stage. To force a rescan of files and resync of database use the following command:

$ syncthing -reset-database

read-only file system error on /etc although run as root

In case Syncthing complains it is a read-only file system although the user (e.g. root on /etc) has write permissions, check the template unit's definition:

$ systemctl cat syncthing@.service

Within the [Service] part, there is a Hardening part and below that, there is a ProtectSystem directive which is set to full by default. See systemd.exec(5) § SANDBOXING for more information on this directive.

Create a drop-in file to override the value to something that suits your needs. If you are trying to sync a sub-folder of /etc, ProtectSystem=true should do the trick.

Others

See Debugging Syncthing.