Title: Port of the Week: Presenting Syncthing | |
Author: Solène | |
Date: 04 October 2023 | |
Tags: syncthing privacy security networking | |
Description: In this article, I will share about the very good | |
synchronization software Syncthing and present its features | |
# Introduction | |
Today's "port of the week" article is featuring Syncthing, a file | |
synchronization software. | |
Syncthing official project website | |
Related blog posts: | |
Blog post about the complementary Relay server | |
Blog post about the complementary discovery server | |
# Quick intro | |
As stated earlier, Syncthing is a network daemon that synchronize files | |
between computers/phones. Each Syncthing instance must know the other | |
instance ID to trust them and find them over the network. The transfer | |
are encrypted and efficient, the storage itself can be encrypted. | |
Some Syncthing vocabulary: | |
* a folder: a local directory that is shared with a remote device, | |
* a remote device: a remote computer running Syncthing, each of them | |
have a unique ID and a user-defined name, you can choose which shared | |
folders you want to synchronize with them | |
* an item: this word appears when syncing two remotes, an item can be | |
either a directory or a file that isn't synchronized yet | |
* a discovery server: a server which helps remotes finding known | |
remotes over the Internet, or in the worst case scenario, relays data | |
from a remote to another if they can't communicate directly | |
# Interesting features | |
I gathered a list of interesting features that you may find interesting | |
in Syncthing. | |
## Security: authentication and encryption | |
When you need to add a new remote, you need to add the remote's ID on a | |
Syncthing and trust that one on the remote one. The ID is a human | |
representation of the Syncthing instance certificate fingerprint. When | |
you exchange ID, you are basically asked to review each certificate and | |
allow each instance to trust the other. | |
All network transfers occurring between two Syncthing are encrypted | |
using TLS, as the remote certificate can be checked, the incoming data | |
integrity can be verified and authenticated. | |
Syncthing official documentation about security principles in the software | |
## Relaying | |
I guess this is Syncthing killer feature. Connecting two remotes is | |
very easy and file transfer between them can bypass firewalls and NATs. | |
This works because the Syncthing offers a default discovery server | |
which has two purposes: | |
* if the two servers could potentially communicate to each other but | |
are behind NATs, it does what we call "hole punching" to establish a | |
connection between the two remotes and allow them to transfer directly | |
from one to the other | |
* if the two servers can't communicate to each other, the discovery | |
server acts as a relay for the data | |
The file transfer is still encrypted, but having a third party server | |
involved may rise privacy issues, and security risks if a vulnerability | |
can be exploited. | |
My next blog post will show how to self-host your own Syncthing relay, | |
for better privacy and even more complicated setups! | |
Note that the discovery server or the relaying can be disabled! You | |
could also build a mesh VPN and run Syncthing on each node without | |
using any relay or discovery server. | |
## Built-in file versioning | |
This may be my preferred feature in Syncthing! | |
On a given Syncthing instance, you can enable per shared folder a | |
retention policy, aka file versioning in the interface. | |
Basically, if a file is modified / removed in the share by a remote, | |
the local instance can keep a hidden copy for a while. | |
There are different versioning modes, from a simple "trash bin" style | |
keeping the files for n days, or more elaborated policies like you | |
could have in backup tools. | |
Syncthing official documentation about file versioning | |
## Partial share synchronization | |
For each share, it's possible to write an exclusion filter, this allows | |
you to either discard sync changes for some pattern (like excluding vim | |
swap files) or entire directories if you don't want to retrieve all the | |
shared folder. | |
The filter works in both way, if you accept a remote, you could write a | |
filter before starting the synchronization and remove some huge | |
directories you may not want locally. But this could also allow | |
preventing a directory to be sent to the remotes, like a temporary | |
directory for instance. | |
This is a topic I covered with a very specific use case, only sync a | |
single file in a directory. | |
Earlier blog post: Configure Syncthing to sync a single file | |
## Encrypted remotes | |
A pretty cool feature I found recently was the support for encrypted | |
shared folders per remote. I'm using syncthing to keep my KeepassXC | |
databases synchronized between my computers. | |
As I don't always have at least two of my computers turned ON at the | |
same time, they can't always synchronize directly with each other, so I | |
use a remote dedicated server as a buffer to hold the files, Syncthing | |
encryption is activated for this remote, both my computers can exchange | |
data with it, but on the server itself you can't get my KeepassXC | |
databases. | |
This is also pretty cool as it doesn't leave any readable data on the | |
storage drive if you use 3rd party systems. | |
Taking the opportunity here, KeepassXC has a cool feature that allows | |
you to add a binary file as a key in addition to a password / FIDO key. | |
If this binary file isn't part of the synchronized directory, even | |
someone who could access your KeepassXC database and steal your | |
password shouldn't be able to use it. | |
## Data chunk based | |
When Syncthing scans a directory, it will hash all the file into chunks | |
and synchronize all these chunks to other remotes, this is basically | |
how BitTorrent work too. | |
This may sound boring, but basically, this allows Syncthing to move or | |
rename files on a remote instead of transferring the data again when | |
you rename / move files in a local shared directory. Indeed, only the | |
changed paths list is sent, and the chunks used in the files, as the | |
files already exist on the remote, the data chunks don't have to be | |
retrieved. | |
Note that this doesn't work for encrypted remotes as the chunks contain | |
some path information, once encrypted, the same file with different | |
paths will look as two different encrypted chunks. | |
## Bandwidth control | |
Syncthing GUI allows you to define inbound or outbound bandwidth | |
limitation, either globally or per remote. If like me, you have a slow | |
ADSL line with slow upload, you may want to limit the bandwidth used to | |
send data to set the non-local remotes. | |
## Support for all attributes synchronization | |
This may sound more niche, but it's important for some users: Syncthing | |
can synchronize file permissions, ownership or even extended | |
attributes. This is not enabled by default as Syncthing requires | |
elevated privileges (typically running as root) to make it work. | |
## Runs everywhere | |
Syncthing is a Go program, it's a small binary with no dependencies, | |
it's quite portable and runs on Linux, all the BSD, Android, Windows, | |
macOS etc... There is nothing worse than a synchronization utility | |
that can't be installed on a specific computer... | |
# Conclusion | |
I really love this software, especially since I figured the file | |
versioning and the encrypted remotes, now I don't fear conflicts or | |
lost files anymore when syncing my files between computers. | |
My computers also use a local discovery server that allows my Qubes OS | |
to be kept in sync together over the LAN. | |
# Note for SystemD users | |
When you install Syncthing on your system, you can enable the service | |
as your user, this will make Syncthing start properly when you log in | |
with your user: | |
``` | |
systemctl enable --user syncthing.service | |
``` | |
# Note for OpenBSD users | |
Syncthing has to listen for each file change, you will need to increase | |
the maximum opened files limit for your user, and maybe the limit in | |
the kernel using the according sysctl. | |
You can find more detailed information about using Syncthing on OpenBSD | |
in the file `/usr/local/share/doc/pkg-readmes/syncthing`. |