Synology NAS DS220j. OS Kernel and findhostd Service Security

Privacy and security are the main reasons why people choose self-hosted NAS over cloud-based solutions.
This article is the second part about Synology NAS DS220j security. It covers OS kernel and findhostd service security.

Note: This article was originally written in Feb 2021 and based on some DSM v6 image. Highlighted bugs were fixed in Aug 2021 with DSM 7 release. I believe that almost everything described in this article is outdated now.

General info

In the first part you can find a lot of information about low level staff of Synology NAS DS220j. Luckily, you don’t need to read it. The only two things you should know to understand this part:

• it has Linux-based OS (called “DiskStation Manager”, DSM),
• you can enable SSH and log in with root privileges.

In this part we’ll check OS kernel and dig into one proprietary binary service.

TL;DR

The kernel is outdated, but it looks like critical fixes are applied.

DSM could run huge number of services with huge number of opened ports. The majority of the services are web based, but there are still some proprietary binary services.

I’ve found several bugs in proprietary discovery service (DoS, security mechanism bypass and sensitive information leakage). It’s (almost) always on but can only be accessed from the local network.

OS kernel

Synology NAS DS220j is based on Linux kernel version 4.4.59+ for aarch64 (ARM64).

The kernel is outdated in general, but it has critical security related fixes (at least some of them). But if you know some critical bug in Linux kernel, that was fixed without “security fix” mark, your exploit should work.

I was lazy in checking kernel for some vendor specific patches or custom drivers, sorry.

Services

There are many active services on my NAS and even more can be run, depends on your configuration and installed software. Here is the list of services on my NAS.

Let’s look at findhostd service.

findhostd

This services is responsible for communication with “Synology Assistant” software. It uses a proprietary protocol over UDP (three ports are occupied: 9997, 9998 and 9999). It’s binary can be found at /usr/syno/bin/findhostd.

This binary has no debug symbols, but it’s small and has a lot of strings. So it’s easy to RE it.

Packet processing

Incoming packet passes the next steps during processing:

• Binary /usr/syno/bin/findhostd receives broadcast UDP packets on specified port (packet’s max size is 0x2000).
• Binary makes basic checks (it checks magic, next few bytes and packet length) to filter out broadcast messages from other NASes.
• If the message has passed the filter it’s parsed by FHOSTPacketRead function of /usr/lib/libfindhost.so.1.
  - Library function checks packet, parses fields and returns special structure (let’s call it find_host_s), where some field are set by values from input message or zeroed.
• Binary takes the structure and process it depending on command field.

Packet structure

Packet structure is shown on the figure below.

As you can see, every packet has magic and several fields. Every field has

• ID which is used to determine field type (one of integer, string, array) and its max length,
• field length.

During parsing procedure (in function FHOSTPacketRead) field_data field content is copied to some offset in the special structure (let’s call it find_host_s). Offset depends on field_id.

I’ve checked parsing logic for errors and found nothing interesting. If you want to find more information about packet structure, take a look at my wireshark dissector, my packet builder script and check section below or RE library \usr\lib\libfindhost.so.1. If you don’t, just go ahead to “Commands” section.

/usr/lib/libfindhost.so.1

Spoiler: no bugs here! You don’t need to read this part to understand further parts!

Function FHOSTPacketRead checks magic and enters fields parsing loop.
In the loop it searches field_id in the predefined list of field descriptions (via binary search) and parses it. Parsing process is different for different field types. Field can be one of the following type:

• string,
• integer (max 8 bytes),
• array (max length is 0x20).

Array elements are always of string type, strings are parsed like normal C strings (with zero termination). Integers are parsed like integers. Nothing special.

Also some field types are not in the predefined list of field descriptions (and, thus, skipped), but still nothing interesting.

Here is the dump of static array of field descriptions, where:

• fid – field ID;
• ftype – field type: 0 for string, 1 for integer and 2 for array,
• struct_off – offset in output structure;
• max_len – max field value length;
• ns_mask – used to help caller understand, either some fields were provided or not.

Some message fields overlap structure fields, but no type collisions or smth like that happen.

Commands

There is a command field in find_host_s structure. By command service determines the action that it’s asked to do.
You can find all commands and action described in table below.

As you can see, there is a lot of interesting commands and many of them are available without authentication.

Command Authentication

The most interesting commands require authentication. Service uses PAM for authentication. It has several advantages:

• all failed authentication attempts are carefully logged,
• global restrictions are also applied to this service (e.g. limit for failed auth attempts, disabling users or blocking some IPs).

But there are also some disadvantages:

• It’s extremely easy to forge source IP. This means that:
  - if NAS is configured to block by IP of anyone who exceeded failed auth attempts limit, attacker in local network can force NAS to block almost any IP (DoS).
  - attacker in local network can do much more auth attempts than you expect (by changing UDP source IP — security mechanism bypass).
• Password in clear text is required (no challenge-response mechanisms are supported) and there is no sessions, so user should send its credentials for every command. Synology tried to address this problem by masking password, but attacker has everything to unmask it. (leaking user credentials).

You can find password masking and unmasking algorithm here.

Finally here are the bugs (all from local network):

• Security mechanism bypass
  As you can see at the screenshot below, it’s possible to configure “Auto Block” security feature. When configured, it will block IP after N unsuccessful login attempts (5 in this case).

With findhostd service attacker could do N * num_hosts_in_local_network attempts which is much more than you could have expected, but still not enough to bruteforce even a weak password.

• DoS
  When “Auto Block” feature enabled, attacker can add any local IP (or all local IPs) to the blocked list by sending N packets with wrong password and fake source IP to the discovery service.
  Thus NAS will become unavailable for legitimate users from that IP.
  Note: if NAS is behind the NAT, attacker could block router’s IP and no remote access will be possible.
• User credentials leak
  If attacker can monitor local network traffic, they can grab legitimate packets with auth block inside and extract users credentials from it.

Check user exists

With command 9 it’s possible to check that user exists (by name). There is no restrictions on the number of failed attempts (even when “Auto Block” is enabled) and no records are added to logs (at least in default configuration).

So it’s possible to get user names for the NAS (even names for disabled users). For more info see script check-user.py.

Getting list of shared folders

With commands 5 and 13 it’s possible to get list of shared folders for any user (even if you’ve disabled this user, but it got some permissions because of the group).

Command 13 will show all shared folders, including read only ones. Command 5 will show only read-write folders.

On the screenshots below you can see configuration for 2 folders: “test-rw” and “test-ro”. As you can see, “test-rw” is accessible to ‘guest’ with read-write permission, “test-ro” is accessible to ‘guest’ with read-only permission.

Below is the result of execution of commands 5 and 13 for user ‘guest’ (and ‘guest’ user is disabled). For more info see script check-shared.py.

Commands 2 and 6

With commands 2 and 6 it’s possible to add device to the list of discovered Synology devices (synods) and to the list of Synology devices group (synogrinst).

Decompiled code is shown below.

As you can see, we can add almost arbitrary record in these lists (e.g. by injecting invalid record with hostnames, including '\n' symbol).
Config injection results are shown below (it was done by scripts test-cmd-2.py and test-cmd-6.py).

I’ve checked services synonetd and synogrinst (they process files “/tmp/synods.list” and “/tmp/grinst_chche.list”), but I haven’t found anything interesting.

Actually, there is one interesting moment. There is a path traversal in synogrinst when processing “/volume1/publicgrinst_chche” (it could contain some data from “/tmp/grinst_chche.list”). But it’s unclear how one can abuse it: content of the wrong file is printed to the console.

Limitations

Incoming packets won’t be processed if all the next conditions are true:

• Synology High Availability (Synology HA) service is running,
• Synology HA service isn’t in safe mode,
• Synology HA service isn’t active,
• any of the files /.vscan_confirmed or /.e2fs_volume exists.

As you can see, these restrictions are quite strict and will rarely happen (maybe it’s some NAS configuration in cloud environment).

Conclusion

All these bugs were reported to Synology in autumn 2021 and fixed in August-September 2021with DSM 7 release. I checked neither the fix nor the architecture of the new findhostd service (if it still exists).