Cluster Crash: Learning from Recent Vulnerabilities in Communication

Stacks

Anne Borcherding

1,3 a

, Philipp Takacs

1

and J

¨

urgen Beyerer

1,2,3

includes blackbox network fuzzing, where fields in network packets are filled with unexpected values to test

the device’s behavior in edge cases. Due to resource constraints, the tests need to be efficient and such the

input values need to be chosen intelligently. Previous solutions use heuristics based on vague knowledge from

previous projects to make these decisions. We aim to structure existing knowledge by defining Vulnerabil-

ity Anti-Patterns for network communication stacks based on an analysis of the recent vulnerability groups

Ripple20, Amnesia:33, and Urgent/11. For our evaluation, we implement fuzzing test scripts based on the

Vulnerability Anti-Patterns and run them against 8 industrial device from 5 different device classes. We show

(I) that similar vulnerabilities occur in implementations of the same protocol as well as in different protocols,

(II) that similar vulnerabilities also spread over different device classes, and (III) that test scripts based on the

Vulnerability Anti-Patterns help to identify these vulnerabilities.

ing digitalisation and complexity, modern industrial

device consist of a plethora of components. In order

to provide reliable and secure services, thorough tests

of the components need to be conducted. One impor-

tant part of a thorough test of an industrial device is

to conduct blackbox tests, which do not consider any

internal details of the Device under Test (DUT), but

test the DUT from the outside. In the domain of in-

dustrial device, blackbox tests are usually performed

via the Ethernet interface. Amongst other, these tests

a

https://orcid.org/0000-0002-8144-2382

ware errors that have been created during the design

or development of the industrial device. One way

to describe the design or development decisions that

lead to these errors are Anti-Patterns or Vulnerability

Anti-Patterns (VAPs) (Nafees et al., 2018). VAPs help

to communicate design and coding practices that po-

tentially lead to vulnerable code. Although VAPs are

intended to prevent errors and vulnerabilities during

et al., 2020). For blackbox network fuzzing, one usu-

ally tests several expected or unexpected values for

the different fields of a network packet. Due to time

and resource restrictions, it is usually not possible to

test all fields with all possible values. We propose to

use the information derived from previous vulnerabil-

ity groups represented as VAPs to guide the prioriti-

zation of fields and values.

Our aim is to understand which VAPs form the ba-

sis of recently published vulnerabilities in communi-

cation stacks of industrial device, and how this infor-

mation can be used to test industrial device more effi-

ciently. In order to achieve this, we analyze three vul-

nerability groups that have been published recently:

Ripple20 (Kohl and Oberman, 2020; Kohl et al.,

2020), Amnesia:33 (dos Santos et al., 2021), and Ur-

gent/11 (Seri et al., 2019). All of these vulnerabili-

ties concern communication stacks for standard Inter-

ing from these VAPs.

For the evaluation of our test scripts, we use 8

industrial device from 5 device classes. We run the

test scripts against the devices and observe their be-

havior. As a result, we observed three crashes and

9 anomalies in the behavior of the industrial device.

Our evaluation shows that similar vulnerabilities oc-

cur in implementations of the same protocol as well as

in different protocols. In addition, we show that sim-

ilar vulnerabilities also spread over different device

implementations of the VAPs in a blackbox set-

ting, and evaluate the test scripts using 8 industrial

devices.

The rest of this paper is structured as follows. Sec-

tion 2 presents related work in the domains of black-

box fuzzing for industrial device, vulnerability scan-

ning, and Anti-Patterns. Our analysis of the recent

vulnerability groups Ripple20, Urgent/11, and Am-

nesia:33 is described in Section 3. The test scripts we

Our work is located in the domain of blackbox

fuzzing for industrial device and also touches the do-

mains of vulnerability scanning, and Anti-Patterns.

framework for blackbox network fuzzing for in-

dustrial device ISuTest is presented by Pfrang et

al. (Pfrang et al., 2017). The included fuzzer gener-

ates new test cases based on heuristics which are used

to determine which values should be used for a packet

field with a certain data type. As a basis for these

heuristics, the authors use knowledge from previous

vulnerability scanning generally is less intrusive. The

aim of vulnerability scanning is to check whether the

DUT contains vulnerable software by identifying the

software running on the DUT. For the vulnerability

groups considered in this work, vulnerability scan-

ners have been developed. These scanners aim to

detect whether the considered DUT includes one of

all three scanners use active fingerprinting to deter-

mine whether the DUT includes one of the vulnerable

stacks. Due to the high impact that one single small

corrupted component can have on a wide range of do-

mains, it is especially challenging to find all vulnera-

ble communication stacks. In addition, fingerprinting

for ICSs has some special challenges (Caselli et al.,

2013). In contrast, our work aims to actively test for

the implementations of VAPs. With this, it has poten-

bility groups consist of 63 vulnerabilities. One of the

vulnerabilities (CVE-2020-11904) is not concerned

Figure 1: Number and type of vulnerabilities present in Rip-

ple20, Urgent/11, and Amnesia:33, sorted by protocol.

with a specific protocol but is a protocol-independent

issue of the Treck stack. This is why we exclude the

vulnerability from our analysis. Another vulnerabil-

ity (CVE-2020-13987) is concerned with TCP as well

as with UDP which we consider as two different vul-

nerabilities for our analysis.

Our analysis shows that more than half of the vul-

nerabilities are caused by a length / offset field, or

the domain name. For both fields, the checks and the

parsing is complex and error prone (see Sections 3.1

and 3.2 for more details). Figure 1 shows how the

vulnerabilities are spread over the different protocols.

The vulnerabilities that have been reported in the vul-

nerability groups are represented by one bar for each

protocol. Within one bar, the vulnerabilities are again

classified by the field the vulnerability occurs in. The

cerned with the domain name. The next most occur-

ring field, id, only occurred 2 times (3.17%).

ing a length / offset field spread over almost all proto-

cols. This already points towards a spreading of simi-

lar issues over different protocols. In contrast, vulner-

abilities concerning the domain name only occurred

in DNS and LLMNR even though domain names are

also used in DHCP, DHCPv6 and ICMPv6.

Since the length / offset and domain name are re-

sponsible for more than 50% of the reported vulnera-

bilities, we will use these field types for the formula-

tion of VAPs. We will then use these Anti-Patterns to

design and develop test scripts in order to find imple-

mentations of these VAPs.

Our analysis shows that 46.03% of the reported vul-

boundary checks (Fetzer and Xiao, 2002) which often

are triggered by unexpected or misfit lengths and off-

sets. The second reason is that there are many lengths

and offsets in the investigated protocols. For exam-

ple, TCP includes six length or offset fields. Each of

these six fields poses a parser the challenge to verify

and parse them correctly.

In addition, protocol standards usually focus on

the parsing of packets that are compliant to the stan-

dard and not all requirements for the lengths and off-

As an example, we want to have a closer look to

the option length field of TCP. A necessary insight

is that a valid option length value cannot be higher

than the length of the remaining data of the header.

The option that is described by the option length

is itself contained in the header and such a value

longer than the remainder of the header would surely

be invalid. In order to check whether the option

four to receive the full length of the data in Bytes,

data. Then, the validity check of the option length

can be conducted by comparing the calculated possi-

necessary checks are for example that (I) the already

parsed data is at least 20 bytes long, since this is the

length of a header without options, and (II) that the

the available data from the IP layer. This small exam-

ple makes apparent that different checks are necessary

that partly depend on different fields of the current

packet.

names are also concerned with length, offsets and ter-

mination. Nevertheless, we handle these issues sepa-

rately. The first reason for this is that DNS is a widely

used protocol and in many cases, DUTs will just parse

any crafted DNS packet. Second, domain names com-

bine many ways to use length and offsets such that it

poses even more challenges to a parser.

The specification of DNS can be found in

RFC 1034 and RFC 1035 (Mockapetris, 1987a;

Mockapetris, 1987b). Domain names are usually

A valid domain name encoding is always terminated

with a length byte of zero, which is the length of the

root domain.

In order to reduce redundancy, DNS supports a

feature called compression. This feature acts like a

pointer and allows to reference a prior list of labels.

With this, it is not necessary to encode the same list

of labels twice.

A full DNS packet contains different sections,

where each has its own lengths, offsets and other spe-

source record specifies the domain name, the type, the

class, and the time to live for a resource. This com-

plexity is reflected by the vulnerabilities found, which

concern out-of-bounds read and write, remote code

execution and denial of service amongst others.

Our analysis indicates that the vulnerabilities reported

by Ripple20, Amnesia:33, and Urgent/11, show sim-

ilarities over different implementations but also over

different protocols. For example, CVE-2020-11912

put validation and is based on a missing sanity check

regarding the IPv6 payload length and the actual

length of the payload. It was reported as part of Am-

nesia:33. Both vulnerabilities are caused by a missing

sanity check regarding the given length and the actual

length of the data. They occur in different protocols

and at different places.

The six VAPs resulting from our analysis are pub-

lished on GitHub

. We use these VAPs and the de-

the one hand that we do not have any insight into the

source code during the development and on the other

hand that the test scripts interact with the DUT only

via the Ethernet interface. In general, the aim of the

test scripts is to provoke anomalies or crashes in the

tested devices.

In order to make our test scripts usable for contin-

uous testing, we designed the test scripts in a way that

they can easily be integrated into the security testing

framework ISuTest (Pfrang et al., 2017). To show the

vulnerability-anti-patterns

mentations of the test scripts are provided upon re-

quest.

Since most of the fields only contain a few bytes, we

are able to test the full input space for these fields.

terface, but for our test scripts, we intentionally want

as a payload to the IP layer. In addition, the IP header

length and the checksum must be set properly.

From the protocols considered for the test scripts,

DHCP and DNS can be categorized as request-

response protocols. That means that no complex state

machine is required to test these protocols. Neverthe-

less, implementing the test scripts for DHCP and DNS

is more challenging than the test scripts for stateless

protocols. Since most industrial device implement the

client of the protocol, the test script needs to repre-

the test script and sent to the DUT.

For the implementation, we encapsulate the mu-

tation of the packet defined by the test script from

the implementation of the protocol server. As a basis

for the implementation of the protocol server, we use

Scapy’s AnsweringMachine. This part implements

the general protocol behavior and is the same for each

test script. The other part, the mutation of the packet,

depends on the test script and interacts with the server

implementation via an interface. For DHCP, we im-

packets. With the interface, we are able to control

each part of the DNS packet. For example, it is possi-

ble to change the length of a label or to set a compres-

sion pointer to an arbitrary value. With this design

and implementation, we can build the test scripts for

the different fields (see Table 2).

Depending on the protocol, there are several ways

to trigger the DUT to generate a request. Regarding

DHCP, one can choose between two reasonable ways.

On the one hand, the DUT usually will generate a

DHCP request after booting. On the other hand, a

DUT will send a new DHCP request after a timeout

which can be set by the DHCP server (Droms, 1993).

the lease time to 20 seconds. As a result, the DUT

seconds, depending on the implementation. This still

is a long time and such the tests need some time to

Regarding DNS, the possibilities to trigger a re-

quest highly depend on the concrete industrial device,

its device type and implemented features. For exam-

ple, one of the DUTs used in our evaluation provides

a dDNS implementation which can be used to trigger

a DNS request on a configurable frequency. A dif-

ferent DUT can be configured to send mails based on

different events. This will trigger a DNS request as

We also consider TCP, which is a highly stateful pro-

tocol. That means that out implementation needs to

generate, keep and cleanup the current state. Fuzzing

state during the tests since the tests may interact with

the state. Each test script needs to establish a spe-

cific state, run the tests and reset the state afterwards.

Some test scripts additionally need to interact with the

general state machine. Depending on the payload of

the TCP packet, the test script also needs to be aware

of the payload’s protocol. For example, it might be

necessary to make a valid HTTP request on top of the

tested TCP packet. Only with this valid payload the

packet will be accepted und further processed by the

The aim of our work is to understand whether simi-

lar vulnerabilities occur in implementations of differ-

ent protocols, and whether blackbox test scripts can

8 industrial device from 5 device classes as a basis.

scripts and analyse the resulting behavior of the in-

dustrial device. The following section describes our

Since our aim is to generally explore the spreading

of vulnerabilities and not to explicitly point out single

vendors, the used industrial device are identified by

pseudonyms. The vulnerabilities we find during our

work are disclosed responsibly and in collaboration

with the vendors.

Before the presentation of the results, we present

our methodology. This includes the formulation of

our hypotheses and the presentation of our evaluation

strategy.

Our evaluation is driven by the following hypotheses:

plementations of the same protocol but also in

different protocols.

test the devices for vulnerabilities. During the whole

process, the industrial device are blackboxes. This

represents the view of an attacker who aims to find

vulnerabilities in industrial device, but also the view

of a test late in the development life cycle. With

this, we are able to evaluate realistically how our ap-

proaches and test scripts work in a blackbox setting.

The devices we choose are not known to be vulner-

able with regard to the analysed vulnerability groups

Ripple20, Urgent/11, and Amnesia:33. This supports

We include 8 industrial device from 5 different de-

printing showed that each of the DUTs uses a unique

of the DUTs even more. In the following, we give a

short overview on the functions and specialities of the

device classes considered in this evaluation (firewall,

controller, gateway, I/O device, and sensor). In order

to follow a responsible disclosure strategy, we will not

include details on the manufacturers of the devices.

We reported the vulnerabilities to the concerned man-

ufacturers, and we will publish the vulnerabilities af-

ter the disclosure process is finished.

Webserver for configuration purposes.

monitor the ICS. Additionally, they can collect and

analyse data from the industrial device. For our eval-

uation, we choose two controllers: Ctl1 and Ctl2.

Ctl1 is a safety controller which means that it is espe-

cially suited for applications with high safety require-

ments. This includes higher requirements regarding

redundancy and reliability.

nect analog and digital actuators and sensors to the

controllers. I/O2 is an I/O device that couples

PROFINET communication from the controllers to

digital or analog signals. I/O1 translates the commu-

nication between PROFINET and I/O Link, and digi-

tal and analog signals.

its measurements using different communication pro-

tocols such as FTP, SNMP, and MQTT. In addition,

Sens1 provides a web application which also shows

as to anomalies in their behavior. We chose the term

finding as an umbrella term for crashes and anomalies.

During our evaluation, we observed 11 findings. Fig-

ure 2 presents the findings sorted by the affected pro-

tocol, and the type of the finding (length/offset or do-

main name). The test scripts provoked findings in

each of the considered protocols except UDP. Most

findings are concerned with IPv4.

The more detailed overview of the results pre-

sented in Table 3 shows that we observed anomalies in

IPv4 and TCP, and crashes in DHCPv4 and DNS. In

quests after two responses with compression pointers

triggered. Based on our observations, we assume that

the NTPd process, which generates regular DNS re-

quests, has crashed. After a reboot, the NTPd works

again as expected.

The test script concerning the option length of

DHCPv4 provoked a full crash of Sens1. If the DHCP

ACK packet does not contain the values expected by

the DUT, the DHCP client as well as the other ser-

vices crash. After a reboot, the DUT and all of its

services are up and running again.

would be the same response as for requests without

hypotheses formulated in Section 5.1.1. In the fol-

lowing, we discuss the implications of our results in