REVS: A Vulnerability Ranking Tool for Enterprise Security

Igor Forain

, Robson de Oliveira Albuquerque

and Rafael Tim

oteo de Sousa J

unior

Professional Program in Electrical Engineering (PPEE), Dept. of Electrical Engineering (ENE),

University of Bras

ılia (UnB), Bras

ılia, Brazil

Keywords:

Cybersecurity, Vulnerabilities, Pentest, NVD, CNVD, TOPSIS.

Abstract:

Information security incidents currently affect organizations worldwide. In 2021, thousands of companies

suffered cyber attacks, resulting in billions of dollars in losses. Most of these events result from known vul-

nerabilities in information assets. However, several heterogeneous databases and sources host information

about those ﬂaws, turning the risk assessment difﬁcult. This paper proposes a Recommender Exploitation-

Vulnerability System (REVS) with the Technique for Order Preference by Similarity to Ideal Solution (TOP-

SIS) to rank vulnerability-exploit. The REVS is a dual tool that can pinpoint the best exploits to pentest or

the most sensitive vulnerabilities to cybersecurity staff. This paper also presents results in the GNS3 emulator

leveraging data from the National Vulnerability Database (NVD), the China National Vulnerability Database

(CNVD), and Vulners. They reveal that the CNVD, despite data issues, has 23,281 vulnerabilities entries

unmapped in the NVD. Moreover, this work establishes criteria to link heterogeneous vulnerability databases.

1 INTRODUCTION

Today, most organizations need to provide services in

computing environments. Beyond the impact on the

healthcare industry worldwide, the COVID-19 epi-

demic has also had a disruptive effect on the way busi-

nesses operate (Ferreira et al., 2021). Once performed

in person, services and processes had to undergo an

almost instantaneous digital re-adaptation to remote

access (Khan et al., 2020).

The digitalization of processes has been underway

since the Internet became ubiquitous. However, it in-

creased the attack surface (Pimenta Rodrigues et al.,

2017; Chowdhary et al., 2020; Gualberto et al., 2020).

The rush to provide services based on Cloud Com-

puting has been accompanied by the introduction of

a series of vulnerabilities in the IT environments of

organizations, especially with the advent of services

based on the Internet of Things (IoT) (Liu et al., 2019;

Thamilarasu and Chawla, 2019).

Although there is a plethora of data regarding in-

formation security, several heterogeneous databases

and sources host those data, turning the risk assess-

ment difﬁcult (Du et al., 2019). Besides, most or-

ganizations still lack effective methods to choose the

https://orcid.org/0000-0002-9965-8077

https://orcid.org/0000-0002-6717-3374

https://orcid.org/0000-0003-1101-3029

best option and path in real scenarios (Bertoglio and

Zorzo, 2017). The National Vulnerability Database

(NVD) from the National Institute of Standards

and Technology (NIST) provides reliable informa-

tion about software and hardware ﬂaws (Mavroei-

dis and Bromander, 2017; Hemberg et al., 2020).

However, there might be a delay between NVD and

other open sources (Rodriguez et al., 2018). For

this reason, new databases like China National Vul-

nerability Databases (CNVD) (CNCERT/CC, 2021),

Metasploit, and Vulners (VULNERS, 2021) can help

achieve better situational awareness of enterprise

risks.

Also, picking the optimal attack action and exploit

in the wild is still an open question (Kanakogi et al.,

2021a). To this end, Vulnerability Assessment (VA)

and Penetration Testing (PT) are essential steps (Shah

and Mehtre, 2015; Yaqoob et al., 2017; Ghanem and

Chen, 2020; Zhou et al., 2021). However, choosing

the most critical and threatening outcomes provided

by those steps is a decision process subjected to sev-

eral constraints.

There are several approaches to solve decision

process issues. They can use from Markov chain to

Deep Learning algorithms (Awiszus and Rosenhahn,

2018). However, the VA is bounded by several at-

tributes from the Common Vulnerability Scoring Sys-

tem (CVSS) (Cheng et al., 2012). So, it becomes

126

Forain, I., Albuquerque, R. and Sousa Júnior, R.

REVS: A Vulnerability Ranking Tool for Enterprise Security.

DOI: 10.5220/0011068600003179

In Proceedings of the 24th International Conference on Enterprise Information Systems (ICEIS 2022) - Volume 2, pages 126-133

ISBN: 978-989-758-569-2; ISSN: 2184-4992

an optimization problem that can leverage operations

research methods like the Multi-Criteria Decision-

Making (MCDM) (Do

c, 2019). The Technique

for Order Preference by Similarity to Ideal Solution

(TOPSIS) is an algorithm of the MCDM family used

for the cybersecurity of 5g networks (Kholidy, 2022),

power control systems (Liu et al., 2010) and Intrusion

Detection System (IDS) (Alharbi et al., 2021)

Based on the described open issues, this work

aims to optimize the vulnerability-exploit choosing

process and integrate new vulnerability databases.

To this end, this work proposes the Recommender

Exploitation-Vulnerability System (REVS) based on

a MCDM approach. It leverages data from the NVD,

CNVD, and Vulners to create a vulnerability-exploit

ranking using the TOPSIS algorithm. REVS is a

dual tool that can pinpoint the best exploits to attack-

ers or the most sensitive vulnerabilities to cyberse-

curity staff. The experimental results reveal that the

CNVD, despite data issues, has 23,281 vulnerabilities

entries unmapped in the NVD. Moreover, this work

establishes criteria to link heterogeneous vulnerabil-

ity databases.

This work is structured as follows. Section 2

presents background and related work. Section 3

describes the proposed architecture. Section 4 uses

GNS3 and open source tools to implement the archi-

tecture. Finally, section 5 concludes this article.

2 LITERATURE REVIEW

This section presents a literature review related to this

work. It presents vulnerabilities and exploits and their

available databases. Besides, it reviews approaches

regarding attack path prediction and pentest automa-

tion. Finally, it shows state-of-the-art recommenda-

tion algorithms applied in the cybersecurity ﬁeld.

2.1 Vulnerabilities and Exploits

Wang and Guo proposed an Ontology for Vulnerabil-

ity Management (OVM) to achieve knowledge repre-

sentation of the Common Vulnerabilities Exposures

(CVE) of the NVD (Wang and Guo, 2009). It was the

ﬁrst attempt to create a knowledge base (KB) of vul-

nerabilities, but it did not consider the exploits possi-

bility, different from this work. GAO et al. created a

taxonomy and ontology for network attack classiﬁca-

tion using description logic (DL) (Gao et al., 2013).

Kanakogi et al. used natural language processing

(NLP) to match CVE to Common Attack Pattern Enu-

merations and Classiﬁcations (CAPEC) (Kanakogi

et al., 2021a; Kanakogi et al., 2021b). These works

guided the CVE data model that REVS uses for vul-

nerability databases.

Householder et al. evaluated a systematic analysis

of the relation between vulnerabilities and exploits.

By the end of 2019, only around 4.1% of the vulner-

abilities exposed after 2013 had an exploit publica-

tion (Hu et al., 2020). It supported this work in giving

more weight to vulnerability features than exploit.

Rodriguez et al. showed that vulnerability disclo-

sure delay between the NVD and other open sources,

e. g. in SecurityFocus, may reach 244 days (Ro-

driguez et al., 2018). Rytel et al. compared sev-

eral databases of vulnerabilities, like the NVD and

CNVD, but only regarding the IoT devices (Rytel

et al., 2020). They presented the necessity of using

more vulnerability databases besides the NVD.

The works presented in this subsection proposed

vulnerability ontologies or evaluated vulnerability

database assessment. Different from them, this work

leverages and integrates those databases for pentest

and security assessment.

2.2 Attack Path and Pentest

Valea and Oprisa proposed pentest automation us-

ing Nmap and Metasploit. Unlike this work, it cov-

ered only vulnerabilities regarding a root shell with

Meterpreter and used decision trees to avoid overﬁt-

ting (Valea and Opris¸a, 2020). REVS leverages the

TOPSIS algorithm. Polatidis et al. (Polatidis et al.,

2017; Polatidis et al., 2020) proposed an attack path

prediction algorithm to achieve an information risk

assessment. Their work used only CVE data from the

MITRE Corporation (MITRE, 2021) on IT maritime

infrastructure to generate the attack graphs. This work

uses more databases with a different optimization ap-

proach.

Huo et al. used the multi-host Multi-Stage Vul-

nerability Analysis (MulVAL) algorithm to generate

the attack tree. It leveraged a Deep Q-Learning Net-

work (DQN) to choose the best attack path based on

the CVSS of the CVE as a reward function. Their

experiment evaluated a small topology without speci-

ﬁcation about the information assets and exploits only

CVE-2012-0053 (Hu et al., 2020). It showed the high

computational cost of deep learning approaches for

extensive networks and sometimes convergence is-

sues. This work uses an operation research method

and improves the integration between Nmap and ex-

ternal data sources.

REVS: A Vulnerability Ranking Tool for Enterprise Security

127

2.3 Recommender Systems

Pawlicki et al. wrote a comprehensive sur-

vey about recommendation systems for Cybersecu-

rity (Pawlicka et al., 2021). It showed to this work

that type of systems as a promising approach to the

vulnerability-exploit recommendation. Polatidis et al.

used a recommendation system with a multi-level col-

laborative ﬁltering method (Polatidis et al., 2018; Po-

latidis and Georgiadis, 2017) to Microsoft Windows.

REVS leverages TOPSIS targeting any platform.

Some works treated the recommendation problem

with an MCDM approach. It relies on minimizing

or maximizing the geometric distance from an ideal

solution like the classical recommendations systems.

Most of them leveraged the TOPSIS for security as-

sessment and Intrusion Detection System (IDS), but

not for pentesting (Kholidy, 2022; Alharbi et al.,

2021; Liu et al., 2010).

Table 1: Comparison with Related Works.

Work Data Sources Algorithm Scope

(Pawlicka et al., 2021) Doesn’t apply Doesn’t apply Survey

(Polatidis et al., 2018) Mitre CVE CF MS Windows

(Polatidis and Georgiadis, 2017) Mitre CVE CF MS Windows

(Kholidy, 2022) NVD, Metasploit TOPSIS 5G networks

(Alharbi et al., 2021) Doesn’t apply TOPSIS IDS attributes

(Liu et al., 2010) Private TOPSIS Power Systems

This work

NVD, CNVD,

Metasploit, Vulners

TOPSIS IPv4, IPv6

Table 1 presents an outline comparison between

the works cited in this subsection and this work. It

shows that it leverages more vulnerability database

sources to a broader network scope.

3 PROPOSED SYSTEM

This section describes the proposed architecture of

the Recommender Exploitation-Vulnerability System

(REVS). Figure 1 presents the system diagram, which

has four main modules.

Four modules comprise the REVS: database,

scanner, matcher, and recommender. The following

subsections describe these four modules.

3.1 Database

The database module comprises two main sets: vul-

nerabilities and exploits. Their function is to provide

numeric features to feed the TOPSIS decision matrix.

REVS gathers the ﬁrst set from two sources: the NVD

and CNVD. The second set comes from Metasploit’s

database. This layer is a batch process to set up the

Recommendation Flow

Database

Data

Gathering

Vulnerabilities

Exploits

Scanner

Network

Probing

Start

CPE

Matcher

Vulnerability

Matrix

Vulnerabilities

Exploits

Decision Matrix

Recommender

TOPSIS Stop

Vulnerability-Exploit

Ranking

Figure 1: REVS Diagram Process.

databases before the Matcher step. REVS uses the

NVD and CNVD in an ensemble, matching registers

with the same CVE number and joining the scores of

the two databases.

3.1.1 NVD

The NIST from the USA supports the NVD. It

is a well-known and de facto authority regard-

ing system vulnerabilities for the research commu-

nity (NIST, 2021). That database comprises several

features: Common Weakness Enumeration (CWE),

Common Platform Enumeration (CPE), vendors, and

ratings (MITRE, 2021). The NVD uses the CVSS to

rate the characteristics and severity of system vulner-

abilities. The CVSS has two versions, v2.0 and v3.0,

with a different range for similar attributes. Table 2

shows the features from CVSS used by REVS.

Table 2 presents the six CVSS features that com-

prise the criteria used by the TOPSIS decision matrix.

3.1.2 CNVD

The National Computer Network Emergency Re-

sponse Technical Team/Coordination Center of China

(CNCERT/CC) sponsors the CNVD. China has been

a global actor in the information security ﬁeld, mak-

ing the CNVD a relevant source of vulnerabili-

ties (CNCERT/CC, 2021). However, it did not pro-

vide an interface for data feeding, nor did it provide

ICEIS 2022 - 24th International Conference on Enterprise Information Systems

128

Table 2: CVSS Features.

CVSSv2 CVSSv3

accessComplexity attackComplexity

accessVector attackVector

availabilityImpact availabilityImpact

conﬁdentialityImpact conﬁdentialityImpact

integrityImpact integrityImpact

authentication privilegesRequired

any documentation. REVS implements a data crawler

to get XML ﬁles from the CNVD. Moreover, it is sup-

posed to use CVSS v2.0.

3.1.3 Metasploit

Metasploit works as a framework integrating recon-

naissance tools, exploits, and payloads in the same

environment. The metasploit source is the Exploit-

DB maintained by the Offensive Security (Security,

2021). Moreover, Metasploit implements only a sub-

set of exploits from Exploit-DB, most of them based

on Ruby programming language. Besides, REVS

is responsible for matching the framework’s exploits

with the gathered vulnerabilities because Metasploit

is not entirely CVE oriented.

3.2 Scanner

REVS implements two scanning methods: network

hosts status and CPE for the target services and OSs.

The former uses the Nmap function to detect the net-

work hosts’ situation (up or down). The latter also

leverages Nmap to identify CPEs of the opening ser-

vices in each up machine. The CPE is the key used by

the Matcher step to search for vulnerabilities and ex-

ploits in the gathered database. The more detailed the

CPE information, the more likely the chance to match

vulnerabilities and exploits for the target.

3.3 Matcher

This module uses the results from the Scanner to

search for security data in the following databases:

Vulnerabilities, Exploits, and Vulners API. The last

is a web service provided by Vulners (VULNERS,

2021) an Information Security Company from Rus-

sia. The other two are databases collected by the ﬁrst

module described in subsection 3.1.

There is an interface to Vulners database em-

bedded in Nmap and coded in Lua programming

language (vulns script). The work of (Valea

and Opris¸a, 2020) leveraged it to carry out the

Metasploit automation. However, REVS requires

more vulnerability features to create the deci-

sion matrix for the TOPSIS algorithm. So,

this work creates a new wrapper to Vulners API

(https://vulners.com/api/v3/burp/software?) through

Python classes.

3.4 Recommender

This work approach chooses the best vulnerability-

exploit pair using a decision-making method. In this

case, REVS uses the TOPSIS algorithm based on

the optimization technique described in (Hwang and

Yoon, 1981; Chen and Hwang, 1992; Opricovic and

Tzeng, 2004). The attack is a one-layer problem of

picking up the best vulnerability-exploit pair, i.e., the

lowest cost and highest impact. So, the decision ma-

trix has m vulnerabilities (rows) compared to n fea-

tures (columns).

A = (a

i j

) i = 1,2,...,m; j = 1,2,...,n (1)

Equation 1 shows that the a

i j

is the jth feature

value of the ith vulnerability. After that, the A is nor-

malized and weighted by columns to its X form:

X = (x

i j

) x

i j

= λ

∗ a

i j

∑

j=1

i j

(2)

The technique requires choosing the best and

worst options from the attacker role.

= (z

,...,z

) (3)

−

= (z

−

,...,z

−

) (4)

It is essential to understand if the column feature

is a beneﬁt or a cost. If it is a beneﬁt, the z

value for

the best option is the maximum column value. Oth-

erwise, it must be the minimum value. Equations 5

e 6 indicate the calculus for beneﬁt and cost features,

respectively.

= max(x

i j

) z

−

= min(x

i j

) (5)

= min(x

i j

) z

−

= max(x

i j

) (6)

Now, there are m vectors with dimension n, which

are the rows of the matrix X. There are also two new

vectors with size n, Z

and Z

−

, which are the ideal

solutions. The TOPSIS requires calculating the Eu-

clidean distance between the m vectors and the ideal

solutions.

REVS: A Vulnerability Ranking Tool for Enterprise Security

129

− X

)

(7)

−

− X

−

)

(8)

Finally, calculate the performance ratio to rank

each one of the vulnerabilities, from highest to lowest.

= d

/(d

−

+ d

), i = 1,2,3,...,m (9)

4 PRELIMINARY RESULTS

REVS is an ongoing project with preliminary results

regarding the following modules: database, scanner,

and matcher. This section presents the test environ-

ment based on the GNS3 and discusses those results.

4.1 Test Environment

This work emulates a medium enterprise Secu-

rity Operation Center (SOC) using Graphical Net-

work Simulator-3 (GNS3). It uses Quick Emulator

(QEMU) on kernel-based virtual machines (KVM) in

Linux Ubuntu 20.04. Furthermore, KVM in Linux

performs better than type 2 hypervisors like Virtual-

Box and VMware because of hardware acceleration

and kernel embedded commands. Figure 2 presents

the test environment.

REVS

Figure 2: Emulated SOC with QEMU/KVN.

The GNS3 makes it possible to build an environ-

ment with different Operating Systems (OSs): Kali

Linux, Ubuntu 18.04, CentOS 8, Windows Server,

and Metasploitable III (Rapid7, 2021) machine. This

last is a vulnerable public VM based on Ubuntu 14.04.

Table 3 lists the environment hardware and VMs.

Table 3: Environment hardware and VMs.

Machine Description

Host Ryzen 7 4800h / 16gb RAM

Emulator GNS3 2.2.28

Metasploitable III Ubuntu 14.04

Firewall pfSense 2.5.2

Routers VyOS 1.1.8

Switches Open vSwitch 2.4.0

4.2 Database Results

This work implements two data scrapers to collect

vulnerabilities data from the NVD and CNVD. It

downloads the two databases simultaneously on Jan-

uary 24, 2022, for a fair comparison. The ﬁrst scraper

uses the REST API supported by NIST, which pro-

vides vulnerability registers with unique CVE id and

CVSS v2 and v3 scores. Besides, some of these reg-

isters also contain CPEs and CWEs related to the

vulnerability. REVS makes use of the OPENCVE

tool (Crocfer, 2020) to retrieve data from the REST

API and store it locally in a PostgreSQL relational

database.

The second scraper requires implementation from

scratch. The CNVD does not provide any API to

return vulnerability data. Otherwise, it provides

a set of XML ﬁles lacking schema deﬁnition with

part of the data scored by CVSS v2. Moreover,

CNCERT/CC generates a new XML ﬁle every Mon-

day 18h:00 (CST) with vulnerability registers from

the past week. Different from the work of (Rytel

et al., 2020), REVS uses the python requests library

with custom “User-Agent” tag and cookie parameters

( jsluid and jsl clearance s) to bypass the CNVD

blocking system. Table 4 presents a summary of the

NVD and CNVD.

Table 4: Summary of the National Databases.

Feature NVD CNVD

Vulnerability 178,906 99,261

Missing Weeks 6 0

Duplicated ID 0 88

Without CVE 0 23,281

Duplicated CVE 0 108

Nonexistent CVE 0 193

Table 4 shows that the CNVD dataset has the fol-

ICEIS 2022 - 24th International Conference on Enterprise Information Systems

130

lowing issues: six missing week ﬁles, 23,281 vulner-

abilities without corresponding CVE, 88 vulnerabili-

ties with duplicated id, 108 CVEs linked to more than

one vulnerability, and 193 entries linked to nonexis-

tent CVEs. They look like malformed CVEs during

string copy from other databases. Moreover, preced-

ing ﬁles are not up-to-date; they started in January

2015, and most of the XML ﬁles have scape charac-

ters. REVS corrected these last issues and stored the

CNVD data in a relational database on PostgreSQL.

Figure 3 compares the NVD and the CNVD regarding

the CVSS, CPE, and external references.

Figure 3: Databases Comparison.

Figure 3 shows that despite being the main respon-

sible for CVE Mitre framework implementation, the

NVD has 6.11% of the entries without any CVSS met-

ric, while the CNVD has 0.33%. Moreover, the NVD

and the CNVD have 6.23% and 0.11% of the entries

without CPE, respectively. This last issue prevents a

comprehensive identiﬁcation of the vulnerable hard-

ware or software. The NVD carries more information

regarding external URL references, with 13.43% of

the entries without this information, while the CNVD

shows 19.15% missing this feature.

This work uses two approaches for exploits gath-

ering: parsing the entire Metasploit source code and

scraping the Exploit-DB website. The former exe-

cutes a string pattern search looking for CVE men-

tions in the Metasploit source code structure. The lat-

ter scrapes the Exploit-DB website to get the relations

between exploits and CVEs. After that, REVS also

stores the results in PostgreSQL. Moreover, it lever-

ages the Vulners API on the ﬂy to search for vulnera-

bilities and exploits.

4.3 Scanner and Matcher Results

Figure 2 shows that the attack scenario uses REVS

outside the topology against the Metasploitable III

VM on the DMZ. In this scenario, REVS runs in

the host machine against the guest VM emulated by

GNS3 using the NVD and the CNVD as vulnerability

database and Vulners as exploit source. It ﬁnds the

results listed in table 5.

Table 5: Vulnerabilities and Exploits per CPE.

CPE NVD CNVD Exploits

proftpd:proftpd:1.3.5 9 8 2

linux:linux kernel 6 1 0

apache:http server:2.4.7 44 38 53

samba:samba 1 1 0

apple:cups:1.7 6 0 1

mysql:mysql 2 0 0

68 48 56

Table 5 shows that most of the vulnerabilities be-

long to the Apache webserver. The table also presents

the same situation about the exploits available in the

Metasploit framework. Table 5 also shows that REVS

found 48 vulnerabilities using the CVE id as the

search key in the CNVD. The 68 vulnerabilities de-

tected by REVS using the NVD include these 48.

The 20 unmatched vulnerabilities in the CNVD

are before 2015. It explains why they are not in

the CNVD. However, a future approach is a new

search for these unmatched results using CPE as

the search key in the CNVD. Furthermore, a sec-

ond ofﬁcial database, the China National Vulnerabil-

ity Database of Information Security (CNNVD), also

requires comparison against NVD and CNVD.

Figure 4 presents the data returned from

REVS-Vulners interface regarding the CPE

proftpd:proftpd:1.3.5.

Figure 4: REVS and Vulners Interface Results.

Figure 4 shows that REVS-Vulners interface re-

turns 9 CVEs and two exploits from Metasploit. Be-

sides, it returns data regarding obsolete zero-days and

forum messages.

REVS: A Vulnerability Ranking Tool for Enterprise Security

131

5 CONCLUSIONS

The CNVD has several protection mechanisms to pre-

vent downloads despite being a public database. Also,

there are missing data ﬁles and 23,281 vulnerabilities

without CVE mapping. It indicates process issues or

the existence of vulnerabilities known only to the Chi-

nese community because most of the text data are in

mandarin.

REVS is an ongoing project that is working on the

results of the Recommender module. It has already

downloaded and normalized the NVD and CNVD

databases. Furthermore, REVS integrated those two

national vulnerability databases, Nmap and Vulners,

using CPE and CVE as the search keys. The vul-

nerability assessment against the VM behind a SOC

emulated in GNS3 showed the NVD as a more com-

prehensive database than the CNVD.

As Future work, the authors suggest using the

TOPSIS fuzzy version with attack paths calculated by

the MULVAL algorithm in the recommender module.

The batch translation to English of the already down-

loaded database and an automatic one for new data

are necessary improvements for REVS. This research

will go further into integrating with the CNNVD and

NLP approaches to vulnerability search.

ACKNOWLEDGEMENTS

The authors acknowledge the support of ABIN grant

08/2019. R.d.O.A. gratefully acknowledges the Gen-

eral Attorney of the Union - AGU grant 697.935/2019

and the General Attorney’s Ofﬁce for the Na-

tional Treasure - PGFN grant 23106.148934/2019-

67. R.T.S.J. gratefully acknowledges the support from

EC Horizon 2020 HEROES project grant 101021801,

CNPq grants 465741/2014-2 and 312180/2019-5, the

Administrative Council for Economic Defense grant

08700.000047/2019-14, and the National Auditing

Department of the Brazilian Health System SUS

(Grant 23106.118410/2020-85).

REFERENCES

Alharbi, A., Seh, A. H., Alosaimi, W., Alyami, H., Agrawal,

A., Kumar, R., and Khan, R. A. (2021). Analyzing the

impact of cyber security related attributes for intrusion

detection systems. Sustainability, 13(22):12337.

Awiszus, M. and Rosenhahn, B. (2018). Markov chain neu-

ral networks. In Proceedings of the IEEE Conference

on Computer Vision and Pattern Recognition Work-

shops, pages 2180–2187.

Bertoglio, D. D. and Zorzo, A. F. (2017). Overview and

open issues on penetration test. Journal of the Brazil-

ian Computer Society, 23(1):1–16.

Chen, S.-J. and Hwang, C.-L. (1992). Fuzzy multiple at-

tribute decision making methods. Fuzzy multiple at-

tribute decision making, pages 289–486.

Cheng, P., Wang, L., Jajodia, S., and Singhal, A. (2012).

Aggregating cvss base scores for semantics-rich net-

work security metrics. In 2012 IEEE 31st Symposium

on Reliable Distributed Systems, pages 31–40. IEEE.

Chowdhary, A., Huang, D., Mahendran, J. S., Romo, D.,

Deng, Y., and Sabur, A. (2020). Autonomous security

analysis and penetration testing. In 2020 16th Interna-

tional Conference on Mobility, Sensing and Network-

ing (MSN), pages 508–515. IEEE.

CNCERT/CC (2021). China National

Vulnerability Database (CNVD).

https://www.cnvd.org.cn/shareData/list. [Online;

accessed 24-January-2022].

Crocfer, N. (2020). CVE Alerting Platform.

https://github.com/opencve/opencve. [Online;

accessed 06-December-2021].

c, S. (2019). Multi-criteria decision making methods:

Application in the aviation industry. Journal of Air

Transport Management, 79:101683.

Du, G., Long, C., Yu, J., Wan, W., Zhao, J., and Wei, J.

(2019). A real-time big data framework for network

security situation monitoring. In ICEIS (1), pages

167–175.

Ferreira, A., Cruz-Correia, R., et al. (2021). Covid-19

and cybersecurity: ﬁnally, an opportunity to disrupt?

Jmirx med, 2(2):e21069.

Gao, J.-b., Zhang, B.-w., Chen, X.-h., and Luo, Z. (2013).

Ontology-based model of network and computer at-

tacks for security assessment. Journal of Shanghai

Jiaotong University (Science), 18(5):554–562.

Ghanem, M. C. and Chen, T. M. (2020). Reinforcement

learning for efﬁcient network penetration testing. In-

formation, 11(1):6.

Gualberto, E. S., De Sousa, R. T., Thiago, P. D. B.,

Da Costa, J. P. C., and Duque, C. G. (2020). From

feature engineering and topics models to enhanced

prediction rates in phishing detection. Ieee Access,

8:76368–76385.

Hemberg, E., Kelly, J., Shlapentokh-Rothman, M., Rein-

stadler, B., Xu, K., Rutar, N., and O’Reilly, U.-M.

(2020). Linking threat tactics, techniques, and pat-

terns with defensive weaknesses, vulnerabilities and

affected platform conﬁgurations for cyber hunting.

arXiv preprint arXiv:2010.00533.

Hu, Z., Beuran, R., and Tan, Y. (2020). Automated pene-

tration testing using deep reinforcement learning. In

2020 IEEE European Symposium on Security and Pri-

vacy Workshops (EuroS&PW), pages 2–10. IEEE.

Hwang, C.-L. and Yoon, K. (1981). Multiple attribute deci-

sion making: a state of the art survey. Lecture Notes

in Economics and Mathematical Systems, 186(1).

Kanakogi, K., Washizaki, H., Fukazawa, Y., Ogata, S.,

Okubo, T., Kato, T., Kanuka, H., Hazeyama, A., and

Yoshioka, N. (2021a). Tracing capec attack patterns

ICEIS 2022 - 24th International Conference on Enterprise Information Systems

132

from cve vulnerability information using natural lan-

guage processing technique. In Proceedings of the

54th Hawaii International Conference on System Sci-

ences, page 6996.

Kanakogi, K., Washizaki, H., Fukazawa, Y., Ogata, S.,

Okubo, T., Kato, T., Kanuka, H., Hazeyama, A., and

Yoshioka, N. (2021b). Tracing cve vulnerability infor-

mation to capec attack patterns using natural language

processing techniques. Information, 12(8):298.

Khan, N. A., Brohi, S. N., and Zaman, N. (2020). Ten

deadly cyber security threats amid covid-19 pan-

demic.

Kholidy, H. A. (2022). Multi-layer attack graph analysis in

the 5g edge network using a dynamic hexagonal fuzzy

method. Sensors, 22(1):9.

Liu, N., Zhang, J., Zhang, H., and Liu, W. (2010). Secu-

rity assessment for communication networks of power

control systems using attack graph and mcdm. IEEE

Transactions on Power Delivery, 25(3):1492–1500.

Liu, X., Qian, C., Hatcher, W. G., Xu, H., Liao, W., and Yu,

W. (2019). Secure internet of things (iot)-based smart-

world critical infrastructures: Survey, case study and

research opportunities. IEEE Access, 7:79523–79544.

Mavroeidis, V. and Bromander, S. (2017). Cyber threat in-

telligence model: an evaluation of taxonomies, shar-

ing standards, and ontologies within cyber threat intel-

ligence. In 2017 European Intelligence and Security

Informatics Conference (EISIC), pages 91–98. IEEE.

MITRE (2021). The MITRE Corporation.

https://cve.mitre.org/. [Online; accessed 06-

December-2021].

NIST (2021). National Vulerability Database (NVD).

https://nvd.nist.gov/vuln/data-feeds. [Online; ac-

cessed 24-January-2022].

Opricovic, S. and Tzeng, G.-H. (2004). Compromise so-

lution by mcdm methods: A comparative analysis of

vikor and topsis. European journal of operational re-

search, 156(2):445–455.

Pawlicka, A., Pawlicki, M., Kozik, R., and Chora

s, R. S.

(2021). A systematic review of recommender sys-

tems and their applications in cybersecurity. Sensors,

21(15):5248.

Pimenta Rodrigues, G. A., de Oliveira Albuquerque, R.,

Gomes de Deus, F. E., De Oliveira J

unior, G. A.,

Garc

ıa Villalba, L. J., Kim, T.-H., et al. (2017). Cyber-

security and network forensics: Analysis of malicious

trafﬁc towards a honeynet with deep packet inspec-

tion. Applied Sciences, 7(10):1082.

Polatidis, N. and Georgiadis, C. K. (2017). A dy-

namic multi-level collaborative ﬁltering method for

improved recommendations. Computer Standards &

Interfaces, 51:14–21.

Polatidis, N., Pavlidis, M., and Mouratidis, H. (2018).

Cyber-attack path discovery in a dynamic supply

chain maritime risk management system. Computer

Standards & Interfaces, 56:74–82.

Polatidis, N., Pimenidis, E., Pavlidis, M., and Mouratidis,

H. (2017). Recommender systems meeting security:

From product recommendation to cyber-attack pre-

diction. In International Conference on Engineer-

ing Applications of Neural Networks, pages 508–519.

Springer.

Polatidis, N., Pimenidis, E., Pavlidis, M., Papastergiou, S.,

and Mouratidis, H. (2020). From product recommen-

dation to cyber-attack prediction: generating attack

graphs and predicting future attacks. Evolving Sys-

tems, 11(3):479–490.

Rapid7 (2021). The Metasploitable 3 Linux VM.

https://github.com/rapid7/metasploitable3. [Online;

accessed 10-January-2022].

Rodriguez, L. G. A., Trazzi, J. S., Fossaluza, V., Campiolo,

R., and Batista, D. M. (2018). Analysis of vulnera-

bility disclosure delays from the national vulnerabil-

ity database. In Anais do I Workshop de Seguranc¸a

Cibern

etica em Dispositivos Conectados. SBC.

Rytel, M., Felkner, A., and Janiszewski, M. (2020). To-

wards a safer internet of things—a survey of iot vul-

nerability data sources. Sensors, 20(21):5969.

Security, O. (2021). The Exploit Database.

https://www.exploit-db.com/. [Online; accessed

10-January-2022].

Shah, S. and Mehtre, B. M. (2015). An overview of

vulnerability assessment and penetration testing tech-

niques. Journal of Computer Virology and Hacking

Techniques, 11(1):27–49.

Thamilarasu, G. and Chawla, S. (2019). Towards deep-

learning-driven intrusion detection for the internet of

things. Sensors, 19(9):1977.

Valea, O. and Opris¸a, C. (2020). Towards pentesting au-

tomation using the metasploit framework. In 2020

IEEE 16th International Conference on Intelligent

Computer Communication and Processing (ICCP),

pages 171–178. IEEE.

VULNERS, I. (2021). Vulners, inc. https://vulners.com/.

[Online; accessed 10-January-2022].

Wang, J. A. and Guo, M. (2009). Ovm: an ontology for

vulnerability management. In Proceedings of the 5th

Annual Workshop on Cyber Security and Information

Intelligence Research: Cyber Security and Informa-

tion Intelligence Challenges and Strategies, pages 1–

Yaqoob, I., Hussain, S. A., Mamoon, S., Naseer, N., Akram,

J., and ur Rehman, A. (2017). Penetration test-

ing and vulnerability assessment. Journal of Net-

work Communications and Emerging Technologies

(JNCET) www. jncet. org, 7(8).

Zhou, S., Liu, J., Hou, D., Zhong, X., and Zhang, Y. (2021).

Autonomous penetration testing based on improved

deep q-network. Applied Sciences, 11(19):8823.

REVS: A Vulnerability Ranking Tool for Enterprise Security

133