Towards Rigorous Foundations for Metaheuristic Research

Thimershen Achary

1,2 a

, Anban W. Pillay

1,2 b

and Edgar Jembere

1,2 c

University of KwaZulu-Natal, Westville 4001, South Africa

Centre for AI Research (CAIR), South Africa

Keywords: Metaheuristics, Component-based View, Definition, Rigor.

Abstract: Several authors have recently pointed to a crisis within the metaheuristic research field, particularly the

proliferation of metaphor-inspired metaheuristics. Common problems identified include using non-standard

terminology, poor experimental practices, and, most importantly, the introduction of purportedly new

algorithms that are only superficially different from existing ones. In this paper, we argue that although

metaphors may be good sources of inspiration and creativity, being the only reason for publication is

insufficient. Instead, adopting a formal, mathematically sound representation of metaheuristics is a valuable

path to follow. We believe this will lead to more insightful research.

1 INTRODUCTION

The recent past has seen an increase in research that

is critical of numerous trends and practices observed

in the field of metaheuristics (Aranha et al., 2021;

Fister jr et al., 2016; Molina et al., 2020; Sörensen,

2015; Stegherr et al., 2020; Tzanetos & Dounias,

2021). An influential study by (Sörensen, 2015)

points out several broad issues, including

irresponsible metaphor usage, poor experimental

practices, and misconceptions of what a metaheuristic

is.

Others have lamented the poor quality and lack of

rigor and insights in published works (see

https://github.com/fcampelo/EC-Bestiary).

According to (Campelo & Aranha, 2021; Fister Jr et

al., 2016; Sörensen, 2015), this has severe

consequences for productivity, the credibility of the

field, and the capability to stimulate new, valuable

insights effectively.

In this paper, we review the issues raised by

various researchers, consider proposed solutions, and

argue that metaheuristic studies should adopt a

mathematically formulated metaheuristic definition

where the underlying philosophy is mindful of the

issues affecting the metaheuristic field. We also agree

with recent sentiments that metaphors are useful to

https://orcid.org/0000-0002-6033-7065

https://orcid.org/0000-0001-7160-6972

https://orcid.org/0000-0003-1776-1925

inspire creativity but are insufficient on their own. We

then propose a mindful and rigorous core

understanding of metaheuristics.

1.1 Metaheuristics

The term 'meta-heuristic' was coined by Glover in

(Glover, 1986), where the authors suggested that

Tabu Search could be viewed as a metaheuristic

"superimposed" on another heuristic. The suggestion

is that metaheuristics operate on a higher level than

heuristics.

Early definitions of the term metaheuristic were

critically analyzed in (Voß, 2001). These definitions

generally suggest that a metaheuristic is a higher-

level strategy that guides subordinate heuristics, with

some auxiliary constituents such as information for

the guiding process and intelligent combinations of

various exploration and exploitation concepts.

The meta-level is described as dealing with

applying control and strategy to a given domain

(Ostrowski & Schleis, 2008). In the context of

heuristics being the domain, metaheuristics can then

be defined as entities that apply control and strategy

to heuristics, as depicted in Figure 1. Metaheuristics

consists of a base plan, an integrated learning

component, and strategic heuristics. The base plan

Achary, T., Pillay, A. and Jembere, E.

Towards Rigorous Foundations for Metaheuristic Research.

DOI: 10.5220/0011552600003332

In Proceedings of the 14th International Joint Conference on Computational Intelligence (IJCCI 2022), pages 151-157

ISBN: 978-989-758-611-8; ISSN: 2184-3236

151

and integrated learning component are utilized only

in hyper-parameter tuning, while strategic heuristics

are required for all metaheuristic activities. The

strategic heuristics apply control and strategy to the

heuristics.

Figure 1: Metarules framework (Ostrowski & Schleis, 2008).

Metaheuristics are described in (Sörensen & Glover,

2013) as frameworks that can be used to derive

heuristic optimization algorithms and notes that, in

literature, the frameworks and the heuristic

optimization algorithms are both referred to as

metaheuristics. An elaboration of why the distinction

between framework and algorithm is essential when

discussing novelty can be found in (Lones, 2020); it

can be inferred that, often, a novelty at the algorithm

level is hardly a significant feat.

The rest of the paper is structured as follows: a

review of several criticisms of the field is given in

Section 2. Section 3 briefly reviews potential

solutions to these problems discussed in literature. A

proposal for instilling rigor in the metaheuristics

research space is given in Section 4. This is discussed

in Section 5, and Section 6 concludes.

2 METAHEURISTICS AND ITS

DISCONTENTS

Several authors have recently pointed to problems

afflicting metaheuristic research. This section

summarizes these issues.

Irresponsible metaphor usage, in the

metaheuristic field, is the use of sources of

inspiration, e.g., nature, physics, and human behavior,

to be the most, if not the only, pivotal aspect to justify

the algorithm as a "new" metaheuristic to the field

(Aranha et al., 2021; Sörensen, 2015). These works

usually include practices that obscure details by using

non-standard terminology (terminology specific to

the metaphor/inspiration used). Doing so adds to the

challenge of positioning the proposed contribution in

literature and may give the impression that the

research output is novel. Symptoms of this activity

are, according to (Aranha et al., 2021; de Armas et al.,

2021; Molina et al., 2020; Sörensen, 2015; Tzanetos

& Dounias, 2021), a flood of metaheuristics,

numerous cases of very similar/overlapping work,

lack of novelty, and according to (Molina et al., 2020)

instances where inspirational source and algorithm

behaviour are disconnected.

Researchers have also pointed to poor

experimental practices. Reports such as (Aranha et

al., 2021; Sörensen, 2015; Stegherr et al., 2020;

Tzanetos & Dounias, 2021) suggest unfair and biased

comparisons such as comparing new proposals to

older metaheuristics instead of state-of-the-art and

tweaking hyperparameters in favor of a metaheuristic

to lift its performance above the rest.

Comparative studies are not transparent enough,

resulting in difficulties in extending past studies and

existing data (Aranha et al., 2021; Sörensen, 2015). A

lack of proper motivation for selecting metaheuristics

to compare is common (Stegherr et al., 2020). There

is also a lack of rigorous data analytics (Sörensen,

2015). Competitive studies produce very little insight

and do not answer or aid in answering the how and

why (Birattari et al., 2003; Hooker, 1995), yet

comparative studies are still widely setup as

competitive ones (Campelo & Aranha, 2021;

Sörensen, 2015).

The proliferation of metaphor-inspired

metaheuristics is also a cause for concern. A GitHub

project called the Evolutionary Computational

Bestiary lists a vast and ever-growing number of bio-

inspired metaheuristics (with only a few exceptional

bio-inspired metaheuristics being exempt) (Campelo

& Aranha, 2021). The aforementioned project

opposes the flood of metaheuristics, especially the

creation of new bio-inspired metaheuristics. Articles

and other projects that criticize certain metaheuristic

research trends are listed, some of which are intended

to parody or ridicule the fact that these trends still

exist.

The above criticisms have not been universally

accepted. One such counter-argument is that

metaheuristics are currently being applied in various

domains from numerous disciplines and have also

been applied to real-world problems (Torres-Jiménez

& Pavón, 2014). The view that metaheuristic research

is of poor quality may very well be overly pessimistic

and aims to make capital out of flaws in research

techniques that are merely pragmatic. The pursuit of

ECTA 2022 - 14th International Conference on Evolutionary Computation Theory and Applications

152

being theoretically optimal has little benefit to the real

world.

Also, the argument goes, there is a long history of

using nature to inspire the development of

metaheuristic algorithms. Thus, to reject work that

uses natural inspiration is to hinder creativity. The

researcher pool has a diverse skill set, i.e., not all

possess an advanced mathematical background, and

researchers have skills/talents which may lie more in

creativity than analytics. Therefore, the move to

abandon natural inspiration or inspirational sources,

in general, can be interpreted as a move to

discriminate against researchers that are more

creative than analytical.

To refute these arguments, we refer to a study by

(Ven & Johnson, 2006) that explores the relationship

between scholarly and practical knowledge. It

analyses ways in which the discrepancies between

these domains have been framed and discusses

methods to address this, such as a method of engaged

scholarship (proposed by the aforementioned study).

From the study, it can be understood that practical and

scholarly knowledge have different contexts and

objectives. Practical knowledge deals with specific

circumstances in certain scenarios, while scholarly

knowledge deals with viewing specific circumstances

as instances of a more general case to further

understand and explain how what is done works.

Reaping both benefits can be achieved through

methods of communication between both spaces.

This entails that the scholarly domain must be robust

so that new knowledge can be framed efficiently

amongst existing knowledge and communicated

effectively to practical domains and other scholarly

domains.

Recent studies have shown instances of scholarly

work claiming to be novel, but the novelty does not

stand up to scrutiny. Comparative studies have been

questioned regarding their transparency and choice of

experimental practices. The overloading of well-

known concepts with non-standard terminology is

creating confusion in literature. In summary, the

issues highlighted by several publications are

indicators that the metaheuristic research space falls

extremely short of ideal conditions for a scholarly

domain.

According to (Swan et al., 2015), expressing

metaheuristics via mathematical formulations

facilitates a rigorous definition of the term

metaheuristic. Some may criticize and label this

decision as systematically marginalizing creative

research because mathematical definitions often use

cryptic notation that may not be friendly to

researchers without an advanced mathematical

background and with a different skill set. However,

the benefits of using mathematical formulations

(more specifically, functional descriptions), as listed

and discussed in (Swan et al., 2015), include

promoting better communicability, reproducibility,

interoperability, facilitating automated metaheuristic

assembly, and promoting scientific advancement.

Therefore, using mathematical formulations does not

marginalize creative research; instead, it guides

creativity.

The No Free Lunch theorem (Wolpert &

Macready, 1997) being a valid premise in the

argument for justifying the existence of a vast number

of metaheuristics in the research space, is viewed as

unclear in (Lones, 2020). The study also speculates

that the argument may have substance as the

performance of different optimizers varies when

subjected to different problems. However, a

discussion is presented in (Camacho‐Villalón et al.,

2022) that criticizes the aforementioned argument as

being based on a misunderstanding of the No Free

Lunch theorem for optimization and that the vast

number of published metaheuristics based on

metaphors are creating confusion in the research

space, leading it away from proper scientific goals.

Therefore, relying on the No Free Lunch theorem is

not advisable to support the creation of a novel

metaheuristic.

3 A REVIEW OF POTENTIAL

SOLUTIONS

Several authors have not only given critical

commentary on the field but have also suggested

potential solutions.

The solutions to the metaheuristic research

quality issues require adoption by researchers so that

their impact, as argued in the respective research

publications, may influence the metaheuristic

research space. Increasing awareness about issues

associated with metaphor-based research is therefore

essential to stimulate the adoption of these solutions

(Campelo & Aranha, 2021), and it is a recurring

theme in many such publications, e.g., (Lones, 2020;

Sörensen, 2015; Stegherr et al., 2020; Tzanetos &

Dounias, 2021). Projects such as the Evolutionary

Computational Bestiary are also ways to raise

awareness.

A component-based view of metaheuristics, as a

solution to the issues afflicting the metaheuristic

research space, is highlighted in (Sörensen, 2015).

This view suggests understanding metaheuristics as

sets of general concepts, accompanied by the decision

Towards Rigorous Foundations for Metaheuristic Research

153

to distinguish metaheuristics from the optimization

algorithms derived from them. Its widespread

adoption may help resolve several of the problems

discussed above. The component-based view of

metaheuristics deals with conceptualization at the

foundational layer, i.e., where definitions,

taxonomies, ontologies etc., are crucial.

Applying mathematical formulations to express

metaheuristics facilitates a rigorous definition of the

term metaheuristic (Swan et al., 2015). Several

definitions of the term metaheuristic incorporate

tuples. Tuples encapsulate the specifications, main

components, and sometimes structures that hold the

relationships between the specifications and

components.

The study by (Wang, 2010) provides worded

definitions for the terms metaheuristic and

metaheuristic computing. The study provides a

rigorous definition of metaheuristic computing using

tuples, in which the elements are concept algebra

structures.

A tuple definition for population-based

metaheuristics is presented as part of the unified

framework for population-based metaheuristics

introduced in (Liu et al., 2011).

The work done in (Cruz-Duarte et al., 2020)

defines a metaheuristic as a map (expressible in terms

of three components: initializer, search operator, and

finalizer heuristics) from an arbitrary domain to a

feasible domain of an optimization problem.

As part of the proposed design of a software

framework to solve combinatorial optimization

problems presented in (Peres & Castelli, 2021), a

metaheuristic – actually an abstract metaheuristic – is

defined as a map from a domain of specifications

(encapsulated in a tuple) to a set of possible variations

of the metaheuristic.

Swan et al. (Swan et al., 2015) advocate for

metaheuristics to be described entirely in terms of

functions (which are essentially maps), in which

metaheuristics are parameterized by their

environment, state, and the environments of the

employed components. The environment, in this

sense, refers to information required during

execution, and the state refers to the solution in

chosen representation form. The component

heuristics are also parameterized with their

environment and state.

The component-based view proposed by

(Sörensen, 2015) is meritorious but has drawbacks if

not used properly (Achary & Pillay, 2022).

Definitions such as those presented by (Cruz-Duarte

et al., 2020; Liu et al., 2011) express metaheuristics

in terms of components, but as emphasized above, the

ambiguity present in the definitions by (Cruz-Duarte

et al., 2020) may lead to conflicting understandings.

The definition by (Liu et al., 2011) uses biological

terminology and thereby promotes the metaphor-

based philosophy of metaheuristics. However,

metaphor usage, non-standard terminology, and

natural inspiration have been criticized in literature,

indicating that the perspective used may nullify the

long-term advantages of using the component-based

view.

The framework proposed in (Peres & Castelli,

2021) resolves this ambiguity by providing

mathematically formulated definitions of conceptual-

level and concrete-level metaheuristics. Both are

formulated as maps. The former maps from a tuple of

abstract specifications to a set of concrete heuristic

optimization algorithms, and the concrete heuristic

optimization algorithms map from their concrete

specifications to an optimal solution.

4 A PROPOSED SOLUTION

4.1 Towards a Rigorous Foundation

for Metaheuristic Research

Conducting meaningful metaheuristic research for

both the long and short term requires metaheuristic

research to adopt strong foundations and a rigorous

core.

The study by (Campelo & Aranha, 2021)

summarizes some promising alternative approaches

to conducting research in metaheuristics rather than

relying on metaphor-based techniques. They propose

understanding metaheuristics as frameworks of semi-

independent modules that influence one or more

intrinsic algorithmic structures. This is similar to the

proposal made in (Sörensen, 2015) to see

metaheuristics as frameworks and not concrete

heuristic optimization algorithms. Defining

metaheuristics as functions is advocated in (Swan et

al., 2015), which also suggested a specific template

for expressing these functions. Describing metaphor-

based metaheuristics using standard terminology that

effectively describes similarities and differences

between metaheuristics is motivated in (Lones,

2020). Comparing metaheuristics with structure-wise

similarity metrics, which facilitates determining

special-case and general-case relationships between

metaheuristics, is made possible by the work in (de

Armas et al., 2021). Using existing taxonomies from

literature rigorously is facilitated by work done in

(Achary & Pillay, 2022).

Each of the above contributions has little overlap

and a strict scope. Using these contributions together

ECTA 2022 - 14th International Conference on Evolutionary Computation Theory and Applications

154

may be effective for establishing strong foundations

for metaheuristic research and stimulating good

quality, insightful research.

A rigorous foundation for metaheuristic research

that makes use of the contributions, advice, and

guidelines of existing literature is proposed below.

A philosophy of metaheuristics that is mindful of

the issues affecting the field is provided by (Sörensen

& Glover, 2013) and further explained in (Sörensen,

2015). In this view, metaheuristics are problem-

independent frameworks that provide a set of

guidelines to create heuristic optimization algorithms

and are not the heuristic optimization algorithms

themselves.

Mathematical definitions are known to be

rigorous, and there are also added benefits to

expressing metaheuristics as functions (Swan et al.,

2015). Metaheuristics could be formulated as:

𝑀: 𝑆 →

𝐴

(1)

Where M is an arbitrary metaheuristic and S is a

set of tuples of specifications. The metaheuristic M

has an influence on the tuple format, and a tuple of

the set S must contain at least one heuristic operator.

A is the set of heuristic optimization algorithms, each

of which the rules of M can construct using a certain

element of S. A proof-of-concept for the formulation

in (1) can be found in Section 4.2.

The format and values of the tuples in the set S

may be determined using the works of (Lones, 2020)

and (Achary & Pillay, 2022). The novelty and

influence of metaheuristics can be determined by

applying the work of (de Armas et al., 2021) to

metaheuristics defined in terms of (1).

This map formulation (1) aligns with the

component-based view, as it guides the researcher to

elucidate which components are variable in the

specification tuple, thus providing scope for

experiments in future research.

The restriction that an element of S must contain

at least one heuristic operator enforces the

component-based view and avoids scenarios where

hyper-parameter values are the only elements of a

specification tuple.

This map is very abstract and does not have many

restrictions on how one may specialize it with details.

Its intended use is to be a rigorous underlying

conceptualization of what a metaheuristic is when

proposing a concrete formulated definition for future

research; this underlying conceptualization enforces

alignment with the component-based view and

considers the insights, advice, suggestions, and

guidelines from existing literature on the problems

within the metaheuristic field.

4.2 Proof of Concept

The Genetic Algorithm (GA), Particle Swarm

Optimization (PSO), Bat Algorithm (BAT), and

Differential Evolution (DE) metaheuristics are used

to illustrate how the formulation in (1) could be used;

a description of each of the aforementioned

metaheuristics can be found in (Yang, 2020).

4.2.1 Genetic Algorithm

1. Substitute GA in place of M.

2. An element of S would then contain the initializer,

crossover operator, mutation operator, selector,

and terminating condition.

3. An element of A will be a resulting concrete

Genetic Algorithm.

4.2.2 Particle Swarm Optimization

1. Substitute PSO in place of M.

2. An element of S would then contain the initializer,

location update, velocity update, and terminating

condition.

3. An element of A will be a resulting concrete

Particle Swarm Optimization algorithm.

4.2.3 Bat Algorithm

1. Substitute BAT in place of M.

2. An element of S would then contain the initializer,

position update, velocity update, local search

technique, and terminating condition.

3. An element of A will be a resulting concrete Bat

Algorithm.

4.2.4 Differential Evolution

1. Substitute DE in place of M.

2. An element of S would then contain the initializer,

crossover operator, mutation operator, and

terminating condition.

3. An element of A will be a resulting concrete

Differential Evolution algorithm.

5 DISCUSSION

The definition of metaheuristics adopted by a

researcher will significantly influence their

metaheuristic research.

A contributing factor to the proliferation of novel

metaheuristics is arguably the ambiguity of whether

metaheuristics are frameworks, concrete heuristic

Towards Rigorous Foundations for Metaheuristic Research

155

optimization algorithms, or both. The study in

(Sörensen, 2015) remarks that it is unfortunate that

the term "metaheuristic" is used for both general,

problem-independent, algorithmic frameworks and

concrete heuristic optimization algorithms derived

from these frameworks and further expresses that

metaheuristics are not algorithms, but they are each a

set of ideas, concepts, and operators from which

heuristic optimization algorithms can be derived.

The definitions presented in (Cruz-Duarte et al.,

2020; Liu et al., 2011; Voß, 2001; Wang, 2010) fail

to resolve this ambiguity. Difficulty in determining

the novelty of new proposals may result from this

ambiguity since a heuristic optimization algorithm

could be related to a few or many concepts, ideas, or

operators of a framework, garnished with a metaphor

and non-standard terminology, then published as a

novel metaheuristic.

Comparative studies of metaheuristics have

received criticism in the literature (Aranha et al.,

2021; Sörensen, 2015). A flaw that has been

highlighted is that the implementations of

metaheuristics, whose selections are poorly

motivated (Stegherr et al., 2020), are compared, and

the results could be misunderstood as representative

of the framework.

Using metaphors and natural sources of

inspiration has led to the creation of well-known,

influential, and disruptive contributions such as

Particle Swarm Optimization, Genetic Algorithm,

Simulated Annealing, and Ant Colony Optimization,

as indicated in (Camacho‐Villalón et al., 2022).

However, the incorporation of natural inspiration in

research must outweigh the cost.

Research into the trends of metaphor and

inspirational source usage (Aranha et al., 2021;

Campelo & Aranha, 2021; Fister jr et al., 2016;

Sörensen, 2015; Tzanetos & Dounias, 2021) has

shown that metaphors and non-standard terminology

introduce challenges when trying to frame

metaheuristics amongst existing literature. It

facilitates work similar to existing literature to be

published as novel work. Non-standard terminology

confuses readers and clouds the relevance and the link

of the phenomenon described by the terms to the

metaheuristic.

A flood of metaheuristics has been linked to

metaphor and inspiration source usage. Research by

(Molina et al., 2020) showed that there are many more

inspiration sources than algorithmic behaviors.

Hence, it can be said that inspiration source usage is

a heuristic, in the general sense, for creativity, similar

to the exploration of ideas. However, there is too

much exploration and not enough rigor. Since

metaphor/inspiration usage enhances creativity, it is

insufficient on its own; this analogy is similar to those

used in (Fister jr et al., 2016; Lones, 2020) with the

similar computational optimization terminology

Although various publications argue that new

novel metaheuristics are not needed at this point in

the field's timeline, if a metaheuristic is to be

published, it should be accompanied by a formulation

of the metaheuristic in the format of (1). M represents

the abstract pseudocode, ideas, and concepts that

make up the metaheuristic. The format of elements of

S will convey which components are variable, i.e.,

different concrete components can be substituted in

their respective placeholders, which is then passed to

M to create a concrete optimization algorithm of the

set denoted by A in the formulation.

6 CONCLUSION

In this study, it is argued that metaheuristics studies

should adopt a mathematically formulated

metaheuristic definition where the underlying

philosophy is mindful of the issues affecting the

metaheuristic field; in other words, adopt definitions

that sustain good quality research. Mathematical

formulated definitions are rigorous and leave less

room for vagueness that can lead to convenient

interpretations. Ambiguities in adopted or proposed

definitions can potentially allow choosing a

definition/perspective/interpretation of the shelf that

suits a requirement for publication, leading to low-

quality research. The underlying philosophy of the

mathematically formulated definition must be

mindful of issues affecting metaheuristic research to

prevent the definition from having the potential to

stimulate problematic trends.

This work takes the stance that inspiration source

usage is a good heuristic for creativity but is not

needed right now; it has the capacity to become

saturated, which is detrimental to the field.

Intensifying research on existing work would be a

better practice at present.

Increasing theoretical insight, better analytical

techniques, and solid foundations should be a top

priority of metaheuristic research.

REFERENCES

Achary, T., & Pillay, A. W. (2022). A Taxonomy Guided

Method to Identify Metaheuristic Components. In D.

Groen, C. de Mulatier, M. Paszynski, V. V.

Krzhizhanovskaya, J. J. Dongarra, & P. M. A. Sloot

ECTA 2022 - 14th International Conference on Evolutionary Computation Theory and Applications

156

(Eds.), Computational Science – ICCS 2022 (Vol.

13352, pp. 484–496). Springer International

Publishing. https://doi.org/10.1007/978-3-031-08757-

8_41

Aranha, C., Camacho Villalón, C. L., Campelo, F., Dorigo,

M., Ruiz, R., Sevaux, M., Sörensen, K., & Stützle, T.

(2021). Metaphor-based metaheuristics, a call for

action: The elephant in the room. Swarm Intelligence.

https://doi.org/10.1007/s11721-021-00202-9

Birattari, M., Paquete, L., & Stützle, T. (2003).

Classification of Metaheuristics and Design of

Experiments for the Analysis of Components.

Camacho‐Villalón, C. L., Dorigo, M., & Stützle, T. (2022).

Exposing the grey wolf, moth‐flame, whale, firefly, bat,

and antlion algorithms: Six misleading optimization

techniques inspired by bestial metaphors. International

Transactions in Operational Research, itor.13176.

https://doi.org/10.1111/itor.13176

Campelo, F., & Aranha, C. (2021). Sharks, Zombies and

Volleyball: Lessons from the Evolutionary

Computation Bestiary. Proceedings of the LIFELIKE

Computing Systems Workshop 2021, 3007.

Cruz-Duarte, J. M., Ortiz-Bayliss, J. C., Amaya, I., Shi, Y.,

Terashima-Marín, H., & Pillay, N. (2020). Towards a

Generalised Metaheuristic Model for Continuous

Optimisation Problems. Mathematics, 8(11), 2046.

https://doi.org/10.3390/math8112046

de Armas, J., Lalla-Ruiz, E., Tilahun, S. L., & Voß, S.

(2021). Similarity in metaheuristics: A gentle step

towards a comparison methodology. Natural

Computing. https://doi.org/10.1007/s11047-020-

09837-9

Fister jr, I., Mlakar, U., Brest, J., & Fister, I. (2016,

October). A new population-based nature-inspired

algorithm every month: Is the current era coming to the

end?

Glover, F. (1986). Future paths for integer programming

and links to artificial intelligence. Computers &

Operations Research, 13(5), 533–549.

https://doi.org/10.1016/0305-0548(86)90048-1

Hooker, J. N. (1995). Testing heuristics: We have it all

wrong. Journal of Heuristics, 1(1), 33–42.

https://doi.org/10.1007/BF02430364

Liu, B., Wang, L., Liu, Y., & Wang, S. (2011). A unified

framework for population-based metaheuristics. Annals

of Operations Research, 186(1), 231–262.

https://doi.org/10.1007/s10479-011-0894-3

Lones, M. A. (2020). Mitigating Metaphors: A

Comprehensible Guide to Recent Nature-Inspired

Algorithms. SN Computer Science, 1(1), 49.

https://doi.org/10.1007/s42979-019-0050-8

Molina, D., Poyatos, J., Ser, J. D., García, S., Hussain, A.,

& Herrera, F. (2020). Comprehensive Taxonomies of

Nature- and Bio-inspired Optimization: Inspiration

Versus Algorithmic Behavior, Critical Analysis

Recommendations. Cognitive Computation, 12(5),

897–939. https://doi.org/10.1007/s12559-020-09730-8

Ostrowski, D., & Schleis, G. (2008). New Approaches for

MetaHeuristic Frameworks: A Position Paper. AAAI

Workshop - Technical Report.

Peres, F., & Castelli, M. (2021). Combinatorial

Optimization Problems and Metaheuristics: Review,

Challenges, Design, and Development. Applied

Sciences, 11(14), 6449.

https://doi.org/10.3390/app11146449

Sörensen, K. (2015). Metaheuristics-the metaphor exposed.

International Transactions in Operational Research,

22(1), 3–18. https://doi.org/10.1111/itor.12001

Sörensen, K., & Glover, F. W. (2013). Metaheuristics. In S.

I. Gass & M. C. Fu (Eds.), Encyclopedia of Operations

Research and Management Science (pp. 960–970).

Springer US. https://doi.org/10.1007/978-1-4419-

1153-7_1167

Stegherr, H., Heider, M., & Hähner, J. (2020). Classifying

Metaheuristics: Towards a unified multi-level

classification system. Natural Computing.

https://doi.org/10.1007/s11047-020-09824-0

Swan, J., Adriaensen, S., Bishr, M., Burke, E. K., Clark, J.

A., Durillo, J. J., Hammond, K., Hart, E., Johnson, C.

G., Kocsis, Z. A., Kovitz, B., Krawiec, K., Martin, S.,

Merelo, J. J., Minku, L. L., Pappa, G. L., Pesch, E.,

Garc, P., Schaerf, A., … Wagner, S. (2015). A Research

Agenda for Metaheuristic Standardization.

Torres-Jiménez, J., & Pavón, J. (2014). Applications of

metaheuristics in real-life problems. Progress in

Artificial Intelligence, 2(4), 175–176.

https://doi.org/10.1007/s13748-014-0051-8

Tzanetos, A., & Dounias, G. (2021). Nature inspired

optimization algorithms or simply variations of

metaheuristics? Artificial Intelligence Review, 54(3),

1841–1862. https://doi.org/10.1007/s10462-020-

09893-8

Ven, A., & Johnson, P. (2006). Knowledge for Theory and

Practice. Academy of Management Review, 31, 802–

821. https://doi.org/10.2307/20159252

Voß, S. (2001). Meta-heuristics: The State of the Art. In G.

Goos, J. Hartmanis, J. van Leeuwen, & A. Nareyek

(Eds.), Local Search for Planning and Scheduling (Vol.

2148, pp. 1–23). Springer Berlin Heidelberg.

https://doi.org/10.1007/3-540-45612-0-1

Wang, Y. (2010). A Sociopsychological Perspective on

Collective Intelligence in Metaheuristic Computing:

International Journal of Applied Metaheuristic

Computing, 1(1), 110–128.

https://doi.org/10.4018/jamc.2010102606

Wolpert, D. H., & Macready, W. G. (1997). No free lunch

theorems for optimization. IEEE Transactions on

Evolutionary Computation, 1(1), 67–82.

https://doi.org/10.1109/4235.585893

Yang, X.-S. (2020). Nature-inspired optimization

algorithms (2nd ed.). Elsevier Inc.

Towards Rigorous Foundations for Metaheuristic Research

157