Platform-Pilot-Supervisor (PPS) when Mixed-Initiative Become Entity

Agnostic

Weifei Xie

, C

edric Herpson

, Christophe Cambier

and Jean-Marie Dembelle

Sorbonne Universit

e, CNRS, LIP6, F-75005 Paris, France

Sorbonne Universit

e, IRD, UCAD, UGB, UMI, UMMISCO, F-75006 Paris, France

Universit

e Gaston Berger de Saint-Louis, Laboratoire d’Analyse Num

erique et Informatique, BP.234 Saint Louis, Senegal

{ﬁrstname.lastname}@lip6.fr,

{ﬁrstname.lastname}@ird.fr,

{ﬁrstname.lastname}@ugb.edu.sn

Keywords:

Mixed Initiative, Autonomy, Design Phase.

Abstract:

Adaptive autonomy, adjustable autonomy or mixed initiative are introduced to allow the intelligent system to

cope with a dynamic environment. In these approaches, the system is allowed to change the level of autonomy

by itself or by an external actor. In this research, we introduce PPS, an analysis grid focused on the role of

the entities that compose a human-non-human team. It evaluates the genericity and collaboration capabilities

of existing architectures and mechanisms and can be used as a guide to design architectures covering all

collaboration conﬁgurations...

1 INTRODUCTION

Any collaboration between entities with a certain de-

gree of autonomy requires coordination. Whether

these entities intervene jointly or alternately, are hu-

man, non-human or a combination of the two makes

no difference. Indeed, every autonomous entity has

a certain capacity for initiative. This capacity can be

controlled or limited, either by the degree of auton-

omy the entity has in a given context (intrinsic limita-

tion), or by rules, norms or protocols of coordination

and interaction (extrinsic limitation). The challenge,

widely described in the literature, is to ensure that

teamwork brings more beneﬁts than difﬁculties(C

2013).

The various works that address these issues fall

under the concepts of Adjustable Autonomy(C

2013), Adaptive Autonomy(Frasheri et al., 2018) and

Mixed-Initiative(Bevacqua et al., 2015). Intuitively,

the concept of Adjustable Autonomy introduces the

fact that the degree of autonomy of an entity can be

changed according to the situation or in response to

an event but does not make explicit who is respon-

sible for this change (the entity itself or an external

actor)(C

e, 2013). The concept of adaptive auton-

omy goes a step further and allows an entity to take

the initiative to vary its level of autonomy by itself,

and therefore the perimeter of its decisions (Frasheri

et al., 2018). In both cases, the better reactivity of the

system is achieved at the expense of the risk of error.

Work on mixed-initiative mainly focus on the ways

that allow different actors to intervene proactively in

the progress of a mission in order to guarantee its suc-

cess and maximise its performance. The degree of

autonomy of each entity and the deﬁnition of what

initiatives can be taken by them greatly inﬂuence the

performance of the teams.

However, despite the efforts of the community to

deﬁne a taxonomy (Frasheri et al., 2018; C

e, 2013;

Bevacqua et al., 2015) and clear perimeters for these

three concepts, there is no commonly accepted deﬁni-

tion of each of its terms, many work that fall explicitly

within one of the above concepts have features that

overlap with the other two. Rather than focusing on

deﬁning the boundaries between these domains, we

thought it would be interesting to look at the concept

that links them: the notion of role.

To place us not at the level of the pilot of the plat-

form but at the level of the roles – Pilot(s) and Super-

visor(s) – associated to the Platform(s) offers an anal-

ysis grid of existing approaches that allows to evalu-

ate both their genericity and their compatibility with

evolutions likely to be encountered. We refer here as

much about the direct uses of a platform as about its

integration within a larger system. In this work in

progress we introduce the proposed Platform-Pilot-

Supervisor (PPS) analysis grid and illustrate how it

could be use as a guide to design an architecture cov-

ering all collaboration conﬁgurations.

Xie, W., Herpson, C., Cambier, C. and Dembelle, J.

Platform-Pilot-Supervisor (PPS) when Mixed-Initiative Become Entity Agnostic.

DOI: 10.5220/0011961600003622

In Proceedings of the 1st International Conference on Cognitive Aircraft Systems (ICCAS 2022), pages 69-72

ISBN: 978-989-758-657-6

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

2 PLATFORM-PILOT-

SUPERVISOR APPROACH

The goal of a collaborative system is to achieve the

task given efﬁciently and safely. In our proposal (ﬁg-

ure 1), the mission’s designer does not necessarily

determine ofﬂine the actors of each role. At a given

time, a platform is driven by an entity A which has

been given this role, and is supervised by an entity B

(potentially the same, but not necessarily) which has

been given a supervisory role. These roles can indif-

ferently and dynamically be entrusted to one or more

operators, human or artiﬁcial. And what is true for a

platform is also true for a set of – homogeneous or not

– platforms.

An entity is thus associated to a given set of roles ac-

cording to the current needs, estimated beneﬁts and

capabilities available. An entity can be both the su-

pervisor and pilot of one or several platforms for the

whole mission, or linked to a given role for a certain

amount of time.

Figure 1: Illustration of platform-pilot-supervisor model.

In this system, any platforms (aerial, terrestrial, submarine,

subterranean) can be controlled by one or several operators,

human or IA, either alternatively or concurrently. The su-

pervisor role follow the same reasoning.

To ensure the performance of the system, several

requirement should be veriﬁed:

• Information Collection. Both pilots and super-

visors should be able to gather information with

an acceptable delay. The information the pilot(s)

require comes from both the supervisor and the

environment. The information the supervisors

should gather are related to the current pilots’ ca-

pabilities and the situational awareness.

• Communication. Both roles should be able to ex-

press their opinion, in case of conﬂict between the

two roles, the supervisor makes the decision. The

interaction must be able to evolve according to the

state of communication, for better or for worse.

• Ability of Analysis. Both roles need to be able

to analyse the information gathered within an ac-

ceptable time frame. In both cases, they must deal

with the noise and extract the essential informa-

tion. In the case of the pilot role, he must be able

to estimate the state of the platform. The super-

visor must be able to detect if the platform is in

danger and assess if the current entity(ies) assum-

ing the pilot role are adapted to the situation.

• Planning Ability. This is an essential ability

of both the pilot and supervisor roles. With the

knowledge of the platform-pilot state, the pilot

must be able to evaluate if it is able to cope with

the situation and if so, move on to the planning

of the movement (planning a sequence of actions

to perform the task). In the case of a necessary

re-planning, the pilot must insure that this could

be done under a given time-frame. The supervi-

sor must be able to determine if the pilot is able to

cope with this situation. The supervisor must also

be able to determine is it is necessary to change

the pilot by taking into account both the current

performance, the cost and risks of a role-change,

and the expected performance. Both decisions

must be explainable (in the case of a human, the

human must be able to justify itself, in the case of

an algorithm, the decision must be interpretable).

• Execution. The supervisor must have the ability

to execute decisions. Pilot must be able to ask for

help (transfer its role partially or fully) or execute

the planned actions.

• Knowledge. Both roles must be able to update

their knowledge to adapt their behaviour to un-

certainty, incompleteness and imprecision which

compose any real mission.

• Record: The system must be able to record all ac-

tions performed by any component of the triptych

– Platform(s), Pilot(s), Supervisor(s) – so that any

return of experience process could be realised.

2.1 Illustrative Examples

• Mono-platform. Unmanned Aerial Systems are

used in many warfare of civilian context : Ex-

ploration, surveillance and reconnaissance, search

and rescue,... Consider the case of one Option-

ally Piloted UAVs that should be used by a non-

expert user to complete surveillance missions. In

autonomous mode, the user takes on the role of

supervisor to ensure that the auto-pilot is follow-

ing the deﬁned plan. When the human wants to

switch roles to become the remote-pilot, the user

takes on the role and the AI takes on the role of su-

pervisor. It keeps the capability to take over con-

trol in the eventuality that the user’s action puts

the drone in danger of crashing or of violating

ﬂight rules. In the case of communication-losses

the (embedded) AI assumes both the pilot and su-

ICCAS 2022 - International Conference on Cognitive Aircraft Systems

pervisor roles. The advantage of allowing an AI to

take on the role of supervisor is that we can insure

that the safety level of the drone will be at least

the minimum between the safety levels of the hu-

man and of the AI.

In the case where the pilot role is shared between

several entities, for example an AI and a human,

the AI can take responsibility for a subset of ac-

tions so that the human can more easily manage

the platform. The AI can ensure that the quantity

of information transmitted to the human does not

overﬂow its processing capabilities and dynami-

cally adapt the tasks to its level of expertise.

• Multi-platform. The PPS approach can also be

used to describe multi-platforms conﬁgurations.

Consider multiple drones working together to ﬁnd

an objective on a map. In the case of Optionally

Piloted UAVs, supervisors could be both embed-

ded humans or AIs. Once a point of interest is

founded, a human can take control of the closest

drone to speciﬁcally observe it, in this case he be-

comes the pilote of one platform, and the supervi-

sor of this OPUAV can be either a human or an AI

while the conﬁguration can remain different for

the other components of the ﬂeet.

3 RELATED WORK

As previously introduced, the 3 main concepts that

relates to the question of the autonomy of AI-Human

team and to collaboration between autonomous enti-

ties are Adaptive autonomy (ADA), Adjustable au-

tonomy (AJA) and Mixed initiative (MI). The 3 ap-

proaches are deﬁned differently by different authors.

For (Singh, 2021), there is no difference between

ADA and AJA, and MI is a special case of ADA. In

the case of (Bevacqua et al., 2015) MI is restricted to

the case where a human and AI share the control of

a robot and the AI is the human assistant. In (Chanel

et al., 2020), MI means that each agent, human or not,

can seize the initiative, the role of each entity is de-

cided by its current capabilities. For (Frasheri et al.,

2018), ADA means that the AIs could adapt their au-

tonomy during the session by themselves. In this doc-

ument, we choose to follow the deﬁnitions proposed

by (Hardin and Goodrich, 2009) where the level of au-

tonomy, and thus the initiative capability of an AI, is

primarily controlled by the human in the AJA model

whereas the entity mainly controls its autonomy in

ADA. In this work, MI tries to deﬁne the coordina-

tion conditions between Human and artiﬁcial entities.

Table 1 synthesise the 6 basic conﬁgurations that

can be met when the focus on the roles instead of the

Table 1: Different roles conﬁgurations that should be con-

sidered with one unique platform: 1) Pilot and supervisor

are assumed by the same AI. 2)Pilot and supervisor are dif-

ferent AIs. 3)Human pilot with AI supervisor. 4) AI pi-

lot with human supervisor 5) Pilot and supervisor are one

unique Human 6) Pilot and supervisor are different Hu-

mans.

Supervisor

Pilot

AI Human

Human 4

actors. At these conﬁgurations should be added the

cases where several entities are allowed to simultane-

ously collaborate to assume a given role.

The combination of theses modes cover the 3 pre-

viously introducted concepts. ADA, as presented in

(Abraham et al., 2021) can be presented with a combi-

nation of cases 1,4,5. The ADA perspective proposed

by (Frasheri et al., 2018) for the search and rescue

scenario can be presented as a combination of modes

1 and 3 while the (Rubio et al., 2004) model for path

planning restricted to the cases 1 and 2.

AJA generaly integrates a human supervision di-

mension. In the work of (C

e, 2013), the human may

choose to assist the system by providing information

about the waypoints or directly takes control, which

can be done by a combination of cases 1, 3 and 4.

The work of (Crandall and Goodrich, 2001) presents

a model that allows human integration in both high

and low autonomy level. Taking into account com-

munication delay, the supervision can be delegated to

the AI to avoid serious defects. This model cover the

cases 1,3,4.

The MI model can also be represented by the com-

bination of the basic modes. (Allen et al., 1999) pro-

pose a model in which each participant must con-

stantly monitor the situation to make the best deci-

sion. This model can be represented by a combination

of cases 1, 3, 4 and 5. (Adams et al., 2004) propose to

monitor the human state to achieve better cooperation,

this could represent a combination of cases 1,3,4,5.

The model proposed by (Bruemmer et al., 2003) can

be covered by the cases 1,3,4,5.

Some work also attempts to reduce the needs of

the human operator, with the aim of achieving full

autonomy. Their approach is also similar to ADA

or mixed initiative, such as the model proposed by

(Schwerd and Schulte, 2021) which be represented by

a combination of the cases 1, 3, 4 and 5.

The integrated guidance system in a multi-drone

system (Donath et al., 2010) can be represented by a

combination of modes 1,3,4,5. In other words, these

Platform-Pilot-Supervisor (PPS) when Mixed-Initiative Become Entity Agnostic

approaches attempt to build a system where humans

only play the role of supervisor, without the capacity

of pilot.

PPS allows to categorise existing approaches and

highlight the fact that several combination that make

sense operationally are forgotten. Allowing a super-

visor to not be a pilot or to call for different experts

during a given mission depending of situation to be

managed could improve the overall performance.

4 SYNTHESIS AND FUTURE

WORK

This paper introduce PPS, an analysis grid that fo-

cus on the roles of the entities that composes a team.

PPS allows to evaluate the genericity and collabora-

tion capabilities of existing architectures and mecha-

nisms that tries to improve the efﬁciency of team of

heterogeneous entities, either human or artiﬁcial. In

particular, PPS showed that there exist several use-

cases, either mono or multi platform, where a dynam-

ically changing supervisor could offer overall perfor-

mance improvements. In the near future, we will pro-

pose and instantiate a management architecture that

will cover the different cases described by PPS on a

mono-uav surveillance set-up before extending it to

multi-platform use-cases.

REFERENCES

Abraham, S., Carmichael, Z., Banerjee, S., VidalMata,

R., Agrawal, A., Islam, M. N. A., Scheirer, W., and

Cleland-Huang, J. (2021). Adaptive Autonomy in

Human-on-the-Loop Vision-Based Robotics Systems.

arXiv:2103.15053 [cs]. arXiv: 2103.15053.

Adams, J. A., Rani, P., and Sarkar, N. (2004). Mixed-

Initiative Interaction and Robotic Systems. page 8.

Allen, J., Guinn, C., and Horvtz, E. (1999). Mixed-initiative

interaction. IEEE Intelligent Systems and their Appli-

cations, 14(5):14–23. Conference Name: IEEE Intel-

ligent Systems and their Applications.

Bevacqua, G., Cacace, J., Finzi, A., and Lippiello, V.

(2015). Mixed-Initiative Planning and Execution

for Multiple Drones in Search and Rescue Missions.

page 9.

Bruemmer, D., Marble, J., Dudenhoeffer, D., Anderson, M.,

and McKay, M. (2003). Mixed-initiative control for

remote characterization of hazardous environments.

In 36th Annual Hawaii International Conference on

System Sciences, 2003. Proceedings of the, page 9 pp.,

Big Island, HI, USA. IEEE.

Chanel, C. P. C., Roy, R. N., Drougard, N., and Dehais, F.

(2020). Mixed-Initiative Human-Automated Agents

Teaming: Towards a Flexible Cooperation Frame-

work. In Harris, D. and Li, W.-C., editors, Engi-

neering Psychology and Cognitive Ergonomics. Cog-

nition and Design, Lecture Notes in Computer Sci-

ence, pages 117–133, Cham. sc.

Crandall, J. and Goodrich, M. (2001). Experiments in

adjustable autonomy. In 2001 IEEE International

Conference on Systems, Man and Cybernetics. e-

Systems and e-Man for Cybernetics in Cyberspace

(Cat.No.01CH37236), volume 3, pages 1624–1629,

Tucson, AZ, USA. IEEE.

e, N. (2013). Raisonnement et d

ecision mixte pour

l’autonomie ajustable et le partage d’autorit

e. The-

ses, Universit

e de Caen.

Donath, D., Rauschert, A., and Schulte, A. (2010). Cogni-

tive assistant system concept for multi-UAV guidance

using human operator behaviour models. page 16.

Frasheri, M., Curuklu, B., Esktrom, M., and Papadopoulos,

A. V. (2018). Adaptive Autonomy in a Search and

Rescue Scenario. In 2018 IEEE 12th International

Conference on Self-Adaptive and Self-Organizing Sys-

tems (SASO), pages 150–155, Trento, Italy. IEEE.

Hardin, B. and Goodrich, M. A. (2009). On Using Mixed-

Initiative Control: A Perspective for Managing Large-

Scale Robotic Teams. page 8.

Rubio, J. C., Vagners, J., and Rysdyk, R. (2004). Adaptive

Path Planning for Autonomous UAV Oceanic Search

Missions. In AIAA 1st Intelligent Systems Technical

Conference, Chicago, Illinois. American Institute of

Aeronautics and Astronautics.

Schwerd, S. and Schulte, A. (2021). Operator State Es-

timation to Enable Adaptive Assistance in Manned-

Unmanned-Teaming. Cognitive Systems Research,

67:73–83.

Singh, G. (2021). Enhancing manned unmanned teaming

with physiological computing and adaptive interac-

tion. These de doctorat, Toulouse, ISAE.

ICCAS 2022 - International Conference on Cognitive Aircraft Systems