Human-Machine Interfaces Based on EMG and EEG
Applied to Robotic Systems
Andre Ferreira
1
, Wanderley Cardoso Celeste
1
, Fernando Auat Cheein
2
,
Teodiano Freire Bastos-Filho
1
and Mario Sarcinelli-Filho
1
1
Electrical Engineering Department, Federal University of Espirito Santo,
Av. Fernando Ferrari, 514, 29075-910, Vitoria-ES, Brazil
2
Institute of Automatica, National University of San Juan,
Av. San Martin, 1109-Oeste, 5400, San Juan, Argentina
Abstract. Two different electro-biological signal based Human-Machine Inter-
faces (HMIs) were developed: EMG and EEG based. Such interfaces present like
main characteristics relatively simple acquisition and processing systems, which
need of few hardware and software resources, so that they are computational and
financial low cost solutions. Both interfaces have been applied to robotic systems
and their performance have been shown up in such applications. The EMG based
HMI was tested in a mobile robot, while the EEG based HMI was tested as much
in a mobile robot as a robotic manipulator.
1 Introduction
Electro-biological signals have recently become the focus of several research insti-
tutes, stimulated, mainly, by recent findings in the areas of cardio, muscular and neu-
rosciences, by the availability of more efficient and low cost computational resources,
and by the increasing knowledge and comprehension about motor dysfunctions [2][5].
Electrical signals of different parts of the human-body can be used as command
signals for controlling systems. However, it is necessary that the user can intentionally
generate such signals. Also it is necessary that the interface adopted (Human-Machine
Interface - HMI) can ”understand” and process such signals, setting the command that
better fits the wish of the user. Then, a HMI can be used to improve the movement
capacity of people with motor dysfunctions, using, for example, a robotic wheelchair to
carry them.
Among the many kinds of electro-biological signals which can be used in HMIs,
those more commonly employed in such devices are: Electro-MyoGraphical (EMG),
Electro-OculoGraphical (EOG) and Electro-EncephaloGraphical (EEG) signals.
EMG signals have a standard behavior, what is a great characteristic to take into
account as a starting point of a HMI project. However, there are inherent problems in
using it. Taking into account that the assistive technology proposed here is also directed
to people with neuromotor disabilities, some muscle spasms, for example, can occur on
these people, which represent a very serious problem (unless the HMI has a robust way
Freire Bastos-Filho T., Cardoso Celeste W., Auat Cheein F., Freire Bastos-Filho T. and Sarcinelli-Filho M. (2006).
Human-Machine Interfaces Based on EMG and EEG Applied to Robotic Systems.
In Proceedings of the 2nd International Workshop on Biosignal Processing and Classification, pages 116-125
DOI: 10.5220/0001225001160125
Copyright
c
SciTePress
to reject these disturbances) when using a system controlled by EMG signals. Severe
neuromotor injuries can also cause loss of muscle mobility, being impossible to use any
kind of EMG based control.
Thus, other communication channels (electro-biological signals) are explored in this
work, in order to develop systems that give better life conditions to those people. Brain
signals can be a solution to these problems. Good temporal resolution, portability and
low cost are some advantages related to EEG signals when compared to other ways to
acquire brain signals [6].
There exist several methods to use EEG signals as a communication channel. The
goal of this work is based on the ERS/ERD (Event Related Synchronization / Desyn-
chronization) complex, because such one is easier to be identified using relatively sim-
ple processing signal techniques. These patterns represent band power changes on the
brain rhythms in a specific frequency band. The ERD is related to a band power de-
crease while ERS represents a band power increase. These patterns used here are those
present in the alpha band of the occipital region (region related to visual information
processing). The ERD is related to concentration or existence of visual stimulus while a
condition of a relaxed visual area with few, or absence, of visual stimulus, characterizes
an ERS. Using a simple and low cost HMI [2] based on ERD/ERS, it was possible to
control devices such as a mobile robot [2] and also a robotic manipulator [1].
This work presents the development sequence of HMIs in which is taken into ac-
count the previous considerations, so that the difficult degree in relation to both signal
acquisition and processing is gradually increased, according to it is shown in Section 2.
This way, in the first stage of implementation of the HMI, an eye blink signal (EMG)
based system was done, like is shown in Section 2.1. Such system was used to control
a mobile robot, which was able to navigate in a semi-structured environment. Next, a
module capable to acquire and process EEG signals was developed (Section 2.2). Cur-
rently it is been explored the ERS/ERD complex of the signals acquired by electrodes
placed on the subject’s occipital region (O
1
and O
2
electrodes, according 10-20 stan-
dard), which are related to visual activity. Such modules have been used to control a
mobile robot and a robotic manipulator. The results are shown in Section 3. In Sec-
tion 4 the final considerations are done in addition to future plans for development of
the electro-biological signals based HMI.
2 Acquisition Systems
Two different electro-biological signals based Human-Machine Interfaces (HMIs) were
developed: EMG and EEG. The first one allows a person to command devices through
eye blinks [4]. The other one allows brain commands as well as to control devices [1].
In this section a succinct presentation of such systems is done.
2.1 EMG
Fig. 1 shows the structure of the EMG based on HMI developed. Such HMI is com-
posed of a signal acquisition and processing system. No complex practical preparation
is required to use the system. The subject is supposed to use a commercial cap (for
117
convenience) with the correct electrodes position according to 10-20 International Sys-
tem. The respective head positions to be used should be clean and it is not necessary
to shave the hair, but applying a gel to improve head/electrode contact. The reference
is connected to the ear. The acquisition is done by electrodes which send the user’s
electro-biological signals to be amplified and filtered on the signal conditioning and
power supply board. Afterwards, these signals are sent to other board which does A/D
conversion. Finally, such signals are transmitted to a desktop computer, where their
processing is done, generating or not a specific command for controlling a mobile ro-
bot. A subject closes the control loop, providing the necessary biological feedback.
Fig.1. The structure of the proposed system.
The board’s power source is designed to reduce the interference of the electric ap-
pliances frequency and other external electronic equipments, such as switching sup-
plying, on the acquisition system. The A/D converter board has four analog channels
allowing expansion through cascade connections, increasing the number of channels
being processed. The interface for the user/machine communication is programmed in
the desktop computer, as well as the signal processing software, that sends the control
commands to the mobile robot. These commands are transmitted to the robot through
an Ethernet Radio.
The experiments here reported were carried out with a Pioneer 2DX wheeled mobile
robot. This robot has a microcontroller for the low level instructions and an embedded
PC (Intel Pentium MMX 266 MHz, 128 MB RAM).
For generating a command, the user should be able to blink his/her eyes. To help
him/her in this task an electronic board with automatic scanning (desktop microcom-
puter) was implemented. It represents the area from a robot’s navigation environment
divided in cells, according to Fig. 2. This way, when the wished cell is swept, the user
blinks a determined eye and the EMG signal is captured and processed by the acquisi-
tion and processing systems.
Since the EMG signals due to eye blinks have a well-defined standard behavior
Fig. 3, the necessary processing system is relatively simple. It works as follow: Firstly,
118
Fig.2. System’s electronic board.
a threshold is experimentally established for each user, based on the changes observed
in a signal interval that contains a set of eye blinks (training stage). During the system
run, whenever the signal generated by an eye blink of the user goes above the threshold
established, a counter starts counting the number of samples received ever since. When
the signal falls below the threshold, the number of samples counted is compared with
a predefined one: if it is greater than the pre-defined number, the HMI detects an eye
blink. Otherwise, the HMI detects that there was not an eye blink. After that, the counter
is reset and a new cycle starts.
Fig.3. Eye blink detection.
2.2 EEG
The hardware developed to work with EEG data was used to operate a mobile robot [2]
and also an industrial manipulator [1]. The last one, includes a teleoperation via TCP/IP.
119
The hardware is composed basically of a signal conditioning board (filter/amplification
function) followed by an acquisition board, responsible for A/D conversion. Signal
processing and the generation of control actions in order to operate the robots are done
by an application running on a PC. This application also implements the user/machine
software interface.
Electrodes placed on O
1
and O
2
positions (according to 10-20 system) of the op-
erator’s head acquire EEG signals that are filtered (band pass: 0.5 Hz to 40 Hz) and
amplified.
Fig.4. 10-20 International System for electrodes placement.
Once conditioned, the signal is digitalized and pre-processed to remove artifacts.
Afterwards, a power analysis of the signal allows the identification of a low (ERD) or
high (ERS) energy in alpha band (8-13 Hz) as illustrated in Fig. 5. Using this informa-
tion the machine makes decisions.
In the first case (operating a mobile robot), the alpha band power analysis is used to
change states of a Finite State Machine (FSM) and to generate commands (front, right,
left, back) to the robot.
In the case of operating an industrial manipulator (BOSCH SR800) via TCP/IP, it is
presented to the operator the manipulator’s workspace divided in cells. The application
scans all cells and the user’s alpha band power analysis is used to select one of them.
This is accomplished by determining an ERD or ERS pattern. When the user has the
eyes open the signal’s power is low (ERD), but when the eyes are closed an ERS can
be identified by the increasing of the band power over a established threshold, and thus,
the cell is selected. When it is done, the coordinates of this cell are sent via TCP/IP to
a remote computer which controls the manipulator, moving its end effector toward the
desired position. At the same time, the data incoming from encoders are sent back to
the user’s PC (the local one) in order to update the screen with the current positions of
the manipulator. The whole system is illustrated in Fig. 6.
In both cases a calibration process is necessary before starting the experiments. This
procedure consists of acquiring about 10 seconds EEG data to analyse ERD level. Based
on this information, the threshold used to detect an ERS is set to 5 up to 10 times the
120
Fig.5. ERD and ERS energy levels [3].
Fig.6. EEG based system.
121
ERD’s level. This is very important because these levels change constantly in time and
from person to person.
3 Results
Both HMIs presented in Section 2 have been used to command robotic devices by sub-
ject previously trained to operate them. The EMG based HMI was used to command a
mobile robot, while the EEG based HMI was used to command both a mobile robot and
a robotic manipulator. In this section results from each test done are shown.
3.1 EMG
Firstly, eight subjects were asked to accomplish ten eye blinks with each eye, in order to
test the eye blink identification algorithm. The results of these experiments are shown in
Table 1. The subjects could also choose to wear the electrode cap or individual surface
electrodes. Only results of subjects which were able to perform blinks with both eyes
are considered, according Table 1.
Table 1. Success results using right and left eye.
Volunteer Right Eye Left Eye
1 8 9
2 10 10
3 10 8
4 10 10
5 10 9
6 10 10
7 10 10
So, an average rightness of 95.71% about the results reached by subjects with ability
to blink both eyes allowed to conclude that the system could be used to command
devices.
One among the subjects that presented a good performance in the experiment with
eye blinks system was asked to determine a destination point on the electronic board
described in Section 2. After such subject to select a destination point through eye blinks
(see Section 2), the control software started to guide the robot to such point, following
the path determined by a path planning algorithm [4], which is based on the Dijkstra’s
Algorithm, that determines a secure path, that is, more distant of walls and obstacles.
Fig. 7 shows the map of a navigating environment and the path generated by the
system to go from an initial position to a destination position selected by the user. That
path is transmitted to the mobile robot, which knows its navigating environment. Fig. 8
shows the result of the navigation executed by the robot during the experiments.
122
Fig.7. Path generated by the system.
Fig.8. Result of the experiment with unpredicted obstacle distant of the robot’s navigation path.
123
3.2 EEG
This HMI was proved by a group of 25 people (age between 20 and 50 years old)
without neuromotor disabilities. The results are presented in Fig. 9. As can be seen,
most of people learned how to use the HMI in less than 15 minutes with only one
experiment.
Additionally the operation of complex systems such as an industrial manipulator
and a mobile robot with a considerable low cost HMI is shown, also providing efficient
results.
Fig.9. Number of successful subjects when learning how to use the HMI versus training time
required (in minutes).
4 Conclusions
A review of two different HMIs developed is done in this work: EMG and EEG based.
Both interfaces show oneself to be simple implementation and low cost solutions.
The EMG signal was initially chosen as electro-biological signal due to be a be-
haved signal of easy acquisition and processing compared to other electro-biological
signals as, for example, EEG. The results demonstrate the HMI is easy to operate by
users which can blink their eyes according their wishes. This HMI was tested in a mo-
bile robot, so that a user could point to a destination position, through an eye blink, to
be reached by the robot. In all tests, the mobile reached to the destination pointed by
the user.
The EEG based HMI can be seen like an evolution of the EMG based HMI due to
the increasing of the difficult degree presents as much in the acquisition system as in
the processing system. It has been used the called ERS/ERD complex, which can be
relatively easy identifying it, what allowed, in any case, a simple and low cost solution,
124
as previously commented. This HMI was tested as much in a mobile robot as in a robotic
manipulator. In both cases, the command was executed by the robotic devices.
Such works are initial parts of a system to help people with neuromotor diseases,
including those with severe dysfunctions. The next steps are: to convert a commer-
cial wheelchair in an autonomous mobile vehicle; to implement the HMI onboard such
autonomous wheelchair to assist people with motor diseases; to explore more charac-
teristics of EEG signals, making the Brain Computer Interface (BCI) more robust and
faster, allowing a secure use by people with severe motor dysfunctions.
Acknowledgements
The authors thank CAPES (Brazil) and SPU (Argentina) for their financial support
to the partnership between Federal University of Espirito Santo, Vitoria, Brazil, and
National University of San Juan, San Juan, Argentina, through the binational program
CAPG-BA. As part of this financial support, Andre Ferreira got a scholarship to stay six
months in San Juan, Argentina, where part of this work was developed. The authors also
thank FAPES/Brazil (Process 30897440/2005) for its financial support to this research.
References
1. Ferreira, A., Bastos-Filho, T. F., Sarcinelli-Filho, M., Cheein, F. A., Postigo, J. F., Carelli, R.:
Teleoperation of an Industrial Manipulator Through a TCP/IP Channel Using EEG Signals.
In: ISIE2006, Montreal, Canada, July, 2006.
2. Cheein, F. A., Postigo, J. F.: A fast finite state machine design for a brain computer interface.
In: Proceedings of XI RPIC, Rio Cuarto, Argentina, 2005.
3. Cheein, F. A.: Dise
˜
no de una Interfaz Cerebro-Computadora para la navegaci
´
on de un Robot
M
´
ovil. In: Master Thesis, San Juan, Argentina, August, 2005.
4. Neto, A. F., Celeste, W. C., Martins, V. R., Bastos-Filho, T. F., Sarcinelli-Filho, M.: Human-
Machine Interface Based on Electro-Biological Signals for Mobile Vehicles. In: ISIE2006,
Montreal, Canada, July, 2006.
5. Mill
´
an, J., Renkens, F., Mouri
˜
no, J., Gerstner, W.: Non-invasive brain-actuated control of a
mobile robot. In: Proceedings of the 18th International Joint Conference on Artificial Intelli-
gence, August, 2003.
6. Mouri
˜
no, J. E.: EEG-based analysis for the design of adaptive brain interfaces. In: Ph.D. The-
sis, Barcelona, Spain, 2003.
125
