Model-Based Prediction for Navigation Assistance Using a Set of

Sensors

David Sanders

, Giles Tewkesbury

, Shikun Zhou

and Malik Haddad

Faculty of Technology, University of Portsmouth, Anglesea Road, Portsmouth, U.K.

Northeastern University – London, St. Katharine’s Way, London, U.K.

Keywords: Model-Based, Prediction, Navigation, Assistance, Sensors.

Abstract: Navigation assistance is a vital step towards higher level automation and intelligence. This paper presents a

new conceptual design for an assistive system. It helps to manoeuvre powered wheelchairs in challenging

situations so that more people can drive them. The method is based on dynamic motion prediction using

sensors to provide information about the surroundings. Generic components of the assistive system are

presented based on manoeuvre prediction. Wheelchair user behaviour was analysed while they were driving

their wheelchairs. Other aspects are discussed, including the precision of the models built upon actuator and

environmental data collected from a strategically selected set of sensors.

1 INTRODUCTION

Powered wheelchairs and other mobility platforms

have been equipped with assistive systems to avoid

collisions (Haddad et al, 2020a; 2020b; Okonor et al,

2019; Sanders et al, 2019a; 2019b; 2019c; 2020) and

aid with steering (Sanders et al, 2010; 2015; 2016;

2017; 2018a; 2020; Haddad and Sanders, 2020). Such

systems have assisted wheelchair drivers, especially

when manoeuvring through doorways and through

narrow gaps, or in complicated environments or

adverse weather. Research described in this paper

aims to create new control systems for powered

wheelchairs (Haddad et al, 2019a; 2019b; 2020c;

2020d; Sanders et al, 2018b; Sanders, 2017).

Wheelchairs require high levels of safety and

efficiency, especially when approaching obstacles or

entering narrow spaces such as doorways. The

prototype systems were created during the first phase

of the project to process data when cope with an

obstacle, optimise paths and automatically assist the

user with selecting a safer path using a set of sensors.

2 BACKGROUND

Mobility problems can be tackled in many ways. A

powered wheelchair or scooter could cause many

problems if people could drive them. This paper

considers ways of helping people to drive. However,

many other solutions, devices and systems exist to

assist people with disabilities including personal

devices such as crutches, wheelchairs, white canes,

and systems or devices built into the surrounding

environment such as wheelchair ramps, textured

platform slabs in railway stations and Pelican

crossing beepers. Local bus services could be

modified to be more accessible. Disabled people

could equip with computers so that they can do more

without the needing of more carers. Self-mobility has

many advantages and some will be addressed in this

work. The best solution would be one that was

feasible, preferred by the person operating the system

and affordable. That may not always be archived by a

single technological solution.

In the UK, shops and other public places are

obliged to be accessible to disabled people. It is often

assumed that “disabled” means “wheelchair user” so

that making doorways wider and adding a ramp might

be considered enough. Although a lot of disabled

people are not in wheelchairs. Most environmental

adaptations address problems and deficiencies in

wheelchair performance. For example, lifts and

ramps have been fitted instead of or as well as steps,

doorways have been widened, thresholds made level,

and door handles moved or automatic doors installed.

Other adaptations include the addition of grab handles

and handrails, and clear markings of accessible

routes. People with sensory disabilities can usually

Sanders, D., Tewkesbury, G., Zhou, S. and Haddad, M.

Model-Based Prediction for Navigation Assistance Using a Set of Sensors.

DOI: 10.5220/0011903500003612

In Proceedings of the 3rd International Symposium on Automation, Information and Computing (ISAIC 2022), pages 92-97

ISBN: 978-989-758-622-4; ISSN: 2975-9463

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

move themselves. Adaptations made for people with

sensory disabilities usually provide extra information,

for example through high contrast signs. Audio signs

can provide information and textured flooring, which

with tactile and audio cues can improve safety

greatly. People with cognitive disabilities can have

information provided to them in a more accessible

way. Buildings and internal spaces can be constructed

to make route finding easily memorable and intuitive.

Colour coding of locations can make them more

memorable and distinct. Simple changes may have

significant impacts. Removing steps from shops

could significantly improve accessibilities whilst

wheelchairs are still the most important mobility aids.

2.1 Manual Wheelchair

Manual wheelchairs are used by many millions of

people even today. There are many different models.

The typical design of the majority of such

wheelchairs are shown in Figure 1 from “Disability

and Mobility: Problems of Navigation, Orientation

and Locomotion, and Internal Report from the Bath

Institute of Medical Engineering.

Figure 1: Main features of a manual chair (Sanders, 2017).

The design is a compromise. The driving wheels

are large to place the rims within easy reach of the

wheelchair user, to move over uneven ground more

easily and to have a low rolling resistance. Driving

wheels are usually narrow and pneumatic but can be

filled with puncture-proof polyurethane foam

(although they have a higher rolling resistance).

Castors are small and at the front with solid and hard

rubber tyres. Hard tyres are less comfortable but

reduce rolling resistance and make the wheelchair

more manoeuvrable. But they make the wheelchair

more difficult to operate on uneven or softer ground.

Footrests are important for users’ comfort. The

backrest keeps the user upright in the chair, while its

heights can vary. Some manual chairs are fitted with

pushing handles. Wheelchair frames are similar to

bicycle frames and are constructed in similar ways

using steel, aluminium alloy, titanium or carbon fibre.

There are specialist designs for special occasions,

such as, off-road, skiing, rugby, basketball and tennis.

2.2 Powered Wheelchairs

If wheelchair users cannot use manual wheelchairs or

find manual wheelchairs tiring, then powered

wheelchairs are an alternative. A typical design of a

powered wheelchair is shown in Figure 2.

Figure 2: Features of a powered chair (Sanders, 2017).

There have been three main configurations of

powered wheelchairs, with driving wheels at the

back, in the front or in the middle. Front wheel drive

powered wheelchairs are more manoeuvrable and can

climb curbs more easily. However, they can lose their

traction on steeper gradients, especially on wet and

slippery surfaces. They do allow a user to approach

objects more closely. Centre wheel driven powered

wheelchairs provide good traction and are

manoeuvrable as the wheelchair user sits directly

above the driving wheels but they can feel `tippy'

when weight is transferred from the back to the front

or vice versa when speeding up or slowing down.

Rear wheel drive is the most common type because it

is the most compact design. They are easier to be built

and have the best traction when climbing. Most

folding powered wheelchairs are rear wheel drive.

The chair used in this research was front wheel drive.

There are many designs of frame for powered

wheelchairs. Frames are rigid and able to support the

user and powertrain while keeping the centre of

gravity low. They are usually made form welded steel

tube (although aluminium is used as well). Driving

wheels are smaller than a manual wheelchair, while

just large enough to climb curbs. Tyres are most often

treaded. Motors usually run on 24 Volts DC and are

energy efficient, with ease of control (with a wide

speed range) and can deliver high torques. Two 12

Model-Based Prediction for Navigation Assistance Using a Set of Sensors

Volt sealed lead acid batteries tend to be used in series

to deliver high currents, can be trickle charged for

long periods, and are affordable. However, they are

relatively heavy and could be damaged if completely

discharged. Alternatives include Lithium and Nickel

Metal Hydride batteries. Lithium can deliver high

current, charge quickly, are lighter and have higher

energy density but are expensive.

Wheelchair controllers are complex. They take

inputs from a human wheelchair user to manoeuvre

wheelchair typically via a pair of motors. The

controller must be able to make smooth operations in

both directions and speeds. A joystick is usually a

standard user interface and used by many disabled

people. Pushing a joystick forward moves a powered

wheelchair forward. Pulling a joystick sideways turns

a powered wheelchair left or right. The further a

joystick moves away from a neutral central position,

the faster a powered wheelchair travel. However, a

group of disabled people are not able to use joysticks,

therefore other control options are made available

such as head operated switches, chin operated

joysticks and suck/blow controllers. Some power

wheelchairs have additional features, for example,

there are wheelchairs that can rise, wheelchairs with

many different special seating configurations, and

specific designs of off road powered wheelchair.

The research described in this paper used a Bobcat

II wheelchair as shown in Figure 3. The Bobcat II

wheelchair was controlled by a joystick and had

ultrasonic sensors mounted on it to help a user to

safely drive, avoid obstacles, and negotiate gaps,

narrow spaces, and doorways.

Figure 3: Bobcat II Wheelchair.

3 SYSTEM COMPONENTS

Assistance for wheelchair navigation has been

defined in this research as a visualization of the

predicted dynamic motion of the wheelchair based on

current joystick settings and relevant environmental

forces. A ‘dynamic motion model’ formed the main

basis for a manoeuvre prediction system, as shown in

figure 4.

Figure 4: Components of the assistance system.

The dynamic model represented the relationship

between the joystick configurations and the

corresponding actions of the wheelchair. It

considered the actuators and their collaborated

effects, such as the different effects at various

accelerations and velocities. The influences from

external forces against actuator forces and matching

timings should be considered. These can be

formulated by the equations of motion with at least 2

Degrees Of Freedom. The relationship between

joystick configurations and actuators’ responding

forces, as well as the model parameters, such as those

representing external forces, were collected and

included in a number of lookup tables. The

wheelchair-model parameters were tuned using

programs in MATLAB as the wheelchair was

simulated undertaking different manoeuvres to

achieve a best outcome based on measured motion

states and model data.

Lookup tables were generated from measurement

data, their interpolation and extrapolation covering

the complete setting ranges. The data covers whether

they described the characteristic behaviour in all

motion states of the wheelchair, especially when

manoeuvring close to obstacles or in narrow gaps, for

example doorways. Data was collected during typical

wheelchair operations. The measurement equipment

on board the wheelchair together with data collecting

interfaces decided the quality of the data.

Supplemental sensors may be needed to strengthen a

sufficiently accurate model, in particular sufficiently

accurate orientation and position data.

The measured actuator and sensor data needed to

be synchronised and fused to with a set of consistent

time stamps for a complete modelling. Each

supplemental sensor had a different sampling rate and

the data was synchronised. The model depended on

fusing data, including assignment of weights to the

data according to their quality in each situation. There

ISAIC 2022 - International Symposium on Automation, Information and Computing

were different options to assist in navigation using

proximity recognition systems. Potential sensors

were ultrasonics, cameras, radar or Light Detection

and Ranging based sensors, which are often used in

automotive industry. Ultrasonic sensors were selected

for this work because they were robust and cheap.

Distances to objects needed to be correlated with

wheelchair position. When close to obstacles, the

proximity data was given the highest weighting

within the sensor fusion modules, partially because of

the application and its high reliability.

An essential component of the model was the

environmental data as well as their impact on the

manoeuvrability of the wheelchair. An assistance

system offered the opportunity to integrate

environmental impacts within the dynamic motion

model. Responding motion actions from the joystick

commands with the impact from the environmental

forces could be modelled. Adequacy would be

evaluated by correlating the predicted path and the

required space. This needed an appropriate map.

Automatic adaptation of these monitoring parameters

was needed.

It was possible that each wheelchair user could

have a personal configuration set, which would allow

alarms and hints to be given. The degree of

automation was also increased by including collision

avoidance and automatically calculated evasive

paths.

4 MANOEUVRING BEHAVIOUR

Interviews, observations and automated data

collection were used in this work. A questionnaire-

based survey was created to collect and analyse

individual behaviour data when driving. The first part

of this survey addressed the conventional use of

joysticks.

Automatic data collection evaluated joystick and

sensor usages in terms of effectiveness and

efficiency. The usage was classified on a scale from

0 (= never) to 10 (= always), depending on how

frequently each parameter was used. Similarly, the

data from carers and researchers observing tests were

collected and evaluated. A list of the usual parameters

were compiled, which included obstacles’ size,

number of collisions, location of obstacles and

distances to obstacles.

In the second phase of the project, interviewing

with wheelchair users took place to elicit additional

information. Different pictures were shown to users

to learn about their possible reactions in different

situations.

The personal opinions in the questionnaire-based

survey varied widely and demonstrated the divested

individual approaches to driving. Tendentiously,

wheelchair users appeared to use absolute values than

the relative. The location of the wheelchair, the

reactive correction for veer and the distance to the

target location were the most common values

considered by the users. In addition, different

distances were monitored regularly such as distance

to obstacles or to other powered wheelchairs and

people. The frequency of assessing the environmental

factors depended on the ability and skills of the

driver. Participants commented that the sensors were

usually helpful but sometimes not. In some certain

circumstances, the usage of some sensors slowed

down the wheelchair unnecessarily and made

inefficient manoeuvres.

The majority of clinicians and engineers

considered the saving of manoeuvring data including

manoeuvre points with velocities and actuator values

as useful. The data were especially useful in the

analysis of collisions or critical manoeuvres as well

as in training sessions.

A further part of the survey addressed alarms

depending on velocity and distance to obstacles.

Most people thought it was a good idea but did not

want to use it themselves. For the distances to

obstacles, the majority preferred combinations of

optical and acoustic alarms.

There was a need (at least in psychological terms)

to personalise the assistance systems. Personalisation

had different advantages. Each user could choose

their own additional parameters depending on their

individual circumstances. The user continued to be

responsible for manoeuvring and the manoeuvre

prediction calculated the future motion of the

wheelchair resulting from joystick settings and

external forces. The prediction provided a greater

certainty in guidance accompanied by fewer actuator

variations. Because of the precision, the mental

knowledge base of an experienced user was further

improved. During training, the prediction helped

build a personal dynamic model without any extra

risk.

5 PROXIMITY RECOGNITION

Proximity recognition can be especially important in

dark environment or if the user is visually impaired.

A new proximity recognition system was created. A

combination of ultrasonic and infrared sensors were

installed on the wheelchair to detect hazardous

Model-Based Prediction for Navigation Assistance Using a Set of Sensors

objects surrounding the vehicle. Sensor fusion was

used to fuse proximity recognition data.

Time of flight served as a measurement of the

distance between sensors and obstacles. Sixteen

independent active channels were used for

simultaneous acquisition arranged with a 90° field of

view. The module was mounted at the front of the

wheelchair between the footrests. The accuracy of

the sensor system could allow for precise object

identifications in the future.

6 QUALITATIVE STUDY

A qualitative exploratory study was undertaken as

part of a group project during an extended

undergraduate degree at the University of

Portsmouth. Individual interviews were conducted

using a semi-structured interview guide with probes

and open-ended questions. Participants were asked

about their wheelchair use and any unmet mobility

needs. Participants were then shown demonstrations

of the new systems. After the demonstrations,

participants were questioned regarding the new

systems.

Two main themes emerged, namely,

“Overcoming challenges” and “Useful features”.

6.1 Overcoming Challenges

These challenges most commonly related to the

following categories and circumstances:

• Hazardous environments:

- inaccessible buildings,

- narrow entrances,

- small elevators,

- pavements in poor condition,

- and narrow store aisles;

• Hazardous weather conditions:

- Such as rain, fog, and snow,

- or any condition causing poor visibility;

• Special venues

- shopping centers,

- stadiums and

- festivals.

After the demonstrations, volunteers reported that

the new features could alleviate some of challenges.

Obstacle avoidance was particularly highlighted,

praised and recognised to reduce possible difficulties

of wheelchair users.

Volunteers did express concerns regarding their

ability of learning and using the new technology.

They didn’t want the new technologies to weaken

their own abilities. Volunteers mostly only perceived

some of the features as being relevant. Several were

skeptical about trusting the new systems more than

their own knowledge and abilities, or were concerned

about reliability. Others thought the new systems

could respond more quickly in emergencies.

6.2 Useful Features

Two features were potentially the most helpful:

obstacle avoidance and path following. Volunteers

felt the new systems could be especially useful when

they were tired as the new systems could reduce

physical and cognitive effort. Volunteers thought the

systems could be especially useful for people with:

slow reactions, tiredness, poor upper extremity motor

control or reduced vision. The new systems could also

become increasingly useful as people age, and

cognitive and visual impairments increase. The

systems could be especially useful to people with

slow progressive conditions that deteriorate so that

manual control may be ineffective or even totally lost.

7 DISCUSSION & CONCLUSIONS

Assistive devices and changes to the environment

could help provide independent mobility for people

with disabilities. If a disabled person needed help in

order to become mobile, then they were much less

independent. This paper presented a new conceptual

design and generic components for an assistive

system. The method was based on dynamic motion

prediction using sensors to provide information about

the surroundings.

Relevant data must be acquired by sensors with

sufficient precision to characterize motion and

position in relation to obstacles. Collected data were

filtered, synchronised and fused for modelling and

online in real time for driving.

Conventional behaviour during driving was

surveyed using a questionnaire-based survey

followed up with interviews. Highly individual

approaches were obvious in the responses, discussion

about driving and in observing the driving.

It was concluded that the next developments of the

system must offer customisable functionality. A

personalized system could also provide an

opportunity to introduce automation in a gentle

manner.

In the second phase of the project, a higher level

of automation and intelligence is planned. The

developed methods will be demonstrated in real time

and using real powered wheelchairs.

ISAIC 2022 - International Symposium on Automation, Information and Computing

Proximity recognition systems will be redesigned

to improve sensor fusion, considering the number of

sensors and their combination (with different types)

to adapt the sensor system. Processing of the sensor

data was complicated because of the distribution and

different types of sensors. Adequate proximity

recognition is a precondition for assistive automated

systems.

REFERENCES

Haddad, M., Sanders, D., 2019a Selecting a best

compromise direction for a powered wheelchair using

PROMETHEE. IEEE Trans. Neur. Sys. Rehab. 27 2 pp

228-235.

Haddad, M., Sanders, D., Gegov, A., Hassan, M., Huang,

Y., Al-Mosawi, M., 2019b Combining multiple criteria

decision making with vector manipulation to decide on

the direction for a powered wheelchair Proc. SAI

Intelligent Systems Conference, (London) pp 680-693.

Haddad, M., Sanders, D., Langner, M., Omoarebun, P.,

Thabet, M., Gegov, A., 2020a Initial results from using

an intelligent system to analyse powered wheelchair

users’ data Proc. 2020 IEEE 10th International

Conference on Intelligent Systems 2 pp 241-245.

Haddad, M., Sanders, D., Langner, M., Thabet, M.,

Omoarebun, P., Gegov, A., Bausch, N., Giasin, K.,

2020b Intelligent system to analyze data about powered

wheelchair drivers Proc. SAI Intelligent Systems

Conference (Amsterdam) pp 584-593.

Haddad, M., Sanders, D., Langner, M., Thabet, M.,

Omoarebun, P., Gegov, A., Bausch, N., Giasin, K.,

2020c Intelligent system to analyze data about powered

wheelchair drivers Proc. SAI Intelligent Systems

Conference (Amsterdam) pp 584-593.

Haddad, M., Sanders, D., Langner, M., Ikwan, F.,

Tewkesbury, G., Gegov, A., 2020d Steering direction

for a powered-wheelchair using the Analytical

Hierarchy Process Proc. IEEE 10th International

Conference on Intelligent Systems (Varna)pp 229-234.

Haddad, M., Sanders, D., Thabet, M., Gegov, A., Ikwan, F.,

Omoarebun, P., Tewkesbury, G., Hassan, M., 2020e

Use of the Analytical Hierarchy Process to Determine

the Steering Direction for a Powered Wheelchair Proc.

SAI Intelligent Systems Conf (London) pp 617-630.

Okonor, O., Gegov, A., Adda, M., Sanders, D., Haddad, M.,

Tewkesbury, G., 2019 Intelligent approach to

minimizing power consumption in a cloud-based

system collecting sensor data and monitoring the status

of powered wheelchairs. Intelligent Systems and

Applications. Advances in Intelligent Systems and

Computing 1037 Springer pp 694-710.

Sanders, D., Langner, M., Tewkesbury, G., 2010,

Improving wheelchair‐driving using a sensor system

to control wheelchair‐veer and variable‐switches as

an alternative to digital‐switches or joysticks Ind Rob:

An int' jnl 32 2 157-167.

Sanders, D., Bausch, N., 2015 Improving steeri.ng of a

powered wheelchair using an expert system to interpret

hand tremor Proc. International Conf on Intelligent

Robotics and Applications (Portsmouth) pp 460-471.

Sanders, D., 2017 Using self-reliance factors to decide how

to share control between human powered wheelchair

drivers and ultrasonic sensors IEEE Trans Neur Sys

Rehab 25 8 pp 1221-1229.

Sanders, D., Sanders, B., Gegov, A., Ndzi, D., 2018 Results

from investigating powered wheelchair users learning

to drive with varying levels of sensor support Proc.

Intelligent Systems Conference (London) pp 241-245.

Sanders, D., Sanders, B., Gegov, A., Ndzi, D., 2018 Using

confidence factors to share control between a mobile

robot tele-operater and ultrasonic sensors Proc.

Intelligent Systems Conf (London) pp 1026-1033.

Sanders, D., Gegov, A., Haddad, M., Ikwan, F., Wiltshire,

D., Tan, Y., 2018 A rule-based expert system to decide

on direction and speed of a powered wheelchair Proc.

SAI Intelligent Systems Conf (London) pp 822-838.

Sanders, D., Langner, M., Bausch, N., Huang, Y.,

Khaustov, S., Simandjuntak. S., 2019 Improving

human-machine interaction for a powered wheelchair

driver by using variable-switches and sensors that

reduce wheelchair-veer Intelligent Systems and

Applications Advances in Intelligent Systems and

Computing 1038 Springer pp 1173-1191.

Sanders, D., Tewkesbury, G., Parchizadeh, H., Robertson,

J., Omoarebun, P., Malik, M., 2019 Learning to drive

with and without intelligent computer systems and

sensors to assist Intelligent Sys & App’s 1. Advances in

Intelligent Sys & Comp 868 Springer pp 1171-1181.

Sanders, D., Gegov, A., Tewkesbury, G., Khusainov, R.,

2019 Sharing driving between a vehicle driver and a

sensor system using trust-factors to set control gains

Intelligent Sys & Applications. Advances in Intelligent

Systems & Computing 868 Springer pp 1182-1195.

Sanders, D., Haddad, M., Tewkesbury, G., Thabet, G.,

Omoarebun, P., Barker, T., 2020 Simple expert system

for intelligent control and HCI for a wheelchair fitted

with ultrasonic sensors Proc. IEEE 10th International

Conference on Intelligent Systems (Varna) pp 211-216.

Sanders, D., Ndzi, D., Chester, S., Malik, M., 2016

Adjustment of tele-operator learning when provided

with different levels of sensor support while driving

mobile robots Lecture Notes in Networks and Systems

16 Springer pp 548-558.

Sanders, D., Haddad, M., Omoarebun, P., Ikwan, F.,

Chiverton, J., Zhou, S., Rogers, I., Vatchova., B 2020

Intelligent control and HCI for a powered wheelchair

using a simple expert system and ultrasonic sensors Int’

Sys & Applications 3. Advances in Intelligent Systems

& Computing 1252 Springer pp 571-583.

Model-Based Prediction for Navigation Assistance Using a Set of Sensors