Study on VR Application Efficiency of Selected Android OS

Mobile Devices

Przemyslaw Falkowski-Gilski

and Karol Fidurski

Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology,

Narutowicza 11/12, Gdansk, Poland

Keywords: Android OS, HCI (Human-Computer Interface), Mobile Devices, Multimedia Content, UX (User Experience).

Abstract: Currently, the number of scenarios for using VR (Virtual Reality) technology grows every year.

Yet, there are still issues associated with it, related with the performance of the mobile device itself.

The aim of this work is to perform an analysis of the effectiveness of virtual reality applications in case of

mobile platforms. We put the main emphasis on examining the performance and efficiency of four different

hardware and software platforms, evaluated in a number of research scenarios, related with typical user

activities. The performance of various consumer devices running Android OS was assessed using selected

benchmark applications. Additionally, a custom-build environments was also created to facilitate further

testing, including an enhanced HCI (Human-Computer Interface) linking the mobile device, head-mounted

googles, and a powerful desktop PC. The performed tests and obtained results can aid any interested individual

when choosing the right mobile device, as well as configuring the VR environment, for various UX

(User Experience) purposes.

1 INTRODUCTION

Nowadays, there are many definitions of VR (Virtual

Reality) that more or less overlap in different areas

and fields of study. When we use the word VR today,

it refers precisely to computer-generated images that

have been specifically designed to provide us with the

most immersive experience possible.

Many definitions also say that VR must be

interactive. This would distinguish it from

applications such as: 3D movies, 360 videos and other

similar enhanced media (Fan et al., 2019).

The problem is that many computer-generated VRs

are not interactive at all. In practice, a 360 video may

not be computer-generated and functionally not

different from a non-interactive VR experience.

In the historical context, what is perceived as or

adjacent to VR should be widened. Some of the

milestones were therefore precursors to other forms

of media and entertainment (Dixon, 2016). The aim

of this paper is to perform an investigation of VR

application efficiency on selected mobile devices

running Android OS.

https://orcid.org/0000-0001-8920-6969

2 BENEFITS OF VR

TECHNOLOGY

Research conducted by PricewaterhouseCoopers

(PwC, 2021) shows that the use of virtual reality

technology in conducting training and education

(Kamińska et al., 2019; Radianti et al., 2020) brings

many benefits:

 Employees using VR were able to complete

training in 4 times less time than people who

chose the lecture and about 30% faster than

people using distance learning.

 People who completed training via VR showed

40% more confidence in using new abilities than

remote students.

 VR users felt 4 times more emotionally attached

to the tasks performed than during remote

learning.

 The use of VR in learning, almost 5 times,

reduces the number of distractions during

classes, which significantly improves the user’s

focus.

Falkowski-Gilski, P. and Fidurski, K.

Study on VR Application Efﬁciency of Selected Android OS Mobile Devices.

DOI: 10.5220/0011536400003318

In Proceedings of the 18th International Conference on Web Information Systems and Technologies (WEBIST 2022), pages 407-414

ISBN: 978-989-758-613-2; ISSN: 2184-3252

407

So why are virtual reality classes not conducted in

every school or university? According to a survey

conducted by VR First in 2017 (Graham, 2017),

VR laboratories were present only in 26 out of 553

surveyed universities. Others (Getting smart, 2020)

show that approx. 97% of students would like to

attend such classes. So the demand greatly exceeds

the supply.

Of course, there are several advantages of using

VR in education (Huang et al., 2019), but the cost of

such a solution seems to be the most important factor.

While browsing for VR goggles offered in the most

popular electronics stores, it was noticed that the

prices range from a couple of hundreds to thousands

of Dollars or Euro. Now do assume a scenario where

the school would like to purchase 30 sets to conduct

classes with a group of students simultaneously at the

same time.

Yet, there are affordable devices, costing less than

100 Dollars/Euro. So where is the catch? Such a

device requires a smartphone to operate that will

generate and present the entire image that we see

through the lenses.

Since by 2022 almost every person has a mobile

phone, and consumes multimedia content on a daily

basis (Falkowski-Gilski and Uhl, 2020),

nothing prevents from using it with such goggles.

If a significant part of the costs were eliminated in

this way, many institutions could start classes with

the aid of virtual reality. The question is whether VR

on smartphones is effective enough to meet the needs

of modern-day users.

3 VR ON MOBILE ANDROID OS

DEVICES

The target image that we see on mobile devices is

rendered using the theme. Usually there are several

independent parts that generate images,

e.g., SystemUI responsible for creating the status bar

and tools, and foreground applications rendering the

image within their own buffers. Then, such a buffer

is absorbed by the client, who is responsible for their

proper distribution and finally for displaying the

appropriately arranged final effect on the screen.

In case of the Android operating system (Gilski and

Stefański, 2015), this is usually a system application

called SurfaceFlinger (Android Developer, 2020),

as shown in Figure 1.

Figure 1: Flowchart of graphical data in Android OS.

Both parts are highly independent. In fact, they are

different processes that, thanks to inter-process

communication, talk to each other and synchronize

their actions. Incorrect synchronization may lead to

corruption on the screen. An example of such

corruption is called tearing, when the display shows

information from more than one frame. Such a

phenomenon is easy to identify thanks to the visible

cut line. This happens when the client renders to a

buffer that is currently displayed to the user. In this

case, one part of the screen shows the old content and

the other shows the new content.

To prevent image tearing, in addition to

synchronization, double buffering is also a key

element. It allows the producer to render new content

while the old content is still used by the client.

When rendering is finished, the buffers are switched

over and the new content can be presented.

This mechanism is necessary for a smooth and

comfortable image reception in the Android system.

Unfortunately, it has its cost, which is additional

delay, therefore optimization is very important,

especially in the case of VR technology (Shi et al.,

2019).

4 VR PERFORMANCE

Reaching the target number of frames is a particularly

important element to provide the user with a

comfortable experience in virtual reality. VR is

computationally more expensive compared to other

projects, mainly due to the need to render per each

eye separately. In case of mobile platforms, this task

is particularly demanding not only because of the VR

overhead, but also because the power of mobile

devices is much lower than that of desktop PC units

(Ba et al., 2013).

WEBIST 2022 - 18th International Conference on Web Information Systems and Technologies

408

4.1 Optimization Methods

The most common optimization methods include:

 Static batching – combines static objects into

one large mesh. Thanks to this, when we have,

e.g., 100 models of chairs made of the same

material, we can combine them into one draw

call. This significantly reduces the amount of

memory used, compared to drawing each such

object in a separate draw call.

 Light baking – this technique refers to the initial

calculation of the lighting of a scene prior to its

launch. Usually, Unity renders each object

separately for each light that shines on a

particular object. This means that if an object

has, e.g., three lights on it, it will be rendered

three times in a given view. Thanks to the earlier

preparation of the scene, and the preparation of

textures with lighting,

we significantly reduce the number of

drawing calls.

 Occlusion culling – another optimization

technique is based on the fact that objects

outside the field of view of the camera are not

rendered at all.

4.2 Common Problems

The most common problems of VR technology,

in case of mobile devices, include:

 Limited power – the most obvious and well-

discussed challenge facing mobile virtual reality

applications is a much more limited power

budget and thermal constraints compared to the

desktop PC. Moreover,

the proximity of the processing equipment to the

user means that the thermal budget also cannot

be higher. By comparison, mobile devices

typically run below 4 W, while a single desktop

GPU can easily consume 150 W or even more.

 Device overheating – when the device detects

that it is getting too hot, it begins to intentionally

limit the amount of energy the app can use. This

will obviously result in slower computation and

will negatively affect the

FPS (Frames Per Second). Thermal limitation is

an important factor to keep in mind when

developing mobile VR applications. This is a

common problem, mainly due to the fact that

mobile virtual reality often pushes devices to

their limits. Devices enclosed in goggles,

while designed to be handheld, also contribute

to the increased thermal impact. Optimizing

CPU and GPU processing, reducing the use of

network and location services, e.g., related with

the handover mechanism, could improve energy

efficiency.

 Delay – it is very important to minimize the

delay of the head movement to its actual

representation on the device screen to give the

user the impression of real-time presence,

and being physically connected to the virtual

world. It is recommended to obtain 20 ms or less

motion latency.

In this experiment, different aspects were tested

and examined, including: obtained framerate,

device temperature and RAM consumption.

5 TEST ENVIRONMENT

The investigation was carried out using a laboratory

stand composed of a desktop PC as well as a set of 4

mobile devices.

5.1 Desktop Stand

The laboratory stand included a desktop PC with the

following configuration:

 Processor – Intel i9-9900K, 8-core, 3.60 GHz.

 GPU – NVIDIA RTX 2080Ti.

 RAM – 16 GB.

 Operating system – Windows 10.

 Googles – DreamzVR 2.0.

Next, we have performed an evaluation of

a selected group of mobile devices, namely

4 smartphones running Android OS.

5.2 Tested Mobile Devices

The tested set was composed of 4 devices, including

different manufacturers, hardware components and

version of the operating system. Their principle

technical specification is described in Table 1.

Those devices included a multi-core

configuration, either 4 or 8 units, with more than

1 GB of RAM, as well as a version of Android OS

equal to 5.1 and higher. Those models were released

on the market as a medium segment, just to meet the

needs of a typical user. It should be mentioned that

none of them was advertised as a flagship model

when being released on the market, with high-end

integrated hardware.

Study on VR Application Efﬁciency of Selected Android OS Mobile Devices

409

Table 1: Principle technical specification of tested mobile

devices.

No. Model CPU, GPU, RAM Android

1 Huawei

4-core, 1.10 GHz

Adreno 304

1 GB

5.1

2 Xiaomi

Mi9 SE

8-core, 2.30 GHz

Adreno 616

6 GB

9.0

3 Xiaomi

Mi6

8-core, 2.45 GHz

Adreno 540

4 GB

7.1

4 Xiaomi

Note 10

8-core, 2.20 GHz

Adreno 618

6 GB

9.0

Next, each device was subjected to a number of

benchmarks and stress test, in order to determine their

performance under rich multimedia rendering and

3D image processing.

6 RESULTS

As shown, the most important element of virtual

reality technology is the image itself. For this reason,

at the very beginning, it was decided to investigate

how efficient the devices are in rendering 3D scenes

in relation to each other. On each smartphone,

the Android Sling Shot test was launched in the

3DMark application (3DMark, 2022). The results are

shown in Figure 2.

Figure 2: Performance in Android Sling Shot.

As shown, the performance of each device differs

significantly. We have chosen this particular

software, because 3DMark is currently the top

benchmark for both computers and mobile devices

and has a public database of results for virtually

every device.

Next, we have decided to check the performance

in case of the most popular VR-compatible mobile

games. They were selected on the basis of top

recommended Android games (Hughes, 2022).

In case of every application, the test lasted 5 minutes.

In order to minimize the influence of other apps

running in the background, the only application

opened on the smartphone, apart from system

applications, were Gamer Bubbles and the tested

game itself. The results, describing framerate in FPS,

obtained temperature and consumed RAM resources,

are shown in Figures 3, 4 and 5, respectively.

At first glance, a typical smartphone is feasible of

obtaining a stable framerate of 60 FPS, giving a

feeling of a smooth and uninterrupted playback.

However, a noticeable difference may be observed

in case of heating of the device itself. It should be

noted that different manufacturers utilize various

materials for building the chassis as well as case.

Figure 3: Framerate in selected mobile games.

As we know, some provide a more sturdy grip,

whereas on the other side they may affect the heat

dissipation capabilities. In some cases, the placement

of the user’s hands may cause additional attenuation

of the signal, leading to an interruption in wireless

communication. Yet, this aspect was not the topic of

our study.

Figure 4: Temperature in selected mobile games.

WEBIST 2022 - 18th International Conference on Web Information Systems and Technologies

410

As shown, the typical temperature of a device put

into stress has raised to a level between 30°C and

38°C. Such a temperature will not cause discomfort

to the user.

Figure 5: RAM consumption in selected mobile games.

Surprisingly, the amount of integrated memory

proved to be sufficient when it comes to handling

complex graphical content. As noticed, neither device

did not occupy 100% of available RAM resources.

One of the reasons, aside from the mere size of this

memory, may be the upgrades made in newer releases

of the Android OS.

7 DISCUSSION

After carrying out the main objective of this study,

we have started to wonder whether there is another

way to enhance the user experience (UX) when it

comes of VR technology (Kim et al., 2020). Each and

every person can appreciate high-quality resolution,

smooth framerate, especially when one desires to

stare at the screen for a long period of time.

Eventually, we have decided to investigate what are

the possibilities of enhancing the gaming experience.

After preliminary examinations, we have agreed

to check VRidge, according to which gaming is one

of the most important needs of VR technology users.

With this software we tried to create an environment

in which we would be able to play games available

only on stationary VR-compatible PCs.

At first, it was necessary to have the RiftCat client

installed on a desktop computer (Riftcat, 2022). The

software also required a mobile client downloaded

from Google Play (VRidge, 2021).

The next step was to configure the LeapMotion

camera. For this purpose, it was necessary to download

the SDK (Software Development Kit) from the

manufacturer’s website (Leap Motion SDK, 2022). For

the camera to work with SteamVR, the Leap Driver

(Leap Driver, 2022) driver and modification of the

configuration file were also required.

Thanks to the combination of the above software

and hardware configuration, along with some custom

modifications, it was possible to run one of the most

popular VR games, namely Half-Life: Alyx, in 60

FPS, and with the possibility of hand and gesture

tracking. The photo report from the research is

presented in Figures 6-8.

Figure 6: DreamzVR 2.0 googles with an installed

smartphone and a front-mounted Leap Motion camera

running Half-Life: Alyx.

Figure 7: Half-Life: Alyx VR application with gesture

detection – gesture 1.

Study on VR Application Efﬁciency of Selected Android OS Mobile Devices

411

Figure 8: Half-Life: Alyx VR application with gesture

detection – gesture 2.

This hardware and software configuration worked

very well, especially when the user was in a static

sitting position. Therefore, several additional

applications were tested using the described

configuration, enriching it with supplementary input

devices, as shown in Figures 9 and 10.

Figure 9: Assetto Corsa car driving simulation VR

application with Leap Motion enabled.

Figure 10: VTol flight simulation VR application with Leap

Motion enabled.

It is worth mentioning that streaming delays

related with generating image sequences as well as

interpreting gestures were imperceptible. This aspect

becomes even more important in 5G networks

(Krogfoss et al., 2020).

8 SUMMARY

When analyzing the effectiveness of the set of user

terminals, it was found that mobile VR technology

still has a great potential for development. With the

use of appropriate software solutions, one can create

a full-sized virtual environment, thus fulfilling the

most important user needs. In addition, thanks to

mobile VR, in the nearest future we could, i.e.,

organize a mass show in virtual reality at a much

lower cost. Furthermore, mobile VR could serve as a

good introduction for the newest and most up to date

technologies in a very simple way. Compared to

desktop stationary solutions, it does not require

additional wiring, etc. As shown, current capabilities

offer a smooth rendering of high-quality images.

Of course, there are still other aspects, namely

problems with gesture control, high battery

consumption, heat dissipation, etc. Another issue is

the inability to use the phone when it is fixed inside

the head-mounted goggles. However, many of these

problems could be solved in the near future, with a set

of compatible wireless accessories. We do believe

that with the right support, this technology will have

its so-called second youth.

WEBIST 2022 - 18th International Conference on Web Information Systems and Technologies

412

This research investigation has shown that even

a typical mid-range smartphone is able to provide

stable VR image rendering. It is also possible to

relieve the mobile device from the rendering process,

as the whole handling of complex image sequences

may be realized with the aid of VRidge. Thanks to

this we were able to show any virtual reality

composition on the smartphone. Streaming delays

were practically imperceptible.

Recently, cloud gaming technologies are also

being developed, where computers bear the burden of

rendering the image. In this case, the mobile device

becomes only the stream recipient. In such a scenario,

delay plays an even more important role (Soliman,

et al., 2013; Huang et al., 2014; Laghari et al., 2019).

As shown, VR technology is very capable of

enhancing the user experience, even with a portable

mobile device. With the aid of appropriate goggles,

even a mid-range device becomes a good replacement

for expensive dedicated accessories. Undeniably,

a broader range of consumer application could speed

up the whole process and make VR even more

popular among people.

Future studies may and should include a broader

range of consumer devices, googles, as well as test

scenarios, including rich 3D graphics and multimedia

content. Additional source of inspiration may be

found in (Kopczyński, 2021; Langer et al., 2021;

Sermet and Demir, 2022; Cheng, 2022).

REFERENCES

3DMark. (2022). The Gamer’s benchmark for Android.

https://benchmarks.ul.com/3dmark-android

(access:. 21.06.2022).

Android Developer. (2020). https://source.android.com/

devices/graphics (access: 21.06.2022).

Ashtari, N., Bunt, A., McGrenere, J., Nebeling, M.,

Chilana, P. K. (2020). Creating augmented and virtual

reality applications: Current practices, challenges, and

opportunities. In CHI’20, Proceedings of the 2020 CHI

Conference on Human Factors in Computing Systems.

ACM.

Ba, H., Heinzelman, W., Janssen, C. A., Shi, J. (2013).

Mobile computing – A green computing resource.

In WCNC’13, 2013 IEEE Wireless Communications

and Networking Conference. IEEE.

Cheng, Y. (2022). 5G mobile virtual reality optimization

solution for communication and computing integration.

Mobile Networks and Applications, 27, 912-925.

Dixon, S. (2006). A history of virtual reality in

performance. International Journal of Performance

Arts & Digital Media, 2(1), 23-54.

Falkowski-Gilski, P., Uhl, T. (2020). Current trends in

consumption of multimedia content using online

streaming platforms: A user-centric survey. Computer

Science Review, 37, 100268.

Fan, C. L., Lo, W. C., Pai, Y. T., Hsu, C. H. (2019).

A survey on 360 video streaming: Acquisition,

transmission, and display. ACM Computing Surveys,

52(4), 1-36.

Getting smart. (2020). The future of VR & AR in education.

https://www.gettingsmart.com/2020/09/12/the-future-

of-vr-ar-in-education/ (access: 21.06.2022).

Gilski, P., Stefański, J. (2015). Android OS: A review. Tem

Journal, 4(1), 116-120.

Graham, P. (2017). Universities expanding VR hardware

inventories states VR first survey. https://www.gmw3.

com/2017/01/universities-expanding-vr-hardware-

inventories-states-vr-first-survey/ (access: 21.06.2022).

Huang, K. T., Ball, C., Francis, J., Ratan, R., Boumis, J.,

Fordham, J. (2019). Augmented versus virtual reality in

education: An exploratory study examining science

knowledge retention when using augmented

reality/virtual reality mobile applications.

Cyberpsychology, Behavior, and Social Networking,

22(2), 105-110.

Huang, C. Y., Chen, K. T., Chen, D. Y., Hsu, H. J., Hsu, C.

H. (2014). GamingAnywhere: The first open source

cloud gaming system. ACM Transactions on

Multimedia Computing, Communications, and

Applications, 2(3), 1-25.

Hughes, A. (2022). 17 of the best AR and VR apps and

games for iOS and Android. https://www.sciencefocus.

com/future-technology/best-ar-and-vr-apps-and-games

-for-ios-and-android/ (access: 21.06.2022).

Kamińska, D., Sapiński, T., Wiak, S., Tikk, T., Haamer, R.

E., Avots, E., Helmi, A., Ozcinar, C., Anbarjafari, G.

(2019). Virtual reality and its applications in education:

Survey. Information, 10(10), 318.

Kim, Y. M., Rhiu, I., Yun, M. H. (2020). A systematic

review of a virtual reality system from the perspective

of user experience. International Journal of Human-

Computer Interaction, 36(10), 893-910.

Kopczyński, M. (2021). Optimizations for fast wireless

image transfer using H.264 codec to android mobile

devices for virtual reality applications. In: Zamojski,

W., Mazurkiewicz, J., Sugier, J., Walkowiak, T.,

Kacprzyk, J. (eds) Theory and Engineering of

Dependable Computer Systems and Networks.

DepCoS-RELCOMEX 2021, Springer, Cham, 203-212.

Krogfoss, B., Duran, J., Perez, P., Bouwen, J. (2020).

Quantifying the value of 5G and edge cloud on QoE for

AR/VR. In QoMEX’20, 12th International Conference

on Quality of Multimedia Experience. IEEE.

Laghari, A. A., He, H., Memon, K. A., Laghari, R. A.,

Halepoto, I. A., Khan, A. (2019). Quality of experience

(QoE) in cloud gaming models: A review. Multiagent

and Grid Systems, 15(3), 289-304.

Langner, R., Satkowski, M., Büschel, W., Dachselt, R.

(2021). MARVIS: Combining mobile devices and

augmented reality for visual data analysis. In CHI’21,

Proceedings of the 2021 CHI Conference on Human

Factors in Computing Systems. ACM.

Study on VR Application Efﬁciency of Selected Android OS Mobile Devices

413

Leap Driver. (2022). https://github.com/SDraw/driver_leap

(access: 21.06.2022).

Leap Motion SDK. (2022). https://developer.leapmotion.

com/tracking-software-download (access: 21.06.2022).

PwC. (2021). How virtual reality is redefining soft

skills training. https://www.pwc.com/us/en/tech-effect/

emerging-tech/virtual-reality-study.html (access:

21.06.2022).

Radianti, J., Majchrzak, T. A., Fromm, J., Wohlgenannt, I.

(2020). A systematic review of immersive virtual

reality applications for higher education: Design

elements, lessons learned, and research agenda.

Computers & Education, 147, 103778.

Riftcat. (2022). VRidge. https://riftcat.com/vridge

(access: 21.06.2022).

Sermet, Y., Demir, I. (2022). GeospatialVR: A web-based

virtual reality framework for collaborative

environmental simulations. Computers & Geosciences,

159, 105010.

Shi, S., Gupta, V., Hwang, M., Jana, R. (2019).

Mobile VR on edge cloud: a latency-driven design.

In MMSys’19, 10th ACM Multimedia Systems

Conference. ACM.

Soliman, O., Rezgui, A., Soliman, H., Manea, N. (2013).

Mobile Cloud Gaming: Issues and Challenges.

In: Daniel, F., Papadopoulos, G.A., Thiran, P. (eds)

Mobile Web Information Systems. MobiWIS 2013,

Springer, Berlin, 121-128.

VRidge 2.0. (2021). https://play.google.com/store/apps/

details?id=com.riftcat.vridge2 (access: 21.06.2022).

Khundam, C., Vorachart, V., Preeyawongsakul, P., Hosap,

W., Noël, F. (2021). A comparative study of interaction

time and usability of using controllers and hand

tracking in virtual reality training. Informatics, 8(3), 60.

WEBIST 2022 - 18th International Conference on Web Information Systems and Technologies

414