A Self-adaptive Mechanism for Serious Quiz Games

Michael Striewe

a

University of Duisburg-Essen, Essen, Germany

Keywords:

Serious Quiz Games, Self-Adaptation, Quiz Difficulty.

Abstract:

Serious quiz games can provide an entertaining way to assess knowledge and promote learning. Motivation of

the players can be increased by presenting quiz questions in the order of increasing difficulty or by presenting

1 INTRODUCTION

Serious quiz games can be an entertaining and mo-

tivating tool for self-assessment. They can pro-

mote learning in various educational scenarios or help

teachers and students to track progress. Although em-

pirical results reveal some downsides or limited ef-

fects, a general positive effect can be confirmed in

several studies (Heitmann et al., 2021; Becker-Blease

and Bostwick, 2016; Simon-Campbell and Phelan,

2016; Wang, 2008; Wickline and Spektor, 2011). Se-

tions and answers right (Wang and Tahir, 2020). Neg-

ative effects e. g. on motivation can be expected both

for presenting too easy questions to advanced players

as well as for presenting too hard questions for be-

ginners. Another negative effect on motivation could

possibly be observed if questions are shown repeat-

edly to the same player within a short time frame. If

little is known about the target audience and thus it

cannot be foreseen which difficulty levels will be used

most, constant effort is required to adjust the size of

be used as there is no additional extrinsic motivation

be less resources available for maintaining and cali-

brating the question pool. Finally, there may also little

be known about the level of knowledge within the tar-

get audience. Hence, sophisticated methods for gen-

erating quizzes (like e. g. proposed by (Lin, 2020))

are hardly applicable in these contexts.

nism to solve these challenges: The mechanism auto-

matically maintains a difficulty rating for each ques-

tion and adjusts the number of questions used per

a public website and been used in more than 100’000

quiz sessions. Based on these data and a close inspec-

tion of all quiz sessions from the year 2021, the pa-

per evaluates whether both the difficulty rating of the

quiz questions and the behaviour of players in recent

quiz sessions show the expected characteristics. The

evaluation aims to find out whether the mechanism is

indeed able to solve the aforementioned challenges.

lows: Section 2 provides a detailed description of the

site. Section 4 provides usage data for the quiz imple-

mentation and discusses whether the behaviour is as

expected. Section 5 concludes the paper.

2 QUIZ MECHANISM

reached. To ensure large coverage of a domain, at

most one question per category or topic is to be used

during one session. Categories can also be used if

there are questions that are mutually exclusive in the

sense that one question text may directly reveal the

answer to another question. Nevertheless, the use

of categories has no further implications for the quiz

mechanism and can thus be considered optional.

The quiz mechanism could also be used for other

forms of quizzes, such as serving sessions of fixed

tructure that can be implemented in form of two data

tables: One data table contains the quiz questions and

particularly stores the category or topic for each ques-

tion and a difficulty rating in the range from 0 to 1,

where higher numbers denote easier questions. The

second data table contains an entry for each difficulty

level and stores the number of questions delivered so

far for that level.

New questions can be inserted into the question set

=

(

Hence, a question with rating 0.5 will get a new

rating of 0.505 if answered correctly and a new rating

a more recent answer has a larger impact on the rating

than an older one. Hence, a question that is first an-

swered wrong and then answered correctly will have

a higher rating than a question that is first answered

be considered specifically important in informal set-

tings, where the actual knowledge of players is based

on random sources and not bound the any well known

learning materials. Hence, changes in the knowledge

sources are not under any control and it may thus be

helpful if the quiz mechanism can adapt quickly to

such changes.

While it is tempting to map difficulty ratings directly

to difficulty levels, doing so can cause serious prob-

may miss some categories and populating a quiz ses-

sion in the desired way may become harder. Hence,

the algorithm populates the quiz session in the re-

versed order from the hardest to the easiest question.

a question is picked randomly. As mentioned above,

an implication of that mechanism is that there should

be at least as much different categories available as

their are levels in the quiz game.

associated with a small award, level 10 is associated

with a medium award. If players answer correctly in

the final question on level 15, they get a large award.

content will be discussed in section 4.2 below.

actually started. The average value is 2.1 quiz starts

per quiz page visit, but the actual values range be-

tween less than 0.7 in April 2021 and more than 5.0

Figure 1: Screenshot from the website presenting the quiz (https://www.roemercohorte.de/de/quiz). The quiz page provides

a textual introduction on how to use the quiz. Below the text, it presents the current level indicated on the bar, the current

question, four answer options (A - D), and two buttons to use a joker and restart the quiz. Although the website also contains

some English texts, the quiz is only available in German.

Visits

Starts

Answers

Questions

Rating

Rating

Rating

1 66 0.9523 0.9336 0.9976

2 64 0.9193 0.9054 0.9335

3 60 0.8936 0.8818 0.9052

4 57 0.8718 0.8627 0.8817

5 52 0.8515 0.8390 0.8626

6 47 0.8250 0.8138 0.8387

7 42 0.8013 0.7891 0.8134

8 38 0.7781 0.7662 0.7890

9 34 0.7579 0.7509 0.7645

quiz starts per quiz page visit in June 2021. The av-

erage length of a quiz session is about 6.2 answers

and the players use about 0.6 jokers per session on

average. More detailed usage figures are examined in

section 4.3 below.

As mentioned above, the quiz currently contains 596

questions that are distributed over 25 categories. The

question set has not seen much changes in recent

years and only few questions have been added to the

as well as the minimum, maximum and average rat-

ing per level as of February 14th, 2022. Since the

quiz mechanism calculates the available questions

per level directly from the number of actually shown

questions per level, numbers decrease monotonously

similar to figure 2, but softened due to the use of the

square root function.

The first and the last level show the largest dif-

ferences between minimum and maximum rating per

level with values of 0.064 for the first level and 0.26

mum and maximum is only 0.0055. A question from

level 13 may thus drop to level 15 directly if answered

wrong. Likewise, a question from level 15 may jump

to level 13 if answered correctly.

be a need for some more easy questions for the lower

levels. Adding such questions will also prevent ques-

tions from moving more than one level, because level

14 will than cover a wider range of ratings.

number of questions in each pool and their minimum,

maximum and average rating. The ratings show that

where the minimum rating is 0.78 or lower in all

cases and the maximum rating is 0.92 or higher in all

cases. The easiest topic seems to be “Famous Quotes”

with an average rating of 0.88, while the hardest topic

seems to be “Art and Literature” with an average rat-

ing of 0.76. The latter also contains the hardest ques-

tion of the quiz with a rating of 0.43: “Who wrote the

’Dialogus de oratoribus’?” - (A) Sueton, (B) Ovid,

(C) Pliny the Elder, (D) Tacitus. Notably, that is the

only question with a rating below 0.5.

mer one or some harder questions to the latter one.

files for a detailed analysis.

each session in terms of given answers. The x-axis in-

cludes a bar for 0, because visitors may end a session

without giving any answer at all. Since the log files do

not reveal whether an answer was correct or wrong, it

cannot be distinguished where a session ends due to a

wrong answer or because a visitor stopped playing or

15 levels in any case, independent of the correctness

of the given answer.

Section 4.1 already mentioned that the average

from levels 3 to 10 and a lower decrease on the earlier

and later levels.

The box plot in figure 4 provides median values

and quartiles for the time between showing a ques-

tion and getting an answer. The y-axis is cut-off at

80 seconds and thus the plot does not include larger

outliers. The server log files capture timestamps for

events in the granularity of seconds and thus there are

often the same values for two or more adjacent lev-

els. Nevertheless, the plot shows a general trend of

a player uses a joker on a given level. Probability is

calculated by the number of jokers used on a particu-

response theory method for alleviatingthe cold-start

problem in adaptive learning environments. Behavior

Research Methods, 51:895–909.

Simon-Campbell, E. and Phelan, J. (2016). Effectiveness of

an Adaptive Quizzing System as an Institutional-Wide

Strategy to Improve Student Learning and Retention.

Nurse Educator, 41(5):246–251.

Wang, A. I. and Tahir, R. (2020). The effect of using Ka-

hoot! for learning – A literature review. Computers &

Education, 149:103818.

ing of Psychology, 38(2):98–101.