Table 1: Performance (on 14 sequences) of self-adaptive
tracking and benchmarks.
Method AUC
recall
AUC
precision
Manual tuning 0.429 0.532
(on 6 sequences)
Auto-regulation 0.437 0.629
using KM-F (batch)
Auto-regulation 0.431 0.648
using Histo-EM-F
No regulation 0.414 0.391
(low thresholds)
No regulation 0.243 0.426
(high thresholds)
by evaluating the output of the system that uses the
parameter setting chosen by the auto-regulation algo-
rithm. Table 1 shows the results of the different meth-
ods on the 14 test sequences. The auto-regulation
methods are compared to three benchmarks. The up-
per benchmark is our tracking system with manually
tuned parameters. We only have the results for 6 test
sequences due to the tedious manual task. The lower
benchmarks are provided by using static parameter
settings for all test sequences. Low thresholds give
good recall but bad precision, high thresholds give
bad recall and bad precision.
We compare the auto-regulation method using a
batch model KM-F and the best performing incremen-
tal method Histo-EM-F. The KM-F model is obtained
using a KMeans clustering with 1000 initial clusters
that are subsequently merged. All clustering solutions
from 100 to 5 clusters are transformed to a GMM.
The model obtained by the method Histo-EM-F uses
a fine grain histogram for initialisation of EM. The
Gaussians computed by EM are subsequently merged
to provide a fusion tree. In both methods, we choose
the least complex model with a P
err
value below the
acceptance threshold of 5% (see paragraph below).
The incremental model and the batch model both
match the performance of a manually tuned system
(although the manually tuned system is evaluated only
on 6 sequences, the AUC values give an idea of the
performance range). The increase in performance us-
ing automatic parameter regulation with respect to us-
ing no regulation is demonstrated clearly by the com-
parison to the lower benchmark performance. An-
other important result is that no significant differ-
ence can be noted between the incremental and non-
incremental model. This motivates the use of incre-
mental models in the future due to their ability of fur-
ther refinement with additional data.
Link between P
err
and tracking performance
In section 3.2 we proposed a measure for model se-
lection based on the probability of classification er-
ror. This is a convenient and fast measure for model
selection. The definition and validation of such a fast
quality measure makes possible the automatic gener-
ation and selection of a scene reference model in a
non-supervised commercial tracking application. For
validation of this measure, we need to show the rela-
tion between P
err
and the tracking performance.
We perform following experiment. 4 sequences are
selected among the test sequences with different de-
gree of difficulty (1 easy, 2 intermediate and 1 hard).
We compute the P
err
scores for a family of mod-
els extracted from different levels of the fusion tree
produced by the GNGN-EM-F approach (a growing
neural gas network (Fritzke, 1995) is used for initial-
isation of EM with fusion). P
err
requires the selec-
tion of a representative set of positive and negative
examples. The positive examples are composed of 7
trajectories of the most common paths of the scene
(2420 measurements). The negative examples are ob-
tained by monitoring the tracking output and collect-
ing tracking errors (1997 measurements). Each model
is then used for auto-regulation of parameters. The
XML output of the tracking system using the selected
parameter setting is evaluated and the AUC values of
precision and recall are computed.
Table 2 shows these results. The results confirm the
relation between the P
err
value and the global per-
formance of the system using auto-regulation. Small
P
err
values (good model quality) yield good track-
ing performance (high precision and high recall). The
results show also that no further improvement is ob-
served for P
err
values below 5%. We observed a sim-
ilar behaviour for model families obtained by other
learning approaches. The performance stabilises for
P
err
values below 5%. As a consequence, in this par-
ticular system setup, a model with a P
err
value below
5% is appropriate for parameter regulation.
5 CONCLUSION AND OUTLOOK
We described an architecture for a self-adaptive
tracking system that uses a control component to
implement the abilities of auto-criticism and auto-
regulation. Both modules require a metric with re-
spect to a reference model. Our approach allows to
automatically generate and select such a reference
model with good quality without human supervision.
The quality of the model is determined by a fast eval-
uation measure based on the classification error with
respect to a validation data set.
The experiments show that a tracking system with
auto-regulation of parameters has the same or better
performance than a tracking system with manually
tuned parameters. We also demonstrate that our au-
tomatic parameter selection scheme selects parame-
ter settings with very high performance. The exper-
AN AUTOMATIC APPROACH FOR PARAMETER SELECTION IN SELF-ADAPTIVE TRACKING
25