of 0.989 and recall of 0.888 on this task.
While previous approaches focused on artificially
generated corruption, our study presents a workflow
for collecting real display corruption of various types
from a vast range of video games while using Struc-
tural Similarity Index Measure (SSIM) to ensure the
collected images are visually diverse. We presented
a two-stage training procedure and demonstrated its
effectiveness through a variety of neural networks
achieving high validation performance. We presented
the distribution of correct and incorrect predictions on
corrupted images for our top-performing model and
argued that, with sufficient training data per corrup-
tion type, a DCNN can be successfully trained to de-
tect a wide variety of graphical malfunctions. Finally,
we showed that Grad-CAM can be leveraged to pro-
vide interpretability in our neural net’s predictions.
7 FUTURE WORK
The main shortcoming of our pipeline is the gameplay
being highly scripted throughout the game tests. In
the future, we plan on extending our method to pro-
vide a gameplay testing experience close to human
behavior. This allows for more in-game exploration,
thus providing more visually varied gameplay scenar-
ios and a potentially diverse source of corrupted im-
ages.
Moreover, our current method detects corruption
in individual images without accounting for adja-
cent game frames. Incorporating video understanding
with Long-term Recurrent Convolutional Networks
(J. Donahue and Darrell, 2017) could provide valu-
able insights on the game context and potentially im-
prove the overall performance.
In this study, we treated the task at hand as a bi-
nary classification problem. Given sufficient data per
corruption category, a DCNN could be trained as a
multi-label classifier to effectively detect each corrup-
tion subtype separately. In GPU testing workflow, this
could further minimize the amount of manual triage
required upon the detection of visual corruption.
REFERENCES
A. Krizhevsky, I. S. and Hinton, G. (2012). Imagenet clas-
sification with deep convolutional neural network. In
NIPS.
A. Nantes, R. B. and Maire, F. (2008). A framework for
the semi-automatic testing of video games. In Pro-
ceedings of the Fourth AAAI Conference on AI and
Interactive Digital Entertainment, page 197–202.
A. Nantes, R. B. and Maire, F. (2013). Neural net-
work based detection of virtual environment anoma-
lies. In Neural Computing and Applications, page
1711–1728.
C. Ling, K. T. and Gissl
´
en, L. (2020). Using deep con-
volutional neural networks to detect rendered glitches
in video games. In Proceedings of the AAAI Con-
ference on AI and Interactive Digital Entertainment,
pages 66–73.
E. McLaughlin, N. C. and Narasimhan, S. (2019). Combin-
ing deep learning and robotics for automated concrete
delamination assessment. In Proceedings of the 36th
ISARC, pages 485–492.
G. Huang, Z. Liu, L. V. D. M. and Weinberger, K. Q. (2017).
Densely connected convolutional networks. In IEEE
Conference on Computer Vision and Pattern Recogni-
tion, pages 2261–2269.
J. Donahue, L. A. Hendricks, S. G. M. R. S. V. K. S. and
Darrell, T. (2017). Long-term recurrent convolutional
networks for visual recognition and description. In
IEEE Transactions on Pattern Analysis and Machine
Intelligence, pages 677–691.
K. He, X. Zhang, S. R. and Sun, J. (2016). Deep residual
learning for image recognition. In IEEE Conference
on Computer Vision and Pattern Recognition, pages
770–778.
Kingma, D. and Ba, J. (2014). Adam: A method for
stochastic optimization. In arXiv:1412.6980.
M. Sandler, A. Howard, M. Z. A. Z. and Chen, L. (2018).
Mobilenetv2: Inverted residuals and linear bottle-
necks. In IEEE Conference on Computer Vision and
Pattern Recognition, page 4510–4520.
N. Ma, X. Zhang, H. Z. and Sun, J. (2018). Shufflenet v2:
Practical guidelines for efficient cnn architecture de-
sign. In Proceedings of the European Conference on
Computer Vision, page 116–131.
P. Davarmanesh, K. Jiang, T. O. A. V. S. I. M. K. S. J. and
Malaya, N. (2020). Automating artifact detection in
video games. arXiv:2011.15103.
R. Selvaraju, M. Cogswell, A. D. R. V. D. P. and D.Batra
(2017). Grad-cam: Visual explanations from deep net-
works via gradientbased localization. In IEEE Inter-
national Conference on Computer Vision, pages 618–
626.
Simonyan, K. and Zisserman, A. (2014). Very deep convo-
lutional networks for large-scale image recognition. In
arXiv:1409.1556.
Tan, M. and Le, Q. (2019). Efficientnet: Rethinking model
scaling for convolutional neural networks. In Proceed-
ings of the 36th International Conference on Machine
Learning, pages 6105–6114.
Y. Chen, F. Yang, E. S. and Ho, C. (2019). Automatic defect
detection system based on deep convolutional neural
networks. In International Conference on Engineer-
ing, Science, and Industrial Applications, pages 1–4.
Z. Wang, A. Bovik, H. S. and Simoncelli, E. (2004). Image
quality assessment: from error visibility to structural
similarity. In IEEE Transactions on Image Processing,
pages 600–612.
VISAPP 2022 - 17th International Conference on Computer Vision Theory and Applications
908