TY - GEN
T1 - Evaluating sequence discovery systems in an abstraction-aware manner
AU - Rogers, Eoin
AU - Ross, Robert J.
AU - Kelleher, John D.
N1 - Publisher Copyright:
© IFIP International Federation for Information Processing 2018 Published by Springer International Publishing AG 2018. All Rights Reserved.
PY - 2018
Y1 - 2018
N2 - Activity discovery is a challenging machine learning problem where we seek to uncover new or altered behavioural patterns in sensor data. In this paper we motivate and introduce a novel approach to evaluating activity discovery systems. Pre-annotated ground truths, often used to evaluate the performance of such systems on existing datasets, may exist at different levels of abstraction to the output of the output produced by the system. We propose a method for detecting and dealing with this situation, allowing for useful ground truth comparisons. This work has applications for activity discovery, and also for related fields. For example, it could be used to evaluate systems intended for anomaly detection, intrusion detection, automated music transcription and potentially other applications.
AB - Activity discovery is a challenging machine learning problem where we seek to uncover new or altered behavioural patterns in sensor data. In this paper we motivate and introduce a novel approach to evaluating activity discovery systems. Pre-annotated ground truths, often used to evaluate the performance of such systems on existing datasets, may exist at different levels of abstraction to the output of the output produced by the system. We propose a method for detecting and dealing with this situation, allowing for useful ground truth comparisons. This work has applications for activity discovery, and also for related fields. For example, it could be used to evaluate systems intended for anomaly detection, intrusion detection, automated music transcription and potentially other applications.
UR - http://www.scopus.com/inward/record.url?scp=85049585928&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-92007-8_23
DO - 10.1007/978-3-319-92007-8_23
M3 - Conference contribution
AN - SCOPUS:85049585928
SN - 9783319920061
T3 - IFIP Advances in Information and Communication Technology
SP - 261
EP - 272
BT - Artificial Intelligence Applications and Innovations - 14th IFIP WG 12.5 International Conference, AIAI 2018, Proceedings
A2 - Plagianakos, Vassilis
A2 - Maglogiannis, Ilias
A2 - Iliadis, Lazaros
PB - Springer New York LLC
T2 - 14th IFIP WG 12.5 International Conference on Artificial Intelligence Applications and Innovations, AIAI 2018
Y2 - 25 May 2018 through 27 May 2018
ER -