ALaRI Hang Glider

Search form

Education and Innovation in Embedded Systems Design

USI Università della Svizzera italiana, USI Faculty of Informatics, Advanced Learning and Research Institute USI Università della Svizzera italiana USI Faculty of Informatics USI Advanced Learning and Research Institute
TitleA Kolmogorov-Smirnov Test to Detect Changes in Stationarity in Big Data
Publication TypeJournal Article
Year of Publication2017
AuthorsZhao, D., L. Bu, C. Alippi, and Q. Wei
Pagination14260 - 14265
Keywordschange detection test, KS test, LSDD

The paper proposes an effective change detection test for online monitoring data streams by inspecting the least squares density difference (LSDD) features extracted from two non-overlapped windows. The first window contains samples associated with the pre-change probability distribution function (pdf) and the second one with the post-change one (that differs from the former if a change in stationarity occurs). This method can detect changes by also controlling the false positive rate. However, since the window sizes is fixed after the test has been configured (it has to be small to reduce the execution time), the method may fail to detect changes with small magnitude which need more samples to reach the requested level of confidence. In this paper, we extend our work to the Big Data framework by applying the Kolmogorov-Smirnov test (KS test) to infer changes. Experiments show that the proposed method is effective in detecting changes.