The discrete Fourier transformation for seasonality and anomaly detection of an application to rare data
Purpose The discrete Fourier transformation (DFT) has been proven to be a successful method for determining whether a discrete time series is seasonal and, if so, for detecting the period. This paper deals exclusively with rare data, in which instances occur periodically at a low frequency. Design/methodology/approach Data based on real-world situations is simulated for analysis. Findings Cycle number detection is done with spectral analysis, period detection is completed using DFT coefficients and signal shifts in the time domain are found using the convolution theorem. Additionally, a new method for detecting anomalies in binary, rare data is presented: the sum of distances. Using this method, expected events which have not occurred and unexpected events which have occurred at various sampling frequencies can be detected. Anomalies which are not considered outliers to be found. Research limitations/implications Aliasing can contribute to extra frequencies which point to extra periods in the time domain. This can be reduced or removed with techniques such as windowing. In future work, this will be explored. Practical implications Applications include determining seasonality and thus investigating the underlying causes of hard drive failure, power outages and other undesired events. This work will also lend itself well to finding patterns among missing desired events, such as a scheduled hard drive backup or an employee's regular login to a server. Originality/value This paper has shown how seasonality and anomalies are successfully detected in seasonal, discrete, rare and binary data. Previously, the DFT has only been used for non-rare data.
Year of publication: |
2020
|
---|---|
Authors: | Collins Jackson, Aryana ; Lacey, Seán |
Published in: |
Data Technologies and Applications. - Emerald Publishing Limited, ISSN 2514-9318, ZDB-ID 2935212-5. - Vol. 54.2020, 2, p. 121-132
|
Publisher: |
Emerald Publishing Limited |
Subject: | Discrete Fourier transformation | Rare data | Discrete data | Seasonality detection | Anomaly detection |
Saved in:
Online Resource
Saved in favorites
Similar items by subject
-
Inference without smoothing for large panels with cross-sectional and temporal dependence
Hidalgo, Javier, (2021)
-
Inference without smoothing for large panels with cross-sectional and temporal dependence
Hidalgo, Javier, (2017)
-
Recognizing Driving Behavior and Road Anomaly using Smartphone Sensors
Ali, Aya Hamdy, (2017)
- More ...