Anomaly Detection in Data Streams Using Machine Learning and Deep Learning

Authors

  • Bakhtiyar Mahmood Abdullah Department of Computer Science, Faculty of Science, Soran University, Soran, Iraq
  • Muhammad Amin Daneshwar Department of Computer Science, Faculty of Science, Soran University, Soran, Iraq

DOI:

https://doi.org/10.62760/iteecs.3.4.2024.118

Keywords:

Deep learning, Data stream mining, Anomaly detection, Network intrusion detection systems

Abstract

Data stream mining for movement has emerged as an important area of machine learning because of the huge amount of changing and continuous data coming from diverse sources such as social media, business sensors, and mobile communications. The goal of this anomaly identification in the data streams is to find patterns that deviate substantially from the way things usually work. This will be valuable information for making decisions in a large number of areas, including healthcare, management of financial risk, keeping communities safe, and operating the power grid. This research discusses the intractable ways of finding oddities in a stream of data with the corresponding predicaments of always having a new inflow of data, creating information fast, and also dynamics of information changing. We also observe how distinct deep learning and machine learning approaches are being used in different fields to rapidly detect anomalies. Some examples of the way these techniques have been used to discover network intrusions, malware, IoT outliers, healthcare anomalies, and credit card frauds are a demonstration of the techniques work.

References

?. Korycki, A. Cano and B. Krawczyk, "Active Learning with Abstaining Classifiers for Imbalanced Drifting Data Streams," 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA, pp. 2334-2343, 2019.

https://doi.org/10.1109/BigData47090.2019.9006453

R. A. A. Habeeb, F. Nasaruddin, A. Gani, I. A. T. Hashem, E. Ahmed, & M. Imran, “Real-time big data processing for anomaly detection: A survey”, International Journal of Information Management, Vol. 45, pp. 289-307, 2019.

https://doi.org/10.1016/j.ijinfomgt.2018.08.006

M. A. Daneshwar & N. M. Noh, “Detection of stiction in flow control loops based on fuzzy clustering. Control Engineering Practice, Vol. 39, pp. 23-34, 2015.

https://doi.org/10.1016/j.conengprac.2015.02.002

S. Kunasekaran & C. Suriyanarayanan “Anomaly detection techniques for streaming data–an overview”, Malaya Journal of Matematik, pp. 703-710, 2020.

https://doi.org/10.26637/MJM0S20/0133

L. Rettig, M. Khayati, P. Cudré-Mauroux and M. Piórkowski, "Online anomaly detection over Big Data streams”, 2015 IEEE International Conference on Big Data (Big Data), Santa Clara, CA, USA, pp. 1113-1122, 2015.

https://doi.org/10.1109/BigData.2015.7363865

M. Sughasiny “Zero event anomaly detection in big data using spark for fast and streaming applications”, International Journal of Pure and Applied Mathematics, Vol. 119, No. 15, pp. 3407-3412, 2018. [Cross Ref].

M. A. Daneshwar, & N. M. Noh “Identification of a process with control valve stiction using a fuzzy system: A data-driven approach. Journal of Process Control, Vol. 24, No. 4, pp. 249-260, 2014.

https://doi.org/10.1016/j.jprocont.2014.01.013

D. Varmedja, M. Karanovic, S. Sladojevic, M. Arsenovic and A. Anderla, "Credit Card Fraud Detection - Machine Learning methods," 2019 18th International Symposium INFOTEH-JAHORINA (INFOTEH), East Sarajevo, Bosnia and Herzegovina, pp. 1-5, 2019.

https://doi.org/10.1109/INFOTEH.2019.8717766

H. Yao, D. Fu, P. Zhang, M. Li and Y. Liu “MSML: A Novel Multilevel Semi-Supervised Machine Learning Framework for Intrusion Detection System”, in IEEE Internet of Things Journal, Vol. 6, No. 2, pp. 1949-1959, April 2019.

https://doi.org/10.1109/JIOT.2018.2873125

G. Karatas, O. Demir and O. K. Sahingoz, "Increasing the Performance of Machine Learning-Based IDSs on an Imbalanced and Up-to-Date Dataset," in IEEE Access, Vol. 8, pp. 32150-32162, 2020.

https://doi.org/10.1109/ACCESS.2020.2973219

S. S. Dhaliwal, A. A. Nahid & R. Abbas “Effective intrusion detection system using XGBoost”, Information, Vol. 9, No. 7, pp. 149, 2018.

https://doi.org/10.3390/info9070149

E. M. Roopa Devi, & R. C. Suganthe “Enhanced transductive support vector machine classification with grey wolf optimizer cuckoo search optimization for intrusion detection system”, Concurrency and Computation: Practice and Experience, Vol. 32, No. 4, art. No. e4999, 2020.

https://doi.org/10.1002/cpe.4999

A. Alalousi, R. Razif, M. AbuAlhaj, M. Anbar, & S. Nizam “A preliminary performance evaluation of K-means, KNN and EM unsupervised machine learning methods for network flow classification”, International Journal of Electrical and Computer Engineering, Vol. 6, No. 2, pp. 778 - 784, 2016.

http://doi.org/10.11591/ijece.v6i2.pp778-784

Y. Shen, K. Zheng, C. Wu, M. Zhang, X. Niu and Y. Yang, "An Ensemble Method based on Selection Using Bat Algorithm for Intrusion Detection”, The Computer Journal, Vol. 61, No. 4, pp. 526-538, April 2018.

https://doi.org/10.1093/comjnl/bxx101

X. Gao, C. Shan, C. Hu, Z. Niu and Z. Liu, "An Adaptive Ensemble Machine Learning Model for Intrusion Detection," in IEEE Access, Vol. 7, pp. 82512-82521, 2019.

https://doi.org/10.1109/ACCESS.2019.2923640

C. Yin, Y. Zhu, J. Fei and X. He, "A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks," IEEE Access, Vol. 5, pp. 21954-21961, 2017.

https://doi.org/10.1109/ACCESS.2017.2762418

Y. Jia, M. Wang, & Y. Wang “Network intrusion detection algorithm based on deep neural network”, IET Information Security, Vol. 13, No. 1, pp. 48-53, 2019.

https://doi.org/10.1049/iet-ifs.2018.5258

G. E. Dahl, T. N. Sainath and G. E. Hinton, “Improving deep neural networks for LVCSR using rectified linear units and dropout”, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, pp. 8609-8613, 2013.

https://doi.org/10.1109/ICASSP.2013.6639346

Y. Xiao, C. Xing, T. Zhang and Z. Zhao, "An Intrusion Detection Model Based on Feature Reduction and Convolutional Neural Networks," in IEEE Access, Vol. 7, pp. 42210-42219, 2019.

https://doi.org/10.1109/ACCESS.2019.2904620

N. Chaabouni, M. Mosbah, A. Zemmari, C. Sauvignac and P. Faruki, "Network Intrusion Detection for IoT Security Based on Learning Techniques," in IEEE Communications Surveys & Tutorials, Vol. 21, No. 3, pp. 2671-2701, 2019.

https://doi.org/10.1109/COMST.2019.2896380

H. Ghallab, H. Fahmy, & M. Nasr, “Detection outliers on internet of things using big data technology. Egyptian Informatics Journal, Vol. 21, No. 3, pp. 131-138, 2020.

https://doi.org/10.1016/j.eij.2019.12.001

A. Khraisat, I. Gondal, P. Vamplew, & J. Kamruzzaman “Survey of intrusion detection systems: techniques, datasets and challenges”, Cybersecurity, Vol. 2, No. 1, pp. 1-22, 2019.

https://doi.org/10.1186/s42400-019-0038-7

Z. Zojaji, R. E. Atani, & A. H. Monadjemi “A survey of credit card fraud detection techniques: data and technique oriented perspective”, arXiv preprint arXiv:1611.06439, 2016.

https://doi.org/10.48550/arXiv.1611.06439

Additional Files

Published

2025-01-01

How to Cite

Mahmood Abdullah, B., & Daneshwar, M. A. (2025). Anomaly Detection in Data Streams Using Machine Learning and Deep Learning. International Transactions on Electrical Engineering and Computer Science, 3(4), 207–214. https://doi.org/10.62760/iteecs.3.4.2024.118

Issue

Section

Articles