Development of Security Anomaly Detection Algorithms using Machine Learning

Hyunwoo Hwangbo, Jae Kyung Kim

Abstract


With the development of network technologies, the security to protect organizational resources from internal and external intrusions and threats becomes more important. Therefore in recent years, the anomaly detection algorithm that detects and prevents security threats with respect to various security log events has been actively studied. Security anomaly detection algorithms that have been developed based on rule-based or statistical learning in the past are gradually evolving into modeling based on machine learning and deep learning. In this study, we propose a deep-autoencoder model that transforms LSTM-autoencoder as an optimal algorithm to detect insider threats in advance using various machine learning analysis methodologies. This study has academic significance in that it improved the possibility of adaptive security through the development of an anomaly detection algorithm based on unsupervised learning, and reduced the false positive rate compared to the existing algorithm through supervised true positive labeling.


Full Text:

PDF

References


Ahmed, M., Mahmood, A. N., and Hu, J., “A survey of network anomaly detection techniques”, Journal of Network and Computer Applications, Vol. 60, pp. 19-31, 2016.

Alla, S. and Adari, S. K., “Beginning anomaly detection using python-based deep learning,” Apress, 2019.

Cadez, I., Heckerman, D., Meek, C., Smyth, P., and White, S., “Visuꠓalization of navigation patterns on a web site using model-based clustering” In: Proceedings of the sixth ACM SIGKDD international conference on knowledge discovery and data mining, pp. 280-284, 2000.

Casas, P., Soro, F., Vanerio, J., Settanni, G., and D’Alconzo, A., “Network security and anomaly detection with Big-DAMA, a big data analytics framework,” IEEE 6th International Conference on Cloud Networking (CloudNet), pp. 1-7, 2017.

Cha, B., Park, K., and Seo, J., “Network based anomaly intrusion detection using bayesian network techniques,” Journal of Internet Computing and Services, Vol. 6, No. 1, pp. 27-38, 2005.

Criste, L., “Insider threat market to top $1 billion in fiscal 2020: This is,” Available from: https://about.bgov.com/news/insider-threat-market-to-top-1-billion-in-fiscal-2020-this-is/.

Forrest, S., Hofmeyr, S., Somayaji, A., and Longstaff, T. A., “A sense of self for unix processes,” Proceedings 1996 IEEE symposium on security and privacy, pp. 120-128, 1996.

Habeeb, R. A. A., Nasaruddin, F., Gani, A., Hashem, I. A. T., Ahmed, E., and Imran, M., “Real-time big data processing for anomaly detection: A survey,” International Journal of Information Management, Vol. 45, pp. 289-307, 2019.

Hofmeyr, S., Forrest, S., and Somayaji, A., “Intrusion detection using sequences of system calls,” Journal of computer security, Vol. 6, No. 3, pp. 151-180, 1998.

Hollmen J. and Tresp, V., “Call-based fraud detection in mobile communication networks using a hierarchical regime-switching model,” In Advances in Neural Information Processing Systems, pp. 889-895, 1999.

Kang, G.-H., Sohn, J.-M., and Sim, G.-W., “Comparative analysis of anomaly detection models using AE and suggestion of criteria for determining outliers,” Journal of Korea Society of Computer Information, Vol. 26, No. 8, pp. 23-30, 2021.

Kim, H., Kim, J., Park, M, Cho, S., and Kang, P., “Insider threat detection based on user behavior model and novelty detection algorithms,” Journal of the Korean Institute of Industrial Engineers, Vol. 43, No. 4, pp. 276-287, 2017.

Lee, J. and Lee, K. Y., “An anomalous sequence detection method based on an extended LSTM autoencoder,” The Journal of Society for e-Business Studies, Vol. 26, No. 1, pp.127-140, 2021.

Liang, N. Biros, D. P., and Luse, A., “An empirical validation of malicious insider characteristics,” Journal of Management Information Systems, Vol. 33, No. 2, pp. 361-392, 2016.

Lopez, E. and Sartip, K., “Detecting the insider’s threat with long short term memory (LSTM) neural networks,” arXiv, 2007. 11956.

Roh, K.-W., Kim, J.-S., and Cho, W.-S., “A Study on the design of supervised and unsupervised learning models for fault and anomaly detection in manufacturing facilities,” The Journal of Bigdata, Vol. 6, No. 1, pp. 23-35, 2021.

Smyth, P., “Clustering sequences with hidden markov models,” Advances in Neural Information Processing Systems, pp. 648-654, 1997.

Theoharidou, M., Kokolakis, S., Karyda, M., and Kiountouzis, E., “The insider threat to information systems and the effectiveness of ISO17799,” Computers & Security, Vol. 24, No. 6, pp. 472-484, 2005.

Vanerio, J. and Casas, P., “Ensemble-learning approaches for network security and anomaly detection,” Proceedings of the Workshop on Big Data Analytics and Machine Learning for Data Communication Networks, pp. 1-6, 2017.

Warrender, C., Forrest, S., and Pearlmutter, B., “Detecting intruꠓsions using system calls: Alternative data models,” Proceedings of the 1999 IEEE symposium on security and privacy, pp. 133-145, 1999.

Xu, K., Tian, K., Yao, D., and Ryder, B.., “A sharper sense of self: Probabilistic reasoning of program behaviors for anomaly detection with context sensitivity,” 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 467-478, 2016.

Xu, K., Yao, D. D., Ryder, B. G., and Tian, K., “Probabilistic program modeling for high-precision anomaly classification” Computer Security Foundations Symposium (CSF), IEEE 28th. pp.497–511, 2015.

Yao, D., Shu, X., Cheng, L., and Stolfo, S. J., “Anomaly detection as a service: Challenges, advances, and opportunities,” Morgan & Claypool, 2017.

Yeung, D.-Y. and Ding, Y., “Host-based intrusion detection using dynamic and static behavioral models,” Pattern Recognition, Vol. 36, No. 1, pp. 229-243, 2003.


Refbacks

  • There are currently no refbacks.