Multimodal Sentiment Analysis Using Review Data and Product Information

Hohyun Hwang, Kyeongchan Lee, Jinyi Yu, Younghoon Lee

Abstract


Due to recent expansion of online market such as clothing, utilizing customer review has become a major marketing measure. User review has been used as a tool of analyzing sentiment of customers. Sentiment analysis can be largely classified with machine learning-based and lexicon-based method. Machine learning-based method is a learning classification model referring review and labels. As research of sentiment analysis has been developed, multi-modal models learned by images and video data in reviews has been studied. Characteristics of words in reviews are differentiated depending on products' and customers’ categories. In this paper, sentiment is analyzed via considering review data and metadata of products and users. Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), Self Attention-based Multi-head Attention models and Bidirectional Encoder Representation from Transformer (BERT) are used in this study. Same Multi-Layer Perceptron (MLP) model is used upon every products information. This paper suggests a multi-modal sentiment analysis model that simultaneously considers user reviews and product meta-information.


Full Text:

PDF

References


Agarwal, A., Yadav, A., and Vishwakarma, D. K., “Multimodal sentiment analysis via RNN variants,” 2019 IEEE International Conference on Big Data, Cloud Computing, Data Science & Engineering (BCD), IEEE, 2019.

Ambartsoumian, A. and Popowich, F., “Self-attention: A better building block for sentiment analysis neural network classifiers,” arXiv preprint arXiv:1812.07860, 2018.

Bahdanau, D., Cho, K., and Bengio, Y., “Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473, 2014.

Baltrušaitis, T., Ahuja, C., and Morency, L., “Multimodal machine learning: A survey and taxonomy,” IEEE transactions on pattern analysis and machine intelligence, Vol. 41, No. 2, pp. 423-443, 2018.

Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y., “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” arXiv preprint arXiv:1406.1078, 2014.

Devlin, J., Chang, M., Lee, K., and Toutanova, K., “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.

Gruber, N. and Jockisch, A., “Are GRU cells more specific and LSTM cells more sensitive in motive classification of text?,” Frontiers in Artificial Intelligence, Vol. 3, 2020.

Hochreiter, S. and Schmidhuber, J., “Long short-term memory,” Neural Computation, Vol. 9. No. 8, pp. 1735-1780, 1997.

Huang, Q., Chen, R., Zheng, X., and Dong, Z., “Deep sentiment representation based on CNN and LSTM”, 2017 International Conference on Green Informatics (ICGI), IEEE, 2017.

Hwang, S. and Kim, D., “BERT-based Classification Model for Korean Documents,” The Journal of Society for e-Business Studies, Vol. 25, No. 1, 2020.

Jeon, W., Lee, Y., and Geum, Y., “Airline Service Quality Evaluation Based on Customer Review Using Machine Learning Approach and Sentiment Analysis,” The Journal of Society for e-Business Studies, Vol. 26, No. 4, pp. 15-36, 2021.

Jin, Z., Cao, J., Guo, H., Zhang, Y., and Luo, J., “Multimodal fusion with recurrent neural networks for rumor detection on microblogs,” Proceedings of the 25th ACM international conference on Multimedia, 2017.

Kingma, D. P. and Ba, J., “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.

Mohan, S., Mullapudi, S., Sammeta, S., Vijayvergia, P., and Anastasiu, D. C., “Stock price prediction using news sentiment analysis,” 2019 IEEE Fifth International Conference on Big Data Computing Service and Applications (BigDataService), IEEE, 2019.

Oh, P. and Hwang, B., “Real-time Spatial Recommendation System based on Sentiment Analysis of Twitter,” The Journal of Society for e-Business Studies, Vol. 21, No. 3, pp. 15-28, 2016.

Pannala, N. U., Nawarathna, C. P., Jayakody, J. T. K., Rupasinghe, L., and Krishnadeva, K., “Supervised learning based approach to aspect based sentiment analysis,” 2016 IEEE International Conference on Computer and Information Technology (CIT), IEEE, 2016.

Pires, T., Schlinger, E., and Garrette, D., “How Multilingual is Multilingual BERT?,” Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4996-5001, 2019.

Poria, S., Cambria, E., and Gelbukh, A., “Deep convolutional neural network textual features and multiple kernel learning for utterance-level multimodal sentiment analysis,” Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015.

Severyn, A. and Moschitti, A., “Twitter sentiment analysis with deep convolutional neural networks,” Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2015.

Smeureanu, I. and Zurini, M., “Spam Filtering for Optimization in Internet Promotions using Bayesian Analysis,” Journal of Applied Quantitative Methods, Vol. 5, No. 2, pp. 198-211, 2010.

Sun, C., Huang, L., and Qiu, X., “Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence,” arXiv preprint arXiv:1903.09588, 2019.

Sutskever, I., Vinyals, O., and Le, Q. V., “Sequence to sequence learning with neural networks,” Advances in Neural Information Processing Systems, 2014.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I., “Attention is all you need,” In Advances in Neural Information Processing Systems, pp. 6000-6010, 2017.

Wang, H., Can, D., Kazemzadeh, A., Bar, F., and Narayanan, S., “A system for real-time twitter sentiment analysis of 2012 us presidential election cycle,” Proceedings of the ACL 2012 System Demonstrations, 2012.

Xu, N. and Mao, W., “Multisentinet: A deep semantic network for multimodal sentiment analysis,” Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 2017.

Yang, L., Li, Y., Wang, J., and Sherratt, R. S., “Sentiment analysis for E-commerce product reviews in Chinese based on sentiment lexicon and deep learning,” IEEE Access, Vol. 8, pp. 23522-23530, 2020.


Refbacks

  • There are currently no refbacks.