Recognition of Multi Label Fashion Styles based on Transfer Learning and Graph Convolution Network

Sunghoon Kim, Sunghoon Kim, Yerim Choi, Yerim Choi, Jonghyuk Park, Jonghyuk Park

Abstract


Recently, there are increasing attempts to utilize deep learning methodology in the fashion industry. Accordingly, research dealing with various fashion-related problems have been proposed, and superior performances have been achieved. However, the studies for fashion style classification have not reflected the characteristics of the fashion style that one outfit can include multiple styles simultaneously. Therefore, we aim to solve the multi-label classification problem by utilizing the dependencies between the styles. A multi-label recognition model based on a graph convolution network is applied to detect and explore fashion styles' dependencies. Furthermore, we accelerate model training and improve the model's performance through transfer learning. The proposed model was verified by a dataset collected from social network services and outperformed baselines.


Full Text:

PDF

References


Chen, Z. M., Wei, X. S., Wang, P., and Guo, Y., “Multi-Label Image Recognition with Graph Convolutional Networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5177-5186, 2019.

Deng, J., Dong, W., Socher, R., Li, L. J., Li, K. and Li, F., “Imagenet: A largescale Hierarchical Image Database,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009.

Doughty, H., Damen, D. and Mayol- Cuevas, W., “Who’s Better? Who’s Best? Pairwise Deep Ranking for Skill Determination,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6057-6066, 2018.

Ferreira, B. Q., Costeira, J. P., Sousa, R. G., Gui, L. Y. and Gomes, J. P., “Pose Guided Attention for Multi-Label Fashion Image Classification,” Proceedings of the IEEE International Conference on Computer Vision Workshop, pp. 3125-3128, 2019.

Ge, Y., Zhang, R., Wang, X., Tang, X. and Luo, P., “Deepfashion 2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Images,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5337-5345, 2019.

Gong, Y., Jia, Y., Leung, T., Toshev, A. and Ioffe, S., “Deep Convolutional Ranking for Multilabel Image Annotation,” arXiv preprint arXiv:1312.4894, 2013.

Guo, Y. and Gu, S., “Multi-Label Classification using Conditional Dependency Networks,” International Joint Conference on Artificial Intelligence, Vol. 22, No. 1, pp. 1300-1305, 2011.

He, H. and Xia, R., “Joint Binary Neural Network for Multi-Label Learning with Applications to Emotion Classification,” International Conference on Natural Language Processing and Chinese Computing, pp. 250-259, 2018.

He, K., Zhang, X., Ren, S. and Sun, J., “Deep Residual Learning for Image Recognition,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.

Joachims, T., Swaminathan, A., and Schnabel, T., “Unbiased Learning-to-Rank with Biased Feedback,” Proceedings of the ACM International Conference on Web Search and Data Mining, pp. 781-789, 2017.

Kipf, T. N. and Welling, M., “Semi-Supervised Classification with Graph Convolutional Networks,” International Conference on Learning Representations, 2016.

LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P., “Gradient-Based Learning Applied to Document Recognition,” Proceedings of the IEEE, Vol. 86, No. 11, pp. 2278-2324, 1998.

Lee, D. and Kim, K., “A LSTM Based Method for Photovoltaic Power Prediction in Peak Times Without Future Meteorological Information,” The Journal of Society for e-Business Studies, Vol. 24, No. 4, pp. 119-133, 2019.

Liu, W., Tsang, I. W. and Muller, K. R., “An Easy-to-Hard Learning Paradigm for Multiple Classes and Multiple Labels,” The Journal of Machine Learning Research, Vol. 18, No. 1, pp. 3300-3337, 2017.

Liu, Z., Luo, P., Qiu, S., Wang, X. and Tang, X., “Deepfashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1096-1104, 2016.

Mirzazadeh, F., Ravanbakhsh, S., Ding, N., and Schuurmans, D., “Embedding Inference for Structured Multilabel Prediction,” Advances in Neural Information Processing Systems, pp. 3555-3563, 2015.

Oh, S., Lee, H., Shin, J., and Lee, J., “Antibiotics-Resistant Bacteria Infection Prediction Based on Deep Learning,” The Journal of Society for e-Business Studies, Vol. 24, No. 1, pp. 105-120, 2019.

Pennington, J., Socher, R. and Manning, C. D., “Glove: Global Vectors for Word Representation,” Empirical Methods in Natural Language Processing, pp. 1532- 1543, 2014.

Schroff, F., Kalenichenko, D. and Philbin, J., “Facenet: A Unified Embedding for Face Recognition and Clustering,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815-823, 2015.

Shin, S., “Application of Big Data in the Fashion Industry,” FashionNet Korea, Retrieved February 28, 2016.

Simonyan, K. and Zisserman, A., “Very Deep Convolutional Networks for Largescale Image Recognition,” arXiv preprint arXiv:1409.1556, 2014.

Szymański, P., Kajdanowicz, T., and Chawla, N., “LNEMLC: Label Network Embeddings for Multi-Label Classification,” arXiv preprint arXiv:1812.02956, 2018.

Takagi, M., Simo-Serra, E., Iizuka, S., and Ishikawa, H., “What Makes a Style: Experimental Analysis of Fashion Prediction,” Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 2247-2253, 2017.

Yoo, S. and Jeong, O., “An Intelligent Chatbot Utilizing BERT Model and Knowledge Graph,” The Journal of Society for e-Business Studies, Vol. 24, No. 3, pp. 87-98, 2019.


Refbacks

  • There are currently no refbacks.