A Study on the Relationship between Class Similarity and the Performance of Hierarchical Classification Method in a Text Document Classification Problem
Abstract
The literature has reported that hierarchical classification methods generally outperform the flat classification methods for a multi-class document classification problem. Unlike the literature that has constructed a class hierarchy, this paper evaluates the performance of hierarchical and flat classification methods under a situation where the class hierarchy is predefined. We conducted numerical evaluations for two data sets; research papers on climate change adaptation technologies in water sector and 20NewsGroup open data set. The evaluation results show that the hierarchical classification method outperforms the flat classification methods under a certain condition, which differs from the literature. The performance of hierarchical classification method over flat classification method depends on class similarities at levels in the class structure. More importantly, the hierarchical classification method works better when the upper level similarity is less that the lower level similarity.
Full Text:
PDFReferences
Agnihotri, D., Verma, K., and Tripathi, P., “Variable global feature selection scheme for automatic classification of text documents,” Expert Systems with Applications, Vol. 81, pp. 268-281, 2017.
Bertule, M., Appelquist, L. R., Spensley, J., Traerup, S. L. M., and Naswa, P., “Climate change adaptation technologies for water: A practitioner’s guide to adaptation technologies for increased water sector resilience,” CTCN publications, Copenhagen, Denmark, 2018.
Beyan, C. and Fisher, R., “Classifying imbalanced data sets using similarity based hierarchical decomposition,” Pattern Recognition, Vol. 48, pp. 1653-1672, 2015.
Byun, J. H., “Current Status and Perspectives of Fintech Innovation,” Journal of New Industry and Business, Vol. 26, No. 2, pp. 35-48, 2018
Chen, Y., Craword, M. M., and Ghosh, J., “Integrating support vector machines in a hierarchical output space decomposition framework,” IEEE International Geoscience and Remote Sensing Symposium, Vol. 2, pp. 949-952, 2004.
Cristianini, N. and Shawe-Taylor, J., “An introduction to support vector machines and other kernel-based leartning methods”, Cambridge University Press, MA, 2000.
Du, Y., Liu, J., Ke, W., and Gong, X., “Hierarchy construction and text classification based on the relaxation strategy and least information model,” Expert Systems with Applications, Vol. 100, pp. 157-164, 2018.
Duan, K. B. and Keerthi, S. S., “Which is the best multiclass SVM method? An empirical study,” International Workshop on Multiple Classifier Systems, Vol. 3531, pp. 278-285, 2005.
Gargiulo, F., Silvestri, S., Ciampi, M., and De Pietro, G., “Deep neural network for hierarchical extreme multi-label text classification,” Applied Soft Computing, Vol. 79, pp. 125-138, 2019.
Kang, S., Cho, S., and Kang, P., “Constructing a multi-class classier using one-against-one approach with different binary classifiers,” Neurocomputing, Vol. 149, pp. 677-682, 2015.
Kim, P. J. and Lee, J. Y., “An experimental study on the performance improvement of automatic classification for the articles of korean journals based on controlled keywords in international database,” Journal of the Korean Society for Library and Information Science, Vol. 48, No. 3, pp. 491-510, 2014
Kim, P. J., “An analytical study on automatic classification of domestic journal articles based on machine learning,” Journal of the Korean Society for information Management, Vol. 35, No. 2, pp. 37-62, 2018.
Kim, Y. S. and Lee, B. Y., “Multi-class support vector machines model based clustering for hierarchical document categorization in big data environment,” The Journal of the Korea Contents Association, Vol. 17, pp. 600-608, 2017.
Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., and Brown, D,, “Text classification algorithms: A survey,” Information, Vol. 10, No. 4, 2019.
Lee, J. H., Yi, J. S., and Son, J. W., “Unstructured construction data analytics using R programming: Focused on overseas construction adjudication cases”, Journal of the Architectural Institute of Korea Structure & Construction, Vol. 32, No. 5, pp. 37-44, 2016.
Lee, J. S. and Kwon, J. G., “A hybrid SVM classifier for imbalanced data sets,” Journal of Intelligence and Information Systems, Vol. 19, pp. 125-140, 2013.
Lee, S. K. and Kim, K., “Academic Conference Categorization According to Subjects Using Topical Information Extraction from Conference Websites,” The Journal of Society for e-Business Studies, Vol. 22, No. 2, pp. 61-77, 2017.
Lee, S. J. and Kim, H. J., “Keyword extraction from news corpus using modified TF-IDF,” The Journal of Society for e-Business Studies, Vol. 14, No. 4, pp, 59-73, 2009.
Lorena, A. C., De Carvalho, A. C., and Gama, J. M. P., “A review on the combination of binary classifiers in multiclass problems,” Artificial Intelligence Review, Vol. 30, No. 19, 2008.
Madzarov, G., Gjorgjevikj, D., and Chorbev, I., “A multi-class SVM classifier utilizing binary decision tree,” Informatica, Vol. 33, 2009.
Min, J. H. and Lee, Y. C., “Bankruptcy prediction using support vector machine with optimal choice of kernel function parameters,” Expert Systems with Applications, Vol. 28, pp. 603-614, 2005.
Naik, A. and Rangwala, H., “Improving large-scale hierarchical classification by rewiring: A data-driven filter based approach,” Journal of Intelligent Information Systems, Vol. 52, pp. 141-164, 2019
Park, J. H. and Kim, J. S., “A text classification system for hierarchical categories,” The Korean Institute of Information Scientists and Engineers, Vol. 27, No. 2, pp. 128-130, 2000.
Silla, C. N. and Freitas, A. A., “A survey of hierarchical classification across different application domains,” Data Mining and Knowledge Discovery, Vol. 22, pp. 31-72, 2011
Silva-Palacios, D., Ferri, C., and Raḿırez-Quintana, M. J., “Probabilistic class hierarchies for multiclass classification,” Journal of Computational Science, Vol. 26, pp. 254-263, 2018
Sun, A., Lim, E. P., Ng, W. K., and Srivastava, J., “Blocking reduction strategies in hierarchical text classification,” IEEE Transactions on Knowledge and Data Engineering, Vol. 16, pp. 1305-1308, 2004
Tegegnie, A. K., Tarekegn, A. N., and Alemu, T. A., “A comparative study of flat and hierarchical classification for amharic news text using SVM,” Information Engineering and Electronic Business, Vol. 3, pp. 36-42, 2017.
UNEP, “Technologies for climate change mitigation,” UNEP, 2011.
Vapnik, V., “Estimation of Dependences Based on Empirical Data.” Nauka, Moscow, 1979.
Vapnik, V., “The nature of statistical learning theory”, Chapter 5. Springer-Verlag, New York, 1995.
Williams, T. P. and Gong, J., “Predicting construction cost overruns using text mining, numericaldata and ensemble classifiers,” Automation in Construction, Vol. 43, pp. 23-29, 2014
Yoon, Y. W. Lee, C. K., and Lee, G. B., “Hierarchical text categorization using support vector machine,” Annual Conference on Human and Language Technology, pp. 7-13, 2013.
Zhang, L., Shah, S. K., and Kakadiaris, I. A., “Hierarchical multi-label classification using fully associative ensemble learning,” Pattern Recognition, Vol. 70, pp. 89-103, 2017.
Zhao, Z., Wang, X., and Wang, T., “A novel measurement data classification algorithm based on SVM for tracking closely spaced targets,” IEEE Transactions on Instrumentation and Measurement, Vol. 68, No. 4, pp. 1089-1100, 2019.
Zheng, J., Guo, Y., Feng, C., and Chen., H., “A hierarchical neural network based document representation approach for text classification,” Mathematical Problems in Engineering, Vol. 2018, 2018.
Refbacks
- There are currently no refbacks.