Sparse Topic Classification on Large Sets of Documents via Non-negative Matrix Tri-factorization

Francis Attah; Wah June Leong; Cynthia Mui Lian Kon; Farah Lateef Joey

doi:10.33102/mjosht.599

Authors

Francis Attah Department of Mathematics and Statistics, Faculty of Science, Universiti Putra Malaysia, 43400 Serdang, Malaysia.
Wah June Leong Department of Mathematics and Statistics, Faculty of Science, Universiti Putra Malaysia, 43400 Serdang, Malaysia.
Cynthia Mui Lian Kon Faculty of Computer Science and Information Technology, Universiti Malaysia Sarawak, 94300 Kota Samarahan, Malaysia.
Farah Lateef Joey Mathematics and Computer Applications Department, College of Science, Al-Nahrain University, Jadriya, Bagdad, Iraq.

DOI:

https://doi.org/10.33102/mjosht.599

Keywords:

topic classification, non-negative matrix tri-factorization (NMTF), sparse regularization, linearly independent

Abstract

This paper proposes a co-clustering framework for sparse topic classification, which incorporates orthogonality and sparse regularization via non-negative matrix tri-factorization The proposed model integrates two key components, namely norm regularization on both factor matrices to promote row sparsity, thereby enhancing noise robustness and facilitating the selection of informative features and representative topics; and linearly independent constraints on the data (row) and feature (column) cluster indicator matrices to promote distinct, non-overlapping clustering structures. An norm-based reconstruction loss is employed, and an effective algorithm is developed to obtain the solution. Numerical experiments on some benchmark datasets demonstrate that the proposed model and solution method achieve competitive performance in sparse topic classification, with improvements in clustering accuracy, sparsity, and interpretability compared with existing methods.

Downloads

Download data is not yet available.

References

Y. Chen, G. Qu, and J. Zhao, J. “Orthogonal graph regularized non-negative matrix factorization under sparse constraints for clustering.” Expert Syst. Appl., vol. 249, Art. no. 123797, 2024.

N. Del Buono and G. Pio, “Non-negative matrix tri-factorization for co-clustering: An analysis of the block matrix,” Inf. Sci., vol. 301, pp. 13–26, 2015.

C. Ding, T. Li, W. Peng, and H. Park, “Orthogonal non-negative matrix tri-factorization for clustering,” in Proc. 12th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2006, pp. 126–135.

S. Ge, H. Li, and L. Luo, “Constrained dual graph regularized orthogonal nonnegative matrix tri-factorization for co-clustering,” Math. Probl. Eng., 2019, pp. 1–17.

D. Greene and P. Cunningham, “Practical solutions to the problem of diagonal dominance in kernel document clustering,” in Proc. 23rd Int. Conf. Mach. Learn. (ICML), pp. 377–384, 2006

S. Hoseinipour, M. Aminghafari, and A. Mohammadpour, “Orthogonal parametric non-negative matrix tri-factorization with α-divergence for co-clustering,” Expert Syst. Appl., vol. 231, Art. no. 120680, 2023.

J. Huang, F. Nie, H. Huang, and C. Ding, “Robust manifold nonnegative matrix factorization,” ACM Trans. Knowl. Discov. Data, vol. 8, no. 3, pp. 1–21, 2014.

X. Jin, J. Miao, Q. Wang, G. Geng, and K. Huang, “Sparse matrix factorization with L2,1 norm for matrix completion,” Pattern Recognit., vol. 127, Art. no. 108655, 2022.

L. Labiod and M. Nadif, “Co-clustering under nonnegative matrix tri-factorization,” in Proc. Int. Conf. Neural Information Processing, vol. 7063, 2011, pp. 709–717.

S. Li, Y. Li, and J. Zhang, “Inter- and intra-hypergraph regularized nonnegative matrix factorization with hybrid constraints,” Eng. Appl. Artif. Intell., vol. 130, Art. no. 107758, 2024.

P. Luo, J. Peng, and J. Fan, “L2,1 norm and Hessian regularized non-negative matrix factorization with discriminability for data representation,” Appl. Sci., vol. 7, no. 10, Art. no. 1013, 2017.

Y. Meng, R. Shang, F. Shang, L. Jiao, S. Yang, and R. Stolkin, “Semi-supervised graph regularized deep NMF with bi-orthogonal constraints for data representation,” IEEE Trans. Neural Netw. Learn. Syst., vol. 31, no. 9, pp. 3245–3258, 2020.

Y. Meng, R. Shang, L. Jiao, W. Zhang, and S. Yang, “Dual-graph regularized non-negative matrix factorization with sparse and orthogonal constraints,” Eng. Appl. Artif. Intell., vol. 69, pp. 24–35, 2018.

J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed. Springer, New York, 2006.

Y. Pei, N. Chakraborty, and K. Sycara, “Nonnegative matrix tri-factorization with graph regularization for community detection in social networks,” in Proc. 24th Int. Joint Conf. Artificial Intelligence (IJCAI), 2015, pp. 2083–2089.

Y. Pei, G. Fletcher, and M. Pechenizkiy, “Joint role and community detection in networks via L2,1 norm regularized nonnegative matrix tri-factorization,” in Proc. 2019 IEEE/ACM Int. Conf. Advances in Social Networks Analysis and Mining, 2019, pp. 168–175.

X. Sha and N. Diao, “{mathrm{ell }}_{2,1}-norm-based robust feature extraction method for fault detection,” ACS Omega, vol. 7, no. 48, pp. 43440–43449, 2022.

S. Wang and A. Huang, “Penalized nonnegative matrix tri-factorization for co-clustering,” Expert Syst. Appl., vol. 78, pp. 64–73, 2017.

Q. Yang, X. Yin, S. Kou, and Y. Wang, “Robust structure convex nonnegative matrix factorization for data representation,” IEEE Access, vol. 9, pp. 155087–155102, 2021.

J. Yoo and S. Choi, “Orthogonal nonnegative matrix tri-factorization for co-clustering: Multiplicative updates on Stiefel manifolds,” Inf. Process. Manage., vol. 46, no. 5, pp. 559–570, 2010.