Comparison of LDA and BERTopic in News Topic Modeling

A Case Study of The New York Times’ Reports on China

Abstract

The comparison of LDA and BERTopic in news topic modeling offers a fascinating insight into the evolving field of text analysis. As big data floods our lives, techniques like LDA and BERTopic are vital for understanding hidden patterns and associations of texts. LDA, a classic topic model, excels in dimensionality reduction and modeling analysis, particularly for large text datasets. Its Bayesian structure effectively explores implicit topic information, making it a popular choice across disciplines. Conversely, BERTopic, a more recent approach, leverages the power of BERT for topic extraction, offering enhanced contextual understanding. Both models have their strengths, and their comparison in a case study of The New York Times’ reports on China can provide valuable insights into their relative performance in real-world scenarios. This study aims to bring new insights to topic modeling for news texts by presenting the respective advantages and disadvantages of LDA and BERTopic.