Research Papers

Integrating Topic, Sentiment, and Syntax for Modeling Online Reviews: A Topic Model Approach

[+] Author and Article Information
Min Tang

College of Management,
Chongqing Technology and Business University,
Chongqing 400067, China

Jian Jin

Department of Information Management,
Beijing Normal University,
Beijing 100875, China

Ying Liu

Institute of Mechanical and
Manufacturing Engineering,
School of Engineering,
Cardiff University,
Cardiff CF24 3AA, UK
e-mail: LiuY81@cardiff.ac.uk

Chunping Li

School of Software,
Tsinghua University,
Beijing 100084, China

Weiwen Zhang

College of Mechanical and
Electrical Engineering,
Dongguan Polytechnic,
Dongguan 523808, China

1Corresponding author.

Contributed by the Computers and Information Division of ASME for publication in the JOURNAL OF COMPUTING AND INFORMATION SCIENCE IN ENGINEERING. Manuscript received October 15, 2017; final manuscript received September 9, 2018; published online October 18, 2018. Assoc. Editor: Joshua Summers.

J. Comput. Inf. Sci. Eng 19(1), 011001 (Oct 18, 2018) (12 pages) Paper No: JCISE-17-1227; doi: 10.1115/1.4041475 History: Received October 15, 2017; Revised September 09, 2018

Analyzing product online reviews has drawn much interest in the academic field. In this research, a new probabilistic topic model, called tag sentiment aspect models (TSA), is proposed on the basis of Latent Dirichlet allocation (LDA), which aims to reveal latent aspects and corresponding sentiment in a review simultaneously. Unlike other topic models which consider words in online reviews only, syntax tags are taken as visual information and, in this research, as a kind of widely used syntax information, part-of-speech (POS) tags are first reckoned. Specifically, POS tags are integrated into three versions of implementation in consideration of the fact that words with different POS tags might be utilized to express consumers' opinions. Also, the proposed TSA is one unsupervised approach and only a small number of positive and negative words are required to confine different priors for training. Finally, two big datasets regarding digital SLR and laptop are utilized to evaluate the performance of the proposed model in terms of sentiment classification and aspect extraction. Comparative experiments show that the new model can not only achieve promising results on sentiment classification but also leverage the performance on aspect extraction.

Copyright © 2019 by ASME
Your Session has timed out. Please sign back in to continue.


O'Reilly, T. , 2005, “ What is Web 2.0,” https://www.oreilly.com/pub/a/web2/archive/what-is-web-20.html
Li, X. , Hitt, L. M. , and Zhang, Z. J. , 2011, “ Product Reviews and Competition in Markets for Repeat Purchase Products,” J. Manage. Inf. Syst., 27(4), pp. 9–42. [CrossRef]
Li, M. , Huang, L. , Tan, C.-H. , and Wei, K.-K. , 2013, “ Helpfulness of Online Product Reviews as Seen by Consumers: Source and Content Features,” Int. J. Electron. Commer., 17(4), pp. 101–136. [CrossRef]
Jin, J. , Ji, P. , and Liu, Y. , 2016, “ Prioritising Engineering Characteristics Based on Customer Online Reviews for Quality Function Deployment,” J. Eng. Des., 4828(7–9), pp. 303–324.
Kangale, A. , Kumar, S. K. , Naeem, M. A. , Williams, M. , and Tiwari, M. K. , 2016, “ Mining Consumer Reviews to Generate Ratings of Different Product Attributes While Producing Feature-Based Review-Summary,” Int. J. Syst. Sci., 47(13), pp. 3272–3286. [CrossRef]
Moen, Ø. , Havro, L. J. , and Bjering, E. , 2017, “ Online Consumers Reviews: Examining the Moderating Effects of Product Type and Product Popularity on the Review Impact on Sales,” Cogent Bus. Manage., 4(1), p. 1368114. [CrossRef]
Hu, M. , and Liu, B. , 2004, “ Mining and Summarizing Customer Reviews,” ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '04), Seattle, WA, Aug. 22–25, p. 168.
Popescu, A. M. , and Etzioni, O. , 2005, “ Extracting Product Features and Opinions From Reviews,” HLT '05 Conference on Human Language Technology and Empirical Methods in Natural Language Processing, Vancouver, BC, Canada, Oct. 6–8, pp. 339–346.
Blair-Goldensohn, S. , Hannan, K. , McDonald, R. , Neylon, T. , Reis, G. A. , and Reynar, J. , 2008, “ Building a Sentiment Summarizer for Local Service Reviews,” WWW Workshop on NLP in the Information Explosion Era, Beijing, China, Apr. 21–25, pp. 339–348.
Lamar, C. , 2010, “ Linguistic Analysis of Natural Language Engineering Requirement Statements,” M.S. thesis, Clemson University, Clemson, SC. https://tigerprints.clemson.edu/all_theses/671/
Hein, P. H. , Voris, N. , and Morkos, B. , 2017, “ Predicting Requirement Change Propagation Through Investigation of Physical and Functional Domains,” Res. Eng. Des., 29(2), pp. 309–328. [CrossRef]
Turney, P. D. , 2001, “ Thumbs Up or Thumbs Down?,” 40th Annual Meeting on Association for Computational Linguistics (ACL '02), Toulouse, France, July 6–11, p. 417.
Pang, B. , Lee, L. , and Vaithyanathan, S. , 2002, “ Sentiment Classification Using Machine Learning Techniques,” Conference on Empirical Methods in Natural Language Processing (EMNLP-2002), Philadelphia, PA, July, pp. 79–86. http://www.aclweb.org/anthology/W02-1011
Pang, B. , and Lee, L. , 2004, “ A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts,” 42nd Annual Meeting on Association for Computational Linguistics, Barcelona, Spain, July 21–26, p. 271. http://www.aclweb.org/anthology/P04-1035
Blei, D. M. , Ng, A. , and Jordan, M. , 2003, “ Latent Dirichlet Allocation,” J. Mach. Learn. Res., 3, pp. 993–1022. https://dl.acm.org/citation.cfm?id=944937
Blei, D. M. , and Lafferty, J. D. , 2009, “ Topic Models,” Text Mining: Theory and Applications, A. S. a. M. Sahami , ed., Taylor & Francis, New York, pp. 101–124.
Griffiths, T. L. , Steyvers, M. , Blei, D. M. , and Tenenbaum, J. B. , 2004, “ Integrating Topics and Syntax,” Advances in Neural Information Processing Systems, Vol. 17, MIT Press, Cambridge, MA, pp. 537–544.
Whitelaw, C. , Garg, N. , and Argamon, S. , 2005, “ Using Appraisal Groups for Sentiment Analysis,” 14th ACM International Conference on Information and Knowledge management (CIKM '05), Bremen, Germany, Oct. 31–Nov. 5, p. 625.
Jo, Y. , and Oh, A. H. , 2011, “ Aspect and Sentiment Unification Model for Online Review Analysis,” Fourth ACM International Conference on Web Search and Data mining (WSDM '11), Hong Kong, China, Feb. 9–12, p. 815.
Lin, C. , and He, Y. , 2009, “ Joint Sentiment/Topic Model for Sentiment Analysis,” 18th ACM Conference on Information and Knowledge management (CIKM '09), Hong Kong, China, Nov. 2–6, p. 375.
Mei, M. W. H. S. C. Z. Q. , and Ling, X. , 2007, “ Topic Sentiment Mixture: Modeling Facets and Opinions in Weblogs,” World Wide Web Conference (WWW'07), Banff, AB, Canada, May 8–12, pp. 171–180.
Titov, I. , and McDonald, R. , 2008, “ A Joint Model of Text and Aspect Ratings for Sentiment Summarization,” ACL-08 HLT, pp. 308–316.
Boyd-Graber, J. , and Blei, D. M. , '2009, “ Syntactic Topic Models,” Advances in Neural Information Processing Systems, Curran Associates, Whistler, BC, Canada, pp. 185–192.
Matsumoto, S. , Takamura, H. , and Okumura, M. , 2005, “ Sentiment Classification Using Word Sub-Sequences and Dependency Sub-Trees,” Pacific-Asia Conference on Knowledge Discovery and Data Mining, Hanoi, Vietnam, May 18–20, pp. 301–311.
Ding, X. , Liu, B. , and Yu, P. S. , 2008, “ A Holistic Lexicon-Based Approach to Opinion Mining,” International Conference on Web Search and Web Data mining (WSDM '08), Palo Alto, CA, Feb. 11–12, p. 231.
Joshi, M. , and Penstein-Rosé, C. , 2009, “ Generalizing Dependency Features for Opinion Mining,” ACL-IJCNLP 2009 Conference Short Papers, Suntec, Singapore, Aug. 4, p. 313.
Baccianella, S. , Esuli, A. , and Sebastiani, F. , 2010, “ SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining SentiWordNet,” Analysis, 10(2010), pp. 1–12.
Esuli, A. , and Sebastiani, F. , 2006, “ SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining,” Fifth Conference on Language Resources and Evaluation, pp. 417–422.
Lu, Y. , Castellanos, M. , Dayal, U. , and Zhai, C. , 2011, “ Automatic Construction of a Context-Aware Sentiment Lexicon: An Optimization Approach,” 20th International Conference on World Wide Web, Hyderabad, India, Mar. 28–Apr. 1, pp. 347–356.
Hofmann, T. , 2013, “ Probabilistic Latent Semantic Analysis,” 15th Conference on Uncertainty in Artificial Intelligence, pp. 289–296.
Griffiths, T. L. , and Steyvers, M. , 2004, “ Finding Scientific Topics,” Proc. Natl. Acad. Sci., 101(Suppl. 1), pp. 5228–5235. [CrossRef]
Brody, S. , 2010, “ An Unsupervised Aspect-Sentiment Model for Online Reviews,” Computational Linguistics, HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Los Angeles, CA, June 2–4, pp. 804–812.
Titov, I. , and McDonald, R. , 2008, “ Modeling Online Reviews With Multi-Grain Topic Models,” 17th International Conference on World Wide Web (WWW '08), Beijing, China, Apr. 21–25, p. 111.
Zhao, W. X. , Jiang, J. , Yan, H. , and Li, X. , 2010, “ Jointly Modeling Aspects and Opinions With a MaxEnt-LDA Hybrid,” EMNLP '10 Conference on Empirical Methods in Natural Language Processing, Cambridge, MA, Oct. 9–11, pp. 56–65. https://dl.acm.org/citation.cfm?id=1870664
Wang, X. , and McCallum, A. , 2006, “ Topics Over Time: A Non-Markov Continuous-Time Model of Topical Trends,” 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Philadelphia, PA, Aug. 20–23, pp. 424–433.
Bach, N. X. , Linh, N. D. , and Phuong, T. M. , 2018, “ An Empirical Study on POS Tagging for Vietnamese Social Media Text,” Comput. Speech Lang., 50, pp. 1–15. [CrossRef]
Das, B. R. , Sahoo, S. , Panda, C. S. , and Patnaik, S. , 2015, “ Part of Speech Tagging in Odia Using Support Vector Machine,” Proc. Comput. Sci., 48, pp. 507–512. [CrossRef]
Zeroual, I. , Lakhouaja, A. , and Belahbib, R. , 2017, “ Towards a Standard Part of Speech Tagset for the Arabic Language,” J. King Saud Univ.-Comput. Inf. Sci., 29(2), pp. 171–178.
Carneiro, H. C. C. , França, F. M. G. , and Lima, P. M. V. , 2015, “ Multilingual Part-of-Speech Tagging With Weightless Neural Networks,” Neural Networks, 66, pp. 11–21. [CrossRef] [PubMed]
Carneiro, H. C. C. , Pedreira, C. E. , França, F. M. G. , and Lima, P. M. V. , 2017, “ A Universal Multilingual Weightless Neural Network Tagger Via Quantitative Linguistics,” Neural Networks, 91, pp. 85–101. [CrossRef] [PubMed]
Barrett, N. , and Weber-Jahnke, J. , 2014, “ A Token Centric Part-of-Speech Tagger for Biomedical Text,” Artif. Intell. Med., 61(1), pp. 11–20. [CrossRef] [PubMed]
Wang, Y. , Wu, S. , Li, D. , Mehrabi, S. , and Liu, H. , 2016, “ A Part-of-Speech Term Weighting Scheme for Biomedical Information Retrieval,” J. Biomed. Inf., 63, pp. 379–389. [CrossRef]
Bravo-Marquez, F. , Frank, E. , and Pfahringer, B. , 2016, “ Building a Twitter Opinion Lexicon From Automatically-Annotated Tweets,” Knowl.-Based Syst., 108, pp. 65–78. [CrossRef]
Wang, G. , Zhang, Z. , Sun, J. , Yang, S. , and Larson, C. A. , 2015, “ POS-RS: A Random Subspace Method for Sentiment Classification Based on Part-of-Speech Analysis,” Inf. Process. Manage., 51(4), pp. 458–479. [CrossRef]
Liu, Y. , Jin, J. , Ji, P. , Harding, J. A. , and Fung, R. Y. K. , 2013, “ Identifying Helpful Online Reviews: A Product Designer's Perspective,” Comput.-Aided Des., 45(2), pp. 180–194.
Zhang, R. , and Tran, T. , 2010, “ A Novel Approach for Recommending Ranked User-Generated Reviews,” Advances in Artificial Intelligence, Vol. 6085, Springer, Berlin, pp. 324–327.
Fujimoto, K. , 2011, “ A Computational Account of Potency Differences in eWOM Messages Involving Subjective Rank Expressions,” IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, Lyon, France, Aug. 22–27, pp. 138–142.
Siering, M. , Deokar, A. V. , and Janze, C. , 2017, “ Disentangling Consumer Recommendations: Explaining and Predicting Airline Recommendations Based on Online Reviews,” Neurocomputing, 254, pp. 33–41. [CrossRef]
Xu, X. , Dutta, K. , and Ge, C. , 2018, “ Do Adjective Features From User Reviews Address Sparsity and Transparency in Recommender Systems?,” Electron. Commer. Res. Appl., 29, pp. 113–123. [CrossRef]
Li, L. , Qin, B. , Ren, W. , and Liu, T. , 2017, “ Document Representation and Feature Combination for Deceptive Spam Review Detection,” Neurocomputing, 254, pp. 33–41. [CrossRef]
Hu, Y.-H. , Chen, K. , and Lee, P.-J. , 2017, “ The Effect of User-Controllable Filters on the Prediction of Online Hotel Reviews,” Inf. Manage., 54(6), pp. 728–744. [CrossRef]
Krestel, R. , and Dokoohaki, N. , 2015, “ Diversifying Customer Review Rankings,” Neural Networks, 66, pp. 36–45. [CrossRef] [PubMed]
Lee, P.-J. , Hu, Y.-H. , and Lu, K.-T. , 2018, “ Assessing the Helpfulness of Online Hotel Reviews: A Classification-Based Approach,” Telematics Inf., 35(2), pp. 436–445. [CrossRef]
Porter, M. F. , 2006, “ An Algorithm of Suffix Stripping,” Program, 14(3), pp. 130–137. https://dl.acm.org/citation.cfm?id=275705
Turney, P. D. , and Littman, M. L. , 2003, “ Measuring Praise and Criticism,” ACM Trans. Inf. Syst., 21(4), pp. 315–346. [CrossRef]
Wallach, H. M. , 2006, “ Topic Modeling: Beyond Bag-of-Words,” International Conference on Machine Learning, Pittsburgh, PA, pp. 977–984.
Wei, L. , and McCallum, A. , 2006, “ Pachinko Allocation: DAG-Structured Mixture Models of Topic Correlations,” 23rd International Conference on Machine Learning (ICML '06), Pittsburgh, PA, pp. 577–584. https://people.cs.umass.edu/~mccallum/papers/pam-icml06.pdf
Davidov, D. , Tsur, O. , and Rappoport, A. , 2010, “ Semi-Supervised Recognition of Sarcastic Sentences in Twitter and Amazon,” 14th Conference on Computational Natural Language Learning, Uppsala, Sweden, July 15–16, pp. 107–116.


Grahic Jump Location
Fig. 1

Three versions of implementations of TSA



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In