A Novel Graph Based Frame work for Opinion …A Novel Graph Based Frame work for Opinion...

A Novel Graph Based Frame work for Opinion summarization on Twitter Data

1Syed Muzamil Basha, 2Dharmendra Singh Rajput 1,2Associate Professor, SITE, VIT University, Vellore - 14

ABSTRACT Twitter is an increasingly accepted social media platform for networking people all around the world. In particular, summarizing representative opinions from tweets can help in understanding the semantics of opinions on any given topic. In this paper, we addressed different challenges of opinion summarization and knowledge extraction from tweets, which helps to built extractive and abstractive summaries. we propose a conventional Graph Based (CGB) opinion summarization frame work for obtaining summarization and knowledge of Twitter Data. In addition to that, we demonstrated a Opinion Text Mining approach on recent Tweets using R. Experimental results on two real-world datasets verify the efficiency of the proposed model. Key Words— Summarization, Extractive, Abstractive, Opinion, Twitter.

1.INTRODUCTION The objective of this paper is to produce a natural language text from the given collection of

Tweets. Bollegala et al. (2010) says typical natural language generation (NLG) system is used, which consists of six components: Content determination, Speech planning, Sentence aggregation, Lexicalization, Referring expression generation, and Orthographic realization. Among this Information ordering is given higher priority to concatenate the sentences to built summary. In Summarization, First analyzed the importance of each word and find the semantic relatedness among them, and finally make a summary of the documents.

In single document summarization, it is easy to arrange the extracted information in the same order as in the original document. In contrast, for multi-document summarization, we need to built a strategy to arrange sentences extracted from different documents. Ordering extracted sentences into a summary is a non-trivial task. Lloret et al. (2013) made a study to understand, to what extent such approaches are useful to produce summaries that helps as a substitute of the source document (or) in contrast, if they can only be used in order to provide users to understand what is important in the original document. Moens et al. (2005) discussed the generic summarization technologies and focus on the detection of important sentences at different levels of topical detail, on the compression of sentences and on the detection of redundant content. The proposed technologies are implemented in the SUMMA summarization system of the K.U. Leuven are implemented in separate modules in C and Perl.

International Journal of Pure and Applied MathematicsVolume 118 No. 18 2018, 4317-4327ISSN: 1311-8080 (printed version); ISSN: 1314-3395 (on-line version)url: http://www.ijpam.euSpecial Issue ijpam.eu

4317

Fig 1. Word Cloud of Tweets.

Figure 2: Text Summarization Approaches.

In this paper, we propose a conventional Machine Learning (CML) based opinion summarization method for capturing the association of sentences in the con-text of single-document summarization for Twitter using open NLP library in R. The fig. 1 is a example of word-cloud obtained from recent tweets. They are trainable Auto-systems in which models learns how to generalize its parameters to extract mostly useful terms. Most of Machine Learning approaches in text summarization and knowledge extractions are inspired from information retrieval and adapted into the task such as Bayesian models, Hidden Markov Models (HMM), Support Vector Machines (SVM) and Support Vector Regression (SVR). In general, many supervised and un-supervised approaches have been proposed and they can be categorized as follows: Latent topic models as unsupervised techniques, classification and regression as the supervised techniques. The fig. 2 gives the complete details of so far adopted approaches in text summarization task. A. Single Document Chen et al. (2016) proposed a principle framework to unify the relationships among several mostly used unsupervised query models by adapting them to the speech recognition and summarization tasks of opinion Mining. Yousefi Azar and Hamey (2017) introduced an unsupervised approach for extractive summarization using Automatic Encoders by sentence Ranking. From the above statements, It is clear that the amount of research carried out on single document have limited scope. B. Multiple Document(MD) He et al. (2016) proposed a novel compressive sensing based multi document summarization framework which can select summary sentences in batch mode by considering group structure information among sentences and Evaluated the proposed framework on two summarization benchmarks. Heu et al. (2015) proposed a MD summarization system, called FoDoSu (Folksonomy-

International Journal of Pure and Applied Mathematics Special Issue

4318

based Multi-Document Summarization), that employs the tag clusters used in Folksonomy systems to detect key sentences in multiple documents, Canhasi and Kononenko (2016) proposed a Archetypal analysis algorithm called Weighted Hierarchical Archetypal analysis Algorithm for MD summarization, and made Deeper investigation on different ways of selecting the most salient sentences from sub-topic originating from significant topics. Marujo et al. (2016) idea is to introduce an Event-based MD summaries that are more useful for humans. Qiang et al. (2016) proposed a pattern-based method which includes the following steps. First step is mining all closed sequential patterns from a corpus. Then, presenting a novel method that represents all sentences using these closed patterns. The model of sentence representation covers the main content of the document collection by calculating pattern weights with respect to the distribution of the closed patterns. Finally, iteratively choosing informative and non-redundant sentences by adopting a variant of the Maximal Marginal Relevance (MMR) evaluation strategy. Yang et al. (2015) demonstrated that the correlations between general and specific topics were important features to evaluate the relevance of sentences.

2.RELATED WORK Wang and Liu (2015) proposed a pronoun related features and make use of them in the

supervised summarization approaches, Results obtained from the experiment prove that both the unsupervised and supervised methods achieve better results compared to the baseline.

Zhang et al. (2016) build a computational model based on a popular cognitive model of narrative text comprehension, establishing its computational counterparts in the model’s cognitive process. In which, Coherence is used as feature to summarize narrative text.

Zhao et al. (2009) proposed an approach of combining query expansion in the graph based algorithm for query focused multiple document summarization, used both inter and intra sentence relations to find the query biased informative words in the document. Ouyang et al. (2011) applied regression models in Ranking sentences in query-focused MD Summarization using SVR. Moen et al. (2016) presented a study of Automated Text Summarization (ATS) methods and the result state that ATS are really providing useful insights.

Plaza et al. (2011) presented A graph-based summarizer that uses the Unified Medical Language System to identify concepts and the semantic relations between them to construct a rich semantic representation of the document that to be summarized. Plaza et al. (2012) describes the application of various strategies for selecting Unified Medical Language System (UMLS), that uses UMLS Metathesaurus to create conceptual representations. Hijikata et al. (2007) proposed "Social Summarization Method" (SSM) for summarizing feedback comments based on Seller and buyer relationship.

Mosa et al. (2016) developed a new strategy called Ant Colony Optimization (ACO) mixed with Jensen-Shannon Divergence, to improve the efficiency in summarizing the document by varying parameters comment’s lengths and high Correlation between comments. Lloret and Palomar (2013b) focused on state of the art Text Summarization techniques which are appropriate for generating a summary from a large piece of news in the form of a tweet. Perea Ortega et al. (2013) analyzed in detail about the impact of the use of automatic summaries in the context of Geographic Information Systems, focused on summarizing text documents that contain different locations or geographic places. Zhong et al. (2015) utilizes deep learning in query-oriented multi-document summarization task. Peng et al. (2016) proposed an query oriented social network summarization method to brief answer for the query automatically in social networks.

3.TEXT SUMMARIZATION APPROACHES


4319

Here by we give different approaches to summarize the text from original document. Text

Summarization approaches are broadly classified in to Extractive and abstractive approaches. where Extractive Summarization (ES) is to select a set of most vital sentences from the source documents. In contrast, Abstractive Summarization (AS) builds an internal semantic representation, and then uses natural language processing techniques to generate a summary. Tayal et al. (2017) used, semantic word and sentence similarity mechanism, semantic representation using word disambiguation through parser mechanism to find the summary of a text document. Ye et al. (2007) represent the sources as a document concept lattice(DCL). In which, data structure organizes the set of all concepts present in the input into a Direct Acyclic Graph (DAG), where nodes represent overlapping sets of concepts. Nodes in the DCL store the frequency of its set of concepts within the input, and contain pointers to subsequent nodes. Li et al. (2016) aims to opinion sentence, which help users better understand the semantics of an opinion, suggested that summarization algorithms like Maximal Marginal Relevance (MMR) and Text Rank (TR) can generate better Topic labels. A. Abstractive Bhargava et al. (2016) proposed a method for compressing and merging information based on

word graphs. Which does not require any Domain Knowledge to provide explicitly. Gerani et al. (2016) proposed a summarization framework that generates an aspect based abstract from multiple reviews which use NLP Technique in generating new sentences.

B. Extractive Reeve et al. (2007) presented three extractive summarization methods (Bio Chain for concept

chaining, Freq Dist for frequency distribution, and a hybrid of the two, Chain Freq) using domain specific concepts as a feature for identifying salient sentences in bio-medical texts. Song et al. (2011) proposed a sentence extraction summarization method called fuzzy evolutionary optimization model based on concept of sentence clustering. C. General Techniques Zajic et al. (2008) described two separate approaches to email thread summarization, First

Individual Message Summarization (IMS), and the other treats is Collective Message Summarization (CMS). Smyth et al. (2010) described the challenges in generate query-focused snippets based on the implicit preferences of a community of like-minded searchers to generate Community Based Summaries (CBS). Sankarasubramaniam et al. (2014) addressed Wikipedia-based summarization problem, and propose an iterative ranking algorithm for selecting summary sentences. Huang et al. (2015) proposed a summarization technique to display the decision table as a new summarization table that helps in quick decision making. Chatterjee and Sahoo (2015) proposed a model called Modified Random Indexed Summarization (MRIS). In which a separate Index is given for both positive and negative words. Mutual cancellation of positive ones and negative one’s occur, while summing the index vectors to create the context related words.

4.METHODOLOGY

Shah et al. (2016) introduced an Event Sensor system, which leverages Event Builder as its semantics engine to produce syntax based event summarization. It extracts concepts from the visual content and textual metadata of a photo and exploits them to determine the syntax details of the Image. Kar et al. (2015) considered Wikipedia where data is dynamic by nature, focused on extractive summarization using Latent Dirichlet Allocation model to address hidden/latent topic structures of changes. Later made a comparison with the manual summaries using ROUGE


4320

metrics. Figure 3 gives the conventional Machine Learning (CML) based opinion summarization method for obtaining summarization and knowledge of Twitter Data.

Figure 3: Basic Frame work of Opinion Summarization on Twitter.

A. preprocessing

Algorithm

1. Start 2. Removing portions of the document(Re-Tweets, HTTP Links and punctuations’) that are not

considered suitable for including in the summary. 3. Cleaning HTTP Links. 4. Removing punctuations. 5. Any occurrences of the abbreviation in the document are then replaced with the expansion.

Abbreviations are then substituted with their expansions in the same way. 6. The text in the body section is split into sentences using sentence Tokenize. 7. Stop words are removed since they are not useful in discriminating between relevant and

irrelevant sentences. 8. Stop

B. Concept Identification The next stage is to map the terms in the document to concepts called: Quantitative, Qualitative, Temporal, Functional, Idea or Concept, Intellectual Product, Mental Process, Spatial Concept and Language.

Fig. 4 Arranging sentences A to D in Bottom-up Approach.

C. Document Representation The next step is to construct a graph representing the document (Bollegala et al., 2010). We begin by creating a sentence graph for each sentence in the document. A bottom up approach is used to construct a sentence graph as shown in the Figure 4. In this approach, We define notation a > b to


4321

represent that sentence a pre-cedes sentence b. We use the term segment, to describe a sequence of ordered sentences. When segment D consists of sentences d1; d2; :::; dm in this order, we denote it as:

),...,( 21 mdddD (1)

we introduce a function f (A > B) to represent the direction and strength of the association of two segments, A and B.

)(0

)()( ABif

BAifpBAf

(2)

where (0>p>1)denotes the association strength of the segments A and B. The association strengths of the two segments with different directions, e.g. f (a > b) and f (b > a) are not always identical by our definition. D. Concept Clustering and Topic Recognition A degree-based clustering method is applied to the document graph. This identifies sets of concepts that are strongly related in meaning and it is assumed that each set represents a different topic in the document Plaza et al. (2012). where connect e(vi, vj) denotes that the edge e connects vertices vi and vj. Salience ranks(S) the nodes according to their structural importance in the graph as in Equation 3.

),,(|

)()(jii vveconnectve

i eweightvS

(3) E. Sentence Selection

The final step of the summarization process consists of selecting the sentences for the summary. Lloret et al. (2015) proposed an approach for identification and extraction of the most relevant sentence using sentence simplification and language regeneration stage that minimizes the effect of cohesion and coherence. Glavas and Snajder (2014) uses similarity between the sentence and the cluster. Computed as the sum of the votes assigned by all the vertices in the sentence to the cluster(C) from equation 3, as shown in equation 4.

ikk svv

ijkji WscSimilarity|

,,),(

(4) We compute a single score for each sentence as the sum of its similarity to each cluster normalized by the clusters’ size equation 5

iC

i

ji

jC

scSimilaritysScore

),()(

(5) The scoring function selects most of the sentences from the most populated cluster and also includes some sentences from other clusters when the sentences assign high scores to them. Perea-Ortega et al. (2013) states that the use of synonyms, words with similar semantics increases Ambiguity and Redundancy. Ouyang et al. (2013) proposed method towards achieving the target of selecting novel and salient sentences by, Ensure the novelty of the sentences and Saliency of sentences. V. CHALLENGES A. Redundancy It is caused by duplicate sentences appearing in the document. This can be solved by double-perspective method. Lloret and Palomar (2013a) quantified the influence of the proposed levels of language analysis for detecting redundancy using cosine similarity for the lexical level, textual entailment for the syntactic, and sentence alignment for the semantic one.


4322

B. Feature sparseness problem It is generally caused by extracting features from only one sentence for sentence extraction. This problem can be partially solved by using bi-gram pseudo sentences; the bi-gram pseudo sentences are generated by binding two consecutive sentences with a sliding window technique. Ko and Seo (2008) proposed method carries out two different sentence extraction tasks for solving the feature sparseness problem.

6.RESULTS

Fig. 5 Opinion Analysis on Demonetization. A. Experimental Setup A data set of 8000 rows and 14 columns on demonetization is collected from Twitter, the another is about 10000 rows and 13 columns on disaster management from Twitter, both are in .csv form is considered to generate sentiment analysis as shown in the figure 5 and 6.

7. DISCUSSION Boran et al. (2016) used statistical characteristics of attributes are usually described by median, mean, standard deviation. statistical summarization is easy to implement for extracting knowledge it provides limited knowledge.

Fig. 6 Opinion Analysis on Disaster management.


4323

8.CONCLUSION In our work, we identified that a Graph based approach can be used to summarize any topic, that can be classified in to five stages that may provide useful theoretical framework for future research on how tweets can be best summarized. Moreover, We proposed an framework to determine the importance of each tweet by assigning them appropriate weights as term Vector, and how to select the top ranked predicate argument structures from each cluster for summary generation phase. In future, we plan to integrate this Graph based approach with opinion Extraction from single/multi-document to built abstractive/extractive summarization.

ACKNOWLEDGE we would like to thank our VIT University to pro-viding us the all the research facilities requirement for publishing SCI journals. At the same time we would like to thank all the reviewers, who helps us to improve the quality of our paper.

REFERENCES

[1] Bhargava, R., Sharma, Y., Sharma, G., 2016. Atssi: Abstractive text summarization using sentiment infusion. Procedia Computer Science 89, 404–411.

[2] Bollegala, D., Okazaki, N., Ishizuka, M., 2010. A bottom-up approach to sentence ordering for multi-document summarization. Information processing & management 46 (1), 89–109.

[3] Boran, F. E., Akay, D., Yager, R. R., 2016. An overview of methods for linguistic summarization with fuzzy sets. Expert Systems with Applications.

[4] Canhasi, E., Kononenko, I., 2016. Weighted hierarchical archetypal analysis for multi-document summarization. Computer Speech & Language 37, 24–46.

[5] Chatterjee, N., Sahoo, P. K., 2015. Random indexing and modified random indexing based approach for extractive text summarization. Computer Speech & Language 29 (1), 32–44.

[6] Chen, K.-Y., Liu, S.-H., Chen, B., Wang, H.-M., Chen, H.-H., 2016. Exploring the use of unsupervised query modeling techniques for speech recognition and summarization. Speech Communication 80, 49–59.

[7] Gerani, S., Carenini, G., Ng, R. T., 2016. Modeling content and structure for abstractive review summarization. Computer Speech & Language.

[8] Glavas, G., Šnajder, J., 2014. Event graphs for information retrieval and multi-document summarization. Expert systems with applications 41 (15), 6904–6916.

[9] He, R., Tang, J., Gong, P., Hu, Q., Wang, B., 2016. Multi-document summarization via group sparse learning. Information Sciences 349, 12–24.

[10] Heu, J.-U., Qasim, I., Lee, D.-H., 2015. Fodosu: multi-document summarization exploiting semantic analysis based on social folksonomy. Information Processing & Management 51 (1), 212–225.

[11] Hijikata, Y., Ohno, H., Kusumura, Y., Nishida, S., 2007. Social summarization of text feedback for online auctions and interactive


4324

presentation of the summary. Knowledge-Based Systems 20 (6), 527– 541.

[12] Huang, T. C.-K., Chen, Y.-L., Chang, T.-H., 2015. A novel summarization technique for the support of resolving multi-criteria decision making problems. Decision Support Systems 79, 109–124.

[13] Kar, M., Nunes, S., Ribeiro, C., 2015. Summarization of changes in dynamic text collections using latent dirichlet allocation model. In-formation Processing & Management 51 (6), 809–833.

[14] Ko, Y., Seo, J., 2008. An effective sentence-extraction technique using contextual information and statistical approaches for text summarization. Pattern Recognition Letters 29 (9), 1366–1371.

[15] Li, Q., Jin, Z., Wang, C., Zeng, D. D., 2016. Mining opinion summarizations using convolution neural networks in Chinese micro blogging systems. Knowledge-Based Systems 107, 289–300.

[16] Lloret, E., Boldrini, E., Vodolazova, T., Martínez-Barco, P., Muñoz, R., Palomar, M., 2015. A novel concept-level approach for ultra-concise opinion summarization. Expert Systems with Applications 42 (20), 7148–7156.

[17] Lloret, E., Palomar, M., 2013a. Tackling redundancy in text summarization through different levels of language analysis. Computer Standards & Interfaces 35 (5), 507–518.

[18] Lloret, E., Palomar, M., 2013b. Towards automatic tweet generation: A comparative study from the text summarization perspective in the journalism genre. Expert Systems with Applications 40 (16), 6624–6630.

[19] Lloret, E., Romá-Ferri, M. T., Palomar, M., 2013. Compendium: A text summarization system for generating abstracts of research pa-pers. Data & Knowledge Engineering 88, 164–175.

[20] Marujo, L., Ling, W., Ribeiro, R., Gershman, A., Carbonell, J., de Matos, D. M., Neto, J. P., 2016. Exploring events and distributed representations of text in multi-document summarization. Knowledge-Based Systems 94, 33–42.

[21] Moen, H., Peltonen, L.-M., Heimonen, J., Airola, A., Pahikkala, T., Salakoski, T., Salanterä, S., 2016. Comparison of automatic sum-marisation methods for clinical free text notes. Artificial intelligence in medicine 67, 25–37.

[22] Moens, M.-F., Angheluta, R., Dumortier, J., 2005. Generic technolo-gies for single-and multi-document summarization. Information Processing & Management 41 (3), 569–586.

[23] Mosa, M. A., Hamouda, A., Marei, M., 2016. Ant colony heuristic for user-contributed comments summarization. Knowledge-Based Systems.

[24] Ouyang, Y., Li, W., Li, S., Lu, Q., 2011. Applying regression models to query-focused multi-document summarization. Information Processing & Management 47 (2), 227–237.

[25] Ouyang, Y., Li, W., Zhang, R., Li, S., Lu, Q., 2013. A progressive sentence selection strategy for document summarization. Information Processing & Management 49 (1), 213–221.

[26] Peng, M., Gao, B., Zhu, J., Huang, J., Yuan, M., Li, F., 2016. High quality information extraction and query-oriented summarization for automatic query-reply in social network. Expert Systems with Applications 44, 92–101.

[27] Perea-Ortega, J. M., Lloret, E., Ureña-López, L. A., Palomar, M., 2013. Application of text summarization techniques to the geo-


4325

graphical information retrieval task. Expert systems with applications 40 (8), 2966–2974.

[28] Plaza, L., Díaz, A., Gervás, P., 2011. A semantic graph-based approach to biomedical summarization. Artificial intelligence in medicine 53 (1), 1–14.

[29] Plaza, L., Stevenson, M., Díaz, A., 2012. Resolving ambiguity in biomedical text to improve summarization. Information Processing & Management 48 (4), 755–766.

[30] Qiang, J.-P., Chen, P., Ding, W., Xie, F., Wu, X., 2016. Multi-document summarization using closed patterns. Knowledge-Based Systems 99, 28–38.

[31] Reeve, L. H., Han, H., Brooks, A. D., 2007. The use of domain-specific concepts in biomedical text summarization. Information Processing & Management 43 (6), 1765–1776.

[32] Sankarasubramaniam, Y., Ramanathan, K., Ghosh, S., 2014. Text summarization using wikipedia. Information Processing & Management 50 (3), 443–461.

[33] Shah, R. R., Yu, Y., Verma, A., Tang, S., Shaikh, A. D., Zimmermann, R., 2016. Leveraging multimodal information for event summarization and concept-level sentiment analysis. Knowledge-Based Systems.

[34] Smyth, B., et al., 2010. Social summarization in collaborative web search. Information processing & management 46 (6), 782–798.

[35] Song, W., Choi, L. C., Park, S. C., Ding, X. F., 2011. Fuzzy evolutionary optimization modeling and its applications to unsupervised categorization and extractive summarization. Expert Systems with Applications 38 (8), 9112–9121.

[36] Tayal, M. A., Raghuwanshi, M. M., Malik, L. G., 2017. Atssc: Development of an approach based on soft computing for text summarization. Computer Speech & Language 41, 214–235.

[37] Wang, D., Liu, Y., 2015. Opinion summarization on spontaneous conversations. Computer Speech & Language 34 (1), 61–82.

[38] Yang, G., Wen, D., Chen, N.-S., Sutinen, E., et al., 2015. A novel contextual topic model for multi-document summarization. Expert Systems with Applications 42 (3), 1340–1352.

[39] Ye, S., Chua, T.-S., Kan, M.-Y., Qiu, L., 2007. Document concept lattice for text understanding and summarization. Information Processing & Management 43 (6), 1643–1662.

[40] Yousefi-Azar, M., Hamey, L., 2017. Text summarization using un-supervised deep learning. Expert Systems with Applications 68, 93–105.

[41] Zajic, D. M., Dorr, B. J., Lin, J., 2008. Single-document and multi-document summarization techniques for email threads using sentence compression. Information Processing & Management 44 (4), 1600–1610.

[42] Zhang, R., Li, W., Liu, N., Gao, D., 2016. Coherent narrative summarization with a cognitive model. Computer Speech & Language 35, 134–160.

[43] Zhao, L., Wu, L., Huang, X., 2009. Using query expansion in graph-based approach for query-focused multi-document summarization. Information Processing & Management 45 (1), 35–41.

[44] Zhong, S.-h., Liu, Y., Li, B., Long, J., 2015. Query-oriented unsupervised multi-document summarization via deep learning model. Expert Systems with Applications 42 (21), 8146–8155.


4326

A Novel Graph Based Frame work for Opinion …A Novel Graph Based Frame work for Opinion...

Documents

Transcript of A Novel Graph Based Frame work for Opinion …A Novel Graph Based Frame work for Opinion...