A survey on phrase structure learning methods for text classification

A survey on phrase structure learning methods for text classification

• 1.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 DOI : 10.5121/ijnlc.2014.3203 33 A SURVEY ON PHRASE STRUCTURE LEARNING METHODS FOR TEXT CLASSIFICATION Reshma Prasad1 and Mary Priya Sebastian2 1 Department of Computer Science and Engineering, Rajagiri School of Engg. & Technology, Kakkanad, Kochi, Kerala 2 Department of Computer Science and Engineering, Rajagiri School of Engg. & Technology, Kakkanad, Kochi, Kerala ABSTRACT Text classification is a task of automatic classification of text into one of the predefined categories. The problem of text classification has been widely studied in different communities like natural language processing, data mining and information retrieval. Text classification is an important constituent in many information management tasks like topic identification, spam filtering, email routing, language identification, genre classification, readability assessment etc. The performance of text classification improves notably when phrase patterns are used. The use of phrase patterns helps in capturing non-local behaviours and thus helps in the improvement of text classification task. Phrase structure extraction is the first step to continue with the phrase pattern identification. In this survey, detailed study of phrase structure learning methods have been carried out. This will enable future work in several NLP tasks, which uses syntactic information from phrase structure like grammar checkers, question answering, information extraction, machine translation, text classification. The paper also provides different levels of classification and detailed comparison of the phrase structure learning methods. KEYWORDS Text classification, Phrase structure, Phrase patterns, Natural Language Processing (NLP) 1. INTRODUCTION Text classification or categorization includes automatic classification of documents or texts into predefined categories. Different application of text classification includes spam filtering, email routing, language identification, genre classification, readability assessment etc. There are different methods for text classification which includes decision trees [2], rule based classifiers [5], SVM classifiers [7], neural network classifiers [4], bayesian classifiers [3] and nearest neighbour classifiers [1]. Text classification can be improved if phrase patterns are used in the classification task and phrase pattern identification progresses with already extracted phrases. Phrase structure is the grammatical arrangement of words in a sentence. The words in a sentence are not arranged in just any order, but language has constraints on word order. Words are organized into phrases, groupings of words that are clumped as a unit and a sentence can be modeled as a set of phrases. Syntactic knowledge can be modeled by phrase structure. Phrase
• 2.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 34 structure is the backbone of many models of syntax of natural language. It can be a powerful way to express sophisticated relations among the words in a sentence. NLP activities like grammar checkers, question answering, information extraction, machine translation, text classification uses syntax information form phrase structure. The advantage of using phrase patterns for text classification is that phrases helps to identify long distant dependencies, the structure can support distant relationships between words. This method provides flexibilities of modeling local word-reordering or grouping which is the main problem of free order languages. Also, phrase extraction brings in some semantic value, which is suitable when NLP activities like text classification, or machine translation is considered. Another advantage of the phrase pattern identification is that it filters words occurring frequently in isolation that does not have much weight towards classification. The different methods of phrase structure learning or extraction have to be studied, classified and compared. 2. SURVEYED TECHNIQUES The goal of the phrase structure extraction is to automatically extract phrase structures from a given corpus. The different techniques surveyed here are based on phrase structure learning. Many methods of phrase structure learning have already been developed for languages like English, Chinese, German, Japanese, Swedish etc. Phrase extraction techniques based on both bilingual and monolingual corpus are discussed in this paper. Different methods surveyed here are: 2.1. Basic N-gram based approach N-gram based approach is a statistical approach which includes application of n-gram models to obtain phrases. William B. Cavnar and John M. Trenkle proposed an approach which extracted phrases using n-gram model and the phrases thus obtained are used for text categorization [6]. Gulila Altenbek, Ruina Sun used N-gram models for phrase structure extraction from unannotated monolingual corpus [31]. Bigram and trigram models are applied to extract phrases from the corpus. The monolingual corpus is roughly segmented and N-gram model is applied followed by a normalization process. Equation (2) represents the probability [31]. Accuracy is measured in terms of number of phrases correct to total phrases extracted. The accuracy is measured around 51%, which is low. Among the two models, bigram model has more accuracy than trigram [31]. 2.2. Rule based method Ramshow and Marcus used transformation rule based learning for extracting the noun phrases[9]. Gulila Altenbek, Ruina Sun used rule based method [31] for noun phrase extraction from monolingual corpus. The method is a non-statistical approach which uses annotated monolingual corpus. The approach is based on the basic rules of the target monolingual language; therefore developing a rule set for the corresponding language is a necessary condition. The phrases are extracted according to the rules defined, the corpus is searched for a matched rule and the phrases thus found are extracted. Accuracy is measured in terms of number of phrases correct to total phrases extracted. The accuracy for rule based approach is found to be around 80% while that of N-gram based approach is found to be around 51%. (2)
• 3.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 35 2.3. Word alignment based method Word alignment based method is a statistical method. Phrase alignments are learned from a corpus that has been word aligned. The basic idea is to align the parallel corpus in both directions and to take an intersection so that an alignment matrix is generated. The alignment points in the alignment matrix are expanded based on different heuristics. Franz Josef Och, Christoph Tillman and Hermann Ney developed an improved alignment model [11] which uses alignment templates. An alignment template is a triple (F,E,A) where A is an alignment matrix with binary values. The template describes the alignment between source class sequence F and target class sequence E. The initial step of the alignment template approach is to align the parallel corpus in the two translation directions, source to target and target to source. Expectation maximization algorithm is applied in both directions to obtain two alignment vectors. The two alignment vectors are combined to form the alignment matrix A. Iteratively checking and adding neighboring links extend the alignment. All the consistent phrase pairs of the training corpus are determined by checking if the source phrase words are aligned only to target phrase words. The advantage of this approach is the fully automatic learning using bilingual training corpus [11]. The disadvantage is that it selects all templates without checking whether it is good or bad [11]. The measuring score used is word error rate (WER), position independent error rate (PER) and subjective sentence error rate (SSER). In terms of efficiency, the error rate decreased to about 6% than baseline. Philip Koehn, Franz Josef Och and Daniel Marcu modified the alignment template approach later [16]. The heuristics for expansion in the alignment template approach is modified by permitting diagonal neighborhood in the expansion stage [5]. Giza++ toolkit is used for word alignment. Lexical weighting and maximum phrase length scores are applied to the model. Top performance is obtained when the phrase length is three. The method shows better performance when lexical weighting score is applied. The method has an improvement of about 0.01 BLEU score than alignment template approach. 2.4. Phrase alignment based method Phrase alignment based method is another statistical approach in which phrases are extracted from the phrase alignment using phrase-based joint probability. Daniel Marcu and William Wong developed a phrase based joint probability model [14]. The model captures simultaneous generation of source and target sentences in a parallel corpus rather than alignment between them. In this method, each sentence pair in our corpus is generated by the idea of generation of a bag of concepts (each concept is a phrase pair) and the bag of concepts can be arranged linearly to obtain source and target sentences [14]. The initial step of the method is to find high frequency n-grams and t-distribution table initialization. Expectation maximization learning on the Viterbi alignment is then applied iteratively which yields joint probability distribution. The performance of the phrase-based method has an average improvement of about 0.02 BLEU score than word alignment based method. A modification of the base model has been proposed by Philip Koehn, Franz Josef Och and Daniel Marcu [16]. The base model is modified by marginalizing the joint probabilities to conditional probability [16]. The performance of the method is high when phrases are of length three.
• 4.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 36 2.5. Syntactic approach Philip Koehn, Franz Josef Och and Daniel Marcu proposed a syntactic method [16], which involves parsing of the sentences in bilingual parallel corpus. In this method, a syntactic phrase is defined as a sequence of words which is covered in a single subtree of a syntactic parse tree [15]. The first step of the method is to word align the parallel corpus. Both side of the corpus is parsed using syntactic parsers. For consistent phrase pairs, it is checked whether both the phrases are subtrees in the parse trees generated. The measuring score used is BLEU score. A BLEU score of 0.243 is obtained when efficiency is measured [16]. 2.6.. Mutual Information based method Ying Zhang, Stephan Vogel and Alex Waibel developed an integrated phrase segmentation and alignment algorithm [17] for statistical machine translation which uses Mutual Information (MI). The algorithm segments sentences into phrases and thus can be used as a phrase extraction technique. In this method, an initial word alignment or initial segmentation on the monolingual text is not required [17]. The phrases are identified by similarity of point wise mutual information and thus the sentences are segmented into phrases. A two dimensional matrix is constructed to represent the source and target sentence pairs where value of each cell corresponds the point wise mutual information score between source and target words. From the matrix, phrase pair with high MI value is selected and it is expanded to rectangle regions such that the expanded region has a similar MI value. The rectangular region is considered as phrase pair. Repeating this step iteratively identifies all the phrase pairs. Equation (1) is used to calculate the point wise mutual information [17]: After the segmentation of sentence pairs into phrase pairs, joint probabilities are calculated for these phrase pairs using monolingual conditional probability. The advantage of this method is that it does not require to find high frequency N-grams. Precision and length penalty is used as measuring scores here and a confidence level of 99.99% over baseline HMM represents the efficiency of this approach. 2.7. Bilingual N-gram based approach Ashish Venugopal, Stephan Vogel and Alex Waibel developed an approach for phrase translation extraction using N-grams and the method builds phrase lexicons from bilingual corpus [18]. The method consists of three phases: generation, scoring, and pruning. In the generation phase, all source phrases are identified using N-gram and all possible candidate target phrases corresponding to source phrases are identified. This set is scored and pruned using various scores to remove unwanted target phrases. In the scoring phase, the phrases are scored using measures from three models, maximum approximation, word based translation lexicon and language specific measures [18]. In the pruning step, maximum likely phrase pairs are selected using maximal separation criteria [18]. The advantage of this method is that it is less computationally expensive and recovers well from noisy alignments, but it lacks an explanatory framework. When performance is considered, the method shows a NIST score improvement of 0.05 over baseline HMM word alignment method. (1)
• 5.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 37 2.8. Block based method Ashish Venugopal, Stephan Vogel and Alex Waibel developed a phrase translation extraction from alignment models in 2005, which is based on blocks [21]. In the block based method [21], phrase pair within parallel sentence is considered as a block. The method does not use alignment. An example of a block is shown in fig.1 [21]. In the block, y-axis is the source sentence, x axis is the target sentence. The block is defined by source phrase and its projection, which can be represented as the left and right boundaries in the target sentence. The source phrase is bounded by the start and end positions in the source sentence. Fig. 1. Example of a block Three models are used in this method. The first one is Fertility model, which predicts the width of the block by computing phrase length. A dynamic programming algorithm using the source word fertilities is employed in this model and given the candidate target phrase e and a source phrase of length J, the model gives the estimation of P(J/e1). Next is the Distortion model. A simple distortion score is computed to estimate how far away the two centers are in a parallel sentence pair in a sense the block is close to the diagonal [21].Another model is Lexicon model, which is computed for translational equivalence. For each candidate block, using word level lexicon, a score within a given sentence pair is computed. For each candidate block, the scores of phrase length, center based distortion and a lexicon-based score are calculated, which is followed by a local greedy search to find best scored phrase pair. The method is a general framework, in which one could plug in other scores and word alignment to get better results, but the computational expense of this method will be higher. 2.9. Clustering method Rile HU, Chengqing ZONG and Bo XU proposed an approach to automatic acquisition of translation templates which is based on phrase structure extraction and alignment [23]. The method is a statistical and data driven approach [23]. The basic idea of the method is to cluster or group words in the corpus using similarity measure. Clustering is performed in two steps, temporal clustering and spatial clustering. Temporal clustering clusters words or entities, which frequently co occurs, into groups. Frequent co occurrence of entities can be obtained by finding the mutual information score between the word pair or entity pair. Spatial clustering clusters words or entities, which have similar left and right contexts which is measured by the Kullback- Leibler distance [23].
• 6.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 38 The initial step of the method is to find the similarity measure in terms of distance for each pair of entities and N-pairs of entities with minimum distance is selected to form semantic class. The entities are then replaced with semantic class label. The next step is to find similarity measure in terms of the Mutual Information for each entity pair and N pairs of entities with highest Mutual Information (MI) are selected to form phrasal structure groups. The entities are replaced with phrasal structure class label. This process is repeated till a stopping criterion (STC) is reached. The method has higher precision, recall and f-score than base approaches Bracketing Transduction Grammar method [8] and parse to parse match[13]. Higher number of phrase groups were obtained when cosine of pointwise mutual information score was used [23]. 2.10. Loose phrase extraction method Xue Yongzeng and Li Sheng developed a loose phrase extraction method with n-best alignments [28]. The method of loose phrase extraction [28] is based on the idea of extracting phrase pairs that are not strictly consistent with word alignments. In normal phrase extraction techniques, which use bilingual corpora, exact phrase extraction is used, that is, all the phrase pairs that are consistent are extracted. But this method allows some relaxation to the rule of consistency. Loose phrase pairs can be aligned to some words outside, provided that the word is also aligned to some words inside the phrase pair [28]. After phrase extraction, constraints are applied to these loose phrase pairs to avoid ill formed phrase pairs. Main constraints applied are intersection-based constraint and heuristic based constraint. Apart from applying constraints, a union between n-best alignments from each translation direction is collected and the two unions of alignment are combined. The method achieves better performance than baseline exact match approach, also N- best alignment results on all constraints shows better BLEU score. 2.11. Word alignment and Rule based approach Andreas Eisele, Christian Federmann, Herv´e Saint Amand, Michael Jellinghaus, Teresa Herrmann and Yu Chen developed a hybrid method integrating a rule based with a hierarchical translation system [30]. This method is a statistical and rule based hybrid approach. The hybrid system inherits the lexicons from both sub-systems as well as other merits of each system [30]. The method uses word alignment method and rule based approach to extract phrases. Phrase tables are generated from both statistical method and rule based method. These phrase tables are later combined so that the hybrid system can exploit knowledge from both methods [30]. The advantage is that the hybrid method can gain extra knowledge from rule-based system but the errors in rule-based system can affect the correct information in statistical system [30]. Another method with variation in combining translation models from various sources has been proposed in 2010 by Yu Chen and Andreas Eisele [32]. In this method, instead of combining phrase tables by adding one binary feature for each individual system, all features in both translation models are retained while combining. BLEU score is used as a measuring score. The hybrid method showed improved performance than the baseline word alignment or rule based approach. The performance difference between the hybrid system and the SMT core improved to nearly 1.5 BLEU [32]. 2.12. N-gram and Rule based approach N-gram and Rule based approach [34] is a statistical and rule based hybrid approach developed by Yoh Okuno. In this method, N-gram model is applied to preprocessed corpus using map reduce framework. The N-gram model application is followed by rule based filtering based on the part of speech patterns. Three types of errors were observed, Judgment inconsistency, Morphological analysis error, Lack of features for additional rules. Measuring scores used are
• 7.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 39 precision, recall and F-measure. When compared with the baseline N-gram, precision improvement of about 0.49 was observed. The method shows better performance than N-gram and rule based approaches. 3. CLASSIFICATION The level 1 classification of the different phrase structure learning methods is given in figure.2. The different methods of phrase structure learning can be broadly classified into three classes, statistical methods, rule-based methods and statistical and rule hybrid methods. The approaches, which use statistical techniques for phrase extraction are classified as statistical methods and include clustering method, mutual information based, N-gram based, syntactic based method, alignment based method, block based method. Statistical methods are based on the statistical modeling of data and depend on statistical theorems and rules. The methods do not require rule set for the language. The approach, which uses basic rules for phrase structure extraction, is classified as rule based method. The rule based approach needs a developed set of rules for the language and the task is performed based on this set of rules. The approaches, which use both statistical techniques and rules for phrase structure extraction, are classified as statistical and rule hybrid methods and include N-gram and rule based method and word alignment and rule based method. 4. OBSERVATIONS AND DISCUSSION The different methods for phrase structure learning can be compared in detail based on several factors. A level 2 classification of the methods based on corpus used, initial alignment requirement of corpus, base approach, tools used, and technique used is given in Table.1. Another level of classification, level 3 based on different scores applied, evaluation metrics used and efficiency is given in Table.2. When the different methods are compared, an observation made is that the mutual information based method and probabilistic based method shows higher efficiency and performance. Mutual information based method shows a confidence value of 99.99% [17], which is promising. Hybrid methods like N-gram and rule based approach and word alignment and rule based approach shows good performance but requires set of rules for the language. Clustering method also shows relatively better results but the concept of clustering and
• 8.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 40 Figure.2. Level 1 classification of methods translation templates does not have much relevance when agglutinative languages like Indian languages, Japanese, Turkish etc are considered, and may need to consider words in its root form [10]. Alignment template approach, which is a word alignment based method, does not distinguish between good or bad templates though it can be learned automatically using bilingual corpus. When N-gram based approached are considered, basic N-gram shows less accuracy as internal structure of phrases are not considered. Bilingual N-grams show good performance than basic and is less computationally expensive, but it lacks an explanatory framework. Block based method is another candidate but it can be computationally expensive. Rule based approaches is not so efficient as the task of developing a rule set for a particular language is cumbersome and also large amount of rules have to be developed manually. Thus it needs time and support from trained linguistics expert. But when compared with N-grams, the accuracy is more for rule-based
• 9.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 41 approach as it takes into consideration the internal structure of phrases. Hybrid methods like N- gram and rule based approach and word alignment and rule based approach shows good performance but requires set of rules for the language. Syntactic method is not much efficient, as syntactic models do not provide important phrase alignments. Weighting of syntactic phrases also does not improve the performance much. Method Author and year Corpus used Initial alignment requireme nt of corpus Base approach Tools used Technique used Basic N- gram based method William B. Cavnar.et. al (1994), Gulila Altenbek.e t.al (2010) Xinjiang daily corpus No informati on available No information available No informati on available N-gram method for unannotated text Basic Rule based approach L. Ramshaw. et.al (1995), Gulila Altenbek.e t.al (2010) Xinjiang daily corpus No informati on available No information available No informati on available Noun phrases extracted based on set of rules Word Alignmen t based method Franz Josef Och.et.al (1999), Philip Koehn.et.a l (2003) Europarl corpus Sentence aligned Single word based approach(uses manual dictionary) GIZA++ tool kit Phrases from word based alignments - alignment template approach and modified alignment template approach Phrase alignment based method Daniel Marcu.et.a l (2002), Philip Koehn.et.a l (2003) Europarl corpus Sentence aligned word alignment method(IBM model 4) No informati on available Phrases are extracted from the phrase alignment using phrase-based joint probability model Syntactic method Imamura, K (2002), Philip Koehn.et.a l (2003) Europarl corpus Sentence aligned Syntactic translation models Syntacti c parser Syntactic phrase pairs extracted from word aligned corpus, word alignment- modified alignment template approach
• 10.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 42 Mutual Informati on based method Ying Zhang .et.al (2003) Xinhua English news corpora Sentence aligned Word-word translations (IBM model1), Phrase-phrase translations from HMM word alignment No informati on available Integrated phrase segmentation and alignment algorithm- phrases identified by similarity of point wise mutual information Method Author and year Corpus used Initial alignment requireme nt of corpus Base approach Tools used Technique used Bilingual N-gram based method Ashish Venugopal .et.al (2003) English- Chinese parallel language corpus Word aligned using IBM alignment model HMM alignment model, word level system( IBM model1) GIZA tool Building phrasal lexicons by N- gram method with generation, scoring, pruning steps Block based method Bing Zhao.et.al (2005) English- French corpus Sentence aligned word alignment method(IBM model 4) GIZA++, pharaoh decoder Fertility model-to predict width of the block. Distortion model- to predict how close centers of source and target phrase are. Lexicon model- for translation equivalence. Clustering method Rile Hu.et.al (2006) English- Chinese parallel spoken language corpus Sentence aligned Phrase alignment method using Bracketing Transduction Grammar, Syntactic method using parse-to-parse match No informat ion available Phrase extraction- temporal and spatial clustering, Alignment- bracketing transduction grammar Loose phrase based method Xue Yongzeng. et.al (2007) IWSLT-04 Chinese- English translation task Sentence aligned Word alignment method using IBM word alignment model Pharaoh trainer and decoder Based on loose phrase extraction, with extensions of word position based constraints and n-best alignments Word alignment and Rule hybrid Andreas Eisele.et.al (2008), Yu Chen.et.al (2010) Europarl corpus Sentence aligned Rule based approach and Word alignment based method(IBM word alignment model) Moses or Joshua decoder, Lucy, SRILM toolkit Based on word alignment method and rule based method
• 11.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 43 N-gram and Rule hybrid Yoh Okuno (2011) Japanese blog corpus No informati on available N-gram based method No informati on available Based on N-gram model and rule based filtering Table 1. Level 2 classification based on the technique and corpus used Method Author and year Scores applied Evaluation metrics used Efficiency Basic N-gram based method William B. Cavnar.et. al (1994), Gulila Altenbek.e tal (2010) No information available Accuracy % measured in terms of number of phrases correct to total phrases extracted Bigram accuracy- 54.1%, Trigram accuracy- 51.5% Basic Rule based approach L. Ramshaw. et.al (1995), Gulila Altenbek.e tal (2010) No information available Accuracy % measured in terms of number of phrases correct to total phrases extracted Accuracy- 80.3% Word Alignment based method Franz Josef Och.et.al (1999), Philip Koehn.et.a l (2003) lexical weight, word alignment heuristics BLEU score Error rate decreased to about 6%, improvement of about 0.01 BLUE score Phrase alignment based method Daniel Marcu.et.a l (2002), Philip Koehn.et.a l (2003) t-counts BLEU score Improvement of 0.02 BLEU score Syntactic method Imamura, K (2002), Philip Koehn.et.a l (2003) syntactic phrase weight,lexical weight BLEU score BLEU: 0.243 Mutual Information based method Ying Zhang .et.al (2003) student’s t-test NIST- precision and length penalty, Chinese English machine translation evaluation package Confidence:99.99% Precision-6.966, Lengthpenality-0.97
• 12.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 44 Bilingual N-gram based method Ashish Venugopal .et.al (2003) Maximum approximation, estimation from word based translation lexicon, language specific measure, score within sentence consistency,across sentence consistency, score on maximal separation criteria NIST and BLEU scores NIST improvement over HMM: 0.05 Scores: BLEU : 0.197 NIST: 7.6 Block based method Bing Zhao.et.al (2005) Fertility probability, distortion center, word level lexicon probability, seven base scores BLEU score Dev.Bleu: 27.44 , Tst.Bleu: 27.65 Clustering method Rile Hu.et.al (2006) Distance, Cosine measure, Cosine of pointwise MI, Dice Coefficient Precision(P), Recall(R), F-measure(F) P-76.7%, R-80.8%, F-78.75% Method Author and year Scores applied Evaluation metrics used Efficiency Loose phrase based method Xue Yongzeng. et.al (2007) Intersection based constraint, Heuristic based constraint BLEU score Improvement over baseline: 0.043 Word alignment and Rule hybrid Andreas Eisele.et.al (2008), Yu Chen.et.al (2010) No information available BLEU score Moses+Lucy: 27.26 Joshua+Lucy: 27.52 N-gram and Rule hybrid Yoh Okuno (2011) No information available Precision, Recall and F-measure P:0.9,R:0.81,F:0.85 Table 2. Level 3 classification based on scores applied and efficiency 5. CONCLUSION Text classification is an important natural language processing task, which has got many useful applications like spam filtering, email routing, language identification, genre classification, readability assessment. The use of phrases helps in capturing non-local behaviors and thus helps in the improvement of text classification task. In this survey paper, different phrase structure learning methods for text classification have been studied. The approaches are classified in a broader aspect into three groups, statistical methods, rule-based methods and statistical and rule hybrid methods. Different techniques are further classified into two more levels based on the technique used and efficiency. The methods are classified and compared in detail based on different factors like corpus used, initial alignment requirement, base approach, tools used, techniques in the level 2 classification. The methods are again compared and classified based on different scores applied, evaluation metrics used and efficiency in level 3 classification. One observation made is that the major works in phrase structure learning are focused on statistical
• 13.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 45 and hybrid methods as rule based approach needs time and trained linguistics personnel. The major observation made is that mutual information based approach is the most promising technique for phrase structure extraction and shows better performance and efficiency. REFERENCES [1] T.Cover & P. Hart (1967) “Nearest neighbor pattern classification”, IEEE transactions on Information Theory, Volume 13, Issue:1 [2] J. R. Quinlan (1986) “Induction of Decision Trees”, Machine Learning,1(1), pp 81–106 [3] Cheeseman, P., Kelly, J., Self, M., Stutz, J., Taylor, W., & Freeman, D. (1988) “Autoclass: A Bayesian classication system”, Proceedings of the Fifth International Conference on Machine Learning (pp. 54{64). Ann Arbor, MI: Morgan Kaufman [4] Hecht-Nielsen,R. (1992) “Theory of backpropagation Neural Networks” Neural Networks for perceptions, vol.2 [5] C. Apte, F. Damerau, S. Weiss (1994) “Automated Learning of Decision Rules for Text Categorization”, ACM Transactions on Information Systems, 12(3), pp. 233–251 [6] Cavnar,W.B.and Trenkle, J. M, (1994) “N-grambased text categorization”, In Proceedings of SDAIR-94, 3rd Annual Symposium on Docu-ment Analysis and Information Retrieval ,161-175 [7] C. Cortes, V. Vapnik (1995) “Support-vector networks”, Machine Learning, 20: pp. 273–297 [8] D. Wu, (1995 )“An algorithm for simultaneously bracketing parallel texts by aligning words,” in Proc. 33th Annu. Meeting Association for Computational Linguistics, pp. 244–251 [9] L. Ramshaw and M. Marcus. (1995) “Text Chunking using Transformation-Based Learning” In Proceedings of the Third Workshop on Very Large Corpora [10]H. Altay G¨uvenir and Ilyas Cicekli (1998 ) “Learning Translation Templates from Examples” Information Systems [11]Franz Josef Och, Christoph Tillmann, and Hermann Ney (1999) “Improved Alignment Models for Statistical Machine Translation”, Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora [12]Fangfang Feng , W. Bruce Croft (2000) “Probabilistic techniques for phrase extraction”, Information Processing and Management [13]K. Imamura, (2001) “Hierarchical phrase alignment harmonized with parsing,” in Proc. 6th Natural Language Processing Pacific Rim Symp., pp.377–384 [14]Imamura, K (2002) “Application of translation knowledge acquired by hierarchical phrase alignment for pattern based machine translation”, Proceedings of the conference on Empirical methods in Natural Language Processing [15]Daniel Marcu and William Wong, (2002) “A Phrase-Based, Joint Probability Model for Statistical Machine Translation,’’ Proceedings of the Conference on Empiric01 Methods in Natural Language Processing (LMNLP-2002) [16]Philipp Koehn, Franz Josef Och & Daniel Marcu (2003) “Statistical phrase based translation”, Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1 [17]Ying Zhang, Stephan Vogel, Alex Waibel (2003) “Integrated Phrase segmentation and alignment algorithm for statistical machine translation”, Proceedings of International Conference on Natural Language Processing and Knowledge Engineering [18]Ashish Venugopal, Stephan Vogel, Alex Waibel (2003) “Effective Phrase Translation Extraction from Alignment Models”, Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1 [19]R. Hu, C. Zong, and B. Xu (2003) “Semiautomatic acquisition of translation templates from monolingual unannotated corpora” Proc. Int. Conf. Natural Language Processing and Knowledge Engineering, Beijng, China [20]Nikitas N. Karanikolas and Christos Skourlas (2004) “Key-phrase extraction for classification”, X Mediterranean Conference on Medical and Biological Engineering [21]Bing Zhao, Stephan Vogel (2005) “A Generalized Alignment-Free Phrase Extraction”, Proceedings of the ACL Workshop on Building and Using Parallel Texts [22]Yoshida M, Nakagawa H (2005) “Automatic term extraction based on Perplexity of compound words”, IJCNLP [23]Rile Hu & Cheng qing Zong, (2006) “An Approach to Automatic Acquisition of Translation Templates Based on Phrase Structure Extraction and Alignment”, IEEE transactions on Audio,
• 14.
  International Journal onNatural Language Computing (IJNLC) Vol. 3, No.2, April 2014 46 Speech and Language Processing, Volume 14 [24]Chris Callison-Burch, Philipp Koehn, Miles Osborne (2006) “Improved Statistical Machine Translation Using Paraphrases” [25]Josep M. Crego, Adria de Gispert, Patrik Lambert (2006) “N-gram-based SMT System Enhanced with Reordering Patterns”, Proceedings of the HLT-NAACL Workshop on Statistical Machine Translation [26]Masaaki Nagata, Kuniko Saito, Kazuhide Yamamoto, Kazuteru Ohashi (2006) “A Clustered Global Phrase Reordering Model for Statistical Machine Translation”, Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL [27]Rong Zhang (2006) “Extraction and Clustering of Term Definition, and Term Recognition”, Beijing Language and Culture University [28]Xue Yongzeng & Li Sheng (2007) “Loose phrase extraction with n-best alignments”, Journal of Electronics, Volume 24 [29]D. Chiang (2007) “Hierarchical phrase based translation”, Computational Linguistics [30]Andreas Eisele, Christian Federmann, Herv´e Saint Amand, Michael Jellinghaus, Teresa Herrmann, and Yu Chen (2008) “Using moses to integrate multiple rule based machine translation engines into a hybrid system”, Third Workshop on Statistical Machine Translation [31]Gulila Altenbek & Ruina Sun, (2010) “Kazakh Noun Phrase Extraction based on N-gram and Rules”, Internation conference on Asian Language Processing [32]Yu Chen, Andreas Eisele (2010) “Integrating a Rule-based with a Hierarchical Translation System”, Proceedings of 7th International Conference on Language Resources and Evaluation [33]Graham Neubig, Taro Watanabe, Eiichiro Sumita, Shinsuke Mori & Tatsuya Kawahara (2011) “An Unsupervised Model for Joint Phrase Alignment and Extraction”, Proceedings of 49th Annual meeting of the Association for Computational Linguistics: Human Language Technologies [34]Yoh Okuno (2011) “Phrase Extraction for Japanese Predictive Input Method as Post-Processing”, Proceedings of the Workshop on Advances in Text Input Methods [35]Bin Zhang, Alex Marin, Brian Hutchinson & Mari Ostendorf, (2013) “Learning Phrase Patterns for Text Classification”, IEEE transactions on Audio, Speech and Language Processing, Volume 21, Issue 6 [36]Dechun Yin , Dakui Zhang (2013) “Semiautomatic Acquisition of Translation Templates from Monolingual Unannotated Chinese Patent Corpus”, Journal of Information & Computational Science Authors Reshma Prasad received her B.Tech degree in Computer Science & Engineering from Amrita School of Engineering, Kollam in 2010. She is currently doing Mtech in Computer Science and Information Systems from Rajagiri School of Engg. & Technology. Her research interests include Natural Language Processing and Text classification. Mary Priya Sebastian received her B.Tech degree in Computer Engineering from College of Engineering, Chengannur in 2002 and M.Tech degree in Computer and Information Science from the Department of Computer Science, CUSAT in 2010. She is currently working as Assistant Professor in the Department of Computer Science, RSET. She is currently doing her Ph.D in the Dept. of Computer Science, CUSAT. Her research area is Natural Language Processing and Machine Learning. She has 12 publications including National and International ones and she is a professional member of ACM.