'[f!0dZddlZddlTddlTejdZejdZejdZejdZGdd Z Gd d Z Gd d e Z dS)a CorpusReader for reviews corpora (syntax based on Customer Review Corpus). Customer Review Corpus information ================================== Annotated by: Minqing Hu and Bing Liu, 2004. Department of Computer Science University of Illinois at Chicago Contact: Bing Liu, liub@cs.uic.edu https://www.cs.uic.edu/~liub Distributed with permission. The "product_reviews_1" and "product_reviews_2" datasets respectively contain annotated customer reviews of 5 and 9 products from amazon.com. Related papers: - Minqing Hu and Bing Liu. "Mining and summarizing customer reviews". Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD-04), 2004. - Minqing Hu and Bing Liu. "Mining Opinion Features in Customer Reviews". Proceedings of Nineteeth National Conference on Artificial Intelligence (AAAI-2004), 2004. - Xiaowen Ding, Bing Liu and Philip S. Yu. "A Holistic Lexicon-Based Appraoch to Opinion Mining." Proceedings of First ACM International Conference on Web Search and Data Mining (WSDM-2008), Feb 11-12, 2008, Stanford University, Stanford, California, USA. Symbols used in the annotated reviews: :[t]: the title of the review: Each [t] tag starts a review. :xxxx[+|-n]: xxxx is a product feature. :[+n]: Positive opinion, n is the opinion strength: 3 strongest, and 1 weakest. Note that the strength is quite subjective. You may want ignore it, but only considering + and - :[-n]: Negative opinion :##: start of each sentence. Each line is a sentence. :[u]: feature not appeared in the sentence. :[p]: feature not appeared in the sentence. Pronoun resolution is needed. :[s]: suggestion or recommendation. :[cc]: comparison with a competing product from a different brand. :[cs]: comparison with a competing product from the same brand. Note: Some of the files (e.g. "ipod.txt", "Canon PowerShot SD500.txt") do not provide separation between different reviews. This is due to the fact that the dataset was specifically designed for aspect/feature-based sentiment analysis, for which sentence-level annotation is sufficient. For document- level classification and analysis, this peculiarity should be taken into consideration. N)*z ^\[t\](.*)$z%((?:(?:\w+\s)+)?\w+)\[((?:\+|\-)\d)\]z\[(?!t)(p|u|s|cc|cs)\]z##(.*)$c2eZdZdZddZdZdZdZdZdS) Reviewz> A Review is the main block of a ReviewsCorpusReader. Nc8||_| g|_dS||_dS)z :param title: the title of the review. :param review_lines: the list of the ReviewLines that belong to the Review. N)title review_lines)selfrrs N/var/www/piapp/venv/lib/python3.11/site-packages/nltk/corpus/reader/reviews.py__init__zReview.__init__Rs-    "D    ,D   cht|tsJ|j|dS)z Add a line (ReviewLine) to the review. :param review_line: a ReviewLine instance that belongs to the Review. N) isinstance ReviewLinerappend)r review_lines r add_linezReview.add_line]s7 +z22222   -----r cRg}|jD]}||j|S)a Return a list of features in the review. Each feature is a tuple made of the specific item feature and the opinion strength about that feature. :return: all features of the review as a list of tuples (feat, score). :rtype: list(tuple) )rextendfeatures)r rrs r rzReview.featuresfs8, 2 2K OOK0 1 1 1 1r c$d|jDS)z Return all tokenized sentences in the review. :return: all sentences of the review as lists of tokens. :rtype: list(list(str)) cg|] }|j S)sent).0rs r z Review.sents..zsFFF[  FFFr )rr s r sentsz Review.sentsssGFD4EFFFFr cBd|j|jS)Nz#Review(title="{}", review_lines={}))formatrrrs r __repr__zReview.__repr__|s$4;; J)   r NN) __name__ __module__ __qualname____doc__r rrrr rr r rrMsq - - - -...   GGG     r rc eZdZdZddZdZdS)rz A ReviewLine represents a sentence of the review, together with (optional) annotations of its features and notes about the reviewed item. NcZ||_|g|_n||_| g|_dS||_dSNrrnotes)r rrr*s r r zReviewLine.__init__s:  DMM$DM =DJJJDJJJr cNd|j|j|jS)Nz*ReviewLine(features={}, notes={}, sent={}))rrr*rrs r r zReviewLine.__repr__s';BB M4:ty   r r!)r"r#r$r%r r rr r rrsA         r rcjeZdZdZeZedfdZd dZd dZ d dZ d dZ d Z d Z d Zd ZdS)ReviewsCorpusReadera Reader for the Customer Review Data dataset by Hu, Liu (2004). Note: we are not applying any sentence tokenization at the moment, just word tokenization. >>> from nltk.corpus import product_reviews_1 >>> camera_reviews = product_reviews_1.reviews('Canon_G3.txt') >>> review = camera_reviews[0] >>> review.sents()[0] # doctest: +NORMALIZE_WHITESPACE ['i', 'recently', 'purchased', 'the', 'canon', 'powershot', 'g3', 'and', 'am', 'extremely', 'satisfied', 'with', 'the', 'purchase', '.'] >>> review.features() # doctest: +NORMALIZE_WHITESPACE [('canon powershot g3', '+3'), ('use', '+2'), ('picture', '+2'), ('picture quality', '+1'), ('picture quality', '+1'), ('camera', '+2'), ('use', '+2'), ('feature', '+1'), ('picture quality', '+3'), ('use', '+1'), ('option', '+1')] We can also reach the same information directly from the stream: >>> product_reviews_1.features('Canon_G3.txt') [('canon powershot g3', '+3'), ('use', '+2'), ...] We can compute stats for specific product features: >>> n_reviews = len([(feat,score) for (feat,score) in product_reviews_1.features('Canon_G3.txt') if feat=='picture']) >>> tot = sum([int(score) for (feat,score) in product_reviews_1.features('Canon_G3.txt') if feat=='picture']) >>> mean = tot / n_reviews >>> print(n_reviews, tot, mean) 15 24 1.6 utf8c\t||||||_d|_dS)ad :param root: The root directory for the corpus. :param fileids: a list or regexp specifying the fileids in the corpus. :param word_tokenizer: a tokenizer for breaking sentences or paragraphs into words. Default: `WordPunctTokenizer` :param encoding: the encoding that should be used to read the corpus. z README.txtN) CorpusReaderr _word_tokenizer_readme)r rootfileidsword_tokenizerencodings r r zReviewsCorpusReader.__init__s1 dD'8<<<-# r Nc|j}nt|tr|g}tfd|dDS)au Return a list of features. Each feature is a tuple made of the specific item feature and the opinion strength about that feature. :param fileids: a list or regexp specifying the ids of the files whose features have to be returned. :return: all features for the item(s) in the given file(s). :rtype: list(tuple) NcPg|]"\}}|j|#S)r6) CorpusView_read_featuresrfileidencr s r rz0ReviewsCorpusReader.features..sB   !VS(;cJJ   r T)_fileidsrstrconcatabspathsr r4s` r rzReviewsCorpusReader.featuressr ?mGG  % % iG    %)]]7D%A%A      r ct|j}tfd|dDS)aS Return all the reviews as a list of Review objects. If `fileids` is specified, return all the reviews from each of the specified files. :param fileids: a list or regexp specifying the ids of the files whose reviews have to be returned. :return: the given file(s) as a list of reviews. NcPg|]"\}}|j|#Sr9)r:_read_review_blockr<s r rz/ReviewsCorpusReader.reviews..sB   !VS(?#NN   r T)r?rArBrCs` r reviewszReviewsCorpusReader.reviewssW ?mG    %)]]7D%A%A      r cdtfd|ddDS)aY Return all sentences in the corpus or in the specified files. :param fileids: a list or regexp specifying the ids of the files whose sentences have to be returned. :return: the given file(s) as a list of sentences, each encoded as a list of word strings. :rtype: list(list(str)) cRg|]#\}}}|j|$Sr9)r:_read_sent_blockrpathr>r=r s r rz-ReviewsCorpusReader.sents..D   'T3d&;cJJ   r TrArBrCs` r rzReviewsCorpusReader.sentsM    +/==$+M+M      r cdtfd|ddDS)aK Return all words and punctuation symbols in the corpus or in the specified files. :param fileids: a list or regexp specifying the ids of the files whose words have to be returned. :return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str) cRg|]#\}}}|j|$Sr9)r:_read_word_blockrKs r rz-ReviewsCorpusReader.words..rMr TrNrCs` r wordszReviewsCorpusReader.wordsrOr cg}tdD]I}|}|s|cS|tjt |J|S)N)rangereadlinerrefindallFEATURES)r streamrilines r r;z"ReviewsCorpusReader._read_featuressbr 8 8A??$$D  OOBJx66 7 7 7 7r c |}|sgStjt|}|r6t |d}nk |}|}|s|gStjt|r|||gStj t|}tj t|}tj t|}|r |j |d}t|||} || )NT)rrr))rWrXmatchTITLErgroupstriptellseekrYrZNOTESSENTr1tokenizerr) r r[r] title_matchreviewoldposfeatsr*rrs r rFz&ReviewsCorpusReader._read_review_blocksV ??$$D  (5$//K %++A..4466  )[[]]F??$$D xxt$$  F###xJx..EJud++E:dD))D >+44T!W==$$eLLLK OOK ( ( (% )r cg}||D]3}|d|D4|S)Ncg|]}|Srr)rrs r rz8ReviewsCorpusReader._read_sent_block..As:::4$:::r )rFrr)r r[rrjs r rJz$ReviewsCorpusReader._read_sent_block>sU--f55 < >::: ; ; ; ; r cg}tdD]e}|}tjt|}|r3||j|df|S)NrUr)rVrWrXrYrgrr1rh)r r[rSr\r]rs r rRz$ReviewsCorpusReader._read_word_blockDstr E EA??$$D:dD))D E T1::47CCDDD r r()r"r#r$r%StreamBackedCorpusViewr:WordPunctTokenizerr rrGrrSr;rFrJrRrr r r-r-s>(J-?,>,@,@6 $ $ $ $    *    $    "    ")))B r r-) r%rXnltk.corpus.reader.api nltk.tokenizecompilerarZrfrgrrr0r-rr r rus66p $$$$ >"" 2:,    ,--rz*2 2 2 2 2 2 2 2 j        0qqqqq,qqqqqr