'[f$dZddlZddlZddlZddlmZddlTddlm Z ddl Tddl Tddl m Z GddeZGd d eZdS) zN A reader for corpora that contain chunked (and optionally tagged) documents. N tagstr2tree)*)BracketParseCorpusReader)TreeceZdZdZdeeddeddfdZdd Zdd Z dd Z dd Z dd Z ddZ ddZddZddZdZdS)ChunkedCorpusReadera& Reader for chunked (and optionally tagged) corpora. Paragraphs are split using a block reader. They are then tokenized into sentences using a sentence tokenizer. Finally, these sentences are parsed into chunk trees using a string-to-chunktree conversion function. Each of these steps can be performed using a default function or a custom function. By default, paragraphs are split on blank lines; sentences are listed one per line; and sentences are parsed into chunk trees using ``nltk.chunk.tagstr2tree``.  T)gapsutf8Nc Vt||||||||f|_dS)z :param root: The root directory for this corpus. :param fileids: A list or regexp specifying the fileids in this corpus. N) CorpusReader__init___cv_args) selfrootfileids extension str2chunktreesent_tokenizerpara_block_readerencodingtagsets N/var/www/piapp/venv/lib/python3.11/site-packages/nltk/corpus/reader/chunked.pyrzChunkedCorpusReader.__init__&s; dD'8<<<&8I6R  A Acbtfd|dDS)z~ :return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str) c Dg|]\}}t||ddddgjRS)rChunkedCorpusViewr.0fencrs r z-ChunkedCorpusReader.words..AI   Q"!S!Q1Et}EEE   rTconcatabspathsrrs` rwordszChunkedCorpusReader.words:sK      $ gt < <      rcbtfd|dDS)z :return: the given file(s) as a list of sentences or utterances, each encoded as a list of word strings. :rtype: list(list(str)) c Dg|]\}}t||ddddgjRSrrr!s rr%z-ChunkedCorpusReader.sents..Or&rTr'r*s` rsentszChunkedCorpusReader.sentsGK     $ gt < <      rcbtfd|dDS)z :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as lists of word strings. :rtype: list(list(list(str))) c Dg|]\}}t||ddddgjRSr.rr!s rr%z-ChunkedCorpusReader.paras..]r&rTr'r*s` rparaszChunkedCorpusReader.parasUr1rcftfd|dDS)z :return: the given file(s) as a list of tagged words and punctuation symbols, encoded as tuples ``(word,tag)``. :rtype: list(tuple(str,str)) c Jg|]\}}t||ddddgjRdi Sr/r target_tagsetrr"r#r$rrs rr%z4ChunkedCorpusReader.tagged_words..kb   Q"sAq!Q)-FL   rTr'rrrs` `r tagged_wordsz ChunkedCorpusReader.tagged_wordscS     !% gt < <       rcftfd|dDS)z :return: the given file(s) as a list of sentences, each encoded as a list of ``(word,tag)`` tuples. :rtype: list(list(tuple(str,str))) c Jg|]\}}t||ddddgjRdi Sr7rr9s rr%z4ChunkedCorpusReader.tagged_sents..{r:rTr'r;s` `r tagged_sentsz ChunkedCorpusReader.tagged_sentssr=rcftfd|dDS)z :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as lists of ``(word,tag)`` tuples. :rtype: list(list(list(tuple(str,str)))) c Jg|]\}}t||ddddgjRdi Sr7rr9s rr%z4ChunkedCorpusReader.tagged_paras..r:rTr'r;s` `r tagged_parasz ChunkedCorpusReader.tagged_parasr=rcftfd|dDS)av :return: the given file(s) as a list of tagged words and chunks. Words are encoded as ``(word, tag)`` tuples (if the corpus has tags) or word strings (if the corpus has no tags). Chunks are encoded as depth-one trees over ``(word,tag)`` tuples or word strings. :rtype: list(tuple(str,str) and Tree) c Jg|]\}}t||ddddgjRdi Sr7rr9s rr%z5ChunkedCorpusReader.chunked_words..r:rTr'r;s` `r chunked_wordsz!ChunkedCorpusReader.chunked_wordsS     !% gt < <       rcftfd|dDS)a6 :return: the given file(s) as a list of sentences, each encoded as a shallow Tree. The leaves of these trees are encoded as ``(word, tag)`` tuples (if the corpus has tags) or word strings (if the corpus has no tags). :rtype: list(Tree) c Jg|]\}}t||ddddgjRdi Sr7rr9s rr%z5ChunkedCorpusReader.chunked_sents..r:rTr'r;s` `r chunked_sentsz!ChunkedCorpusReader.chunked_sentsrGrcftfd|dDS)ao :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as a shallow Tree. The leaves of these trees are encoded as ``(word, tag)`` tuples (if the corpus has tags) or word strings (if the corpus has no tags). :rtype: list(list(Tree)) c Jg|]\}}t||ddddgjRdi S)r/r8rr9s rr%z5ChunkedCorpusReader.chunked_paras..r:rTr'r;s` `r chunked_parasz!ChunkedCorpusReader.chunked_parasrGrc4dt|DS)Nc,g|]}t|Sr)r"ts rr%z3ChunkedCorpusReader._read_block..sEEE1 AEEEr)read_blankline_block)rstreams r _read_blockzChunkedCorpusReader._read_blocksEE(rs   """"""$$$$EEEEEE%%%%pFpFpFpFpF,pFpFpFfDDDDD.DDDDDr