text

Text document display class.

Text

 Text (tokens:numpy.ndarray, has_spaces:numpy.ndarray, metadata:dict={},
       doc_df:polars.dataframe.frame.DataFrame=None)

Class to represent text documents

	Type	Default	Details
tokens	ndarray		list of token strs
has_spaces	ndarray		whether token strs followed by space
metadata	dict	{}	metadata for doc as a dict
doc_df	DataFrame	None	if provided can be used for enhanced display (e.g. keyword highlighting)

The Text class is not intended to be used directly. Functionality is accessible via the Corpus.text method, which provides the necessary inputs to instantiate the class. The examples below illustrate how Text objects can be created and used for a Corpus.

source

Text.as_string

 Text.as_string (max_tokens:int|None=None,
                 highlighted_token_range:tuple|None=None)

Return the text as a string

	Type	Default	Details
max_tokens	int \| None	None	maximum length of text to display in tokens, if None, display all
highlighted_token_range	tuple \| None	None	range of tokens to highlight, note: these token ids are positions within the corpus, not the text itself

source

Text.as_tokens

 Text.as_tokens ()

Return the text as a tokens

source

Text.str

 Text.__str__ ()

Return str(self).

source

Text.tokens_count

 Text.tokens_count ()

source

Text.display_metadata

 Text.display_metadata ()

Output the metadata for a text

source

Text.get_metadata

 Text.get_metadata ()

Output the metadata for a text

source

Text.display

 Text.display (show_metadata:bool=True, max_tokens:int|None=None,
               output_html:bool=True, textwrap_width:int|None=None,
               textwrap_args:dict|None=None, reflow_paragraphs:bool=False,
               paragraph_delimiter_regex:str='(\\s*\\n\\s*){1,}\\n')

Output a text

	Type	Default	Details
show_metadata	bool	True	whether to display Metadata for the text
max_tokens	int \| None	None	maximum length of text to display in tokens, if None, display all
output_html	bool	True	whether to display text with HTML formatting
textwrap_width	int \| None	None	maximum length of text to display in characters, if None, no wrapping
textwrap_args	dict \| None	None	additional args to pass to textwrap.fill
reflow_paragraphs	bool	False	whether to reflow paragraphs individually before text wrapping is applied
paragraph_delimiter_regex	str	(){1,}	regex to split paragraphs for reflow_paragraphs (default looks for whitespace ending with a newline that contains at least one other newline)

gardenparty.text(12).display(max_tokens = 200)

Metadata

Attribute	Value
document_id	12
file	the-singing-lesson.txt

With despair—cold, sharp despair—buried deep in her heart like a wicked knife, Miss Meadows, in cap and gown and carrying a little baton, trod the cold corridors that led to the music hall. Girls of all ages, rosy from the air, and bubbling over with that gleeful excitement that comes from running to school on a fine autumn morning, hurried, skipped, fluttered by; from the hollow class-rooms came a quick drumming of voices; a bell rang; a voice like a bird cried, “Muriel.” And then there came from the staircase a tremendous knock-knock-knocking. Some one had dropped her dumbbells. The Science Mistress stopped Miss Meadows. “Good mor-ning,” she cried, in her sweet, affected drawl. “Isn’t it cold? It might be win-ter.” Miss Meadows, hugging the knife, stared in hatred at the Science Mistress. Everything about her was sweet,… [200 of 2985 tokens]

gardenparty.text(12).display(show_metadata = False, max_tokens = 200, textwrap_width = 100, reflow_paragraphs = True)

With despair—cold, sharp despair—buried deep in her heart like a wicked knife, Miss Meadows, in cap and gown and carrying a little baton, trod the cold corridors that led to the music hall. Girls of all ages, rosy from the air, and bubbling over with that gleeful excitement that comes from running to school on a fine autumn morning, hurried, skipped, fluttered by; from the hollow class- rooms came a quick drumming of voices; a bell rang; a voice like a bird cried, “Muriel.” And then there came from the staircase a tremendous knock-knock-knocking. Some one had dropped her dumbbells. The Science Mistress stopped Miss Meadows. “Good mor-ning,” she cried, in her sweet, affected drawl. “Isn’t it cold? It might be win-ter.” Miss Meadows, hugging the knife, stared in hatred at the Science Mistress. Everything about her was sweet,… [200 of 2985 tokens]

gardenparty.text(12).as_string(max_tokens = 50)

'With despair—cold, sharp despair—buried deep in her heart like a wicked\r\nknife, Miss Meadows, in cap and gown and carrying a little baton, trod\r\nthe cold corridors that led to the music hall. Girls of all ages, rosy'