gm.text

[[Source]]

Text processing utilities.

Symbols

Class

gm.text.ChatSampler	Chat sampler.
gm.text.Gemma2Tokenizer	Tokenizer for Gemma 2.
gm.text.Gemma3Tokenizer	Tokenizer for Gemma 3.
gm.text.Gemma3nTokenizer	Tokenizer for Gemma3n.
gm.text.Gemma4Sampler	Stateless sampler for Gemma4 with variable-aspect-ratio image support.
gm.text.Gemma4Tokenizer	Tokenizer for Gemma 4.
gm.text.Greedy	Greedy sampling.
gm.text.RandomSampling	Simple random sampling.
gm.text.Sampler	Sampler.
gm.text.SamplingMethod	Base class for sampling methods.
gm.text.SpecialTokens	Special tokens ids.
gm.text.Tokenizer	Base class for tokenizers.
gm.text.ToolSampler	Sampler with tool support.
gm.text.TopPSampling	Top-p (Nucleus) Sampling.
gm.text.TopkSampling	Top-k sampling.