peft.quantize

peft.quantize#

gemma.peft.quantize(
params: gemma.peft._quantization_utils.PyTree,
*,
method: gemma.peft._quantization_utils.QuantizationMethod | str,
checkpoint_kernel_key: str = 'w',
in_place_keys: bool = False,
) → gemma.peft._quantization_utils.PyTree[source]

Quantizes the given params.

In ths API, we convert the elements of params in order to actually get quantized values. It is currently limited to INT$ per-channel weight quantization.

Parameters:

  • params – The params to quantize.

  • method – The quantization method to use.

  • checkpoint_kernel_key – The key of the kernel in the checkpoint (in pre-trained checkpoitns that is ‘w’).

  • in_place_keys – Whether to quantize the keys in place.

Returns:

The quantized params.