vllm.model_executor.layers.quantization.utils.gptq_utils
¶
Functions:
-
is_moe_layer_gptq_quantized–Whether the routed experts at
prefixare quantized in the checkpoint.
is_moe_layer_gptq_quantized(prefix, quantized_layers)
¶
Whether the routed experts at prefix are quantized in the checkpoint.
prefix is the absolute name of the fused expert layer
(...mlp.experts), while modules_in_block_to_quantize names the
individual expert projections (...mlp.experts.0.down_proj) or the fused
expert module itself, using either absolute or block-relative names. Match
on the experts portion of the name so that a layer whose experts are
stored unquantized -- e.g. the MTP layer of a partially quantized
checkpoint -- is not given a quantized method.
Configs that enumerate no expert module at all say nothing about the experts, so the layer is assumed to be quantized.