vllm.models.mistral.mistral_large4.amd.specialized_decode.mono.geometry_config
¶
ML4's own geometry, and the widths its kernel is built for.
The vocabulary these are expressed in -- LayerConfig, MoeMode, the weight
and cache layouts -- is model-free and lives in vllm.models.mistral.mistral_large4.amd.specialized_decode.mono.common.weights.config;
what is ML4's alone is here. The shared names are re-exported, so a kernel
source taking both from one place need not care which is which.
Functions:
-
ml4_kernel_samples–Choose an LDS-safe request-aligned launch width.
-
ml4_tp_config–Return the one ML4 shard geometry for
tp_size. -
validate_shard–Validate one model profile before allocating GPU buffers.
ml4_kernel_samples(samples, query_length)
¶
Choose an LDS-safe request-aligned launch width.
Source code in vllm/models/mistral/mistral_large4/amd/specialized_decode/mono/geometry_config.py
ml4_tp_config(tp_size)
¶
Return the one ML4 shard geometry for tp_size.
Source code in vllm/models/mistral/mistral_large4/amd/specialized_decode/mono/geometry_config.py
validate_shard(samples, heads, rank, npes, sparse_attention_topk, model_config=ML4_CONFIG, supported_samples=SUPPORTED_SAMPLES, expected_heads=None)
¶
Validate one model profile before allocating GPU buffers.