vllm.models.minimax_m3.nvidia.specialized_decode.runtime.warmup
¶
Tuning and compiling the specialized decode's stages.
The model state calls warm_up once the executor is built, before memory
profiling and CUDA graph capture, so no kernel compiles or tunes inside a
capture. Each stage implementation warms up for the row counts it runs; the
stock stages need nothing (the eager stock forward before each capture
compiles their kernels). A stage whose warm-up is collective (FlashInfer's
tuner reduces its timings over the TP group) runs the same calls in the same
order on every rank, which this order guarantees.
Functions:
-
step_rows–Every row count an executor step can have: the FULL decode CUDA graph
-
warm_up–Warm every stage implementation up for the step rows it supports.
step_rows(decode, vllm_config)
¶
Every row count an executor step can have: the FULL decode CUDA graph
capture sizes up to the executor's max_rows.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/runtime/warmup.py
warm_up(decode, vllm_config)
¶
Warm every stage implementation up for the step rows it supports.