vllm.models.qwen3_5.amd.model
¶
Inference-only Qwen3.5 Series, AMD ROCm implementation.
Mirrors ../nvidia/model.py (eager + breakable CUDA graphs) with the ROCm
counterparts of its fusions: Triton Gemma RMSNorm instead of FlashInfer's, the
row-parallel all-reduce deferred into AITER's fused all-reduce + Gemma RMSNorm,
and the fused QK-norm + RoPE + gate kernel. FlashInfer's FP8 / NVFP4 norm-quant
fusions have no ROCm counterpart here, so the norms emit unquantized
activations.
Classes:
-
Qwen3_5ForConditionalGeneration– -
Qwen3_5RMSNorm–Gemma-style RMS normalization backed by Triton kernels.
Qwen3_5ForConditionalGeneration
¶
Bases: Qwen3VLForConditionalGeneration, IsHybrid
Methods:
-
forward–Run forward pass for Qwen3.5.
Source code in vllm/models/qwen3_5/amd/model.py
718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 | |
forward(input_ids, positions, intermediate_tensors=None, inputs_embeds=None, **kwargs)
¶
Run forward pass for Qwen3.5.
With mrope enabled (default for Qwen3VL-class models), positions
has shape (3, seq_len), otherwise (seq_len,).
Source code in vllm/models/qwen3_5/amd/model.py
Qwen3_5RMSNorm
¶
Bases: Module
Gemma-style RMS normalization backed by Triton kernels.
When residual is given, the fused add + norm returns the updated
(x, residual) pair.
reduce_inputs: in forward_reduced, an x that comes with a
residual is the un-reduced TP partial of a row-parallel linear, and the
norm absorbs its all-reduce. An x without a residual (first decoder
layer, MTP layer) is always complete.
Methods:
-
forward_reduced–forwardwith the preceding TP all-reduce fused in when
Source code in vllm/models/qwen3_5/amd/model.py
forward_reduced(x, residual)
¶
forward with the preceding TP all-reduce fused in when
reduce_inputs. Returns the normed activation and the residual (x
when residual is None).