vllm.model_executor.model_loader.weight_cache.weight_seed
¶
Daemon-to-daemon weight shard seeding (movers, the seedable, the peer loader).
Modules:
-
core–Weight movers: copy whole storages so views keep their sharing and footprint.
-
loader–Fill a daemon's model with a source replica's post-processed weights.
-
mooncake–The Mooncake weight mover: pull storages through the TransferEngine.
-
peer_ipc–The
peer_ipcweight mover: copy storages through CUDA IPC on one host. -
seed–The weights seedable: a daemon both serves and fetches its shard.
-
utils–Helpers for weight shard seeding (manifest format, fingerprint, launcher logic).
Classes:
-
WeightMover–One mover that copies weight storages from a source daemon.
-
WeightShardSeed–This daemon's weight shard: served to and fetched from peer daemons.
Functions:
-
build_manifest–Describe
tensorsas views over their unique storages. -
get_mover–The mover for a backend;
device_indexpins the peer_ipc device. -
rebuild_tensors–Recreate every manifest tensor on the mirror's copies; bounds-checked first.
WeightMover
¶
Bases: ABC
One mover that copies weight storages from a source daemon.
Methods:
-
copy_into–Copy every source storage into its local byte buffer.
-
fill–Copy the source's storages and rebuild its tensors on them.
-
prepare_seed–Publish how a mirror can read each storage, in manifest order.
-
release–Drop source-side state for the weights the daemon is releasing.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/core.py
copy_into(storages, seed, buffers)
abstractmethod
¶
Copy every source storage into its local byte buffer.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/core.py
fill(manifest, seed, device)
¶
Copy the source's storages and rebuild its tensors on them.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/core.py
prepare_seed(storages)
abstractmethod
¶
Publish how a mirror can read each storage, in manifest order.
WeightShardSeed
¶
Bases: Seedable
This daemon's weight shard: served to and fetched from peer daemons.
Methods:
-
release–Wait for copies, then drop source state without blocking their acks.
-
request_params–Ask this daemon's source for the shard, over the fitting mover.
-
resolve–Describe this rank so a peer daemon can mirror it.
-
validate_payload–Structural checks plus the shard fingerprint against this daemon.
-
validate_remote–Refuse node-local movers on the remote control plane.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/seed.py
38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 | |
release()
¶
Wait for copies, then drop source state without blocking their acks.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/seed.py
request_params(ctx)
¶
Ask this daemon's source for the shard, over the fitting mover.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/seed.py
resolve(request)
¶
Describe this rank so a peer daemon can mirror it.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/seed.py
validate_payload(payload)
¶
Structural checks plus the shard fingerprint against this daemon.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/seed.py
validate_remote(request)
¶
Refuse node-local movers on the remote control plane.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/seed.py
build_manifest(tensors, kinds)
¶
Describe tensors as views over their unique storages.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/utils.py
get_mover(name, *, device_index=None)
¶
The mover for a backend; device_index pins the peer_ipc device.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/core.py
rebuild_tensors(manifest, buffers)
¶
Recreate every manifest tensor on the mirror's copies; bounds-checked first.