vllm.model_executor.model_loader.weight_cache.weight_seed.utils
¶
Helpers for weight shard seeding (manifest format, fingerprint, launcher logic).
Classes:
-
WeightsSeedPayload–What a source daemon's
weightsresolve sends to a peer loader.
Functions:
-
allocate_storage–A fresh byte buffer for one manifest storage, on the source's device type.
-
build_manifest–Describe
tensorsas views over their unique storages. -
mirror_device_indices–A same-host mirror's GPU indices: the free ones past the source's.
-
peer_cache_config–Switch one daemon's config to the
peer_cacheload format. -
rebuild_tensors–Recreate every manifest tensor on the mirror's copies; bounds-checked first.
-
seed_address–Resolve one daemon's rank-matched source socket or remote endpoint.
-
seed_mismatches–Fields on which a seed source's shard differs (model-path fallback).
-
storage_bytes–A flat byte tensor covering the whole storage.
-
validate_weights_payload–Structural checks plus the shard fingerprint against the expected key.
WeightsSeedPayload
¶
Bases: TypedDict
What a source daemon's weights resolve sends to a peer loader.
Attributes:
-
cache_config(dict[str, Any]) –asdict(WeightCacheKey)of the source's shard. -
manifest(dict[str, Any]) –Storage manifest, as
build_manifestproduces. -
seed(dict[str, Any]) –Mover-specific transfer seed;
seed["backend"]picks the mover.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/utils.py
_view_extent(view, itemsize)
¶
Bytes from the start of its storage that a strided view reaches.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/utils.py
allocate_storage(storage, device)
¶
A fresh byte buffer for one manifest storage, on the source's device type.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/utils.py
build_manifest(tensors, kinds)
¶
Describe tensors as views over their unique storages.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/utils.py
mirror_device_indices(addresses, num_local)
¶
A same-host mirror's GPU indices: the free ones past the source's.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/utils.py
peer_cache_config(config, *, is_draft)
¶
Switch one daemon's config to the peer_cache load format.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/utils.py
rebuild_tensors(manifest, buffers)
¶
Recreate every manifest tensor on the mirror's copies; bounds-checked first.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/utils.py
seed_address(addresses, local_rank, global_rank, port_offset=0, *, node_rank=0, is_draft=False)
¶
Resolve one daemon's rank-matched source socket or remote endpoint.
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/utils.py
seed_mismatches(local, source, model, source_model)
¶
Fields on which a seed source's shard differs (model-path fallback).
Source code in vllm/model_executor/model_loader/weight_cache/weight_seed/utils.py
storage_bytes(storage)
¶
A flat byte tensor covering the whole storage.
validate_weights_payload(payload, expected_key, model)
¶
Structural checks plus the shard fingerprint against the expected key.