vllm.distributed.kv_transfer.kv_connector.v1.mooncake.transfer.backend
¶
Classes:
-
BatchTransferResult–Terminal outcome covering
num_ranges, not a count of successful ranges. -
TransferBackend–Moves KV blocks between two workers synchronously.
BatchTransferResult
dataclass
¶
Terminal outcome covering num_ranges, not a count of successful ranges.
Source code in vllm/distributed/kv_transfer/kv_connector/v1/mooncake/transfer/backend.py
TransferBackend
¶
Bases: ABC
Moves KV blocks between two workers synchronously.
Methods:
-
register_regions–Pins local memory for transfers.
-
shutdown–Stops the backend.
-
transfer–Moves every range and returns one terminal event for each.
-
transfer_sync_arrays–Moves equal-length, 1D, contiguous native uint64 columns.
-
unregister_regions–Releases memory pinned by
register_regions.
Attributes:
Source code in vllm/distributed/kv_transfer/kv_connector/v1/mooncake/transfer/backend.py
rpc_port
abstractmethod
property
¶
Port peers use to reach this worker's engine.
register_regions(regions)
abstractmethod
¶
Pins local memory for transfers.
An empty sequence is a no-op. Raises RuntimeError when the engine rejects the memory.
Source code in vllm/distributed/kv_transfer/kv_connector/v1/mooncake/transfer/backend.py
shutdown()
abstractmethod
¶
Stops the backend.
The caller stops submitting before calling this, but work already
queued on its worker threads is not cancelled, so a transfer call may
still be running or start afterwards. An implementation must tolerate
that.
Source code in vllm/distributed/kv_transfer/kv_connector/v1/mooncake/transfer/backend.py
transfer(peer, op, ranges)
abstractmethod
¶
Moves every range and returns one terminal event for each.
The result lines up with ranges by position, and an empty request
returns an empty list. Blocks until every range reached a terminal
state. Called from worker threads, so an implementation must accept
concurrent calls.
Source code in vllm/distributed/kv_transfer/kv_connector/v1/mooncake/transfer/backend.py
transfer_sync_arrays(peer, op, local_addrs, remote_addrs, lengths)
abstractmethod
¶
Moves equal-length, 1D, contiguous native uint64 columns.
Blocks until all ranges are terminal, holding the input buffers for the duration of the call. Callers must not modify them meanwhile. Concurrent calls keep their own inputs and results. Empty input completes without engine work. COMPLETED means all ranges succeeded; FAILED does not mean all failed. Use the first failure in input order when available.
Returns one batch summary without constructing per-range requests, events or status arrays. Invalid inputs and engine exceptions propagate.
Source code in vllm/distributed/kv_transfer/kv_connector/v1/mooncake/transfer/backend.py
unregister_regions(regions)
abstractmethod
¶
Releases memory pinned by register_regions.
An empty sequence is a no-op.