U Jc+ @sPddlmZddlmZddlZddlmZddlmZeGddde Z dS)) dataclass)ListN) ShardMetadata)_remote_devicec@sPeZdZUdZddgZejed<eed<ddZ e eje e e dddZ d S) Shardae Container which holds the data for a shard as a Tensor and also the associated metadata for that shard. Args: tensor(torch.Tensor): Local tensor for the shard. metadata(:class `torch.distributed._shard.sharded_tensor.ShardMetadata`): The metadata for the shard, including offsets, lengths and device placement. tensormetadatacCszt|j|jjkr:tdt|jd|jjd|jj}|dk rv||jjkrvtd|jjd|dS)NzWShard tensor size does not match with metadata.shard_lengths! Found shard tensor size: z, metadata.shard_lengths: z, zhLocal shard tensor device does not match with local Shard's placement! Found local shard tensor device: z), local shard metadata placement device: )listrsizer shard_sizes ValueError placementdevice)selfZplacement_devicerQ/tmp/pip-unpacked-wheel-gikjz4vx/torch/distributed/_shard/sharded_tensor/shard.py __post_init__szShard.__post_init__)r shard_offsetsrankcCs>t|}td|dt|j}t|||d}t||S)ae Creates a Shard of a ShardedTensor from a local torch.Tensor, shard_offsets and rank. Args: tensor(torch.Tensor): Local tensor for the shard. shard_offsets(List[int]): List of integers specify the offset of the shard on each dimension. rank(int): Specify the rank for the shard. zrank:/)rr r )r r rstrrrr)clsrrrr r Z shard_metarrrfrom_tensor_and_offsets(s zShard.from_tensor_and_offsetsN)__name__ __module__ __qualname____doc__ __slots__torchZTensor__annotations__rr classmethodrintrrrrrr s   r) Z dataclassesrtypingrrZ!torch.distributed._shard.metadatarZtorch.distributed.remote_devicerobjectrrrrrs