U <c@sddlmZddlmZddlmZmZmZddgZedddZ ed Gd ddeZ ed Gd ddeee d fZ dS))functional_datapipe) MapDataPipe)SizedTupleTypeVarConcaterMapDataPipeZipperMapDataPipeT_coT) covariantconcatc@sPeZdZUdZeeed<eed<edddZe ddd Z edd d Z d S) ra Concatenate multiple Map DataPipes (functional name: ``concat``). The new index of is the cumulative sum of source DataPipes. For example, if there are 2 source DataPipes both with length 5, index 0 to 4 of the resulting `ConcatMapDataPipe` would refer to elements of the first DataPipe, and 5 to 9 would refer to elements of the second DataPipe. Args: datapipes: Map DataPipes being concatenated Example: >>> # xdoctest: +SKIP >>> from torchdata.datapipes.map import SequenceWrapper >>> dp1 = SequenceWrapper(range(3)) >>> dp2 = SequenceWrapper(range(3)) >>> concat_dp = dp1.concat(dp2) >>> list(concat_dp) [0, 1, 2, 0, 1, 2] datapipeslength)r cGsXt|dkrtdtdd|Ds.tdtdd|DsHtd||_d|_dS) Nr/Expected at least one DataPipe, but got nothingcss|]}t|tVqdSN isinstancer.0dprL/tmp/pip-unpacked-wheel-gikjz4vx/torch/utils/data/datapipes/map/combining.py &sz/ConcaterMapDataPipe.__init__..'Expected all inputs to be `MapDataPipe`css|]}t|tVqdSrrrrrrrr(s!Expected all inputs to be `Sized`len ValueErrorall TypeErrorr r selfr rrr__init__#s zConcaterMapDataPipe.__init__returncCsNd}|jD]0}||t|kr.|||S|t|7}q td|dS)NrzIndex {} is out of range.)r r IndexErrorformat)r"indexoffsetrrrr __getitem__-s  zConcaterMapDataPipe.__getitem__cCs&|jdkr tdd|jD|_|jS)Nrcss|]}t|VqdSrrrrrrr8sz.ConcaterMapDataPipe.__len__..)r sumr r"rrr__len__6s zConcaterMapDataPipe.__len__N) __name__ __module__ __qualname____doc__rr__annotations__intr#r r*r.rrrrr s    zipc@sfeZdZUdZeeedfed<eed<eeddddZ eedfd d d Z ed d d Z dS)ra$ Aggregates elements into a tuple from each of the input DataPipes (functional name: ``zip``). This MataPipe is out of bound as soon as the shortest input DataPipe is exhausted. Args: *datapipes: Map DataPipes being aggregated Example: >>> # xdoctest: +SKIP >>> from torchdata.datapipes.map import SequenceWrapper >>> dp1 = SequenceWrapper(range(3)) >>> dp2 = SequenceWrapper(range(10, 13)) >>> zip_dp = dp1.zip(dp2) >>> list(zip_dp) [(0, 10), (1, 11), (2, 12)] .r r N)r r%cGsXt|dkrtdtdd|Ds.tdtdd|DsHtd||_d|_dS) Nrrcss|]}t|tVqdSrrrrrrrTsz-ZipperMapDataPipe.__init__..rcss|]}t|tVqdSrrrrrrrVsrrrr!rrrr#Qs zZipperMapDataPipe.__init__r$c CsVg}|jD]B}z|||Wq tk rJtd|d|dYq Xq t|S)NzIndex z3 is out of range for one of the input MapDataPipes .)r appendr&tuple)r"r(resrrrrr*[s zZipperMapDataPipe.__getitem__cCs&|jdkr tdd|jD|_|jS)Nrcss|]}t|VqdSrr+rrrrrfsz,ZipperMapDataPipe.__len__..)r minr r-rrrr.ds zZipperMapDataPipe.__len__) r/r0r1r2rrr r3r4r#r*r.rrrrr<s   .N) Z%torch.utils.data.datapipes._decoratorrZ#torch.utils.data.datapipes.datapipertypingrrr__all__r rrrrrrs   1