U <c+@sddlZddlZddlmZddlZGdddeZedddZdd Z d d Z d d Z dZ dddddZ ddZddZdS)N)Enumc@seZdZdZdZdZdZdS)_SnapshotStateaU These are the snapshotting-related states that IterDataPipes can be in. `NotStarted` - allows you to restore a snapshot and create an iterator with reset `Restored` - cannot restore again, allows you to create an iterator without resetting the DataPipe `Iterating` - can restore, will reset if you create a new iterator rN)__name__ __module__ __qualname____doc__Z NotStartedZRestored Iteratingr r M/tmp/pip-unpacked-wheel-gikjz4vx/torch/utils/data/datapipes/_hook_iterator.pyrsr)returncCst|r|jSt|SdS)zn Simplify the display strings of objects for the purpose of rendering within DataPipe error messages. N)inspect isfunctionrrepr)objr r r _simplify_obj_names rcCsvt|j}t}|jD]\}}||qg}t|D]"\}}||kr>||t |fq>d dd|DS)zA Generate a string for the input arguments of an object. z, cSsg|]\}}|d|qS)=r ).0namevaluer r r *sz/_generate_input_args_string..) r signature __class__set parametersitemsadd getmembersappendrjoin)rrZinput_param_names param_name_resultrr r r _generate_input_args_strings  r$cCs|jjdt|dS)N())rrr$datapiper r r _generate_iterdatapipe_msg-sr)cCsdt|dS)NziThis iterator has been invalidated because another iterator has been created from the same IterDataPipe: zt This may be caused multiple references to the same IterDataPipe. We recommend using `.fork()` if that is necessary.)r)r'r r r _gen_invalid_iterdatapipe_msg1sr*z For feedback regarding this single iterator per IterDataPipe constraint, feel free to comment on this issue: https://github.com/pytorch/data/issues/45.FcCs|r2|jdk r|jdkrd}tt||tnht|dr|jdkrt|drv||s~tdt|jdtqtd n|j|krtt|tdS) z Given an instance of a DataPipe and an iterator ID, check if the IDs match, and if not, raises an exception. In the case of ChildDataPipe, the ID gets compared to the one stored in `main_datapipe` as well. NrzR Note that this exception is raised inside your IterDataPipe's a `__next__` method_is_child_datapipeT_check_valid_iterator_idznThis iterator has been invalidated, because a new iterator has been created from one of the ChildDataPipes of .z:ChildDataPipe must have method `_check_valid_iterator_id`.) _valid_iterator_id RuntimeErrorr* _feedback_msghasattrr+r,r)Z main_datapipe)r( iterator_idZnext_method_existsZ extra_msgr r r _check_iterator_valid<s    r3cCs`t|dr2|jdkr2t|dr(|qZtdn(|jdkrDd|_n|jd7_||jS)zQ Given a DataPipe, updates its valid iterator ID and reset the DataPipe. r+T$_set_main_datapipe_valid_iterator_idzFChildDataPipe must have method `_set_main_datapipe_valid_iterator_id`.Nrr)r1r+r4r/r.resetr'r r r _set_datapipe_valid_iterator_idTs    r6csfddGfddddtrRtfdd}|d<nRdkrdtfd d }|d<tfd d }|d<d S)z Hook that is applied to all `__iter__` of metaclass `_DataPipeMeta`. This is done for the purpose of profiling and checking if an iterator is still valid. cstjjSN)torchautogradZprofilerZrecord_functionr ) profile_namer r profiler_record_fn_contextksz1hook_iterator..profiler_record_fn_contextcs<eZdZdZddZddZddZfdd Zd d Zd S) z(hook_iterator..IteratorDecoratorz Wrap the iterator and modifying its `__next__` method. This decorator is applied to DataPipes of which `__iter__` method is NOT a generator function. Those `__iter__` method commonly returns `self` but not necessarily. cSs4||_||_||_tj|_|j|jko,||_dSr7)iterator source_dpr2r8r9_profiler_enabledself_and_has_next_method)selfr<r=r2Zhas_next_methodr r r __init__ts  z1hook_iterator..IteratorDecorator.__init__cSs|Sr7r r@r r r __iter__|sz1hook_iterator..IteratorDecorator.__iter__cSs2t|j|jt|j}|js.|jjd7_|S)z Return next with logic related to iterator validity, profiler, and incrementation of samples yielded. r)r3r=r2nextr<r?_number_of_samples_yielded)r@r#r r r _get_nexts  z2hook_iterator..IteratorDecorator._get_nextc s6|jr*|W5QRSQRXn|SdSr7)r>rFrBr;r r __next__sz1hook_iterator..IteratorDecorator.__next__cSs t|j|Sr7)getattrr<)r@rr r r __getattr__sz4hook_iterator..IteratorDecorator.__getattr__N) rrrr rArCrFrHrJr rGr r IteratorDecoratorns  rKrCc ?s||}|d}|jrT|j}d|_tj|_zt|VWq,tk rPYdSXq,t|}tj }z|r| d}W5QRXn | d}|j d7_ |V}|rԈt ||| |}W5QRXqt ||| |}qWntk r} z WYdSd} ~ XYnt k r} z|d}d} d} t| jdr| d|jjdt|d} t| jdkst| jdtsd | f| _nF| | jdkr| | jdkr| jdd | f| jdd| _W5d} ~ XYnXdS) Nrrzthrown by __iter__ ofz+single iterator per IterDataPipe constraint__len__ r%r&z This exception is )_fast_forward_iteratorrr _snapshot_staterD StopIterationr6r8r9r>sendrEr3 Exceptionr1argsrrr$len isinstancestr) rSkwargsgenr(itr2r>responserequestemsgZsingle_iterator_msgZfull_msg)funcr;r r wrap_generatorsL      " &z%hook_iterator..wrap_generatorrHc sLtjr(||}W5QRXn ||}|d}|jd7_|S)Nrr)r8r9r>rE)rSrWr#r() next_funcr;r r wrap_nexts  z hook_iterator..wrap_nextcsJ||}|d}tj|_|jr0|j}d|_|St|}|||dkS)NrrH)rr rOrNr6)rSrWZiter_retr(r2)rKr^ namespacer r wrap_iters z hook_iterator..wrap_iterN)risgeneratorfunction functoolswraps)rbr:r_rarcr )rKr^rbr`r:r;r hook_iteratorfs ' 0   rg)F)rreenumrZtorch.autogradr8rrVrr$r)r*r0r3r6rgr r r r s