U <║c^╕у @sДddlZddlZddlmZmZmZmZddlZddlm Z ddlm mZddlm Z ddlmZddlmZddlmZmZddlmZddlmZdd lmZmZmZmZdd lmZddlm Z!dd ДZ"ddДZ#ddДZ$ej%j&ee e dЬddДГZ'ej%j&ee(dЬddДГZ)ej%j&eee(ddЬddДГZ*e(e(dЬddДZ+GddДdГZ,ej%j&dd ДГZ-ej%j&d!d"ДГZ.ej%j&d#d$ДГZ/ej%j&e0d%Ьd&d'ДГZ1ej%j&e0e0e d(Ьd)d*ДГZ2ej%j&d+d,ДГZ3ej%j&ee e d-Ьd.d/ДГZ4ej%j&ee ee d-Ьd0d1ДГZ5ej%j&d2d3ДГZ6ej%j&d4d5ДГZ7ej%j&e e0e d6Ьd7d8ДГZ8ej%j&e0e0e d(Ьd9d:ДГZ9ej%j&e e d;Ьdd?Дd?ej%j;ГZej%j&ee e d-ЬdBdCДГZ?GdDdEДdEГZ@GdFdGДdGГZAej%j&GdHdIДdIГГZBej%jCGdJdKДdKejDjEГГZFGdLdMДdMej%j;ГZGdNdOДZHdPdQДZIej%j&eeBe(dЬdRdSДГZJej%j&eeFe dЬdTdUДГZKGdVdWДdWГZLdXdYДZMej%j&eаNdZdZgбeаNd[d[gбfd\d]ДГZOej%j&eаNd^d^gбd_d^fe e0e(e e0e(d`ЬdadbДГZPej%j&dcddДГZQej%j&e0ee e fee0e fdeЬdfdgДГZRej%j&e0ee e fee0e fdeЬdhdiДГZSej%j&e0ee e fee0e fdeЬdjdkДГZTGdldmДdmГZUej%j=e(eFdnЬdodpДГZVej%j&e(eFdnЬdqdrДГZWej%j&eeFe e dsЬdtduДГZXej%j&ee eYdЬdvdwДГZZej%j&ee e0ddxЬdydzДГZ[ej%j&e e e d{Ьd|d}ДГZ\ej]j^ej%j&e0e e ee d~ЬddАДГГZ_ej]j^ej%j&e dЬdБdВДГГZ`dГdДДZaGdЕdЖДdЖe,e@eLeUeAeГZbdS)ЗщN)┌Dict┌List┌Tuple┌Any)┌Tensor)┌record_function)┌RRef)┌RPCExecMode┌_build_rpc_profiling_key)┌Future)┌TemporaryFileName)┌ dist_init┌get_function_event┌ initialize_pg┌worker_name)┌RpcAgentTestFixture)┌profilecCst|аб|ГSйN)┌ isinstance┌local_value)┌rrefZcls_to_checkйr·X/tmp/pip-unpacked-wheel-gikjz4vx/torch/testing/_internal/distributed/rpc/jit/rpc_test.py┌rref_isinstancesrcCstа|бdSr)┌time┌sleep)┌trrrrsrcCstj|tjtаddбdfdНSйNщщй┌args)┌rpc┌remote┌torch┌add┌ones)┌dstrrr┌rpc_return_rref!sr(йr┌returncCs|абSr)rйrrrr┌rref_local_value%sr,йr*cCsdddg}|S)Nrrщr)Zglobal_listrrr┌list_create*s r/cCs0|абаdб|абаdб|аdбаdбdS)Nщщg@щ)r┌append┌to_herer+rrr┌rref_list_mutate0sr5)┌valuer*cCs|Srrйr6rrr┌return_value7sr8c@s\eZdZeddДГZeddДГZeddДГZddДZed d ДГZeddДГZ ed dДГZ dS)┌RRefAPITestcCsPt|jd|jГ}t|Г}tjjttt dЬddДГ}||Г}|а |dбdS)Nrй┌rref_varr*cSs|абSr)Zis_ownerйr;rrr┌rref_tensor_is_ownerAsz.rref_tensor_is_ownerF)r┌rank┌ world_sizer(r$┌jit┌scriptrr┌bool┌assertEqual)┌self┌dst_worker_namer;r=┌resrrr┌test_rref_is_owner<szRRefAPITest.test_rref_is_ownerc Cs~|jdkrdSt|jd|jГ}t|Г}|аtdбПt|ГW5QRXtа|t|fб}}|а |t аt аddбdббdS)Nrrz3Can't call RRef.local_value on a non-owner RRefr) r>rr?r(┌assertRaisesRegex┌RuntimeErrorr,r"┌rpc_syncrCr$r%r&йrDrEr┌retrrr┌test_rref_local_valueHs z!RRefAPITest.test_rref_local_valuecCs@|jdkrdSt|jГ}tа|tdiб}t|Г}|а|dбdS)Nr)r1r1)r>rr"r#r8r,rCrKrrr┌test_local_rref_local_valueXs z'RRefAPITest.test_local_rref_local_valuecCs2|jd|j}tjt|ГtjtаddбdfdНSr)r>r?r"r#rr$r%┌zeros)rDZ owner_rankrrr┌_create_rrefcs zRRefAPITest._create_rrefcCs>|jd|j}|аб}tjt|Гt|fdН}|а|dбdSйNrr T)r>r?rPr"rJr┌script_check_rref_confirmedrC)rD┌dst_rankrrLrrr┌test_user_rrefs_confirmedis z%RRefAPITest.test_user_rrefs_confirmedcCsB|jd|j}|аб}tjt|Гt|fdН}|а|абdбdSrQ) r>r?rPr"r#rrRrCr4)rDrSrZret_rrefrrr┌ test_user_rrefs_confirmed_remoters z,RRefAPITest.test_user_rrefs_confirmed_remotec CsRt|jd|jГ}tа|tб}tj|t|fdН|а|а бddddddgбdS)Nrr rr.r0r1r2) rr>r?r"r#r/rJr5rCr4)rDr'Z list_rrefrrr┌test_rref_list_mutate{sz!RRefAPITest.test_rref_list_mutateN)┌__name__┌ __module__┌__qualname__r rGrMrNrPrTrUrVrrrrr9;s r9cCsdSйNrrrrrr┌no_argДsr[cCs|dSйNrrr7rrr┌one_argЙsr]cCstа|tаdббSr\)r$r%r&)┌xrrr┌script_add_onesНsr_)┌blockc Cs2t|ГП tа|tаdббW5QRгSQRXdSr\)rr$r%r&)r^r`rrr┌$script_add_ones_with_record_functionСs ra)rEr`r*c Cs`tаdб}t|ГПD}tа|t|fб}tа|б}tа|t|fб}|аб|аб|}W5QRX|Sr\)r$r&rr"┌ rpc_asyncr_Z zeros_like┌wait)rEr`r┌rfZfut1ZzeroZfut2rFrrr┌#record_function_on_caller_rpc_asyncЧs recCstjаt|б}tjа|б}|Sr)r$r@┌_forkr_┌_wait)┌tensor┌futr^rrr┌script_fork_wait_udfеsrjr:cCs|абSr)r4r<rrr┌rref_to_hereмsrkcCs|Srrr<rrr┌return_rref▒srlcCs|абdkrtdГВ|dS)Nr·Expected errorr)Znumel┌ ValueErrorr7rrr┌script_raise_func╢srocCstjаt|б}tjа|б}|Sr)r$r@rfrorg)Zinvaluerir6rrr┌script_fork_wait_throw╜srp)┌handlerEr*cCs2tа|ttаdбfб}tjjа||б|аб}|Sr\) r"rbr]r$rh┌ops┌profiler┌_call_end_callbacks_on_jit_futrc)rqrErirLrrr┌call_rpc_with_profiling─srucCs tа|ttаdб|fб}|абSr\)r"rbrar$rhrc)rEr`rirrr┌)call_rpc_torchscript_with_record_function╬srv)rqr*cCs0tjаttаdбб}tjjа||б|аб}|Sr\) r$r@rfr]rhrrrsrtrc)rqrirLrrr┌call_fork_with_profiling╘srwcs2eZdZЗfddДZejjedЬddДГZЗZ S)┌MyScriptModuleWithRRefscs2tГабg|_tdГD]}|jаt|ГбqdS)Nr0)┌super┌__init__┌rrefs┌ranger3r()rDZ dst_worker┌_й┌ __class__rrrzрs z MyScriptModuleWithRRefs.__init__r-cCs(tаddб}|jD]}||аб7}q|SйNr)r$r&r{r4)rDZ res_tensorrrrr┌forwardцs zMyScriptModuleWithRRefs.forward) rWrXrYrzr$r@┌ script_methodrrБ┌ __classcell__rrr~rrx▀srxcCs|Srrr<rrr┌rref_python_annotationяsrДcCst|ГабSr)rДr4r<rrr┌rref_script_annotationЇsrЕc@s0eZdZeddДГZeddДГZeddДГZdS)┌RRefTypingTestc Csр|jd}||j}ttаddбГ}tjt|jГttаddбfdН}tjt|Гt |fdН}|а ||бtjt|Гt|fdН}|а |аб|бtjt|Гt |fdН}|а |аб|бtjt|Гt|fdН}|а |абаб|бdSйNrrr ) r>r?r]r$r&r"r#rrJrkrCrlr4) rD┌nrS┌ local_retrrLZrref1Zrref2Zrref3rrr┌test_rref_as_arg_and_return·s z*RRefTypingTest.test_rref_as_arg_and_returncCsB|jd}||j}tt|ГГ}|Г}|а|tаddбdбdS)Nrrщ )r>r?rxrrCr$r&)rDrИrSZmodule_with_rrefsrFrrr┌ test_my_script_module_with_rrefss z/RRefTypingTest.test_my_script_module_with_rrefscCsD|jd}||j}tt|ГГ}t|Г}|а|tаddбdбdS)Nrr)r>r?r(rrЕrCr$r&)rDrИrSr;rFrrr┌test_rref_python_annotations z*RRefTypingTest.test_rref_python_annotationN)rWrXrYr rКrМrНrrrrrЖ∙s rЖc@s$eZdZeddДГZeddДГZdS)┌FutureTypingTestcCs╛|jd|j}tаddgбtаddgбf}tjt|Гt|dН}tаddgб}tjj t ttdЬddДГ}|а||Г|бtjj t tttft tdЬd d ДГ}|||Г}|а|аб|бdS)Nrrr щ )rir*cSs|абSrйrcйrirrr┌future_wait_in_script.szYFutureTypingTest.test_future_passed_between_python_and_jit..future_wait_in_script)rS┌inputsr*cSstаdа|бt|бS)Nzworker{})r"rb┌format┌two_args_two_kwargs)rSrУrrr┌future_return_to_python4s z[FutureTypingTest.test_future_passed_between_python_and_jit..future_return_to_python)r>r?r$rhr"rbrrХr@rArrrC┌intrrc)rDrSrУZret_fut┌expected_resrТrЦZfut_resrrr┌)test_future_passed_between_python_and_jit's ■ z:FutureTypingTest.test_future_passed_between_python_and_jitcsР|jdkrdSt|jd|jГЙtаddбЙdЙtаИИб}tjjtt dЬЗЗЗfddДГЙtjj t dЬЗfddДГ}|Г}|а||бdS) Nrrrr-cstаИtjИИfiб}|Sr)r"rbr$r%rС)rE┌input_0┌input_1rr┌python_return_futureIszLFutureTypingTest.test_future_python_annotation..python_return_futurecsИГ}|абSrrРrС)rЬrr┌script_use_futureNszIFutureTypingTest.test_future_python_annotation..script_use_future)r>rr?r$r&r%r@┌ignorerrrArC)rDrШrЭrFr)rErЪrЫrЬr┌test_future_python_annotation?s z.FutureTypingTest.test_future_python_annotationN)rWrXrYr rЩrЯrrrrrО&s rОc@s(eZdZedЬddДZedЬddДZdS)┌ MyScriptClassй┌acCs ||_dSrrб)rDrвrrrrzYszMyScriptClass.__init__r-cCs|jSrrбйrDrrr┌ get_value\szMyScriptClass.get_valueN)rWrXrYrЧrzrдrrrrrаWsrаc@seZdZedЬddДZdS)┌MyModuleInterfacer-cCsdSrrrгrrrrБbszMyModuleInterface.forwardN)rWrXrYrrБrrrrrе`srеcsHeZdZЗfddДZejjedЬddДГZejjedЬddДГZ ЗZ S)┌MyScriptModulecstГабtа|б|_dSr)ryrzr$r&rв)rDr>r~rrrzhs zMyScriptModule.__init__r-cCs|jSrrбrгrrrrБlszMyScriptModule.forwardcCs|jSrrбrгrrr┌custom_funcpszMyScriptModule.custom_func)rWrXrYrzr$r@rВrrБrзrГrrr~rrжgs rжcCstаt|ГбSr)r"rrаrбrrr┌!owner_create_rref_my_script_classusrиcCstjt|ГtdНS)N)Z type_hint)r"rrжrеrбrrr┌"owner_create_rref_my_script_moduleysrйcCs|абабSr)r4rдr+rrr┌%script_rref_get_value_my_script_class}srкcCs|абабSrйr4rБr+rrr┌(script_rref_run_forward_my_script_moduleВsrмc@s<eZdZeddДГZeddДГZeddДГZeddДГZd S) ┌ LocalRRefTestcCs<|jdkrdStаt|jГб}|абаб}|а||jбdSrZ)r>r"rrаr4rдrC)rDZrref_script_classrLrrr┌)test_create_local_script_class_rref_in_pyИs z7LocalRRefTest.test_create_local_script_class_rref_in_pyc Csl|jdkrdStаt|jГtб}|абаб}|а|tа |jбб|а tdбПtаt|jГб}W5QRXdS)Nrz[The RRef being created contains a ScriptModule, must provide its ModuleInterface type hint.)r>r"rrжrеr4rБrCr$r&rHrI)rDZrref_script_modulerLrrr┌*test_create_local_script_module_rref_in_pyТs ¤z8LocalRRefTest.test_create_local_script_module_rref_in_pycCsЖ|jdkrdSt|jd|jГ}tj|t|jfdН}tttdЬddД}||Г}|а ||jбt jа|б}||Г}|а ||jбdS)Nrrr r)cSs*|f}i}tа|абt||б}|аб}|Sr)r"rb┌ownerrкrcйrr!┌kwargsrirLrrr┌use_rref_on_owner▓s zdLocalRRefTest.test_return_local_script_class_rref_in_py_and_use_in_script..use_rref_on_owner) r>rr?r"rJrиrrаrЧrCr$r@rAйrDrErr│rLZuse_rref_on_owner_scriptrrr┌;test_return_local_script_class_rref_in_py_and_use_in_scriptжs zILocalRRefTest.test_return_local_script_class_rref_in_py_and_use_in_scriptcCsТ|jdkrdSt|jd|jГ}tj|t|jfdН}tttdЬddД}||Г}|а |t а|jббt jа |б}||Г}|а |t а|jббdS)Nrrr r)cSs*|f}i}tа|абt||б}|аб}|Sr)r"rbZ owner_namerмrcr▒rrrr│╨s№zeLocalRRefTest.test_return_local_script_module_rref_in_py_and_use_in_script..use_rref_on_owner)r>rr?r"rJrйrrеrrCr$r&r@rAr┤rrr┌rr?r$rhr┬r├r─rCйrDrEr!r▓Z script_oprLrrr┌)test_all_kwargs_are_populated_by_defaultss z6JitRpcOpTest.test_all_kwargs_are_populated_by_defaultscCsЖ|jdkrdSt|jd|jГ}tаddgбtаddgбf}dtаddgбi}tttfD]&}||||Г}|а|tаddgббqZdS)Nrrrr╕rЛr╞r╟rrr┌*test_some_kwargs_are_populated_by_defaults-s z7JitRpcOpTest.test_some_kwargs_are_populated_by_defaultscCsТ|jdkrdSt|jd|jГ}tаddgбtаddgбf}tаddgбtаddgбdЬ}tttfD]&}||||Г}|а|tаddgббqfdS)Nrrrr.)r╕r╣щr╞r╟rrr┌(test_no_kwargs_are_populated_by_defaults=s ■ z5JitRpcOpTest.test_no_kwargs_are_populated_by_defaultscCs`|jdkrdSt|jd|jГ}tjjtdЬddДГ}||Г}|а|tаddgбddfбdS)NrrйrEcSsFtаddgбddf}tаddgбdddЬ}tа|t||б}|аб}|S)Nrr╝r.Z _str_kwarg)r╜r║r╛)r$rhr"rbr┐rcr┴rrr┌)script_rpc_async_call_with_assorted_typesVs¤ zlJitRpcOpTest.test_args_and_kwargs_contain_different_types..script_rpc_async_call_with_assorted_typesr0Zstr_arg_str_kwarg) r>rr?r$r@rA┌strrCrh)rDrEr═rLrrr┌,test_args_and_kwargs_contain_different_typesOs z9JitRpcOpTest.test_args_and_kwargs_contain_different_typescCsP|jdkrdSt|jd|jГ}tjjtdЬddДГ}||Г}|а|dбdS)Nrrr╠cSsd}tа|t|б}|аб}|SйNrйr"rbr[rc)rEr!rirLrrr┌+script_rpc_async_call_without_kwargs_passedwszXJitRpcOpTest.test_kwargs_not_passed..script_rpc_async_call_without_kwargs_passedйr>rr?r$r@rAr╬rC)rDrEr╥rLrrr┌test_kwargs_not_passedps z#JitRpcOpTest.test_kwargs_not_passedcCsP|jdkrdSt|jd|jГ}tjjtdЬddДГ}||Г}|а|dбdS)Nrrr╠cSstа|tб}|аб}|Srr╤)rErirLrrr┌0script_rpc_async_call_without_args_kwargs_passedМszjJitRpcOpTest.test_args_kwargs_are_neither_passed..script_rpc_async_call_without_args_kwargs_passedr╙)rDrEr╒rLrrr┌#test_args_kwargs_are_neither_passedЕs z0JitRpcOpTest.test_args_kwargs_are_neither_passedc CsT|jdkrdSt|jd|jГ}|аtdбПtjjtdЬddДГ}W5QRXdS)Nrrz Argument second_arg not providedr╠cSs0tаddgбf}i}tа|t||б}|аб}|Sr\йr$rhr"rbrХrcr┴rrr┌$script_rpc_async_call_with_less_argsгs zcJitRpcOpTest.test_less_than_needed_args_are_specified..script_rpc_async_call_with_less_argsй r>rr?rHrIr$r@rAr╬)rDrEr╪rrr┌(test_less_than_needed_args_are_specifiedЩs z5JitRpcOpTest.test_less_than_needed_args_are_specifiedc CsT|jdkrdSt|jd|jГ}|аtdбПtjjtdЬddДГ}W5QRXdS)Nrrz=Expected at most 4 arguments but found 5 positional argumentsr╠cSs`tаddgбtаddgбtаddgбtаddgбtаddgбf}i}tа|t||б}|аб}|S)Nrrr.r0r1r╫r┴rrr┌$script_rpc_async_call_with_more_args║s√zcJitRpcOpTest.test_more_than_needed_args_are_specified..script_rpc_async_call_with_more_argsr┘)rDrEr█rrr┌(test_more_than_needed_args_are_specifiedнs ■ z5JitRpcOpTest.test_more_than_needed_args_are_specifiedc Csh|jdkrdSt|jd|jГ}tjjtdЬddДГ}|аtdбП||Г}|а |dбW5QRXdS)Nrrr╠cSsJtаddgбtаddgбf}dtаddgбi}tа|t||б}|аб}|S)NrrZthird_kwargr╫r┴rrr┌+script_rpc_async_call_with_unexpected_kwarg╥s zeJitRpcOpTest.test_unexepected_kwarg_is_specified..script_rpc_async_call_with_unexpected_kwargz&Unknown keyword argument 'third_kwarg'й r>rr?r$r@rAr╬rHrIrC)rDrEr▌rLrrr┌#test_unexepected_kwarg_is_specified╩s z0JitRpcOpTest.test_unexepected_kwarg_is_specifiedc Csh|jdkrdSt|jd|jГ}tjjtdЬddДГ}|аtdбП||Г}|а |dбW5QRXdS)Nrrr╠cSs$d}i}tа|t||б}|аб}|Sr╨)r"rbr╖rcr┴rrr┌0rpc_async_call_remote_py_function_in_torchscriptыs zГJitRpcOpTest.test_call_python_function_remotely_from_script_not_supported..rpc_async_call_remote_py_function_in_torchscriptz#attempted to get undefined functionr▐)rDrErрrLrrr┌.rpc_async_call_remote_raising_torchscript_in_torchscriptrmr▐)rDrErтrLrrr┌:test_call_script_function_that_raises_remotely_from_script∙s zGJitRpcOpTest.test_call_script_function_that_raises_remotely_from_scriptc s||jdkrdSt|jd|jГ}tjjddДГЙtjjtdЬЗfddДГ}|аtdбП||Г}|а |dбW5QRXdS) NrrcSsdSrZrrrrr┌nonexisting_scriptszgJitRpcOpTest.test_call_script_function_that_not_exists_remotely_from_script..nonexisting_scriptr╠cs$d}i}tа|И||б}|аб}|Sr╨)r"rbrcr┴йrфrr┌.rpc_async_call_remote_nonexisting_torchscript_in_torchscriptz6attempted to get undefined function nonexisting_scriptr▐)rDrErцrLrrхr┌>test_call_script_function_that_not_exists_remotely_from_scripts zKJitRpcOpTest.test_call_script_function_that_not_exists_remotely_from_scriptN)rWrXrYr r╚r╔r╦r╧r╘r╓r┌r▄r▀rсrуrчrrrrr┼s0 r┼)r>r*cCst|ГSr)rжйr>rrr┌my_script_module_init1srщcCst|ГSr)rщrшrrr┌construct_my_script_module6srъ)┌ref_script_modulerr*cCs|аб}|аб|Srrл)rыr┌modulerrr┌run_ref_script_module;srэcCs|абSr)Zconfirmed_by_ownerr+rrrrRCsrR)r;┌fnamer*cCstа||бdSr)r$┌save)r;rюrrr┌ save_rrefHsrЁ)r^┌yr*cCs||Srr)r^rёrrr┌ script_addMsrЄй┌tor^rёr*cCstа|t||fбSrйr"rbrЄйrЇr^rёrrr┌ async_addRsrўcCs tаdбSrА)r$rOrrrr┌async_wrong_typeXsr°cCstа|б}tjа|б}|ГSr)┌io┌BytesIOr$r@┌load)Zpickled_script_module┌f┌mrrr┌$load_script_module_with_pickled_rref^s r■c@sHeZdZeddДГZeddДГZeddДГZeddДГZed d ДГZeddДГZ ed dДГZ eddДГZeddДГZeddДГZ eddДГZeddДГZeddДГZeddДГZeddДГZedd ДГZed!d"ДГZed#d$ДГZed%d&ДГZd'd(ДZd)d*ДZed+d,ДГZed-d.ДГZed/d0ДГZed1d2ДГZed3d4ДГZed5d6ДГZd7S)8┌ JitRpcTestcCsиt|jd|jГ}ttаddбГ}tj|ttаddбfdН}|а||бtj |ttаddбfdН}|а|а б|бtj t|jГttаddбfdН}|а|а б|бdSrЗ)rr>r?r]r$r&r"rJrCr#r4)rDrErЙrLrZ local_rrefrrr┌test_torchscript_functionls z$JitRpcTest.test_torchscript_functionc Csht|jd|jГ}|аtdбПtj|tddН}W5QRX|аtdбПtj|tddН}W5QRXdS)Nrzone_arg expected at most)rПщr ) rr>r?rHrIr"rJr]r#)rDrErLrrrr┌#test_torchscript_function_exceptionzs z.JitRpcTest.test_torchscript_function_exceptionc Cszt|jd|jГ}t|jГ}t|j|j|jГtабtj |t |jfdН}|аtdбПtj ||jddН}W5QRXdS)Nrr ┌pickler)rr>r?rжrZfile_init_method┌distZbarrierr"rJrаrH┌ TypeErrorrbrБ)rDrEZmy_local_script_modulerLrrr┌(test_torchscript_functions_not_supportedГs z3JitRpcTest.test_torchscript_functions_not_supportedc Cs╠ddlmmm}d|_tа|jбtа|jб}|jd}||j}tj t |Гt|jfdН}tjt |Гt |tа|jбfdН}|а||б|аtdбП(tjt |jГt |tа|jбfdН}W5QRXdS)NrTrr zFis an RRef to a ScriptModule. It can't be sent through RPC from owner,)Ztorch.distributed.rpc.api┌distributedr"┌apiZ_ignore_rref_leakr$r&r>r?r#rrъrJrэrCrHrI)rDrrЙrИrSZ remote_refrLrrr┌test_remote_script_moduleЬs2 ¤■¤z$JitRpcTest.test_remote_script_modulec Cs&t|jd|jГ}tj|t|jfdН}|аt|tj j Гб|Г}|аtа|jб|бtj |t|jfdН}tj|абt|tj j fdН}|а|б|абаб}|а|tа|jбб|абаб}|а|tа|jбб|аб}|аt|tj j Гб|Г}|а|tа|jбб|аб} |а| tа|jббdSйNrr )rr>r?r"rJrж┌ assertTruerr$r@┌ScriptModulerCr&r#r░rrБrзr4) rD┌dst_nameZcreated_script_moduleZrank_ones_tensorZremote_script_moduleZremote_end_is_scriptZremote_forward_outputZremote_func_outputZlocal_script_moduleZlocal_script_func_outputrrr┌#test_create_script_module_on_remote┴s4 ¤ z.JitRpcTest.test_create_script_module_on_remotecCsvt|jd|jГ}t|Г}t|Г}tаб}tабtj а ||бtабtj|t |абfdН}|Г}|а||бdSr )rr>r?rxr∙r·r"Z_enable_jit_rref_pickler$r@rяZ_disable_jit_rref_picklerJr■┌getvaluerC)rDr ┌m1┌m2r№Zout1Zout2rrr┌)test_load_script_module_with_pickled_rrefхs¤z4JitRpcTest.test_load_script_module_with_pickled_rrefc CsX|jd}||j}tt|ГГ}tГП(}|аtdбПt||ГW5QRXW5QRXdS)Nrz2RRef jit pickling is only allowed inside RPC calls)r>r?r(rrrHrIrЁ)rDrИrSr;rюrrr┌"test_rref_jit_pickle_not_supported∙s z-JitRpcTest.test_rref_jit_pickle_not_supportedc CsLtjt|jd|jГttаdбfdН}|аt dбП|а бW5QRXdSйNrrr ·.*Expected error.*)r"r#rr>r?ror$r&rH┌ Exceptionr4йrDrrrr┌test_remote_script_throws ¤z#JitRpcTest.test_remote_script_throwcCsFtjt|jd|jГttаdбfdН}|а|а бtаdбdбdSrЗ) r"r#rr>r?rjr$r&rCr4rrrr┌test_remote_script_udfs ¤z!JitRpcTest.test_remote_script_udfcCsFtjt|jd|jГttаdбfdН}|а|а бtаdбdбdSrЗ) r"rbrr>r?rjr$r&rCrcйrD┌futurerrr┌test_async_script_udfs ¤z JitRpcTest.test_async_script_udfcCsXddД}tjt|jd|jГttаdбfdНа|б}|а |а бtаdбddбdS)NcSs|абdSr\rРrСrrr┌callback"sz1JitRpcTest.test_callback_simple..callbackrrr )r"rbrr>r?rjr$r&┌thenrCrcйrDrrrrr┌test_callback_simple s ¤№zJitRpcTest.test_callback_simplecCsД|jd}t||jГ}ddД}tjt||jГttа||бfdН}d}t|ГD]}|а |б}qP|а |абtа||бd|бdS)NrcSs|абdSr\rРrСrrrr1sz0JitRpcTest.test_callback_chain..callbackr r)r>rr?r"rbr]r$r&r|rrCrc)rDrИr'rriZnum_cbsr}rrr┌test_callback_chain,s zJitRpcTest.test_callback_chaincsМdЙЗfddД}tjt|jd|jГttаdбfdН}|а|б|а ddДб}|а |абtаdбdб|аб|а ИtаdбdбdS) Ncs|абdЙdSrАrРrСйZcallback_calledrrrBsz3JitRpcTest.test_add_done_callback..callbackrrr cSsdS)NTr)r}rrr┌Mєz3JitRpcTest.test_add_done_callback..r0)r"rbrr>r?rjr$r&Zadd_done_callbackrrCrc)rDrrZfuture_thenrr"r┌test_add_done_callback>s ¤ z!JitRpcTest.test_add_done_callbackc CsLtjt|jd|jГttаdбfdН}|аt dбП|а бW5QRXdSr)r"rbrr>r?rpr$r&rHrrcrrrr┌test_async_script_throwVs ¤z"JitRpcTest.test_async_script_throwc s^ЗfddД}tjtИjdИjГttаdбfdНа|б}Иа t dбП|абW5QRXdS)Nc s,ИаtdбП|абW5QRXtdГВdS)Nr·Another expected error)rHrrcrIrСrгrrrbsz9JitRpcTest.test_callback_with_exception..callbackrrr r')r"rbrr>r?rpr$r&rrHrIrcrrrгr┌test_callback_with_exception`s ¤№z'JitRpcTest.test_callback_with_exceptionc CsЖ|jdkrВtГПD}ttjtjаtбddГ}tj j а|бП}t|j dГ}W5QRXW5QRX|j}t||Г}|аtjаtб|jkбdS)NrZworker0Zworker1)r>┌_profiler r ┌ASYNCr$┌ _jit_internal┌_qualified_namer]┌autogradrsrrurq┌function_eventsrr┌name)rD┌profZprof_keyrdrL┌events┌function_eventrrr┌test_call_rpc_with_profilingps № z'JitRpcTest.test_call_rpc_with_profilingc sИjdkРrИjdИj}t|Г}tаddgбtаddgбf}i}tГП}t|||ГW5QRX|j}tjа t бЙЗЗfddД|DГ}Иаt|Гdб|d}t tjИtИjГ|Г}Иа||jбddД|DГ} ddД| DГ} Иа| |hбd dД| DГd}|Ыd Э}Иа|j|бdS)Nrrrcs&g|]}И|jkr|jИjkr|СqSr)r/┌node_idr>й┌.0┌eventйZ qual_namerDrr┌ Чs ■z:JitRpcTest.test_rpc_async_jit_profiled..cSsg|]}|jr|СqSr)Z is_remoter5rrrr9еscSsh|] }|jТqSr)r4йr6Zremote_eventrrr┌ зsz9JitRpcTest.test_rpc_async_jit_profiled..cSsg|]}d|jkr|СqS)· aten::addйr/r:rrrr9нs ■z#remote_op: aten::add)r>r?rr$rhr)r┬r.r+r,rХrC┌lenr r ┌ ASYNC_JITr/) rDrSrEr!r▓r0r.Zrpc_async_jit_eventZ profiled_nameZ remote_eventsZremote_event_node_idsZ remote_addZremote_add_profiled_namerr8r┌test_rpc_async_jit_profiledЖsJ ■№ ■№ z&JitRpcTest.test_rpc_async_jit_profiledc s╥|jdkr╬|jd|j}t|Г}dЙtГП}t|ИГW5QRX|j}ЗfddД|DГ}|аdt|Гб|d}tt j tjа tбt|jГ|ГЙЗfddД|DГ}|аdt|Гб|D]}|а|j|jkбq╢dS)Nrr┌foocsg|]}|jИkr|СqSrr=r5)┌block_scoperrr9┴s zGJitRpcTest.test_record_function_on_caller_rpc_async..csg|]}|jИkr|СqSrr=r5й┌expected_keyrrr9═s r)r>r?rr)rer.rCr>r r r?r$r+r,r_rZcpu_time_total)rDrSrEr0r.Zrecord_function_scope_eventZjit_rpc_eventsZ jit_rpc_eventr)rBrDr┌(test_record_function_on_caller_rpc_async╡s8 № z3JitRpcTest.test_record_function_on_caller_rpc_asyncc s╛d}|jdkr║|jd|j}t|Г}d}tГП}t||ГW5QRX|аб|j}ttj t jаt бt|jГ|Г||ЙЗfddД|DГd}|а||jkб|j}|аddД|DГбdS) Nz#remote_op: rrrAcsg|]}|jИkr|СqSrr=)r6ZevtrCrrr9їs zCJitRpcTest.test_rpc_torchscript_record_function..css|]}d|jkVqdS)r<Nr=)r6┌childrrr┌ ·szBJitRpcTest.test_rpc_torchscript_record_function..)r>r?rr)rvZkey_averagesr.r r r?r$r+r,rarr/Zcpu_children) rDZ REMOTE_OP_STRrSrErBr0r.Zremote_record_function_eventZremote_childrenrrCr┌$test_rpc_torchscript_record_function┌s< ·° ў ■z/JitRpcTest.test_rpc_torchscript_record_functionc CsДd}tГП@}tjjаdбП}tjаt|б}|а|бW5QRX|а бW5QRX|j }t|dГ}|а|j dб|а|jd|бdS)NrrAgНэ╡аў╞░>)r)r$r-rsrr@rfrZ_call_end_callbacks_on_futurercr.rrCr/ZassertGreaterAlmostEqualZcpu_time)rDZsleep_intervalr0rdrir.Zsleep_eventrrr┌0test_record_function_jit_end_callbacks_with_fork№s z;JitRpcTest.test_record_function_jit_end_callbacks_with_forkc CsXtГП*}tjjаdбП}t|jГ}W5QRXW5QRX|j}t|dГ}|а |j dбdS)NrA)r)r$r-rsrrwrqr.rrCr/)rDr0rdrLr1r2rrr┌$test_call_fork_in_jit_with_profiling s z/JitRpcTest.test_call_fork_in_jit_with_profilingc Csjt|jd|jГ}t|jd|jГ}tj|t|tаddбtаddбfdН}|а|tаddбdбdSrЗ) rr>r?r"rJrўr$r&rC)rD┌dst1┌dst2rLrrr┌test_async_function_simples z%JitRpcTest.test_async_function_simplec Cs8|аtdбП"tаt|jd|jГtбW5QRXdS)N·DAsync functions must return an IValue of Future type, but got Tensorr)rHrIr"rJrr>r?r°rгrrr┌%test_async_function_wrong_return_type"s■ z0JitRpcTest.test_async_function_wrong_return_typec CsB|аtбП.tjjtjjtt t t t dЬddДГГ}W5QRXdS)NrєcSstа|t||fбSrrїrЎrrr┌async_wrong_decorator_order;szYJitRpcTest.test_async_function_wrong_decorator_order..async_wrong_decorator_order)┌assertRaisesrIr$r@rAr"┌ functions┌async_executionr╬rr)rDrPrrr┌)test_async_function_wrong_decorator_order,s ■z4JitRpcTest.test_async_function_wrong_decorator_orderc Csnt|jd|jГ}t|jd|jГ}tj|t|tаddбtаddбfdН}|а|а бtаddбdбdSrЗ) rr>r?r"r#rўr$r&rCr4)rDrKrLrrrr┌test_async_function_remoteBs z%JitRpcTest.test_async_function_remotecCsаt|jd|jГ}t|jd|jГ}d}g}t|ГD]4}|аtj|t|tа ddбtа ddб|fdНбq8t|ГD]$}|а ||абtа ddб|бqvdS)Nrrrr )rr>r?r|r3r"r#rўr$r&rCr4)rDrKrL┌numr{┌irrr┌ test_async_function_remote_multiLs z+JitRpcTest.test_async_function_remote_multic Cs@tаt|jd|jГtб}|аtdбП|абW5QRXdS)NrrN) r"r#rr>r?r°rHrIr4rrrr┌,test_async_function_wrong_return_type_remote]s ■z7JitRpcTest.test_async_function_wrong_return_type_remoteN)rWrXrYr rrrr rrrrrrr r!r%r&r(r3r@rErHrIrJrMrOrTrUrXrYrrrrr dsh $ # . $ ! r )crr∙┌typingrrrrr$Ztorch.distributedrrZtorch.distributed.rpcr"rZtorch.autograd.profilerrrZtorch.distributed.rpc.internalr r Z torch.futuresrZ$torch.testing._internal.common_utilsrZ"torch.testing._internal.dist_utilsr rrrZ>torch.testing._internal.distributed.rpc.rpc_agent_test_fixturerZtorch.autograd.profiler_legacyrr)rrr(r@rAr,rЧr/r5r8r9r[r]r_r╬rarerjrkrlrorprurvrwrrxrЮrДrЕrЖrОrаZ interface┌nn┌Modulerеrжrиrйrкrмrнr╖rhrХr┐r└r┬r├r─r┼rщrъrэrBrRrЁrЄrRrSrўr°r■r rrrr┌sI -1_№ ·· ■ ·