U <ºc–ã@s’ddlmZddlmZddlmZmZddlmZ ddl mm Z Gdd„de j ƒZdd„Ze j d e¡d d „Zedd d„ƒZddd„ZdS)é)Úcontextmanager)Ú timedelta)ÚpartialÚwrapsNcs$eZdZ‡fdd„Zdd„Z‡ZS)ÚMockProcessGroupcstt|ƒ ||¡dS©N)ÚsuperrÚ__init__)ÚselfÚrankZworld©Ú __class__©úY/tmp/pip-unpacked-wheel-gikjz4vx/torch/testing/_internal/distributed/distributed_utils.pyr szMockProcessGroup.__init__cCsdS)NÚmock_process_groupr)r rrrÚgetBackendNameszMockProcessGroup.getBackendName)Ú__name__Ú __module__Ú __qualname__r rÚ __classcell__rrr rr s rcCs t||ƒSr)r)Z prefix_storer Ú world_sizeÚtimeoutrrrÚcreate_mock_pgsrrc CsLt ¡r t‚t ¡}| tj›d|d¡tjd|||dtddddS)Nz:0érZfake)Úseconds)Úbackendr rÚstoreZ group_namer) ÚdistZis_initializedÚAssertionErrorZ HashStoreÚaddÚc10dZSTORE_BASED_BARRIER_PREFIXZinit_process_groupr)r rrrrrÚmock_init_dists úr!éccs&t||dz dVW5t ¡XdS)zJ Context manager that initializer c10d with a fake process group. ©r rN)r!rZdestroy_process_groupr#rrrÚ with_dist)s  r$cs2ˆdkrttˆˆdStˆƒ‡‡‡fdd„ƒ}|S)zˆ Function wrapper that inits a fake process group designed for testing. Right now only querying for world size is available Nr#c s*tˆˆƒˆ|f|ž|ŽW5QRXdSr)r$)r ÚargsÚkwargs©Úfuncr rrrÚwrapper<s z with_fake_comms..wrapper)rÚwith_fake_commsr)r(r rr)rr'rr*4s r*)rr")Nrr")Ú contextlibrÚdatetimerÚ functoolsrrZtorch.distributedZ distributedrZ"torch.distributed.distributed_c10dZdistributed_c10dr Z ProcessGrouprrZBackendZregister_backendr!r$r*rrrrÚs