U Jc@sddlZddlmZmZddlmZmZmZddlZ ddl m Z ddl m Z mZmZmZmZddlmZddlmZmZmZdd lmZmZdd lmZdd lmZGd d d eZee dddZ eeeefdddZ!dS)N) b64decode b64encode)OptionalTuplecast)Client)EtcdAlreadyExistEtcdCompareFailed EtcdExceptionEtcdKeyNotFound EtcdResult)Store)RendezvousConnectionErrorRendezvousParametersRendezvousStateError)RendezvousBackendToken) EtcdStore)parse_rendezvous_endpointc@seZdZUdZdZeed<eed<eed<deee ee edddd Z e ed d d Z e e eefd d dZdee ee e eeefdddZee eefdddZdS)EtcdRendezvousBackendaRepresents an etcd-based rendezvous backend. Args: client: The ``etcd.Client`` instance to use to communicate with etcd. run_id: The run id of the rendezvous. key_prefix: The path under which to store the rendezvous state in etcd. ttl: The TTL of the rendezvous state. If not specified, defaults to two hours. i _client_key_ttlN)clientrun_id key_prefixttlreturncCsL|s td||_|r&|d||_n||_|r@|dkr@||_n|j|_dS)Nz&The run id must be a non-empty string./r) ValueErrorrrr _DEFAULT_TTL)selfrrrrr#`/tmp/pip-unpacked-wheel-gikjz4vx/torch/distributed/elastic/rendezvous/etcd_rendezvous_backend.py__init__0s zEtcdRendezvousBackend.__init__)rcCsdS)See base class.zetcd-v2r#r"r#r#r$nameFszEtcdRendezvousBackend.namec Csfz|j|j}WnHtk r(YdSttjjfk rZ}ztd|W5d}~XYnX| |S)r&NCThe connection to etcd has failed. See inner exception for details.) rreadrr r urllib3 exceptions TimeoutErrorr _decode_state)r"resultexcr#r#r$ get_stateKszEtcdRendezvousBackend.get_state)statetokenrc st|}i}fdd}|rHz t|}Wntk rF|YSX|rV||d<nd|d<zjjj|jf|}WnNtt fk rd}Yn4t t j j fk r}ztd|W5d}~XYnX|dkr|S|d }|S) r&cs }|dk r|d}|SdS)NF)F)r1)r/tmpr'r#r$r1`s z2EtcdRendezvousBackend.set_state..get_stateZ prevIndexFZ prevExistNr)T)T)rdecodeintr rwriterrrr r r+r,r-rr.) r"r2r3 base64_statekwargsr1r/r0r4r#r'r$ set_stateXs2     zEtcdRendezvousBackend.set_state)r/rc CsP|j}z t|}Wn.tjk rD}ztd|W5d}~XYnX||jfS)Nz=The state object is corrupt. See inner exception for details.)valueencoderbinasciiErrorrZ modifiedIndex)r"r/r8r2r0r#r#r$r.s  z#EtcdRendezvousBackend._decode_state)NN)N)__name__ __module__ __qualname____doc__r! EtcdClient__annotations__strr6rr%propertyr(rbytesrr1boolr:r r.r#r#r#r$rs0    +r)paramsrc Cst|jdd\}}tt|dd}|dkr4td|dd}|dkr`|d kr`td |d }|r|d }|r||f}|d }zt ||||||ddWSt t j j fk r}ztd|W5d}~XYnXdS)NiK ) default_port read_timeout<rz,The read timeout must be a positive integer.protocolhttphttpsz#The protocol must be HTTP or HTTPS.ssl_cert ssl_cert_keyca_certT)rKrMcertrRZallow_reconnectr))rZendpointrr6Z get_as_intr getstriplowerrCr r+r,r-r) rIhostportrKrMrPrQrRr0r#r#r$_create_etcd_clients:    rYcCs*t|}t||jdd}t|d}||fS)a Creates a new :py:class:`EtcdRendezvousBackend` from the specified parameters. +--------------+-----------------------------------------------------------+ | Parameter | Description | +==============+===========================================================+ | read_timeout | The read timeout, in seconds, for etcd operations. | | | Defaults to 60 seconds. | +--------------+-----------------------------------------------------------+ | protocol | The protocol to use to communicate with etcd. Valid | | | values are "http" and "https". Defaults to "http". | +--------------+-----------------------------------------------------------+ | ssl_cert | The path to the SSL client certificate to use along with | | | HTTPS. Defaults to ``None``. | +--------------+-----------------------------------------------------------+ | ssl_cert_key | The path to the private key of the SSL client certificate | | | to use along with HTTPS. Defaults to ``None``. | +--------------+-----------------------------------------------------------+ | ca_cert | The path to the rool SSL authority certificate. Defaults | | | to ``None``. | +--------------+-----------------------------------------------------------+ z/torch/elastic/rendezvous)rz/torch/elastic/store)rYrrr)rIrbackendstorer#r#r$create_backends r\)"r=base64rrtypingrrrZurllib3.exceptionsr+ZetcdrrCrr r r r Ztorch.distributedr apirrrZdynamic_rendezvousrrZ etcd_storerutilsrrrYr\r#r#r#r$s    t)