U JcA @sjddlmZmZmZddlmZddlmZddlm Z d ej e e eee eej e ffdd d ZdS) )IteratorTupleUnion)ShardedOptimizerN) ShardedTensorT)moduleprefixrecursereturnc cs|r|j|dn||fg}t}|D]X\}}t|D]B\}}t|tr8||kr8||||rfdnd|}||fVq8q$|D]\}}||fVqdS)aReturns an iterator over module parameters (together with the ShardedTensor parameters), yielding both the name of the parameter as well as the parameter itself. This is typically passed to a :class:torch.distributed._shard.sharded_optim.ShardedOptimizer Args: prefix (str): prefix to prepend to all parameter names. recurse (bool): if True, then yields parameters of this module and all submodules. Otherwise, yields only parameters that are direct members of this module. Yields: (str, Union[Tensor, ShardedTensor]): Tuple containing the name and parameter (or ShardedTensor parameter) Example:: >>> # xdoctest: +SKIP >>> model = torch.nn.Linear(*linear_size) >>> shard_parameter(model, "weight", spec) >>> for name, param in named_params_with_sharded_tensor(model): >>> if name in ['weight']: >>> print(param.size()) )r .rN)Z named_modulessetvarsitems isinstanceraddZnamed_parameters) r r r modulesmemoZ mod_prefixmodnamevalrS/tmp/pip-unpacked-wheel-gikjz4vx/torch/distributed/_shard/sharded_optim/__init__.py named_params_with_sharded_tensor s  r)rT)typingrrrapirZtorch.nnnnZ'torch.distributed._shard.sharded_tensorrModulestrbool Parameterrrrrrs