U Kºct-ã@s²ddlZddlZddlZddlZddlmZddlmZddlmZm Z ddl m Z m Z m Z mZddlmZmZmZddlmZd gZejejejhZejhZGd d „d eƒZdS) éN)Únn)Ú parametrize)Ú ModuleDictÚ ModuleListé)ÚPruningParametrizationÚZeroesParametrizationÚActivationReconstructionÚBiasHook)ÚBaseSparsifierÚ module_to_fqnÚ fqn_to_module)Úget_arg_info_from_tensor_fqnÚ BasePrunercsveZdZdZd‡fdd„ Zdd„Zddd „Zeefd d „Z d d „Z ddd„Z ddd„Z ddd„Z ejdd„ƒZ‡ZS)raBase class for all pruners. Abstract methods that need to be implemented: - update_mask: Function to compute a new mask for all keys in the `groups` attribute. Args: - defaults [dict]: default configurations will be attached to the configuration. Only the keys that don't exist in the `config` will be updated. - also_prune_bias [bool]: whether to prune bias in addition to weights (to prune full output channel) or not; default=True. Tcstƒ |¡||_dS©N)ÚsuperÚ__init__Ú prune_bias)ÚselfÚdefaultsZalso_prune_bias©Ú __class__©úV/tmp/pip-unpacked-wheel-gikjz4vx/torch/ao/sparsity/_experimental/pruner/base_pruner.pyr-s zBasePruner.__init__cCsög}g}|rˆt|dƒtkrZt|d|dƒD](\}}t|j|ƒ}| |¡| |¡q.qît|j|dƒ}| |¡|d}| |¡nft|dƒtkrÊt|d|dƒD]\}}| |¡| |¡qªn$|d}| |¡|d}| |¡||fS)NÚmoduleÚ module_fqnÚ tensor_name)ÚtypeÚtupleÚzipr ÚmodelÚappend)rÚconfigÚuse_pathÚmodulesÚ tensor_namesrrrrrrÚ_get_modules_and_tensor_names1s*       z(BasePruner._get_modules_and_tensor_namesFc OsÎg|_g|_|jD]´}| ||¡\}}t||ƒD]f\}}t|ttƒƒsút|ddƒdkrx|  dt   t||ƒj d¡¡|  dt¡} tj||| |jƒddt|jtƒs¬t‚t|jjtƒs¾t‚t|ttƒƒrð|j | tt|j|ƒdƒ¡¡ntdƒ‚nTt|ddƒdkr*|  dt   t||ƒj d¡¡|  dt¡} tj||| |jƒdd|jdk rx| dt  |j !¡¡¡d|_|j | t"|jjd|j#ƒ¡¡q2t$|ƒd kr|djjdj%|d jjd_%qdS) z6Adds mask parametrization to the layer weight ÚmaskNrÚparametrizationT)Zunsafez&This module type is not supported yet.Z_biasér)&Zactivation_handlesZ bias_handlesÚgroupsr&rÚ isinstancerÚ NEEDS_ZEROSÚgetattrZregister_bufferÚtorchZtensorÚshapeÚgetrrZregister_parametrizationr'ÚparametrizationsrÚAssertionErrorÚweightrÚSUPPORTED_MODULESr!Zregister_forward_hookr ÚNotImplementedErrorrZbiasZregister_parameterrÚ ParameterÚdetachr rÚlenÚpruned_outputs) rr#ÚargsÚkwargsr"r$r%rrÚparamrrrÚ_prepareLs6   ÿ   $ zBasePruner._preparec Cs®g|_|g}|rª| ¡}| ¡D]†\}}t|ƒ|krbt||ƒ}t|tƒsLt‚|j d|di¡q |dk rœt|ƒ|krœt |dƒrœ|j rœt   dt|ƒ›d¡| |¡q q dS)NÚ tensor_fqnú.weightrz Models with z% layers have config provided by user.) r"ÚpopZnamed_childrenrr r+Ústrr2r!ÚhasattrrÚwarningsÚwarn) rr r4r,ÚstackrÚnameÚchildZ child_fqnrrrÚmake_config_from_modelus  $z!BasePruner.make_config_from_modelcCs@||_||_|jdkr"| |j¡|jD]}t|ƒtkr|\}}t|tjƒr\t|tjƒs`t ‚t|tƒsnt ‚d|i}t   |j ¡}|  |¡g}g}g} |dD]X} t|| ƒ} | dkrºd} | rÖ| ddkrÖ| dd…} | | ¡| | d¡|  d¡q ||d <||d <| |d <nt|tjƒr,d|i}t   |j ¡}|  |¡| d d¡dk rÈ| d ¡} t| tƒsnt ‚t|| ƒ} |  ¡D]:}||kr€|d ks€| |||ks€t d  |¡ƒ‚q€|  | ¡n^|d} t|| ƒ} | rú| ddkrú| dd…} | |d <d|d <t| tƒst ‚| d|d <|j |¡q(| ¡dS) aUPrepares a model, by adding the parametrizations and forward post-hooks. Note:: The model is modified inplace. If you need to preserve the original model, use copy.deepcopy. Args: - model [nn.Module]: model to configure. The model itself is not saved but used for the state_dict saving / loading. - config [list]: configuration elements could either be instances of tuples of dict maps or dict maps. The dicts must have a key 'tensor_fqn' with the value being the fqn of the tensor to be pruned. NrÚrÚ.rr?r3rr>rz?Given both `{}` and `tensor_fqn`, it is expected them to agree!)r r"rHrrr+rÚConv2dÚ BatchNorm2dr2ÚcopyÚdeepcopyrÚupdater r!ÚModuler0rArÚkeysÚformatr*r=)rr r"Z module_configZ first_layerZ next_layerZ local_argsZmodule_fqn_listZtensor_fqn_listZtensor_name_listrrr>Zinfo_from_tensor_fqnÚkeyrrrÚprepare…sj                  ÿÿ    zBasePruner.preparec Os||jD]p}| ||¡\}}t||ƒD]P\}}tj||ddt|jddƒrT|jd=nt|jddƒrj|jd=t|dƒq$qdS)NT)Zleave_parametrizedr') r*r&rrZremove_parametrizationsr-Ú _parametersÚ_buffersÚdelattr) rr#r:r;r"r$r%rrrrrÚ squash_maskÔs ÿ zBasePruner.squash_maskr3cCs t |¡st‚t|j|ƒdjS)z+Returns the set of pruned indices of moduler)rZis_parametrizedr2r-r1r9)rrrrrrÚget_module_pruned_outputsász$BasePruner.get_module_pruned_outputsc Csn|js dSt ¡R|jD]D}| ||¡\}}i}| ¡|d|d<|d|d<|jf|ŽqW5QRXdS)Nrrr)Zenable_mask_updater.Zno_gradr*r&rOÚ update_mask)rr#r"r$r%Z untupled_argsrrrÚstepæs    zBasePruner.stepcKsdSrr)rrrr;rrrrZõszBasePruner.update_mask)T)F)F)r3)F)Ú__name__Ú __module__Ú __qualname__Ú__doc__rr&r=r4r,rHrTrXrYr[ÚabcÚabstractmethodrZÚ __classcell__rrrrrs )O  )rMrCr`r.rZtorch.nn.utilsrZtorch.nn.modules.containerrrr(rrr r Ztorch.ao.sparsityr r r Z"torch.ao.sparsity.sparsifier.utilsrÚ__all__ZLinearrKrLr4r,rrrrrÚs"   ýÿ