U KcmP @s<ddlZddlZddlZddlmZddlmmmZddl m mmZ ej jZ ddlmZmZmZmZmZmZmZddlmZddlmZddlmZmZddlmZddlm Z dd l!m"Z"m#Z#Gd d d ej$Z%eeeee&ee"fee%e%fd d dZ'eeee&ee"feeeej(e)feej(e*ffdddZ+eeedddZ,eee*dddZ-eee*dddZ.eee&dddZ/e#e&e#dddZ0e#ddd d!Z1d"d#Z2e2ej(ej(ej(d$d%d&Z3e2ej(ej(ej(d$d'd(Z4e2ej(ej(ej(d$d)d*Z5ee6dd+d,Z7eee*ed-d.d/Z8dS)0N)TupleCallableDictSetListOptionalUnion) GraphModule)Node) ObserverBaseFakeQuantizeBase)getattr_from_fqn)is_activation_post_process)NSNodeTargetType NSResultsTypec@s4eZdZeZeZeZeZeZ dS)NodeInputOrOutputTypeN) __name__ __module__ __qualname__enumautoFP32INT8FP16UNKNOWNZ FP32_OR_INT8rr8/tmp/pip-unpacked-wheel-gikjz4vx/torch/ao/ns/fx/utils.pyrs r)nodegm logger_clsnode_type_to_io_type_mapreturncs|d}|d}|d}|d}|d}|d} |d} |d} |jd kr|j|kr`tjtjfS|j|krvtjtjfS|j|krtjtjfS|j|krt||d } t| tst t | |||\} }||fStj tj fSn|jd kr|jd kst t|jt st t ||jtfd d | D}t|ttfs@|rvt||d } t| ts\t t | |||\} }||fStfdd |D}tfdd | D}|rtjtjfS|rtjtjfStj tj fSn|jdkr|jdkr&t||d }t|ts t t ||||\} }|tjfS|jdkrt||d }t|tsNt t ||||\} }t||d}|tjkst |d|tjfS|j| krt||d } t| tst t | |||\} }||fStj tj fStj tj fSdS)NZfuns_io_type_fp32Zfuns_io_type_fp16Zfuns_io_type_int8Zfuns_io_type_fp32_or_int8Zmods_io_type_fp32Zmods_io_type_int8mods_io_type_fp32_or_int8Zmeths_io_type_fp32_or_int8 call_functionr call_modulec3s|]}t|VqdSN isinstance.0 target_typemodrr Osz7get_node_first_input_and_output_type..c3s|]}t|VqdSr&r'r)r,rrr.asc3s|]}t|VqdSr&r'r)r,rrr.ds call_method dequantizetorz handling needs to be added)optargetrrrrget_normalized_nth_inputr(r AssertionError$get_node_first_input_and_output_typerstrr anyr r torchZfloat16)rrr r!ZFUNS_IO_TYPE_FP32ZFUNS_IO_TYPE_FP16ZFUNS_IO_TYPE_INT8ZFUNS_IO_TYPE_FP32_OR_INT8ZMODS_IO_TYPE_FP32ZMODS_IO_TYPE_INT8MODS_IO_TYPE_FP32_OR_INT8ZMETHS_IO_TYPE_FP32_OR_INT8Z first_argZ_prev_node_input_typeZprev_node_output_type"is_known_fp32_or_int8_input_moduleZis_known_fp32_input_moduleZis_known_int8_input_module prev_nodeZcur_node_dtype_targetrr,rr6&s                            r6)rrr!r"csFt||d}t|tsdS|d}dd}|jdkrz|jtjkrN|||ddS|jtjtj tj tj fkrv|||dd SdS|jd krBt|jt st t||jttjtjtjtjtjtjtjtjtjtjtjtjtjtjtjtj tj!tj"tj#tj$tj%tjtj&tj'frj(j)fSt*fd d |D}|rBt+|||SdS) z{ Returns the qparams (scale, zero_point) of the first input to `node`, if they can be inferred from the graph. rNr#cSslt|||}t|||}t|tr.t|jts2tt|trHt|jtsLtt||j}t||j}||fSr&)r4r(r r3r7r5r )rrZ scale_arg_idxZ zp_arg_idxZ scale_nodeZzp_nodeZ scale_objZzp_objrrr _get_scale_zp_from_function_argss    z@get_node_input_qparams.._get_scale_zp_from_function_argsr$rr%c3s|]}t|VqdSr&r'r)Z module_objrrr.sz)get_node_input_qparams..),r4r(r r2r3r9Zquantize_per_tensortoqaddZadd_relumulZmul_relur7r5r nnqZLinearZConv1dZConv2dnniqZ ConvReLU2dZConv3dZ BatchNorm2dZ BatchNorm3dZConvTranspose1dZConvTranspose2dZELUZ GroupNormZInstanceNorm1dZInstanceNorm2dZInstanceNorm3dZ LayerNormZ HardswishZ LeakyReLUZReLU6ZBNReLU2dZBNReLU3dZ ConvReLU1dZ ConvReLU3dZ LinearReLUZscaleZ zero_pointr8get_node_input_qparams)rrr!r<r:r=r;rr@rrFsb         rF)rrr"cCs|jdkrt||j}t|rt|jdks0tt|jdtsDt|jd}t|jt s^tt||j}t|rt|jdkstt|jdtst|jd}|S)a If node is not an observer, returns it. If node is an observer, navigates up the graph and returns the first parent which is not an observer. For example, graph: (node_non_obs), node = node_non_obs : returns node_non_obs graph: (node_non_obs -> obs0), node = obs0 : returns node_non_obs graph: (node_non_obs -> obs0 -> fq0), node = fq0 : returns node_non_obs r%rr) r2r r3rlenargsr5r(r r7rrZnode_objrrrreturn_first_non_observer_nodes     rJcCs*|jdkr&t||j}t|tjr&dSdS)aO Assumes that all non-param args occur first. Returns the number of non-param args expected for a node. For example, for F.linear(x, weight, bias) Returns 1, because x is a non-param arg and weight and bias are params. For lstm_mod(x, hid) Returns 2, because both x and hid are non-param args. r%r>r)r2r r3r(nnZLSTMrIrrrget_number_of_non_param_argss    rL)rr"cCst|jdkrgS|jdkr|jtjtjjjtjfksP|jtj tjjj tj fkrg}t dD] }t |j|t kr\| |q\|SdgS)a- Returns the indices of args of the node which we should attach loggers to, if input logging is enabled. For example, * for (x + y), returns [0, 1] * for (1 + y), returns [1] * for (x + 1), returns [0] * for (linear(x, w, b)) returns [0] * by default, returns [0] rr$r>)rGrHr2r3r9rBops quantizedoperatorrCrangetyper append)rresultirrr get_arg_indices_of_inputs_to_log+s    rUcCsPd}|jdkrt|j}n0|jdkrLt|jts6tt||j}t|}|S)z Returns a string representation of the type of the function or module pointed to by this node, or '' for other node types. )r$r/r%)r2r9typenamer3r(r7r5r )rrr+Z target_modrrrget_target_type_strFs    rX)results model_namer"c Csi}|D]n\}}d}|D]>\}}|D],\}} ||kr0t| sLt| dd}q0q0q0q |dk rr|||<q |||<q |S)a  Rekeys the layer name of a results dictionary to use node names from `model_name`. For example, transforms {'base_op_1_0': {'node_output': {'model_a': [{'ref_node_name': 'linear1', ...}]}}} into {'linear1': {'node_output': {'model_a': [{'ref_node_name': 'linear1', ...}]}}} Note: we cannot use these node names directly because they are not guaranteed to be consistent across models. This is why we extract the results first and rekey afterwards. NrZ ref_node_name)itemsrGr5) rYrZZ new_resultsZold_layer_nameresult_type_to_resultsZnew_layer_nameZ _result_typemodel_name_to_resultsZcur_model_nameZlist_of_resultsrrr'rekey_logger_info_on_node_name_of_modelUs   r^)rYr"c Csd}|D]X\}}|D]B\}}|D],\}}t|dkr,|dddk r,|}qZq,q`qqfq |r|D]h\}}|D]V\}}||}|D]<\}}||krqtt|D]} || d} | || d<qqqqrdS)ay If `fqn` entries are filled in for one of the models in `results`, copies them over to any models which do not have them filled out. A common use case benefitting from this is comparing a model prepared by quantization to a quantized model. In this case, the model prepared by quantization would have `fqn` entries, and the quantized model would not. Nrfqn)r[rGrP) rYZmodel_name_with_fqnsZ layer_namer\Z result_typer]rZZ model_resultsZref_model_resultsrTr_rrrmaybe_add_missing_fqns|s(   r`csfddS)Nc s|^}}}t|trt|ts2t|trjt|trjg}t||D]$\}}||f|}|||q@|St|tjrt|tjr|jr|}|jr|}|j tj ks|j tj krdS||f|}||Sr&) r(tuplelistziprRr9TensorZ is_quantizedr0Zdtypefloat) rHkwargsZa0Za1Za_otherrYZel0Zel1new_argsfinnerrrrjs(   zGmaybe_dequantize_first_two_tensor_args_and_handle_tuples..innerr)rirrhr8maybe_dequantize_first_two_tensor_args_and_handle_tuplessrk)xyr"cCs*t|}t||}dt||S)z Computes the SQNR between `x` and `y`. Args: x: Tensor or tuple of tensors y: Tensor or tuple of tensors Return: float or tuple of floats )r9Znormlog10)rlrmZPsZPnrrr compute_sqnrs rpcCs"t||d|dS)z Computes the normalized L2 error between `x` and `y`. Args: x: Tensor or tuple of tensors y: Tensor or tuple of tensors Return: float or tuple of floats r>)r9sqrtsumrlrmrrrcompute_normalized_l2_errors rtcCs(|dd}|dd}tjj||S)z Computes the cosine similarity between `x` and `y`. Args: x: Tensor or tuple of tensors y: Tensor or tuple of tensors Return: float or tuple of floats r)Zreshaper9rKZ functionalZcosine_similarityrsrrrcompute_cosine_similaritys  rvcCs4|jdkr0|jtjtjtjtjtjtjfkr0dSdS)Nr$FT)r2r3r9rBrCrOcatstack)rrrrop_type_supports_shadowings "ry)rridxr"cCs6z|j|dd}|dk rb|\}}t|t||ks8t|t|krN||WSt||WSnXt|jt|j|ks~t|t|jkr|j|WS|t|j}t|j|WSWnttk r0t|jt|j|kst|t|jkr|j|YS|t|j}t|j|YSYnXdS)zu Given a node, gets the n'th input to that node, normalizing args and kwargs to the best of its ability. T)Znormalize_to_only_use_kwargsN)Znormalized_argumentsrGr5rbvaluesrHrf RuntimeError)rrrzZnorm_args_and_kwargsZ norm_argsZ norm_kwargsZ kwargs_idxrrrr4s,   r4)9rrOr9Ztorch.nnrKZtorch.nn.intrinsic.quantizedZ intrinsicrNrEZtorch.ao.nn.quantizedZaorDrMrAtypingrrrrrrrZtorch.fxr Ztorch.fx.graphr Ztorch.ao.quantizationr r Ztorch.ao.quantization.utilsr Ztorch.ao.quantization.quantizerZns_typesrrEnumrr7r6rdreintrFrJrLrUrXr^r`rkrprtrvboolryr4rrrrsb $     }" S   '"