U Kc]h@sdZddlZddlZddlmZddlmmmZddl m Z ddl m Z ddl mZddlmZmZddlmZdd lmZmZdd lmZmZmZdd lmZmZmZdd lm Z m!Z!m"Z"m#Z#m$Z$m%Z%e!ej&e!ej&ej&ffZ'Gd ddej(Z)Gdddej*Z+d6e,e e#e!e e,fee$e e,e e"e"ffddddZ-d7e,e e,e e$e e,e%efe$e e,e%efe$e e,e e"e"ffedddZ.d8e,ej(e,ej(e$e e,e%efe$e e,e%efe$e e,e e"e"ffedddZ/e,e e#e!e e,e,fe#e!e e,e,fe"ej(dddZ0d9e,e e,e e"e1e$e e,e%efe$e e,e%efe!ej(ej(fd ddZ2d:e,ej(e,ej(e"e1e$e e,e%efe$e e,e%efe!ej(ej(fd! d"d#Z3ej(ee"dd$d%d&Z4ej(ej(e"e,ed'd(d)Z5d;e,e e,e e"e1e$e e,e%efe$e e,e%efe$e e,e%efej(d* d+d,Z6dboolrGr@r)r)r'r*rAsrA)rmodelnodes_and_names_to_instrumentresults"op_to_type_to_weight_extraction_fnrDc Cs\tjd|D]F\}}tjj}t|||}|r||krD|ii||<|g||||<qdS)Nz=quantization_api._numeric_suite_fx._extract_weights_one_model)r._C_log_api_usage_oncerZWEIGHTvaluer) rrKrLrMrNnoderZres_typeZextracted_weightr)r)r*_extract_weights_one_models   rS) model_name_agm_a model_name_bgm_b base_name_to_sets_of_related_opsunmatchable_types_maprNrDcCstjdt||||}g}g} |D]0\} } | \} } || j| f| | j| fq*i}t|||||t||| ||t|t ||}|S)Nz8quantization_api._numeric_suite_fx._extract_weights_impl) r.rOrPritemsr0 base_op_noderSr r )rTrUrVrWrXrYrNmatched_subgraph_pairsZnodes_and_names_to_instrument_aZnodes_and_names_to_instrument_b match_namematch subgraph_a subgraph_brMr)r)r*_extract_weights_impls>  ra)rTmodel_arVmodel_brXrYrNrDcCstjd|dkrt}t|}g}g} t|| } t|| } t|| |} t|dr`|j | _ t|| |} t|dr|j | _ t || || |||S)a Extract weights from model A and model B, and return a comparison. Args: model_name_a: string name of model A to use in results model_a: model A model_name_b: string name of model B to use in results model_b: model B base_name_to_sets_of_related_ops: optional override of subgraph base nodes, subject to change unmatchable_types_map: optional override of unmatchable types, subject to change op_to_type_to_weight_extraction_fn: optional override of function which extracts weight from a type, subject to change Return: NSResultsType, containing the weight comparisons z2quantization_api._numeric_suite_fx.extract_weightsN_node_name_to_scope) r.rOrPrrrArtracehasattrrdra)rTrbrVrcrXrYrNZtype_a_related_to_bskipped_module_namesskipped_module_classestracer_atracer_brUrWr)r)r*extract_weightss2     rk)rrK$nodes_and_names_to_instrument_inputs%nodes_and_names_to_instrument_outputs logger_clsrDc Cs`tjdi}i}|D]\}}} || f||<q|D]\}}} || f||<q4t|||||}|S)Nz9quantization_api._numeric_suite_fx._add_loggers_one_model)r.rOrPr ) rrKrlrmrnZ%node_to_instrument_inputs_to_ref_nameZ&node_to_instrument_outputs_to_ref_namerRrZ ref_node_typer)r)r*_add_loggers_one_modelOs ro) name_arUname_brWrnshould_log_inputsrXrYrDcCstjdt||||}g} g} g} g} |D]p\} \}}t|j|}t|j|}|r~| |j| |f| |j| |f| |j | |f| |j | |fq2t ||| | |}t ||| | |}||fS)Nz4quantization_api._numeric_suite_fx._add_loggers_impl) r.rOrPrrZr r[r0Z start_nodeZend_nodero)rprUrqrWrnrrrXrYr\Z&nodes_and_names_to_instrument_inputs_aZ&nodes_and_names_to_instrument_inputs_bZ'nodes_and_names_to_instrument_outputs_aZ'nodes_and_names_to_instrument_outputs_br]r_r`Zref_node_type_aZref_node_type_bZ new_model_aZ new_model_br)r)r*_add_loggers_implgsT       rsF) rprbrqrcrnrrrXrYrDc Cstjdg}g} t|| } t|| } t|| |} t|drJ|j| _t|| |} t|drl|j| _t|| || ||||dS)aP Instrument model A and model B with loggers. Args: model_name_a: string name of model A to use in results model_a: model A model_name_b: string name of model B to use in results model_b: model B logger_cls: class of Logger to use base_name_to_sets_of_related_ops: optional override of subgraph base nodes, subject to change unmatchable_types_map: optional override of unmatchable types, subject to change Return: Returns a tuple of (model_a_with_loggers, model_b_with_loggers). Modifies both models inplace. z.quantization_api._numeric_suite_fx.add_loggersrd)rrrXrY) r.rOrPrArrerfrdrs)rprbrqrcrnrrrXrYrgrhrirjrUrWr)r)r* add_loggerss*     rt)rKrMrnrDc Cs:tjd|D]\}}t||p>t|tjjo>|jdk}|r|j}||krZi||<|j ||ksxt |j d|j ||kri|||j <|j |||j krg|||j |j <|j }t |jdkr|j}|||j |j |j ||j|j|j|j|j|j|jd |||j |j jdddqdS) NzAquantization_api._numeric_suite_fx._extract_logger_info_one_modelrz is already present in resultsr) typevaluesrrrrr!r"r#cSs|dd|dS)Nr":r!r))resr)r)r*sz0_extract_logger_info_one_model..)key)r.rOrPZ named_modulesr-ZjitZRecursiveScriptModule original_namerrAssertionErrorr rr3rr0rrrrr!r"r#sort)rKrMrnZgm_namemodZ is_loggerrzZ stats_to_user)r)r*_extract_logger_info_one_modelsD    r)rbrcrn!model_name_to_use_for_layer_namesrDcCs@tjdi}||fD]}t|||qt|t||}|S)a Traverse all loggers in `model_a` and `model_b`, and extract the logged information. Args: model_a: model A model_b: model B logger_cls: class of Logger to use model_name_to_use_for_layer_names: string name of model to use for layer names in the output Return: NSResultsType, containing the logged comparisons z6quantization_api._numeric_suite_fx.extract_logger_info)r.rOrPrr r )rbrcrnrrMrKr)r)r*extract_logger_infos  r) rprUrqrWrnrrrXnode_type_to_io_type_maprYrDc Cs6tjdt||||} t||||| |||d} | S)Nz;quantization_api._numeric_suite_fx._add_shadow_loggers_impl)rrr)r.rOrPrr ) rprUrqrWrnrrrXrrYr\Zgm_a_shadows_br)r)r*_add_shadow_loggers_impls$ r) rprbrqrcrnrrrXrrYrDc  Cstjdg} g} t| | } t| | } t|| |} t|drJ|j| _t|| |}t|drl|j|_t|| |||||||d S)a Instrument model A and model B with shadow loggers. Args: model_name_a: string name of model A to use in results model_a: model A model_name_b: string name of model B to use in results model_b: model B logger_cls: class of Logger to use should_log_inputs: whether to log inputs base_name_to_sets_of_related_ops: optional override of subgraph base nodes, subject to change unmatchable_types_map: optional override of unmatchable types, subject to change z5quantization_api._numeric_suite_fx.add_shadow_loggersrd)rrrXrrY) r.rOrPrArrerfrdr)rprbrqrcrnrrrXrrYrgrhrirjrUrWr)r)r*add_shadow_loggers&s,     r)model_a_shadows_brnrrDcCs<tjdtt}t|||t|t||}t|S)al Traverse all loggers in a shadow model, and extract the logged information. Args: model_a_shadows_b: shadow model logger_cls: class of Logger to use model_name_to_use_for_layer_names: string name of model to use for layer names in the output Return: NSResultsType, containing the logged comparisons z=quantization_api._numeric_suite_fx.extract_shadow_logger_info) r.rOrP collections defaultdictdictrr r )rrnrrMr)r)r*extract_shadow_logger_infoRs   r)rM model_name_1 model_name_2 comparison_fncomparison_namerDcCs|D]\}}|D]\}}||ks6t|d||ksLt|d||}||} | D]} | d} | d} d} |D],}|d}|d}|| kr||| kr||} qq|| dk st| d}| d}g| |<t||D] \}}|||}| ||qq`qqdS)ak Compares the logged values from `model_name_2` against the corresponding values in `model_name_1`, using `comparison_fn`. Records the result in `model_name_2`'s results under `comparison_name`. Modifies `results` inplace. Args: results: the result data structure from `extract_logger_info` or `extract_shadow_logger_info`. model_name_1: string name of model 1 model_name_2: string name of model 2 comparison_fn: function to compare two Tensors model_name_to_use_for_layer_names: string name of model to use for layer names in the output z not found in resultsr!r"Nrv)rZr|zipr0)rMrrrr_Zresults_type_to_resultsZmodel_name_to_resultsZ results_1Z results_2Zresult_2Zindex_within_arg_2Zindex_of_arg_2Zresult_1Z cur_result_1Zindex_within_arg_1Zindex_of_arg_1Zvalues_1Zvalues_2Zvalue_1Zvalue_2Zcomparison_resultr)r)r*%extend_logger_results_with_comparisonos<    r)N)NNN)NNN)NN)FNN)NNN)FNNN):r;rr.Ztorch.nnrHZ!torch.ao.quantization.quantize_fxrErFZ quantize_fxZtorch.fxrZtorch.fx.graphrZtorch.ao.ns.fx.mappingsrZtorch.ao.ns.fx.graph_matcherrrZfx.weight_utilsrZfx.graph_passesr r Zfx.utilsr r r Z fx.ns_typesrrrtypingrrrrrrr/r=rIrZQuantizationTracerrAr>rSrarkrorJrsrtrrrrrrr)r)r)r*sV      V  - 2  2 . 1 '  -