U JcT@sddlZddlmZddlmZmZmZmZmZm Z ddl Z ddl Z ddl m Z ddlmZddlmZddlmZdd lmZmZmZmZmZmZmZd d d gZeeZe d dGdd d e Z!e d dGdd d e Z"e d dGdd d e Z#eGdddZ$GdddZ%dS)N) dataclass)AnyCallableDictListOptionalTuple) compatibility)map_arg) ShapeProp) split_by_tags)CALLABLE_NODE_OPSFxNetAccFusionsFinderNamesNodeListNodeSetTensorOrTensorsTensorsFxNetMinimizerBadModuleErrorFxNetMinimizerRunFuncError!FxNetMinimizerResultMismatchErrorF)Zis_backward_compatiblec@seZdZdZdS)rz9 Raised if failed to split out a minimize module N__name__ __module__ __qualname____doc__rr@/tmp/pip-unpacked-wheel-gikjz4vx/torch/fx/passes/net_min_base.pyrsc@seZdZdZdS)rz@ Raised if error occurs during run_a or run_b functions Nrrrrrr(sc@seZdZdZdS)rzJ Raised if comparing function thinks the results are mismatching. Nrrrrrr1sc@sJeZdZUdZdZeed<dZeed<dZ eed<dZ eed<dd Z d S) _MinimizerSettingBasea; Args: `accumulate_error`: Instead of using a's input for both converted module to verify , use the previous outputs of each converted module as input to accumulate the errors. `traverse_method`: "sequential" or "binary" or "accumulate" Determine the way of traverse the nodes in FX module. `find_all`: Minimizer will go through the entire model and return all problematic nodes. `return_intermediate`: If true, when using `run_nodes()` function to run the model, intermediate results of all the ops will be returned as output. Faccumulate_error sequentialtraverse_methodfind_allreturn_intermediatecCs4d}t|D]\}}|d|d|d7}q|S)NzFX Minimizer Settings:  z:  )varsitems)selfZ settings_strkvrrr__str__Psz_MinimizerSettingBase.__str__N) rrrrr bool__annotations__r"strr#r$r,rrrrr:s     rc@seZdZdZejjeee e e ge e e ffedddZejjee dddZejjee ddd Ze e ejjd d d Zejjee eefd ddZedddZee ejjefdddZejjee dddZeeeedddZeedddZeedddZeedd d!Zeeeeed"d#d$Z d0eeeed&d'd(Z!e"ed)d*d+Z#d,d-Z$d1eeeeed"d.d/Z%d%S)2_MinimizerBasea This class is used to automatically find problematic nodes in a model. It takes a FX graphmodule and generate some submodules while traverse the graph. Then two functions `run_a` and `run_b` will be used to run the same submodule and a function `compare_fn` will be used to compare the results. Currently we provides two ways to traverse the graph and generate submodules. 1. Sequential traversal: this will traverse the graph node by node and generate one submodule with one sigle node. 2. Binary searching: this will do a binary search style traversal on the graph. For internal Users, a guide can be found here https://fb.quip.com/HDtuAgiKGfkP. )module sample_input compare_fnsettingsc Cst|tjjst||_||_||_||_i|_ i|_ i|_ g|_ d|_ dd|jjjD}t|jj|jt|j||_dd|jjjD}t|t|jkstt|D]$\}}|||j |<|||j |<qdS)NrcSsh|]}|jtkr|qSr)opr.0noderrr s z*_MinimizerBase.__init__..cSsg|]}|jdkr|jqS placeholderr5namer6rrr s z+_MinimizerBase.__init__..) isinstancetorchfx GraphModuleAssertionErrorr1r2r3r4 a_outputs b_outputsresultsreports iterationgraphnodesr propagaterfusionslen enumerate) r)r1r2r3r4Zcallable_nodes placeholdersir=rrr__init__hs, z_MinimizerBase.__init__)modinputsreturncCs tddS)zz Run `mod` with `inputs` and generate output. The output will be compared with output of run_b(). zrun_a() is not implemented.N RuntimeErrorr)rRrSrrrrun_asz_MinimizerBase.run_acCs tddS)zz Run `mod` with `inputs` and generate output. The output will be compared with output of run_a(). zrun_b() is not implemented.NrUrWrrrrun_bsz_MinimizerBase.run_b)a_resultb_result submodulecCstdd|jjD}t|jdtjjrP||j|jdj <||j |jdj <n8t |jdD](\}}|||j|j <|||j |j <q^dS)a Store the outputs of self.run_a() and self.run_b() into self.a_outputs and self.b_outputs, so that we can use them when execute preceding nodes that use those outputs as inputs. Args: a_result: Output of self.run_a(). Could be a tensor or tensors. b_result: Output of self.run_b(). Could be a tensor or tensors. submodule: The module that generates a_result and b_result. css|]}|jdkr|VqdS)outputN)r5r6rrr s z0_MinimizerBase._store_outputs..rN) nextrIrJr?argsr@rANoderDr=rErN)r)rZr[r\Z output_noderPargrrr_store_outputssz_MinimizerBase._store_outputs) main_module submod_pathrTc sgg}t||}dd|jjD}t||jkrb|D]$}|j|||j|q:nP|jj rzt d|dt j j tdfdd }||}||j|}|jj sˆfS|fS)al Try get submodule inputs from stored outputs. If not found then use torch_glow.get_submod_inputs to get the inputs. If accumulate_error is False, use a_input for run_a() and run_b() otherwise use a_input for run_a and b_input for run_b. Args: main_module: Top-levlel fx module. submod_path: Path to the submodule we want to run and compare results. Returns: a_input: List of tensor(s) that will be used by run_a() as submodule inputs. b_input: List of tensor(s) that will be used by run_b() as submodule inputs. cSsg|]}|jdkr|jqSr:r<r6rrrr>s z5_MinimizerBase._get_submod_inputs..z)Can't find previous stored outputs named !r)rScs|dSNrrga_inputrr get_inputssz5_MinimizerBase._get_submod_inputs..get_inputs)getattrrIrJsetrDkeysappendrEr4r printr@nnModulerZregister_forward_pre_hookr2remove) r)rdreb_inputr\rOr=rkhandlerrir_get_submod_inputss(   z!_MinimizerBase._get_submod_inputs)selected_nodescCsR|jjjD]B}|jtkrq ||kr*d|_q tdd|jDrFd|_q d|_q dS)ag Tag selected nodes with tag "minimize". Nodes with the same tags will be split to the same submodule afterwards. Args: selected_nodes: Nodes that we want to minimize. We will tag those nodes with "minimize", all preceding nodes with "main_0" and all following nodes with "main_1". minimizecss"|]}|jtkr|jdkVqdS)>main_1rxN)r5rtag)r7nrrrr^s z,_MinimizerBase._tag_nodes..rymain_0N)r1rIrJr5rrzanyZall_input_nodes)r)rwr8rrr _tag_nodess  z_MinimizerBase._tag_nodes)rJrTcCsv||t|jdddg}d}|D].\}}d|kr:q(|dkrH|}q(td|q(|dkrntd|||fS)ak Split self.module so that one submodule consists of `nodes` and only `nodes`. Args: nodes: Nodes that we want to include in the minimize submodule. Returns: split_module (torch.fx.GraphModule): the module after split. submodule_name (str): the name of the submodule that consists of `nodes`. r|rxryz0Expected only one minimize submodule with nodes z,Minimize submodule was not found with nodes )r~r r1Znamed_childrenr)r)rJ split_moduleZsubmodule_nameZ child_name_rrr_build_submodules z_MinimizerBase._build_submodule)r submod_name output_namescCst||}|||\}}t|jdkr:|jgd|_|j|jd}|d|rg}|jjD].} | jdkr~|j | | j |krd|| qd|j t|dkr|dnt ||j ||jjD]} | jdkrt| jdd} q|||} |||} || | ||} |dkr4dd | D} || | | \}}||j| <|d ||s|d | td | dS) a Run the submodule in `split_module` that has name `submod_name` using `self.run_a` and `self.run_b` and compare their results. Args: split_module: Main module that contains the minimize submodule. submod_name: Name of the minimize submodule. output_names: Names of the node we want to output. If None, we will use the original output. rr zRun and compare ...r]cSs|jSrhr=)xrrr`z1_MinimizerBase._run_and_compare..NcSsg|] }t|qSr)r/)r7r+rrrr>isz3_MinimizerBase._run_and_compare..zNumerical accuracy = zResult mismatch for )rlrvrMrGrorHrIrJr5Z erase_noder=r]tupleZlintZ recompiler r`rXrYrcr3rFr)r)rrrr\rjrtreportZ output_nodesr8Z result_keyrZr[namesZnumeric_resultZ bool_resultrrr_run_and_compare7sD               z_MinimizerBase._run_and_compare) all_nodes start_idxend_idxrTc Cs|||}g}|j||jd7_|d|jd|d|d|ddt|t|}|D]}||jkrn||j|qnz ||\}} ||| gWnt t fk rt|dkr|d|d| ||YS|d| |t|d } | |||| } t| d krF|j jsF| YS| ||| |} t| d kr|d || || YSX|d | |tSd S)z9 Recursive binary search implementation. r zBinary search iteration .zFrom node index z to z&. Size of the interested node list is zcThis is the last node in the sub-module. Search in the current branch is successful with culprit = zMProceed to split and lower the halves of the current sub-module individually.rzaFurther split and lowering found no errors. Unable to minimize the submodule with list of nodes: zNo discrepancy found.N)rGrorHrMrmrLupdaterrrr print_report_binary_search_implr4r#) r)rrrrJr cur_nodesr8rrZmidculpritsrrrrssP            z"_MinimizerBase._binary_search_implcCs||dt|S)z7 Binary search on `nodes` for culprit. r)rrM)r)rJrrr_binary_traversesz_MinimizerBase._binary_traversec CsPt}|D]>}g}|j||jd7_|d|jd|d|jtd|j|h}||jkr|j|}z.||\}}| |||jg| |Wq t k r| ||d|| ||j js|YSYq tk rH|||d|| ||j jsD|YSYq Xq |S)zX Traverse `nodes` one by one and determine if any of them is a culprit. r zSequential traverse iteration rz Visit node: z"Found culprit from numeric error: zFound culprit from run error: )rmrGrorHr=_LOGGERinforLrrrraddr4r#rr)r)rJrr8rrrrrrr_sequential_traverses8         z#_MinimizerBase._sequential_traversec Cs0t}t}|jjr td|S|D]}g}|j||jd7_|d|jd|||j}|dk rt |t r|d}|dk rt |t st d||d|z,| |\}}||||g||Wq$ttfk r(|||d||||YSXq$|S) Nz9'Find All' mode is not supported in accumulate traversal.r zAccumulate traverse iteration rrzminimize: node_name: z Add node: zFound culprit )rmr4r#rprGrorHrr=r?rr/rCrrrrr) r)rJrZ nodes_to_runr8rZ node_namerrrrr_accumulate_traverses>      z#_MinimizerBase._accumulate_traverse)startendrTcCsVg}|dk}|jjjD]:}|jtkr&q|j|kr4d}|rB|||j|krqRq|S)z Collect nodes in the model that between nodes with name of `start` and `end`. These two nodes are also included. NT)r1rIrJr5rr=ro)r)rrrJadd_noder8rrr_collect_nodess    z_MinimizerBase._collect_nodesN)rrc Cs|||}t|}|D]}||jkr||j|qg}|jjrRdd|D}z ||\}}||||Wn.tt fk r} z t | W5d} ~ XYnXdS)a] Run part of the model from `start` node to `end` node. If `start` is None then we start from the beginning of the model. If `end` is None then we stop at the end of the model. Args: start: The name of the node which is the first node of the submodule we want to run. If set to None, then we'll start with the first node of the model. end: The name of the node which is the last node of the submodule we want to run. If set to None, we'll end with the last node of the model. cSsg|] }|jqSrrr6rrrr>3sz,_MinimizerBase.run_nodes..N) rrmrLrr4r$rrrrrp) r)rrrJrr8rrrerrr run_nodess   z_MinimizerBase.run_nodes)rcCs<tt|D]*}|dkr*td||q t||q dS)Nrz . )rangerMrp)r)rrPrrrr>sz_MinimizerBase.print_reportcCs|jD]}||qdSrh)rGr)r)rrrr print_reportsEs z_MinimizerBase.print_reportscCs|t|jt|jj|||}|jjdkr8||S|jjdkrN||S|jjdkrd||St d|jjddS)a Minimizing the model from node with name `start` to node with name `end` base on self.settings. Find culprits that causes FxNetMinimizerRunFuncError or FxNetMinimizerResultMismatchError errors. Args: start: The name of the node where we want to start minimizing. If set to None, then we'll start with the first node of the model. end: The name of the node where we want to terminate minimizing. If set to None, we'll end with the last node of the model. Returns: nodes: A list of nodes that causes FxNetMinimizerRunFuncError or FxNetMinimizerResultMismatchError errors during minimizing. r!binary accumulatezUnknow traverse method rfN) rpr4r1rIrr"rrrrV)r)rrrJrrrrxIs         z_MinimizerBase.minimize)NN)NN)&rrrrr@rArBrrrrrfloatr-rrQrXrYrcr/rvrr~rrrintrrrrrrrrrrrxrrrrr0YsV 0   4' = ?&'"r0)&loggingZ dataclassesrtypingrrrrrrr@Ztorch.fxZtorch.fx._compatibilityr Z torch.fx.noder Z shape_propr Z split_utilsr Z tools_commonrrrrrrr__all__ getLoggerrr Exceptionrrrrr0rrrrs.      $