U Kc.]@s"ddlmZddlmZddlmZmZmZmZm Z m Z m Z ddl Z ddl mZddlmZddlmZdd d d d gZd ZdZdZdZdZdZdZdZdZdZdZdZdZ dZ!dZ"dZ#dZ$dZ%dZ&d Z'd!Z(d"Z)Gd#d d eZ*eGd$d d Z+eGd%d d Z,Gd&ddZ-Gd'd d Z.dS)() annotations) dataclass)AnyCallableDictListOptionalTypeUnionN)_PartialWrapper)Pattern)Enum BackendConfigBackendPatternConfig DTypeConfigDTypeWithConstraintsObservationType input_dtype output_dtype weight_dtype bias_dtype is_dynamicnameconfigspatternobservation_type dtype_configs root_module qat_moduleZ#reference_quantized_module_for_root fused_module fuser_methodroot_node_getterextra_inputs_getter#num_tensor_args_to_observation_typeinput_type_to_indexinput_output_observedoverwrite_output_fake_quantizeoverwrite_output_observerc@seZdZdZdZdZdS)rzg An enum that represents different ways of how an operator/operator pattern should be observed rN)__name__ __module__ __qualname____doc__&OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUTZ OUTPUT_SHARE_OBSERVER_WITH_INPUTr.r.W/tmp/pip-unpacked-wheel-gikjz4vx/torch/ao/quantization/backend_config/backend_config.pyr3sc@sNeZdZUdZdZded<dZded<dZded<dZded<dZ ded <dS) rz Config for specifying additional constraints for a given dtype, such as quantization value ranges and scale value ranges, to be used in :class:`~torch.ao.quantization.backend_config.DTypeConfig`. NOptional[torch.dtype]dtypezUnion[int, float, None]quant_min_lower_boundquant_max_upper_boundscale_min_lower_boundscale_max_upper_bound) r)r*r+r,r1__annotations__r2r3r4r5r.r.r.r/rEs     c@seZdZUdZded<ded<ded<ded<ded <dd d d ddd d dZeddddZeddddZeddddZ e dddddZ ddddZ d S)ra Config for the set of supported input/output activation, weight, and bias data types for the patterns defined in :class:`~torch.ao.quantization.backend_config.BackendConfig`. Example usage:: >>> dtype_config1 = DTypeConfig( ... input_dtype=torch.quint8, ... output_dtype=torch.quint8, ... weight_dtype=torch.qint8, ... bias_dtype=torch.float) >>> dtype_config2 = DTypeConfig( ... input_dtype=DTypeWithConstraints( ... dtype=torch.quint8, ... quant_min_lower_bound=0, ... quant_max_upper_bound=255, ... ), ... output_dtype=DTypeWithConstraints( ... dtype=torch.quint8, ... quant_min_lower_bound=0, ... quant_max_upper_bound=255, ... ), ... weight_dtype=DTypeWithConstraints( ... dtype=torch.qint8, ... quant_min_lower_bound=-128, ... quant_max_upper_bound=127, ... ), ... bias_dtype=torch.float) >>> dtype_config1.input_dtype torch.quint8 >>> dtype_config2.input_dtype torch.quint8 >>> dtype_config2.input_dtype_with_constraints DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=0, quant_max_upper_bound=255, scale_min_lower_bound=None, scale_max_upper_bound=None) rinput_dtype_with_constraintsoutput_dtype_with_constraintsweight_dtype_with_constraintsr0rzOptional[bool]rNz.Union[torch.dtype, DTypeWithConstraints, None])rrrrrcCsjt|tr||_n t|d|_t|tr0||_n t|d|_t|trN||_n t|d|_||_||_dS)N)r1) isinstancerr7r8r9rr)selfrrrrrr.r.r/__init__s      zDTypeConfig.__init__returncCs|jjSN)r7r1r;r.r.r/rszDTypeConfig.input_dtypecCs|jjSr?)r8r1r@r.r.r/rszDTypeConfig.output_dtypecCs|jjSr?)r9r1r@r.r.r/rszDTypeConfig.weight_dtypeDict[str, Any])dtype_config_dictr>cCs|td}|dk r,t|tjtfs,td|td}|dk rXt|tjtfsXtd|td}|dk rt|tjtfstd|t d}|t d}||||||S)at Create a ``DTypeConfig`` from a dictionary with the following items (all optional): "input_dtype": torch.dtype or ``DTypeWithConstraints`` "output_dtype": torch.dtype or ``DTypeWithConstraints`` "weight_dtype": torch.dtype or ``DTypeWithConstraints`` "bias_type": torch.dtype "is_dynamic": bool Nz@Expected input_dtype to be a torch.dtype or DTypeWithConstraintszAExpected output_dtype to be a torch.dtype or DTypeWithConstraintszAExpected weight_dtype to be a torch.dtype or DTypeWithConstraints) getINPUT_DTYPE_DICT_KEYr:torchr1r ValueErrorOUTPUT_DTYPE_DICT_KEYWEIGHT_DTYPE_DICT_KEYBIAS_DTYPE_DICT_KEYIS_DYNAMIC_DICT_KEY)clsrBrrrrrr.r.r/ from_dicts     zDTypeConfig.from_dictcCsli}|jdk r|j|t<|jdk r,|j|t<|jdk r@|j|t<|j dk rT|j |t <|j dk rh|j |t <|S)z Convert this ``DTypeConfig`` to a dictionary with the items described in :func:`~torch.ao.quantization.backend_config.DTypeConfig.from_dict`. N) rr7rDrr8rGrr9rHrrIrrJ)r;rBr.r.r/to_dicts          zDTypeConfig.to_dict)NNNNN) r)r*r+r,r6r<propertyrrr classmethodrLrMr.r.r.r/rRs* (c@sreZdZdZdddddZddddd Zd dd d d ZdddddZedddddZ ddddZ dS)raConfig that defines the set of patterns that can be quantized on a given backend, and how reference quantized models can be produced from these patterns. A pattern in this context refers to a module, a functional, an operator, or a directed acyclic graph of the above. Each pattern supported on the target backend can be individually configured through :class:`~torch.ao.quantization.backend_config.BackendPatternConfig` in terms of: (1) The supported input/output activation, weight, and bias data types (2) How observers and quant/dequant ops are inserted in order to construct the reference pattern, and (3) (Optionally) Fusion, QAT, and reference module mappings. The format of the patterns is described in: https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/backend_config/README.md Example usage:: import torch from torch.ao.quantization.backend_config import BackendConfig, BackendPatternConfig, DTypeConfig, ObservationType from torch.ao.quantization.fuser_method_mappings import reverse_sequential_wrapper2 weighted_int8_dtype_config = DTypeConfig( input_dtype=torch.quint8, output_dtype=torch.quint8, weight_dtype=torch.qint8, bias_type=torch.float) linear_config = BackendPatternConfig(torch.nn.Linear) .set_observation_type(ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT) .add_dtype_config(weighted_int8_dtype_config) .set_root_module(torch.nn.Linear) .set_qat_module(torch.nn.qat.Linear) .set_reference_quantized_module(torch.nn.quantized._reference.Linear) conv_relu_config = BackendPatternConfig((torch.nn.ReLU, torch.nn.Conv2d)) .set_observation_type(ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT) .add_dtype_config(weighted_int8_dtype_config) .set_fused_module(torch.nn.intrinsic.ConvReLU2d) .set_fuser_method(reverse_sequential_wrapper2(torch.nn.intrinsic.ConvReLU2d)) backend_config = BackendConfig("my_backend") .set_backend_pattern_config(linear_config) .set_backend_pattern_config(conv_relu_config) strrcCs||_i|_dSr?)rrr;rr.r.r/r<szBackendConfig.__init__)rr>cCs ||_|S)z5 Set the name of the target backend. rRrSr.r.r/set_nameszBackendConfig.set_namer)configr>cCs||j|j<|S)z Set the config for an pattern that can be run on the target backend. This overrides any existing config for the given pattern. )rr)r;rUr.r.r/set_backend_pattern_config s z(BackendConfig.set_backend_pattern_configzList[BackendPatternConfig])rr>cCs|D]}||q|S)z Set the configs for patterns that can be run on the target backend. This overrides any existing config for a given pattern if it was previously registered already. )rV)r;rconfr.r.r/set_backend_pattern_configss z)BackendConfig.set_backend_pattern_configsrA)backend_config_dictr>cCsd||td}|tgD]B}t|tr6||qt|trR|t|qtdtq|S)z Create a ``BackendConfig`` from a dictionary with the following items: "name": the name of the target backend "configs": a list of dictionaries that each represents a `BackendPatternConfig` rPz5Expected backend_config_dict['%s'] to be a dictionary) rC NAME_DICT_KEYCONFIGS_DICT_KEYr:rrVrrLrF)rKrYrWdr.r.r/rLs    zBackendConfig.from_dictr=cCst|jtdd|jDiS)z Convert this ``BackendConfig`` to a dictionary with the items described in :func:`~torch.ao.quantization.backend_config.BackendConfig.from_dict`. cSsg|] }|qSr.rM.0cr.r.r/ 9sz)BackendConfig.to_dict..)rZrr[rvaluesr@r.r.r/rM2s zBackendConfig.to_dictN)rP) r)r*r+r,r<rTrVrXrOrLrMr.r.r.r/rs. c@s0eZdZdZddddZddddd Zd dd d d ZdddddZdddddZdddddZ dddddZ dddddZ ddd d!d"Z ddd#d$d%Z ddd&d'd(Zd)dd*d+d,Zd-dd.d/d0Zd1dd2d3d4Zd5dd6d7d8Zd5dd9d:d;Zedd?ZdcCs ||_|S)z Set how observers should be inserted for this pattern. See :class:`~torch.ao.quantization.backend_config.ObservationType` for details )r)r;rr.r.r/set_observation_typeVsz)BackendPatternConfig.set_observation_typer) dtype_configr>cCs|j||S)zo Add a set of supported input/output activation, weight, and bias data types for this pattern. )rappend)r;rlr.r.r/add_dtype_config_s z%BackendPatternConfig.add_dtype_configzList[DTypeConfig])rr>cCs ||_|S)z Set the supported input/output activation, weight, and bias data types for this pattern, overriding all previously registered data types. )r)r;rr.r.r/set_dtype_configsfsz&BackendPatternConfig.set_dtype_configszType[torch.nn.Module])rr>cCs ||_|S)z Set the module that represents the root for this pattern. For example, the root module for :class:`torch.nn.intrinsic.LinearReLU` should be :class:`torch.nn.Linear`. )r)r;rr.r.r/set_root_modulensz$BackendPatternConfig.set_root_module)rr>cCs ||_|S)zY Set the module that represents the QAT implementation for this pattern. )r)r;rr.r.r/set_qat_modulevsz#BackendPatternConfig.set_qat_module)rcr>cCs ||_|S)zw Set the module that represents the reference quantized implementation for this pattern's root module. )rc)r;rcr.r.r/set_reference_quantized_module}sz3BackendPatternConfig.set_reference_quantized_module)rr>cCs ||_|S)z[ Set the module that represents the fused implementation for this pattern. )r)r;rr.r.r/set_fused_modulesz%BackendPatternConfig.set_fused_moduler)r r>cCs ||_|S)z[ Set the function that specifies how to fuse the pattern for this pattern. )r )r;r r.r.r/set_fuser_methodsz%BackendPatternConfig.set_fuser_method)r!r>cCs ||_|Sr?)rd)r;r!r.r.r/_set_root_node_gettersz*BackendPatternConfig._set_root_node_getter)r"r>cCs ||_|Sr?)re)r;r"r.r.r/_set_extra_inputs_gettersz-BackendPatternConfig._set_extra_inputs_getterzDict[int, ObservationType])r#r>cCs ||_|Sr?)rf)r;r#r.r.r/(_set_num_tensor_args_to_observation_typesz=BackendPatternConfig._set_num_tensor_args_to_observation_typezDict[str, int])r$r>cCs ||_|Sr?)rg)r;r$r.r.r/_set_input_type_to_indexsz-BackendPatternConfig._set_input_type_to_indexbool)r%r>cCs ||_|Sr?)rh)r;r%r.r.r/_set_input_output_observedsz/BackendPatternConfig._set_input_output_observedr )r&r>cCs ||_|Sr?)ri)r;r&r.r.r/#_set_overwrite_output_fake_quantizesz8BackendPatternConfig._set_overwrite_output_fake_quantize)r'r>cCs ||_|Sr?)rj)r;r'r.r.r/_set_overwrite_output_observersz3BackendPatternConfig._set_overwrite_output_observerrA)backend_pattern_config_dictr>cCsBddddd}t|kr$tdt||t}t|krF||t|tgD]}|||qR||td| |t d| |t d| |td||td||td||td||ti||ti||td||td||td|S)a  Create a ``BackendPatternConfig`` from a dictionary with the following items: "pattern": the pattern being configured "observation_type": the :class:`~torch.ao.quantization.backend_config.ObservationType` that specifies how observers should be inserted for this pattern "dtype_configs": a list of dictionaries that represents :class:`~torch.ao.quantization.backend_config.DTypeConfig` s "root_module": a :class:`torch.nn.Module` that represents the root for this pattern "qat_module": a :class:`torch.nn.Module` that represents the QAT implementation for this pattern "reference_quantized_module": a :class:`torch.nn.Module` that represents the reference quantized implementation for this pattern's root module. "fused_module": a :class:`torch.nn.Module` that represents the fused implementation for this pattern "fuser_method": a function that specifies how to fuse the pattern for this pattern rr)objr>cSs:t|tr|St|tr"t|Stdtt|fdS)zo Convert the given object into a ``DTypeConfig`` if possible, else throw an exception. zNExpected a list of DTypeConfigs in backend_pattern_config_dict["%s"], got '%s'N)r:rrrLrFDTYPE_CONFIGS_DICT_KEYtype)r~r.r.r/_get_dtype_configs    z9BackendPatternConfig.from_dict.._get_dtype_configz-backend_pattern_config_dict must contain '%s'N)PATTERN_DICT_KEYrFOBSERVATION_TYPE_DICT_KEYrkrCrrnrpROOT_MODULE_DICT_KEYrqQAT_MODULE_DICT_KEYrr#REFERENCE_QUANTIZED_MODULE_DICT_KEYrsFUSED_MODULE_DICT_KEYrtFUSER_METHOD_DICT_KEYruROOT_NODE_GETTER_DICT_KEYrvEXTRA_INPUTS_GETTER_DICT_KEYrw,NUM_TENSOR_ARGS_TO_OBSERVATION_TYPE_DICT_KEYrxINPUT_TYPE_TO_INDEX_DICT_KEYrzINPUT_OUTPUT_OBSERVED_DICT_KEYr{'OVERWRITE_OUTPUT_FAKE_QUANTIZE_DICT_KEYr|"OVERWRITE_OUTPUT_OBSERVER_DICT_KEY)rKr}rrWr\r.r.r/rLs.    zBackendPatternConfig.from_dictr=cCs t|jt|jtdd|jDi}|jdk r4|j|t<|jdk rH|j|t <|j dk r\|j |t <|j dk rp|j |t <|jdk r|j|t<|jdk r|j|t<|jdk r|j|t<t|jdkr|j|t<t|jdkr|j|t<|jdk r|j|t<|jdk r|j|t<|jdk r|j|t<|S)z Convert this ``BackendPatternConfig`` to a dictionary with the items described in :func:`~torch.ao.quantization.backend_config.BackendPatternConfig.from_dict`. cSsg|] }|qSr.r]r^r.r.r/rasz0BackendPatternConfig.to_dict..Nr)rrrrrrrrrrrcrrrr rrdrrerlenrfrrgrrhrrirrjr)r;r}r.r.r/rMs@                      zBackendPatternConfig.to_dictN)r)r*r+r,r<rkrnrorprqrrrsrtrurvrwrxrzr{r|rOrLrMr.r.r.r/r=s( 1)/ __future__rZ dataclassesrtypingrrrrrr r rEZtorch.ao.quantization.observerr Ztorch.ao.quantization.utilsr enumr __all__rDrGrHrIrJrZr[rrrrrrrrrrrrrrrrrrrrr.r.r.r/sR  $     k