U Kcm5@sddlZddlmZddlmZddlmZmZddlmZddlm Z ddd gZ Gd ddejj Z Gd ddejj Z Gd d d e ZdS) N)Tensor)OptionalList)hide_packed_params_repr)_quantize_weightEmbeddingPackedParams Embedding EmbeddingBagcsxeZdZdZejffdd Zejjej ddddZ ejjdd Z d d Z fd d Z fddZddZZS)rrcs~tt|||_|jtjtjfkrltj|tjd}tj |tjd}tj ||g||d|jd}| |nt d|dS)Ndtyperscales zero_pointsZaxisr zSUnsupported dtype on quantized embedding! Supports quint8 and quint4x2. Got dtype: ) superr__init__r torchquint8quint4x2onesfloatzeros#_empty_per_channel_affine_quantized set_weightNotImplementedError)selfnum_embeddings embedding_dimr rrZwq __class__O/tmp/pip-unpacked-wheel-gikjz4vx/torch/ao/nn/quantized/modules/embedding_ops.pyrs  zEmbeddingPackedParams.__init__N)weightreturncCs0|jtjtjfkr$tjj||_ntddS)NzPUnsupported dtype for quantized embedding prepack! Supports quint8 and quint4x2.) r rrrops quantizedZembedding_bag_prepack_packed_weightr)rr"r r r!rsz EmbeddingPackedParams.set_weightcCs.|jtjtjfkr"tjj|jStddS)NzOUnsupported dtype for quantized embedding unpack! Supports quint8 and quint4x2.) r rrrr$r%Zembedding_bag_unpackr&rrr r r!_weight#szEmbeddingPackedParams._weightcCs|SNr )rxr r r!forward*szEmbeddingPackedParams.forwardcs6tt|||||j||d<|||d<dS)Nr r&)rr_save_to_state_dictr r()rZ destinationprefixZ keep_varsrr r!r,2sz)EmbeddingPackedParams._save_to_state_dictc s`||d|_||d||d}||d||tt||||d|||dS)Nr r&F)r poprrr_load_from_state_dict) rZ state_dictr-Zlocal_metadatastrictZ missing_keysZunexpected_keysZ error_msgsr"rr r!r/7s  z+EmbeddingPackedParams._load_from_state_dictcCs |Sr))r(__repr__r'r r r!r1CszEmbeddingPackedParams.__repr__)__name__ __module__ __qualname___versionrrrZjitexportrrr(r+r,r/r1 __classcell__r r rr!r s    c seZdZdZdZddddddejfeeeeee e e e ee dd fdd Z e e d d d Z d d ZddZddZej ddddZddZeddZeddZZS)r a& A quantized Embedding module with quantized packed weights as inputs. We adopt the same interface as `torch.nn.Embedding`, please see https://pytorch.org/docs/stable/nn.html#torch.nn.Embedding for documentation. Similar to :class:`~torch.nn.Embedding`, attributes will be randomly initialized at module creation time and will be overwritten later Attributes: weight (Tensor): the non-learnable quantized weights of the module of shape :math:`(\text{num\_embeddings}, \text{embedding\_dim})`. Examples:: >>> m = nn.quantized.Embedding(num_embeddings=10, embedding_dim=12) >>> indices = torch.tensor([9, 6, 5, 7, 8, 8, 9, 2, 8]) >>> output = m(indices) >>> print(output.size()) torch.Size([9, 12]) rN@F) rr padding_idxmax_norm norm_typescale_grad_by_freqsparser(r#c stt|||_||_| |_|dkrdtj|tjd} tj |tjd} tj ||g| | dtj d} nt |j ||gks~td|} t||| |_|j| dS)Nr rr z?Shape of weight does not match num_embeddings and embedding_dim)rr rrrr rrrrrrlistshapeAssertionErrorr_packed_paramsr) rrrr9r:r;r<r=r(r rrqweightrr r!r]s& zEmbedding.__init__)indicesr#cCs8|jtjkr tjj|jj|Stjj|jj|SdSr)) r rrr$r%Zembedding_4bitrAr&Zembedding_byte)rrCr r r!r+ss zEmbedding.forwardcCsdS)NZQuantizedEmbeddingr r'r r r! _get_nameyszEmbedding._get_namecCs t|tSr))rrr'r r r!r1|szEmbedding.__repr__cCs$d|j|j|jj|}|S)Nz9num_embeddings={}, embedding_dim={}, dtype={}, qscheme={})formatrrrAr r"qscheme)rZextra_repr_strr r r! extra_reprs zEmbedding.extra_repr)wr#cCs|j|dSr))rAr)rrHr r r!rszEmbedding.set_weightcCs |jSr))rAr(r'r r r!r"szEmbedding.weightc Cs:t|drPt|tjjjjksBtd|jddtjjjjj|j }|j }npt|tjksxtd|jdtjjt|dstddd l m }|j d k r|j jd k r|j }n|}|j}|jtjk}|std |tjks|tjkstd |||jt|j|}t|j|j}|||S) zCreate a quantized embedding module from a float module Args: mod (Module): a float module, either produced by torch.ao.quantization utilities or provided by user weight_fake_quantnnq.z .from_float zwith fake quant only works for .from_float only works for qconfigz6Embedding input float module must have qconfig definedr!float_qparams_weight_only_qconfigNzPEmbedding quantization is only supported with float_qparams_weight_only_qconfig.zSThe only supported dtype for nnq.Embedding is torch.quint8 and torch.quint4x2, got )hasattrtyperZaonnZqatr r@r2rIactivation_post_processZtorch.ao.quantizationrNrLr"r rF per_channel_affine_float_qparamsrrrrrrr) clsmodweight_observerrRrNr is_float_qparams_qconfigrB qembeddingr r r! from_floats: "      zEmbedding.from_floatc Cs0||j|j|j|j|j|j|j||j }|Sr)) rrr9r:r;r<r=get_quantized_weight weight_dtype)rTZ ref_embeddingrXr r r!from_references zEmbedding.from_reference)r2r3r4__doc__r5rrintrrboolrrr+rDr1rGrr" classmethodrYr\r7r r rr!r Fs6  'c seZdZdZdZdddddddejfeeee e e e e ee e dd fdd Z de ee ee ee e d d d Zd dZeddZeddZZS)r a A quantized EmbeddingBag module with quantized packed weights as inputs. We adopt the same interface as `torch.nn.EmbeddingBag`, please see https://pytorch.org/docs/stable/nn.html#torch.nn.EmbeddingBag for documentation. Similar to :class:`~torch.nn.EmbeddingBag`, attributes will be randomly initialized at module creation time and will be overwritten later Attributes: weight (Tensor): the non-learnable quantized weights of the module of shape :math:`(\text{num\_embeddings}, \text{embedding\_dim})`. Examples:: >>> m = nn.quantized.EmbeddingBag(num_embeddings=10, embedding_dim=12, include_last_offset=True, mode='sum') >>> indices = torch.tensor([9, 6, 5, 7, 8, 8, 9, 2, 8, 6, 6, 9, 1, 6, 8, 8, 3, 2, 3, 6, 3, 6, 5, 7, 0, 8, 4, 6, 5, 8, 2, 3]) >>> offsets = torch.tensor([0, 19, 20, 28, 28, 32]) >>> output = m(indices, offsets) >>> print(output.size()) torch.Size([5, 12]) rNr8Fsum) rrr:r;r<moder=r(include_last_offsetr#c s4tt|j|||| d||_d|_| |_| |_dS)N)r(r F)rr rrbpruned_weightsrcr ) rrrr:r;r<rbr=r(rcr rr r!rs zEmbeddingBag.__init__)rCoffsetsper_sample_weightscompressed_indices_mappingr#c Cs\|jtjkr2tjj|jj||dd|j|||j Stjj |jj||dd|j|||j SdS)NFr) r rrr$r%Zembedding_bag_4bitrAr&rdrcZembedding_bag_byte)rrCrerfrgr r r!r+s zEmbeddingBag.forwardcCsdS)NZQuantizedEmbeddingBagr r'r r r!rDszEmbeddingBag._get_namecCst|dr|j}npt|tjks:td|jdtjjt|dsLtdddlm}|j dk rz|j j dk rz|j }n| }|j }|j t jk}|std |t jks|t jkstd |||j t|j |}t|j|j|d }|||S) zCreate a quantized embedding_bag module from a float module Args: mod (Module): a float module, either produced by torch.ao.quantization utilities or provided by user rIrJrKrLz9EmbeddingBag input float module must have qconfig definedrrMNzSEmbeddingBag quantization is only supported with float_qparams_weight_only_qconfig.zVThe only supported dtype for nnq.EmbeddingBag is torch.quint8 and torch.quint4x2, got r )rOrIrPrQr r@r2Ztorch.ao.quantization.qconfigrNrLr"r rFrrSrrrrrrr)rTrUrVrNr rWrBqembedding_bagr r r!rYs.      zEmbeddingBag.from_floatc Cs4||j|j|j|j|j|j|j||j|j }|Sr)) rrr:r;r<rbr=rZrcr[)rTZref_embedding_bagrhr r r!r\s zEmbeddingBag.from_reference)NNN)r2r3r4r]r5rrr^rrr_strrrr+rDr`rYr\r7r r rr!r s>   $)rZtorch.nnrQrZtorch._jit_internalrrutilsrr__all__Modulerr r r r r r!s     ;}