U Jc@sddlZddlZddlmZddlmZeejj Z eejj Z GdddeZ ejejdddZdd Zd d Zdd d ZdddZdddZdS)N)Enumc@s&eZdZdZdZdZedddZdS) DQuantTypez Different quantization methods for auto_quantize API are identified here. auto_quantize API currently supports fp16 and bfp16 methods. )Zfp16Zbfp16)returncCs|jSN)value)selfr[/tmp/pip-unpacked-wheel-gikjz4vx/torch/distributed/algorithms/_quantization/quantization.py__str__szDQuantType.__str__N)__name__ __module__ __qualname____doc__FP16BFP16strr rrrr r sr)tensorrcCst|ttSr)torchclampTORCH_HALF_MINTORCH_HALF_MAXZhalf)rrrr _fp32_to_fp16_with_clampsrcCs\t|tjstdt||tjkr0t|S|tjkrHtj j |Std|ddS)Nz;_quantize_tensor expecting torch.Tensor as input but found Quantization type  is not supported) isinstancerTensor RuntimeErrortyperrrrops quantizationZ_FloatToBfloat16Quantized)rqtyperrr _quantize_tensors     r!csDt|trtdd|Ds.tdt|fdd|D}|S)Ncss|]}t|tjVqdSrrrr.0prrr *sz(_quantize_tensor_list..zH_quantize_tensor_list expecting list of torch.Tensor as input but found csg|]}t|qSr)r!r$tr rr 0sz)_quantize_tensor_list..rlistallrr) tensor_listr Zquantized_tensor_listrr)r _quantize_tensor_list)s r/cCst|tjstdt||tjkrr|jtjkrHtd|jdq|jtjkrd|dkrd| S| |SnJ|tj kr|jtjkrtd|jdqtj j |Sntd|ddS)Nz=_dequantize_tensor expecting torch.Tensor as input but found ztensor dtype is z while expected to be FP16.rr)rrrrrrrZdtypefloat16floatrrrZ_Bfloat16QuantizedToFloat)rr quant_lossrrr _dequantize_tensor3s*         r3csDt|trtdd|Ds.tdt|fdd|D}|S)Ncss|]}t|tjVqdSrr"r#rrr r&Osz*_dequantize_tensor_list..zJ_dequantize_tensor_list expecting list of torch.Tensor as input but found csg|]}t|qSr)r3r'r)rr r*Usz+_dequantize_tensor_list..r+)r.r r2Zdequantized_tensor_listrr)r _dequantize_tensor_listNs r4cstfdd}|S)a This is a prototype API that automatically quantize the input tensors, choose the precision types, and pass other necessary arguments and then dequantizes the output. Currently it only supports: . FP16 and BFP16 quantization method supported for gloo and nccl backends . all_gather, all_to_all collective ops Note: BFP16 only supports 2D tensors. Args: func (Callable): A function representing collective operations. qtype (QuantType): Quantization method quant_loss (float, optional): This can be used to improve accuracy in the dequantization. Returns: (Callable): the same collective as func but enables automatic quantization/dequantization. c s|dd}|dd}|dkr(tdtjkr|d}t|d}t|}tj||||dtt|d D]\}}|||<qvntjkr|d}t|d}t|}tj||||dtt|d D]\}}|||<qntj krl|d}|d d} |d d} t|d}t|}tj ||| | |d tt |d D]\}}|||<qVntd ddS)Ngroupasync_opFTz,The async_op=True mode is not supported yet.r)r5r6)r2 out_splits in_splits)r5zThe collective op z is not supported yet) getrdistZ all_gatherr!r/ enumerater4Z all_to_allZall_to_all_singler3) argskwargsr5r6ZtensorsZ input_tensorsZ out_tensorsir(r8r9funcr r2rr wrapperhs@             zauto_quantize..wrapper) functoolswraps)rAr r2rBrr@r auto_quantizeYs&rE)N)N)N)rCrZtorch.distributedZ distributedr;enumrZfinfor0minrmaxrrrrr!r/r3r4rErrrr s