U &cs@sddlZddlZddlZddlmZddlmZddlmZddl m Z m Z m Z m Z mZe de Ze de ZGdd d eZdS) N)ArgumentParser)Enum)Path)AnyIterableNewTypeTupleUnion DataClass DataClassTypecs~eZdZUdZeeed<eeeefdfdd ZedddZ de e d fd ddZ e e e d fdddZZS)HfArgumentParserae This subclass of `argparse.ArgumentParser` uses type hints on dataclasses to generate arguments. The class is designed to play well with the native argparse. In particular, you can add more (non-dataclass backed) arguments to the parser after initialization and you'll get the output back after parsing as an additional namespace. dataclass_types)r c s>tjf|t|r|g}||_|jD]}||q*dS)a! Args: dataclass_types: Dataclass type, or list of dataclass types for which we will "fill" instances with the parsed args. kwargs: (Optional) Passed to `argparse.ArgumentParser()` in the regular way. N)super__init__ dataclassesZ is_dataclassr _add_dataclass_arguments)selfr kwargsdtype __class__=/tmp/pip-unpacked-wheel-ymerj3tt/transformers/hf_argparser.pyrs   zHfArgumentParser.__init__)rcCs6t|D]$}d|j}|j}t|jtr:tdt|j}t t tfD]}|d|j dkrN||_qNt|jtrt |jt rt|j|d<|j|d<|jtjk r|j|d<nn|jtkr|jdkrd nd |d <|jdkrd |j}|j|d <n,|j|d<|jtjk r|j|d<nd|d<|j|f|q dS)Nz--zThis implementation is not compatible with Postponed Evaluation of Annotations (PEP 563),which can be opted in from Python 3.7 with `from __future__ import annotations`.We will add compatibility when Python 3.9 is released.z typing.Union[z , NoneType]choicestypedefaultT store_false store_trueactionz--no-destrequired)rfieldsnamemetadatacopy isinstancerstr ImportErrorintfloat__name__ issubclassrlistrMISSINGbool add_argument)rrfield field_namerZ typestringxrrrr*s4             z)HfArgumentParser._add_dataclass_argumentsNFT.)returnc s|rXttjrXttjdd}|rX|}|dk rF||n|tjdd}|j|d\}}g}|j D]Z} ddt | Dfdd t | D} D]} t|| q| f| } || qrt|jdkr|||r||fS|SdS) a Parse command-line args into instances of the specified dataclass types. This relies on argparse's `ArgumentParser.parse_known_args`. See the doc at: docs.python.org/3.7/library/argparse.html#argparse.ArgumentParser.parse_args Args: args: List of strings to parse. The default is taken from sys.argv. (same as argparse.ArgumentParser) return_remaining_strings: If true, also return a list of remaining argument strings. look_for_args_file: If true, will look for a ".args" file with the same base name as the entry point script for this process, and will append its potential content to the command line args. Returns: Tuple consisting of: - the dataclass instances in the same order as they were passed to the initializer.abspath - if applicable, an additional namespace for more (non-dataclass backed) arguments added to the parser after initialization. - The potential list of remaining argument strings. (same as argparse.ArgumentParser.parse_known_args) rz.argsN)argscSsh|] }|jqSrr".0frrr usz?HfArgumentParser.parse_args_into_dataclasses..csi|]\}}|kr||qSrrr8kvkeysrr vsz@HfArgumentParser.parse_args_into_dataclasses..)lensysargvr with_suffixexists read_textsplitparse_known_argsr rr!varsitemsdelattrappend__dict__) rr5Zreturn_remaining_stringsZlook_for_args_file args_fileZfargs namespaceremaining_argsoutputsrinputsr<objrr>rparse_args_into_dataclassesLs& "      z,HfArgumentParser.parse_args_into_dataclasses) json_filer3csftt|}g}|jD]B}ddt|Dfdd|D}|f|}||q|S)z Alternative helper method that does not use `argparse` at all, instead loading a json file and populating the dataclass types. cSsh|] }|jqSrr6r7rrrr:sz3HfArgumentParser.parse_json_file..csi|]\}}|kr||qSrrr;r>rrr@sz4HfArgumentParser.parse_json_file..) jsonloadsrrFr rr!rJrL)rrUdatarQrrRrSrr>rparse_json_files   z HfArgumentParser.parse_json_file)NFT)r* __module__ __qualname____doc__rr __annotations__r rrrr rTr&rY __classcell__rrrrr s  # 7r )rrVrBargparserenumrpathlibrtypingrrrrr r r r rrrrs