U &ºc ã@sŽddlmZmZddiZddddddd d œiZd d d d d d d œZd d id d id d id d id d id d id œZGdd„deƒZGdd„deƒZdS)é)Ú BertTokenizerÚBertTokenizerFastZ vocab_filez vocab.txtz\https://s3.amazonaws.com/models.huggingface.co/bert/google/electra-small-generator/vocab.txtz[https://s3.amazonaws.com/models.huggingface.co/bert/google/electra-base-generator/vocab.txtz\https://s3.amazonaws.com/models.huggingface.co/bert/google/electra-large-generator/vocab.txtz`https://s3.amazonaws.com/models.huggingface.co/bert/google/electra-small-discriminator/vocab.txtz_https://s3.amazonaws.com/models.huggingface.co/bert/google/electra-base-discriminator/vocab.txtz`https://s3.amazonaws.com/models.huggingface.co/bert/google/electra-large-discriminator/vocab.txt)zgoogle/electra-small-generatorzgoogle/electra-base-generatorzgoogle/electra-large-generatorz"google/electra-small-discriminatorz!google/electra-base-discriminatorz"google/electra-large-discriminatoriZ do_lower_caseTc@s eZdZdZeZeZeZ e Z dS)ÚElectraTokenizeraR Constructs an Electra tokenizer. :class:`~transformers.ElectraTokenizer` is identical to :class:`~transformers.BertTokenizer` and runs end-to-end tokenization: punctuation splitting + wordpiece. Refer to superclass :class:`~transformers.BertTokenizer` for usage examples and documentation concerning parameters. N© Ú__name__Ú __module__Ú __qualname__Ú__doc__ÚVOCAB_FILES_NAMESZvocab_files_namesÚPRETRAINED_VOCAB_FILES_MAPZpretrained_vocab_files_mapÚ&PRETRAINED_POSITIONAL_EMBEDDINGS_SIZESZmax_model_input_sizesÚPRETRAINED_INIT_CONFIGURATIONZpretrained_init_configuration©rrúE/tmp/pip-unpacked-wheel-ymerj3tt/transformers/tokenization_electra.pyr4s  rc@s eZdZdZeZeZeZ e Z dS)ÚElectraTokenizerFasta™ Constructs a "Fast" Electra Fast tokenizer (backed by HuggingFace's `tokenizers` library). :class:`~transformers.ElectraTokenizerFast` is identical to :class:`~transformers.BertTokenizerFast` and runs end-to-end tokenization: punctuation splitting + wordpiece. Refer to superclass :class:`~transformers.BertTokenizerFast` for usage examples and documentation concerning parameters. NrrrrrrDs  rN) Ztokenization_bertrrr r r r rrrrrrÚs4úÿ ú ú