Da

✏️ Editing: charsetprober.cpython-37.pyc

����<_9Y�������������������@���s0���d�dl�Z�d�dlZddlmZ�G�dd��de�ZdS�)�����N����)�ProbingStatec���������������@���sn���e�Zd�ZdZddd�Zdd��Zedd���Zd	d
��Zedd���Z	d
d��Z
edd���Zedd���Z
edd���ZdS�)�
CharSetProbergffffff�?Nc�������������C���s���d�|�_�||�_t�t�|�_d�S�)N)�_state�lang_filter�logging�	getLogger�__name__�logger)�selfr�����r����F/opt/alt/python37/lib/python3.7/site-packages/chardet/charsetprober.py�__init__'���s����zCharSetProber.__init__c�������������C���s���t�j|�_d�S�)N)r���Z	DETECTINGr���)r���r���r���r
����reset,���s����zCharSetProber.resetc�������������C���s���d�S�)Nr���)r���r���r���r
����charset_name/���s����zCharSetProber.charset_namec�������������C���s���d�S�)Nr���)r����bufr���r���r
����feed3���s����zCharSetProber.feedc�������������C���s���|�j�S�)N)r���)r���r���r���r
����state6���s����zCharSetProber.statec�������������C���s���dS�)Ng��������r���)r���r���r���r
����get_confidence:���s����zCharSetProber.get_confidencec�������������C���s���t��dd|��}�|�S�)Ns���([�-])+���� )�re�sub)r���r���r���r
����filter_high_byte_only=���s����z#CharSetProber.filter_high_byte_onlyc�������������C���s`���t���}t�d|��}xH|D�]@}|�|dd����|dd��}|���sN|dk�rNd}|�|��qW�|S�)u9��
        We define three types of bytes:
        alphabet: english alphabets [a-zA-Z]
        international: international characters [-ÿ]
        marker: everything else [^a-zA-Z-ÿ]

The input buffer can be thought to contain a series of words delimited
        by markers. This function works to filter all words that contain at
        least one international character. All contiguous sequences of markers
        are replaced by a single space ascii character.

This filter applies to all scripts which do not use English characters.
        s%���[a-zA-Z]*[�-�]+[a-zA-Z]*[^a-zA-Z�-�]?N��������r���)�	bytearrayr����findall�extend�isalpha)r����filtered�words�wordZ	last_charr���r���r
����filter_international_wordsB���s����
z(CharSetProber.filter_international_wordsc�������������C���s����t���}d}d}x~tt|���D�]n}|�||d���}|dkr>d}n|dkrJd}|dk�r|���s||kr�|s�|�|�||����|�d��|d�}qW�|s�|�|�|d	����|S�)
a���
        Returns a copy of ``buf`` that retains only the sequences of English
        alphabet and high byte characters that are not between <> characters.
        Also retains English alphabet and high byte characters immediately
        before occurrences of >.

This filter can be applied to all scripts which contain both English
        characters and extended ASCII characters, but is currently only used by
        ``Latin1Prober``.
        Fr���r�������>����<Tr���r���N)r����range�lenr���r���)r���r���Zin_tag�prev�currZbuf_charr���r���r
����filter_with_english_lettersg���s"����
z)CharSetProber.filter_with_english_letters)N)r	����
__module__�__qualname__ZSHORTCUT_THRESHOLDr���r����propertyr���r���r���r����staticmethodr���r"���r)���r���r���r���r
���r���#���s���
%r���)r���r���Zenumsr����objectr���r���r���r���r
����<module>���s���

📁 Sid Gifari File Manager