Skip to content

Python cannot run in the ja_JP.sjis locale used windows-31j encoding. #102388

Closed
@moriyama

Description

@moriyama

Bug report

Linux using glibc cannot run Python when ja_JP.sjis locale is set as follows.

$ sudo dnf install glibc-locale-source # RHEL or RHEL compatible Linux distribution $ sudo localedef -f WINDOWS-31J -i ja_JP ja_JP.sjis $ export LANG=ja_JP.SJIS $ python3 Python path configuration: PYTHONHOME = (not set) PYTHONPATH = (not set) program name = 'python3' isolated = 0 environment = 1 user site = 1 import site = 1 sys._base_executable = '/usr/bin/python3' sys.base_prefix = '/usr' sys.base_exec_prefix = '/usr' sys.platlibdir = 'lib64' sys.executable = '/usr/bin/python3' sys.prefix = '/usr' sys.exec_prefix = '/usr' sys.path = [ '/usr/lib64/python39.zip', '/usr/lib64/python3.9', '/usr/lib64/python3.9/lib-dynload', ] Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding Python runtime state: core initialized LookupError: unknown encoding: WINDOWS-31J Current thread 0x00007f49d76ee740 (most recent call first): <no Python frame> 

The charset name "Windows-31J" is registered in the IANA Charset Registry[1].
Windows-31J is supported by perl[2], php[3], ruby[4], java[5], etc.
Python's cp932 is equivalent to Windows-31J, so I propose to add windows_31j to aliases for cp932.

[1] https://www.iana.org/assignments/charset-reg/windows-31J
[2] https://perldoc.perl.org/Encode::JP
[3] https://www.php.net/manual/en/mbstring.encodings.php
[4] https://docs.ruby-lang.org/ja/latest/class/Encoding.html
[5] https://docs.oracle.com/en/java/javase/19/intl/supported-encodings.html

Your environment

  • CPython versions tested on: 3.9.13, 3.12.0a5+
  • Operating system and architecture: MIRACLE LINUX 8.6 x86_64 (RHEL 8.6 compatible)

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    Status

    Done

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions