test_cmd_line_script and test_embed, test_compileall, test_urllib fail if TMPDIR path is non-ASCII

[vstinner]

$ mkdir nonasciié
$ TMPDIR=$PWD/nonasciié/ TEMP=$PWD/nonasciié/ ./python -m test test_urllib -v -m test_file_notexists 
(...)
ERROR: test_file_notexists (test.test_urllib.urlopen_HttpTests.test_file_notexists)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/urllib/parse.py", line 1004, in _to_bytes
    url = url.encode("ASCII").decode()
          ^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 52: ordinal not in range(128)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/test/test_urllib.py", line 535, in test_file_notexists
    with urlopen(tmp_fileurl) as fobj:
         ^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/test/test_urllib.py", line 54, in urlopen
    return opener.open(url)
           ^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 1760, in open
    fullurl = unwrap(_to_bytes(fullurl))
                     ^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/urllib/parse.py", line 1006, in _to_bytes
    raise UnicodeError("URL " + repr(url) +
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeError: URL 'file://localhost//home/vstinner/python/main/nonasciié/tmp3e6una03' contains non-ASCII characters

cc @serhiy-storchaka

[vstinner]

The following tests fail with non-ASCII TMPDIR:

• test_urllib:

  • test_file_notexists()

• test_embed:

  • test_init_pybuilddir()

• test_compileall:

  • test_include_on_stdin()
  • test_include_on_stdin()
  • test_error()

• test_cmd_line_script:

  • test_pep_409_verbiage()
  • test_syntaxerror_indented_caret_position()
  • test_syntaxerror_unindented_caret_position()

[vstinner]

I merged my PR #93909 to create the temporary directory in /tmp but with an ASCII name. Once this issue will be fixed, we can add again os_helper.FS_NONASCII suffix to the temporary directory name to detect such bug.

[vstinner]

Sadly, I don't have the bandwidth to work on this issue, so I just close it.