On Windows 10, Launch of LLDB gets error regarding init_fs_encoding

[KimTheLearner]

Issue Description

On Windows 10, the launch of Swift's LLDB gets error:

Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Possibly related, but perhaps this was submitted to the wrong project: vadimcn/codelldb#566

Note that I had previously accepted the VS Code prompt that created this in my workspace settings.json file, which correctly points to the Swift custom LLDB:

{
    "lldb.library": "C:\\Library\\Developer\\Toolchains\\unknown-Asserts-development.xctoolchain\\usr\\bin\\liblldb.dll"
}

And hey, thanks so much for this work!!

Environment

OS: Windows 10
VSCode version: 1.65.2
Swift extension version: 0.5.1 (released just yesterday)
CodeLLDB version: 1.7.0
Compiler: Swift 5.6.1

Test Case

md hello
cd hello
swift package init --type=executable
place these lines into main.swift:

print("Hello, world!")
print()
print()

open folder with VS Code
compile with: Terminal > Build Debug hello
hello.exe is built, and I can run it just fine from the command prompt.

The Error

set breakpoint on line 2
Run > Start Debugging
this error pops up in VS Code:

"Oops! The debug adapter has terminated abnormally."
Source: CodeLLDB (Extension)          Open log

and that log says:

configuration: {
  type: 'lldb',
  request: 'launch',
  name: 'Debug hello',
  program: '${workspaceFolder:hello}/.build/debug/hello',
  args: [],
  cwd: '${workspaceFolder:hello}',
  preLaunchTask: 'swift: Build Debug hello',
  __configurationTarget: 5,
  relativePathBase: 'k:\\Kim Shared\\CodeLab\\SwiftTests\\hello'
}
Python path configuration:
  PYTHONHOME = 'C:/hostedtoolcache/windows/Python/3.10.2/x64'
  PYTHONPATH = (not set)
  program name = 'python'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = 'c:\\Users\\Kim\\.vscode\\extensions\\vadimcn.vscode-lldb-1.7.0\\adapter\\codelldb.exe'
  sys.base_prefix = 'C:/hostedtoolcache/windows/Python/3.10.2/x64'
  sys.base_exec_prefix = 'C:/hostedtoolcache/windows/Python/3.10.2/x64'
  sys.platlibdir = 'lib'
  sys.executable = 'c:\\Users\\Kim\\.vscode\\extensions\\vadimcn.vscode-lldb-1.7.0\\adapter\\codelldb.exe'
  sys.prefix = 'C:/hostedtoolcache/windows/Python/3.10.2/x64'
  sys.exec_prefix = 'C:/hostedtoolcache/windows/Python/3.10.2/x64'
  sys.path = [
    'C:\\Program Files\\Python\\Python310\\python310.zip',
    'C:/hostedtoolcache/windows/Python/3.10.2/x64\\DLLs',
    'C:/hostedtoolcache/windows/Python/3.10.2/x64\\lib',
    'c:\\Users\\Kim\\.vscode\\extensions\\vadimcn.vscode-lldb-1.7.0\\adapter',
  ]
Fatal Python error: init_fs_encoding: Debug adapter exit code=1, signal=null.
failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00001328 (most recent call first):
  <no Python frame>

Note that sys.path references python310.zip and .../Python/3.10.2/... - neither of which do I have. However I do have these in my Path before the start of VS Code:

C:\Program Files\Python\Python310\;C:\Program Files\Python\Python310\Scripts\

I know this Path addition solved the identical error for me a couple weeks ago when I was using LLDB from the Command Prompt, so I suspect they are not in the Path at the time LLDB is launched by the extension.

Variations

I can run LLDB from the command prompt on this program and the debugger works just fine.

If I delete settings.json, so that the default LLDB is used, then Run > Start Debugging will cause the debugger to run the program and break at the desired breakpoint. While it does allow stepping through the program, no variables are displayable, which of course is why we have the Swift custom version of LLDB.

[adam-fowler]

Hi, the latest release of the extension shows up an issue with the current Windows swift install (5.6). The swift version of lldb is a little borked. lldb is reliant on python to do certain things. The swift version of lldb expects to find the python libs in C:/hostedtoolcache/windows/Python/3.10.2/x64. When we run the code to find the swift lldb library it fails because of this. In vscode-swift 0.5.0 and earlier it would fail silently and just not set the swift lldb library. v0.5.1 tells you it failed.

If the swift lldb lib path is not set then the version of lldb internal to vscode-lldb is used. This displays mangled symbols names but still works.

The swift snapshot from main (trunk) does not have the python issue. I've asked @compnerd to see if he can fix the python issue for both the next patch release of 5.6 and the snapshot for 5.7.

[KimTheLearner]

THANKS so much for responding so quickly!

The swift version of lldb expects to find the python libs in C:/hostedtoolcache/windows/Python/3.10.2/x64

hmm, perhaps I can try to create a symbolic link or junction from that place to the real install folder, as a workaround, till a new release comes out!

This displays mangled symbols names but still works.

This wasn't so useful for me. When I add 3 vars (Int, Double, and String), and print them, I do see 3 mangled vars under side panel > VARIABLES > Static, but the value of each is "Unable to determine byte size.". Despite this being a debug build, perhaps a local optimization has eliminated the assignments. I passed them to a function to circumvent any such optimization, but VARIABLES > Local is empty while I am stepped into the function. More local optimization, perhaps. When I use the 'v' command in the lldb command entry area like this:

v $s5hello3aaaSivp

(variable name is aaa), I get "could not resolve type". So doesn't look like using the default LLDB will help me.

I've asked @compnerd to see if he can fix...

Thanks for that!! I am new to Swift and new to VS Code, and am excited to use these. I have translated a 360-line .bat file into Swift, and am just on hold till GUI debugging is working so I can step through it carefully (I have LLDB cmd line as a backup plan, in case it takes a while).

At any rate, I am grateful for all the work it took to get Swift and tooling working on Windows!

[compnerd]

[redacted string of explictives] Sorry, I forgot to make the change that I had hoped to a couple of days ago :sigh:. I'll do that now. The latest main snapshot should not be embedding PYTHONHOME so that might be a good thing to check. If there is something beyond just that which is needed, please let me know and lets see if we can figure out how to get that addressed. I'd love some ideas on workarounds to help bridge the gap for 5.6 for the time being.

[compnerd]

Okay; swiftlang/swift#58841 is up to add the required change to the 5.7 branch. Thank you for identifying the issue and the reminder!

[KimTheLearner]

Excellent, thanks! I am glad to help. I'll watch for that fix to be in a snapshot. Meanwhile I'll try that symbolic link/junction idea and report back.

[KimTheLearner]

Creating the junction has helped.

md C:\hostedtoolcache\windows\Python\3.10.2
cd there
mklink /j x64 "c:\Program Files\Python\Python310"

Now at Run > Start Debugging, the Swift LLDB comes up and breaks at the breakpoint. All vars are displayed in the left pane with their actual (unmangled) names. Hovering over a var shows its value.

On the flip side, print statements do not work. The Output Pane at LLDB channel has this:

configuration: {
  type: 'lldb',
  request: 'launch',
  name: 'Debug hello',
  program: '${workspaceFolder:hello}/.build/debug/hello',
  args: [],
  cwd: '${workspaceFolder:hello}',
  preLaunchTask: 'swift: Build Debug hello',
  __configurationTarget: 5,
  relativePathBase: 'k:\\Kim Shared\\CodeLab\\SwiftTests\\hello'
}
[adapter\src\terminal.rs:104] FreeConsole() = 1
[adapter\src\terminal.rs:105] AttachConsole(pid) = 1
[adapter\src\terminal.rs:109] FreeConsole() = 1
ERROR(Python) 17:17:11 codelldb: Traceback (most recent call last):
  File "c:\Users/Kim/.vscode/extensions/vadimcn.vscode-lldb-1.7.0/adapter\codelldb.py", line 152, in evaluate
    value = evaluate_in_context(pycode, is_simple_expr, context)
  File "c:\Users/Kim/.vscode/extensions/vadimcn.vscode-lldb-1.7.0/adapter\codelldb.py", line 288, in evaluate_in_context
    return eval(code, eval_globals, eval_locals)
  File "<input>", line 1, in <module>
  File "c:\Users/Kim/.vscode/extensions/vadimcn.vscode-lldb-1.7.0/adapter\codelldb.py", line 269, in __missing__
    raise VariableNotFound(name)
codelldb.VariableNotFound: Variable 'print' not found

[2022-05-11T22:17:11.388Z ERROR codelldb::debug_session] Variable 'print' not found
[2022-05-11T22:18:27.990Z ERROR codelldb::handles] Parent/key combination is not unique (None/[1,0])
[2022-05-11T22:24:10.143Z ERROR codelldb::handles] Parent/key combination is not unique (None/[1,0])
[2022-05-11T22:24:20.055Z ERROR codelldb::handles] Parent/key combination is not unique (None/[1,0])

Do not trouble yourselves to analyze this, since the final solution is around the corner (unless this points to a problem likely to occur even with your fix). What I have working with the junction is enough to let me continue on my project.

[compnerd]

The print issue worries me some - that should work with the junction in place (it effectively makes PYTHONHOME work, which is what removal of PYTHONHOME does in a sense). Waiting until the next snapshot to test doesn't seem entirely terrible, but should be relatively easy to test on main. How does the print thing work exactly? I'm not familiar with VSCode :-(.

[KimTheLearner]

Normally, output from the 'print' statement shows up in the "Terminal" tab of the bottom pane of VS Code. The errors above occur in the "Output" tab of that pane.

I should be able to test the latest main snapshot tonight and let you know.

[compnerd]

Awesome, thank you so much!

[KimTheLearner]

Alrighty. I deleted the junction and installed the latest snapshot:
https://www.swift.org/download/ Snapshots > Trunk Development (main) > Windows 10 - May 4, 2022.

Run > Start Debugging - comparing results to those of 5.6.1 (my previous install):

Identical:

• able to step through the program, able to see all var values, vars had unmangled names
• still no output in [VS Code lower pane > Terminal tab], from 'print' statements (not good), or any other tabs for that matter
• program runs fine when executed from command prompt - all print output is visible there

Different:

• this time there were no errors regarding the 'print' symbol. however there is a new error (see below).
• also, at start of execution, a new command prompt window appeared titled 'hello.exe', totally a blank screen, and it remained blank during the print statements; presumably it is accidentally created, because the output of the prog should be going to the Terminal tab of the lower pane of VS Code. I am 95% sure that window did not come up on 5.6.1.

this is the contents of the VS Code bottom pane, Output tab, LLDB channel.

configuration: {
  type: 'lldb',
  request: 'launch',
  name: 'Debug hello',
  program: '${workspaceFolder:hello}/.build/debug/hello',
  args: [],
  cwd: '${workspaceFolder:hello}',
  preLaunchTask: 'swift: Build Debug hello',
  __configurationTarget: 5,
  relativePathBase: 'k:\\Kim Shared\\CodeLab\\SwiftTests\\hello'
}
[adapter\src\terminal.rs:104] FreeConsole() = 1
[adapter\src\terminal.rs:105] AttachConsole(pid) = 1
[adapter\src\terminal.rs:109] FreeConsole() = 1
ERROR(Python) 20:38:54 codelldb: Traceback (most recent call last):
  File "c:\Users/Kim/.vscode/extensions/vadimcn.vscode-lldb-1.7.0/adapter\codelldb.py", line 152, in evaluate
    value = evaluate_in_context(pycode, is_simple_expr, context)
  File "c:\Users/Kim/.vscode/extensions/vadimcn.vscode-lldb-1.7.0/adapter\codelldb.py", line 288, in evaluate_in_context
    return eval(code, eval_globals, eval_locals)
  File "<input>", line 1, in <module>
  File "c:\Users/Kim/.vscode/extensions/vadimcn.vscode-lldb-1.7.0/adapter\codelldb.py", line 269, in __missing__
    raise VariableNotFound(name)
codelldb.VariableNotFound: Variable 'vars' not found

[2022-05-12T01:38:54.750Z ERROR codelldb::debug_session] Variable 'vars' not found
Debug adapter exit code=0, signal=null.

So, print statements are no longer working - sorry, I know that's not the news you hoped for. On the positive side, my issue has been fixed, the custom debugger is running, and this snapshot will let me resume my project. Let me know if I can help further. Otherwise I will watch for updates.

On a personal note, I have seen your talk on YouTube, “Porting by a 1000 Patches: Bringing Swift to Windows”, and I had read that you put in a year of work to make this happen. An amazing job, and greatly appreciated. You have done an immense service. And I'm thrilled I can thank you directly! :)

[KimTheLearner]

P.S. With the latest snapshot, after Run > Start Debugging gets the program running, the debuggee's std input seems to be closed. With these lines...

print("enter something  ->", terminator: "")
guard let answer = readLine() else {
    // readLine saw EOF
    print("\n\n\nEnd of File was reached - exiting...")
    exit(1)
}

...the control flows to the EOF handling. when I run the code outside the debugger from the command prompt, std input is normal and no EOF occurs..

so with this snapshot under the Swift extension, it seems likely that the print statement machinery is fine, but stdin and stdout have been pre-closed or some related issue.

[compnerd]

Interesting; thanks for the details. 'm pretty clueless about the vars variable, its also unclear what scope the variable is meant to be in.

Thank you! I hope you enjoyed the talk :)

[KimTheLearner]

@adam-fowler or @compnerd,

A bit of advice is needed, regarding whether I should submit a new issue or not...

Using the 5/04/22 dev snapshot, I am still experiencing the issue where stdin and stdout seem to have been closed by the start of program execution, when the Swift LLDB is launched from the Swift extension for VS Code. As described above, no print output appears anywhere and reading stdin gets an EOF-detected.

When I run the Swift LLDB myself, from the command prompt, the program uses stdin/stdout just fine, and that occurs in a command prompt window that lldb launches for the program.

Likewise when I run the program directly from the command prompt (no LLDB), the prog uses stdin/stdout just fine in that command prompt window.

When I comment out the "lldb.library" entry in settings.json, thus forcing the Swift extension (or the LLDB extension) to use the standard LLDB, the print() statement output appears in the lower pane of VS Code, in the Terminal tab; likewise, for readline(), the user enters text in the same Terminal tab. That feels like the correct behavior to me, for VSCode-based debugging.

So the issue might be in the Swift extension or it could be in the LLDB extension.

Now my questions: Would my best option be to submit this as a new issue, since this (#301) has such an unrelated title? Or is someone already planning to look at this and submitting a new issue would just be unnecessary?

And if it should be submitted, should it be to vscode-swift? or over to vscode-lldb?

I am new to github and unsure with normal practices.

Thanks!
Kim