Multi-root workspace with hidden directories uses wrong python interpreter

[Cnoor0171]

Type: Bug

Behaviour

If you create a multi-root workspace, select different interpreters for each workspace and create python files inside a hidden directory (folder whose name starts with '.'), the wrong interpreter gets used for syntax highlighting and code completion. It seems to just use the first workspace's interpreter.

Steps to reproduce:

1. Create a multi-root workspace, and create separate virtual environments for each of them.

mkdir ws1 ws2
python3.11 -m venv ws1/.venv
python3.11 -m venv ws2/.venv
code ws1/ ws2/

2. Install different libraries in each virtual env

./ws1/.venv/bin/python -m pip install PyJwt
./ws2/.venv/bin/python -m pip install pydantic

3. Create 4 files, ./ws1/ws1.py ./ws1/.hidden_folder/ws1_hidden.py, ./ws2/ws2.py ./ws2/.hidden_folder/ws2_hidden.py, with exact same content:

from jwt import decode
from pydantic import model_serializer

4. Select the interpreter for both workspaces in vscode
5. Open all 4 files, and notice that the syntax highlighting is incorrect for ./ws2/.hidden_folder/ws2_hidden.py. "Go to Definition" also open the library files in ws1 instead of ws2.

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2024-12-05 11:54:17.874 [info] Telemetry is disabled
2024-12-05 11:54:17.874 [info] Telemetry level is off
2024-12-05 11:54:17.874 [info] Experiments are disabled, only manually opted experiments are active.
2024-12-05 11:54:17.874 [warning] Dir "/home/cnoor/work/scratch/pythonMultiRoot/ws1/.pixi/envs" is not watchable (directory does not exist)
2024-12-05 11:54:17.874 [warning] Dir "/home/cnoor/work/scratch/pythonMultiRoot/ws2/.pixi/envs" is not watchable (directory does not exist)
2024-12-05 11:54:17.874 [info] Default formatter is set to ms-python.black-formatter for workspace /home/cnoor/work/scratch/pythonMultiRoot/ws1
2024-12-05 11:54:17.874 [info] Default formatter is set to ms-python.black-formatter for workspace /home/cnoor/work/scratch/pythonMultiRoot/ws2
2024-12-05 11:54:17.874 [info] Not enabling terminal env var experiment in multiroot remote workspaces
2024-12-05 11:54:17.874 [info] Starting Environment refresh
2024-12-05 11:54:17.874 [info] Starting Environment refresh
2024-12-05 11:54:17.874 [info] > pixi --version
2024-12-05 11:54:17.874 [info] > hatch env show --json
2024-12-05 11:54:17.874 [info] cwd: ~/work/scratch/pythonMultiRoot/ws2
2024-12-05 11:54:17.874 [info] > hatch env show --json
2024-12-05 11:54:17.874 [info] cwd: ~/work/scratch/pythonMultiRoot/ws1
2024-12-05 11:54:17.874 [info] Python interpreter path: ./.venv/bin/python
2024-12-05 11:54:18.546 [info] > /opt/bb/bin/python3.12 -I ~/.vscode-server/extensions/ms-python.python-2024.14.1/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.14.1/python_files/interpreterInfo.py
2024-12-05 11:54:18.583 [info] > ~/work/scratch/pythonMultiRoot/ws1/.venv/bin/python -I ~/.vscode-server/extensions/ms-python.python-2024.14.1/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.14.1/python_files/interpreterInfo.py
2024-12-05 11:54:18.584 [info] Environments refresh paths discovered: 1069 milliseconds
2024-12-05 11:54:18.586 [info] Environments refresh paths discovered: 1072 milliseconds
2024-12-05 11:54:18.646 [info] > ~/work/scratch/pythonMultiRoot/ws2/.venv/bin/python -I ~/.vscode-server/extensions/ms-python.python-2024.14.1/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.14.1/python_files/interpreterInfo.py
2024-12-05 11:54:18.669 [info] Environments refresh finished (event): 1154 milliseconds
2024-12-05 11:54:18.672 [info] Environment refresh took 1157 milliseconds
2024-12-05 11:54:18.677 [info] > ~/.pyenv/bin/pyenv which python
2024-12-05 11:54:18.677 [info] cwd: ~/work/scratch/pythonMultiRoot/ws1
2024-12-05 11:54:18.704 [info] Environments refresh finished (event): 1190 milliseconds
2024-12-05 11:54:18.705 [info] Environment refresh took 1192 milliseconds
2024-12-05 11:54:18.863 [info] Not enabling terminal env var experiment in multiroot remote workspaces
2024-12-05 11:54:19.691 [info] Starting Environment refresh
2024-12-05 11:54:19.693 [info] Python interpreter path: ./.venv/bin/python
2024-12-05 11:54:19.753 [info] Environments refresh paths discovered: 62 milliseconds
2024-12-05 11:54:19.753 [info] Environments refresh finished (event): 62 milliseconds
2024-12-05 11:54:19.755 [info] Environment refresh took 64 milliseconds
2024-12-05 11:54:19.767 [info] > ~/.pyenv/bin/pyenv which python
2024-12-05 11:54:19.767 [info] cwd: ~/work/scratch/pythonMultiRoot/ws2
2024-12-05 11:54:20.517 [info] Not enabling terminal env var experiment in multiroot remote workspaces
2024-12-05 11:54:20.518 [info] Starting Pylance language server.
2024-12-05 11:54:25.409 [info] Python interpreter path: ./.venv/bin/python
2024-12-05 11:54:29.217 [info] Starting Pylance language server.
2024-12-05 11:54:34.332 [info] Python interpreter path: ./.venv/bin/python
2024-12-05 11:55:09.061 [info] Python interpreter path: ./.venv/bin/python
2024-12-05 12:06:58.579 [info] Python interpreter path: ./.venv/bin/python
2024-12-05 12:07:01.722 [info] Python interpreter path: ./.venv/bin/python
2024-12-05 12:07:03.539 [info] Python interpreter path: ./.venv/bin/python
2024-12-05 12:07:04.784 [info] Python interpreter path: ./.venv/bin/python
2024-12-05 12:07:06.471 [info] Python interpreter path: ./.venv/bin/python
2024-12-05 12:08:10.626 [info] Python interpreter path: ./.venv/bin/python
2024-12-05 12:08:10.867 [info] Not enabling terminal env var experiment in multiroot remote workspaces
2024-12-05 12:08:10.872 [warning] could not find a pixi interpreter for the interpreter at /home/cnoor/work/scratch/pythonMultiRoot/ws1/.venv/bin/python
2024-12-05 12:08:10.998 [info] Send text to terminal: source /home/cnoor/work/scratch/pythonMultiRoot/ws1/.venv/bin/activate

Extension version: 2024.4.1
VS Code version: Code 1.91.1 (f1e16e1e6214d7c44d078b1f0607b2388f29d729, 2024-07-09T22:06:49.809Z)
OS version: Windows_NT x64 10.0.19045
Modes:
Remote OS version: Linux x64 5.15.167.4-microsoft-standard-WSL2
Remote OS version: Linux x64 5.15.167.4-microsoft-standard-WSL2

• Python version (& distribution if applicable, e.g. Anaconda): 3.12.7
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
• Value of the python.languageServer setting: Pylance

User Settings

Multiroot scenario, following user settings may not apply:

languageServer: "Pylance"

Installed Extensions

Extension Name	Extension Id	Version
autoDocstring - Python Docstring Generator	njpwerner.autodocstring	0.6.1
Data Workspace	ms-mssql.data-workspace-vscode	0.5.0
GitLens — Git supercharged	eamodio.gitlens	15.6.1
Increment Selection	albymor.increment-selection	0.2.0
JavaScript Debugger	ms-vscode.js-debug	1.91.0
Jest	Orta.vscode-jest	6.2.5
Prettier - Code formatter	esbenp.prettier-vscode	11.0.0
Pylance	ms-python.vscode-pylance	2024.10.1
Python	ms-python.python	2024.14.1
Python Debugger	ms-python.debugpy	2024.8.0
Python Environment Manager	donjayamanne.python-environment-manager	1.2.4
Python Extension Pack	donjayamanne.python-extension-pack	1.7.0
Table Visualizer for JavaScript Profiles	ms-vscode.vscode-js-profile-table	1.0.9
transpose	v4run.transpose	0.0.6
vscode-styled-components	styled-components.vscode-styled-components	1.7.8
YAML	redhat.vscode-yaml	1.15.0

System Info

Item	Value
CPUs	12th Gen Intel(R) Core(TM) i7-12800H (20 x 2803)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	undefined
Memory (System)	63.67GB (37.66GB free)
Process Argv	--file-uri vscode-remote://dev-container%2B7b22686f737450617468223a225c5c5c5c77736c2e6c6f63616c686f73745c5c5562756e74752d32322e30345c5c686f6d655c5c636e6f6f725c5c776f726b5c5c736372617463685c5c646576436f6d62696e6564436f6e7461696e6572222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f636e6f6f722f776f726b2f736372617463682f646576436f6d62696e6564436f6e7461696e65722f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d/vscode/dev_workspace.code-workspace
Screen Reader	no
VM	0%

Item	Value
Remote	Dev Container: dev_workspace
OS	Linux x64 5.15.167.4-microsoft-standard-WSL2
CPUs	12th Gen Intel(R) Core(TM) i7-12800H (2 x 0)
Memory (System)	24.46GB (20.03GB free)
VM	0%

Item	Value
Remote	WSL: Ubuntu-22.04
OS	Linux x64 5.15.167.4-microsoft-standard-WSL2
CPUs	12th Gen Intel(R) Core(TM) i7-12800H (2 x 0)
Memory (System)	24.46GB (20.03GB free)
VM	0%

[Cnoor0171]

For ./ws1/ws1.py and ./ws1/.hidden_folder/ws1_hidden.py the syntax highlighting correctly shows jwt is installed, but pydantic is not
For ./ws2/ws2.py the syntax highlighting correctly shows pydantic is installed, but jwt is not

But, for ./ws2/.hidden_folder/ws2_hidden.py, it incorrectly shows jwt is installed, but pydantic is not. The file seems to use ws1's interpreter.

[karthiknadig]

@Cnoor0171 When you run the hidden script does it pick the right environment?

[Cnoor0171]

Yes. If I run the script using python.execInTerminal or debugpy.debugInTerminal vscode commands, it does use the correct environment.

[StellaHuang95]

Thanks, I was able to repro this issue. It seems to be a bug in Pylance where the wrong environment is detected in multi-root workspace scenarios.

Update:
Debugging getContainingWorkspacesForFileSync in pylanceWorkspaceFactory.ts, seems like the files starting with . are not considered source files of the workspace, so it's assigned to the default workspace, and the first available python environment is assigned to it.