Use python[w].exe adjacent to launcher #28
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
konstin
added a commit
to astral-sh/uv
that referenced
this pull request
Jan 25, 2024
In virtual environments, we want to install python programs as console commands, e.g. `black .` over `python -m black .`. They may be called [entrypoints](https://packaging.python.org/en/latest/specifications/entry-points/) or scripts. For entrypoints, we're given a module name and function to call in that module. On Unix, we generate a minimal python script launcher. Text files are runnable on unix by adding a shebang at their top, e.g. ```python #!/usr/bin/env python ``` will make the operating system run the file with the current python interpreter. A venv launcher for black in `/home/ferris/colorize/.venv` would look like this: ```python #!/home/ferris/colorize/.venv/bin/python # -*- coding: utf-8 -*- import re import sys from black import patched_main if __name__ == "__main__": sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0]) sys.exit(patched_main()) ``` On windows, this doesn't work, we can only rely on launching `.exe` files. So instead we use posy's rust implementation of trampoline, which is based on distlib's c++ implementation. We pre-build a minimal exe and append the launcher script as stored zip archive behind it. The exe will look for the venv python interpreter next to it and use it to execute the appended script. I've vendored the posy trampoline crate. It is a formatted, renamed and slightly changed for embedding version of njsmith/posy#28. The posy launchers are smaller than the distlib launchers, 16K vs 106K for black. Currently only `x86_64-pc-windows-msvc` is supported. On windows, an application can be launched with a console or without (to create windows instead), which needs two different launchers. The gui launcher will subsequently use `pythonw.exe` while the console launcher uses `python.exe`.
konstin
added a commit
to astral-sh/uv
that referenced
this pull request
Jan 25, 2024
In virtual environments, we want to install python programs as console commands, e.g. `black .` over `python -m black .`. They may be called [entrypoints](https://packaging.python.org/en/latest/specifications/entry-points/) or scripts. For entrypoints, we're given a module name and function to call in that module. On Unix, we generate a minimal python script launcher. Text files are runnable on unix by adding a shebang at their top, e.g. ```python #!/usr/bin/env python ``` will make the operating system run the file with the current python interpreter. A venv launcher for black in `/home/ferris/colorize/.venv` would look like this: ```python #!/home/ferris/colorize/.venv/bin/python # -*- coding: utf-8 -*- import re import sys from black import patched_main if __name__ == "__main__": sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0]) sys.exit(patched_main()) ``` On windows, this doesn't work, we can only rely on launching `.exe` files. So instead we use posy's rust implementation of trampoline, which is based on distlib's c++ implementation. We pre-build a minimal exe and append the launcher script as stored zip archive behind it. The exe will look for the venv python interpreter next to it and use it to execute the appended script. I've vendored the posy trampoline crate. It is a formatted, renamed and slightly changed for embedding version of njsmith/posy#28. The posy launchers are smaller than the distlib launchers, 16K vs 106K for black. Currently only `x86_64-pc-windows-msvc` is supported. On windows, an application can be launched with a console or without (to create windows instead), which needs two different launchers. The gui launcher will subsequently use `pythonw.exe` while the console launcher uses `python.exe`.
konstin
added a commit
to astral-sh/uv
that referenced
this pull request
Jan 25, 2024
In virtual environments, we want to install python programs as console commands, e.g. `black .` over `python -m black .`. They may be called [entrypoints](https://packaging.python.org/en/latest/specifications/entry-points/) or scripts. For entrypoints, we're given a module name and function to call in that module. On Unix, we generate a minimal python script launcher. Text files are runnable on unix by adding a shebang at their top, e.g. ```python #!/usr/bin/env python ``` will make the operating system run the file with the current python interpreter. A venv launcher for black in `/home/ferris/colorize/.venv` would look like this: ```python #!/home/ferris/colorize/.venv/bin/python # -*- coding: utf-8 -*- import re import sys from black import patched_main if __name__ == "__main__": sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0]) sys.exit(patched_main()) ``` On windows, this doesn't work, we can only rely on launching `.exe` files. So instead we use posy's rust implementation of trampoline, which is based on distlib's c++ implementation. We pre-build a minimal exe and append the launcher script as stored zip archive behind it. The exe will look for the venv python interpreter next to it and use it to execute the appended script. I've vendored the posy trampoline crate. It is a formatted, renamed and slightly changed for embedding version of njsmith/posy#28. The posy launchers are smaller than the distlib launchers, 16K vs 106K for black. Currently only `x86_64-pc-windows-msvc` is supported. On windows, an application can be launched with a console or without (to create windows instead), which needs two different launchers. The gui launcher will subsequently use `pythonw.exe` while the console launcher uses `python.exe`.
konstin
added a commit
to astral-sh/uv
that referenced
this pull request
Jan 25, 2024
In virtual environments, we want to install python programs as console commands, e.g. `black .` over `python -m black .`. They may be called [entrypoints](https://packaging.python.org/en/latest/specifications/entry-points/) or scripts. For entrypoints, we're given a module name and function to call in that module. On Unix, we generate a minimal python script launcher. Text files are runnable on unix by adding a shebang at their top, e.g. ```python #!/usr/bin/env python ``` will make the operating system run the file with the current python interpreter. A venv launcher for black in `/home/ferris/colorize/.venv` would look like this: ```python #!/home/ferris/colorize/.venv/bin/python # -*- coding: utf-8 -*- import re import sys from black import patched_main if __name__ == "__main__": sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0]) sys.exit(patched_main()) ``` On windows, this doesn't work, we can only rely on launching `.exe` files. So instead we use posy's rust implementation of trampoline, which is based on distlib's c++ implementation. We pre-build a minimal exe and append the launcher script as stored zip archive behind it. The exe will look for the venv python interpreter next to it and use it to execute the appended script. I've vendored the posy trampoline crate. It is a formatted, renamed and slightly changed for embedding version of njsmith/posy#28. The posy launchers are smaller than the distlib launchers, 16K vs 106K for black. Currently only `x86_64-pc-windows-msvc` is supported. On windows, an application can be launched with a console or without (to create windows instead), which needs two different launchers. The gui launcher will subsequently use `pythonw.exe` while the console launcher uses `python.exe`.
konstin
force-pushed
the
use-adjacent-python
branch
from
70ce6f9 to
918d853
Compare
konstin
added a commit
to astral-sh/uv
that referenced
this pull request
Jan 25, 2024
## Background In virtual environments, we want to install python programs as console commands, e.g. `black .` over `python -m black .`. They may be called [entrypoints](https://packaging.python.org/en/latest/specifications/entry-points/) or scripts. For entrypoints, we're given a module name and function to call in that module. On Unix, we generate a minimal python script launcher. Text files are runnable on unix by adding a shebang at their top, e.g. ```python #!/usr/bin/env python ``` will make the operating system run the file with the current python interpreter. A venv launcher for black in `/home/ferris/colorize/.venv` (module name: `black`, function to call: `patched_main`) would look like this: ```python #!/home/ferris/colorize/.venv/bin/python # -*- coding: utf-8 -*- import re import sys from black import patched_main if __name__ == "__main__": sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0]) sys.exit(patched_main()) ``` On windows, this doesn't work, we can only rely on launching `.exe` files. ## Summary We use posy's rust implementation of a trampoline, which is based on distlib's c++ implementation. We pre-build a minimal exe and append the launcher script as stored zip archive behind it. The exe will look for the venv python interpreter next to it and use it to execute the appended script. The changes in this PR make the `black` entrypoint work: ```powershell cargo run -- venv .venv cargo run -q -- pip install black .\.venv\Scripts\black --version ``` Integration with our existing tests will be done in follow-up PRs. ## Implementation and Details I've vendored the posy trampoline crate. It is a formatted, renamed and slightly changed for embedding version of njsmith/posy#28. The posy launchers are smaller than the distlib launchers, 16K vs 106K for black. Currently only `x86_64-pc-windows-msvc` is supported. The crate requires a nightly compiler for its no-std binary size tricks. On windows, an application can be launched with a console or without (to create windows instead), which needs two different launchers. The gui launcher will subsequently use `pythonw.exe` while the console launcher uses `python.exe`.
konstin
added a commit
to astral-sh/uv
that referenced
this pull request
Jan 26, 2024
## Background In virtual environments, we want to install python programs as console commands, e.g. `black .` over `python -m black .`. They may be called [entrypoints](https://packaging.python.org/en/latest/specifications/entry-points/) or scripts. For entrypoints, we're given a module name and function to call in that module. On Unix, we generate a minimal python script launcher. Text files are runnable on unix by adding a shebang at their top, e.g. ```python #!/usr/bin/env python ``` will make the operating system run the file with the current python interpreter. A venv launcher for black in `/home/ferris/colorize/.venv` (module name: `black`, function to call: `patched_main`) would look like this: ```python #!/home/ferris/colorize/.venv/bin/python # -*- coding: utf-8 -*- import re import sys from black import patched_main if __name__ == "__main__": sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0]) sys.exit(patched_main()) ``` On windows, this doesn't work, we can only rely on launching `.exe` files. ## Summary We use posy's rust implementation of a trampoline, which is based on distlib's c++ implementation. We pre-build a minimal exe and append the launcher script as stored zip archive behind it. The exe will look for the venv python interpreter next to it and use it to execute the appended script. The changes in this PR make the `black` entrypoint work: ```powershell cargo run -- venv .venv cargo run -q -- pip install black .\.venv\Scripts\black --version ``` Integration with our existing tests will be done in follow-up PRs. ## Implementation and Details I've vendored the posy trampoline crate. It is a formatted, renamed and slightly changed for embedding version of njsmith/posy#28. The posy launchers are smaller than the distlib launchers, 16K vs 106K for black. Currently only `x86_64-pc-windows-msvc` is supported. The crate requires a nightly compiler for its no-std binary size tricks. On windows, an application can be launched with a console or without (to create windows instead), which needs two different launchers. The gui launcher will subsequently use `pythonw.exe` while the console launcher uses `python.exe`.
konstin
added a commit
to astral-sh/uv
that referenced
this pull request
Jan 26, 2024
## Background In virtual environments, we want to install python programs as console commands, e.g. `black .` over `python -m black .`. They may be called [entrypoints](https://packaging.python.org/en/latest/specifications/entry-points/) or scripts. For entrypoints, we're given a module name and function to call in that module. On Unix, we generate a minimal python script launcher. Text files are runnable on unix by adding a shebang at their top, e.g. ```python #!/usr/bin/env python ``` will make the operating system run the file with the current python interpreter. A venv launcher for black in `/home/ferris/colorize/.venv` (module name: `black`, function to call: `patched_main`) would look like this: ```python #!/home/ferris/colorize/.venv/bin/python # -*- coding: utf-8 -*- import re import sys from black import patched_main if __name__ == "__main__": sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0]) sys.exit(patched_main()) ``` On windows, this doesn't work, we can only rely on launching `.exe` files. ## Summary We use posy's rust implementation of a trampoline, which is based on distlib's c++ implementation. We pre-build a minimal exe and append the launcher script as stored zip archive behind it. The exe will look for the venv python interpreter next to it and use it to execute the appended script. The changes in this PR make the `black` entrypoint work: ```powershell cargo run -- venv .venv cargo run -q -- pip install black .\.venv\Scripts\black --version ``` Integration with our existing tests will be done in follow-up PRs. ## Implementation and Details I've vendored the posy trampoline crate. It is a formatted, renamed and slightly changed for embedding version of njsmith/posy#28. The posy launchers are smaller than the distlib launchers, 16K vs 106K for black. Currently only `x86_64-pc-windows-msvc` is supported. The crate requires a nightly compiler for its no-std binary size tricks. On windows, an application can be launched with a console or without (to create windows instead), which needs two different launchers. The gui launcher will subsequently use `pythonw.exe` while the console launcher uses `python.exe`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I'm not sure if that is the right thing for posy, i've built this for installing packages into a venv: When creating a launcher for let's say black, we put it in
.venv\Scripts\black.exe. The python interpreter is adjacent in the same directory, either.venv\Scripts\python.exeor.venv\Scripts\pythonw.exe. This replaces thePOSY_PYTHONenvironment variables.