Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why to modify
Recently, I attempted to install sentencepiece using pip on Windows on ARM, but due to
python/setup.py
not taking into accountarm64
in the judgment of Windows arch, cmake mistakenly built the lib for theAMD64
architecture.sentencepiece/python/setup.py
Lines 125 to 137 in 2de10cb
What has been modified
1. For building from source or pip install from source
I added a judgment for
arm64
throughplatform.machine()
, without modifying the judgment statements for x86 and x64, to ensure that it does not have any side effects on the original architecture building.At the same time, I have added a section in the Python build documentation that explains how to build from source code on Windows for the convenience of Windows users.
2. For building wheels for win-arm64 automatically
I've also modified the
.github/workflows/wheel.yml
to build wheels for win-arm64 automatically. But there's a bug in cibuildwheel. pypa/cibuildwheel#1942 (comment)Simply put, for the win arm64 arch in cibuildwheel,
platform.machine()
will retrieveAMD64
instead ofARM64
. This will cause issues with the judgment of arch insetup.py
, resulting in selecting the wrong lib path during build. So I add a special check for win-arm under ciwheelbuild using envPYTHON_ARCH
.Here's the Github Actions building results on my own repo. https://github.com/Nagico2/sentencepiece/actions/runs/10105917616
The outcomes
Now windows on arm users can build and install python wrapper from source. And after the modification of
python/setup.py
have been published to PyPI, windows on arm users can usepip install sentencepiece
to build and install.After the wheels for win-arm published to PyPI, indows on arm users can use
pip install sentencepiece
to install the pre-built wheels.Others
I've also tried to add win-arm build in
cmake.yml
. But there's another bug in setup-python (actions/setup-python#915). So I haven't made any modifications to this yet.Modifying the build code may lead to serious consequences, please conduct comprehensive and full testing.
Thank you all for building the Windows on ARM ecosystem. :)