Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python module mutli-platform setup #519

Merged
merged 7 commits into from
Jan 21, 2024
Merged

Conversation

vathomass
Copy link
Contributor

Hi,

There were three issues when installing pyclblast in windows:

  1. To compile the python module, you should provide to the linker the clblast and opencl include paths and the clblast library path. If not the module will not compile.
  2. The runtime_library_dirs options in distutils.extension.Extension, used by the setup.py script, is not available in windows platforms (see here).
  3. After a successful install, you get an ImportError: DLL load failed message when trying to import the module. This is probably related to this issue.

This PR moves towards installing pyclblast in windows, without breaking the installation on others platforms. The CLBlast library uses cmake for building and installing, and also installs a cmake package for the library. Using cmake for building the python module also should address issues 1 and 2 (let cmake take care of finding the clblast installation and setting up the compiler and linker flags). To solve issue 3, one approach (found in the internet) is to copy the dll along with the binary module to the installation folder.

Moreover, the PR switched from the setup.py style to the pyproject.toml style installation, hopefully for good.

I am a basic user of cmake, so I welcome any comments. I have tested the changes on windows and linux environments.

Best,
Thomas

Copy link
Owner

@CNugteren CNugteren left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution. And apologies for the late review. I did leave one comment below, but it isn't a super important one. One other comment is:

  • Perhaps you can add a note to the CHANGELOG file in the root about this change? Although not really related to CLBlast itself, it is in the same repository so I guess we can also include something like Improved pyclblast builds in Windows or so?

Otherwise it looks good to me. I've tested it on my Linux system and it builds fine and the tests run fine as well. Indeed, using the pyproject.toml should be the more modern version. I didn't test on Windows but I trust you on that, and given that it didn't work before this can't do much harm anyway ;-)

@@ -53,6 +53,6 @@ How to release a new version on PyPi

Following [the guide](https://packaging.python.org/tutorials/packaging-projects/), in essence doing (after changing the version number in `setup.py`):

python3 setup.py sdist bdist_wheel
python3 -m build
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This did not work for me, while the old command does still work:

:~/CLBlast/src/pyclblast$ python3 -m build
python3: No module named build.__main__; 'build' is a package and cannot be directly executed

And if I remove that folder then I get No module named build. This is with the relatively old Python 3.7.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,

No worries for the time taken, it is a festive period after all.
I have updated the changelog file.

Regarding the error, it seems to me that you have not installed the build python package. Running python3 -m pip install --upgrade build (or install it through your linux distro package manager) should make it work.
Keep in mind that the python -m build method for packaging, is the one currently described in the guide in the README.md link.
Let me know if the error persists after you install the build package.

One more unrelated thing, it would be good to add 'hints' (if you have any suggestion) for cmake to find the CLBlast library.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes of course, thanks that fixes it. I got confused because I also had a local build folder, and didn't think of build as a Python package. It does build now and the tests pass.

One more unrelated thing, it would be good to add 'hints' (if you have any suggestion) for cmake to find the CLBlast library.

Hmm yes indeed. Not sure what those locations would be though. Perhaps something similar to what I did for the clBLAS library here: https://github.com/CNugteren/CLBlast/blob/master/cmake/Modules/FindclBLAS.cmake#L25?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some hints for CLBlast in the CMakeLists.txt file and updated the README.md to inform the user about them.

@vathomass
Copy link
Contributor Author

Hi,

I have also switched to scikit-build-core for building the bindings, a module that uses CMake for building python packages. I assume it is better than the custom code I had added in the setup.py file.

Best
Thomas

Copy link
Owner

@CNugteren CNugteren left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks. Looks really neat now!

Thanks again for your contribution, let's merge this.

@CNugteren CNugteren merged commit 162783a into CNugteren:master Jan 21, 2024
5 checks passed
@vathomass vathomass deleted the pysetup_win branch February 2, 2024 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants