Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Would a cython python app benefit from this/how do i do this? #3

Open
hippylover opened this issue Oct 3, 2019 · 9 comments
Open
Labels
question Further information is requested

Comments

@hippylover
Copy link

Hi. I downloaded the precompiled version of build-anywhere earlier today. Then:
source /home/kris/x86_64-anywhere-linux-gnu

Then i made a .c file with cython(embedding the python interpreter):
cython --embed -3 program.py
this depends upon pyqt5 and enum(that again depends on some other standard libs in /usr/lib/python3.7 im not sure how to include).

Then how shall i compile this app making sure i include the right dependencies? I don't care if i have to copy the whole /usr/lib/python3.7 folder and set the sys.path inside the python program to look inside customdir/python3.7 if this is the only option, but first and foremost i just want to compile the app.

Then i will try to copy it to centos6 VM just to see if it runs.

@hippylover
Copy link
Author

I tried this:

$ gcc -O0 -fPIC -I /usr/include/python3.7m  -L/usr/lib -lpython3.7m -lcrypt -lpthread -ldl  -lutil -lm  -o program program.c
ld.lld: error: unknown -format value: elf64-x86-64 (supported formats: elf, default, binary)
clang-8: error: linker command failed with exit code 1 (use -v to see invocation)

but that is what i got.

Flags gotten through:

$ python3.7-config --ldflags
 -L/usr/lib -lpython3.7m -lcrypt -lpthread -ldl  -lutil -lm

@theopolis
Copy link
Owner

Hey @hippylover, I think you'd have to build each of those dependencies with this toolchain.

It you look at how ssdeep's build is configured, it installs itself into the toolchain: https://github.com/theopolis/build-anywhere/blob/master/tests/basic.sh#L49

Then other libraries/projects that depend on ssdeep can use the portable version.

@theopolis theopolis added the question Further information is requested label Oct 5, 2019
@hippylover
Copy link
Author

I think this is going a bit above my paygrade... im primarily a python dev. I don't speak C and all i know of C++ is half of the tutorial on learncpp.com...

Iv'e already spent two months on this madness(should have just learned golang).

If you could give it a shot i would be much obliged, for where you spend 2 hours i spend 2 months. I'm sure a lot of python devs would appreciate it too, because binary distribution is currently a nightmare with python and cython.

@theopolis
Copy link
Owner

Sure I can try later today. :)

@hippylover
Copy link
Author

Found some stuff that might be relevant: https://web.archive.org/web/20180926104719/http://general-purpose.io/2015/12/06/compiling-python-and-libpython-statically-using-musl-libc/
https://gist.github.com/maddouri/c4b97474f21fabc9ef61

These two show some configure flags etc to do it that should be relevant even if one doesn't use musl i think(!)
https://stackoverflow.com/questions/1150373/compile-the-python-interpreter-statically/1155092#1155092

I don't think i can use musl though. Because i need to use the pyqt libs which are built on glibc. To save download time i plan on letting the user install needed deps through a static mythical pip that i may or may not be able to build.

@hippylover
Copy link
Author

hippylover commented Oct 6, 2019

Btw, if i want to use pyqt does that also have to be compiled with build-anywhere or can i avoid it? Maybe alpine linux is a help, they only build musl binaries?

edit: i also need PyQtWebEngine from pip.

@hippylover
Copy link
Author

Well, seems like this will help: https://wiki.python.org/moin/BuildStatically. I will try it and report back. How's your progress? :D

@hippylover
Copy link
Author

I actually managed to make a portable no-dynamic libs python binary with included standard libs... I will outline the steps i took tomorrow when im not so tired.

I have two issues left though(no doubt before i get more issues xD). What on god's green earth shall i put in Setup/Setup.dist/Setup.local to get _decimal module to build?? This must be portable of course. I don't know if i need to build libmpdec portable too, if so i probably need help with that too.

That is one problem. Another problem is which line do i need for _sqlite3 module... I haven't tried that one so far. Will attempt now.

@theopolis
Copy link
Owner

Hey @hippylover, my approach was to build python using the build-anywhere toolchain then try to perform the small test you mentioned at the top.

It looks like the scope of what you are trying to accomplish keeps increasing. I do not know python well or what you are trying to ultimately achieve so I'm not sure I can be very helpful. I do recommend that if you want to ship something portable, it, and all of it's dependencies need to be built with build-anywhere.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants