Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Python 3.13's GIL-Free Mode #675

Open
gitctrlx opened this issue Aug 26, 2024 · 9 comments
Open

Support for Python 3.13's GIL-Free Mode #675

gitctrlx opened this issue Aug 26, 2024 · 9 comments

Comments

@gitctrlx
Copy link

Hello TA-Lib maintainers,

With the upcoming release of Python 3.13, there is an experimental feature allowing Python to run without the Global Interpreter Lock (GIL). This feature is described in PEP 703, which introduces the "free-threaded" mode for CPython.

You can find more details about this feature in the Free-threaded CPython documentation.

Since TA-Lib is a widely used library for technical analysis, especially in multithreaded environments, it would be beneficial to ensure compatibility with Python's GIL-free mode. This will allow users to fully utilize multi-core processors for parallel processing without the limitations of GIL.

Would it be possible to test and, if necessary, update TA-Lib to support this new mode? Your attention to this matter would be greatly appreciated by the community.

Thank you for your continuous support and development of TA-Lib!

@gitctrlx
Copy link
Author

image

@mrjbq7
Copy link
Collaborator

mrjbq7 commented Aug 28, 2024

I don't know who maintains the conda-forge package. I'd be happy to do so, if someone wants to work with me to transfer that... two things stick out:

  1. ta-lib-0-4.19 is old (tagged in September 2020, almost 4 years ago)
  2. ta-lib works fine with python 3.13 (https://github.com/TA-Lib/ta-lib-python/actions/runs/10589287581/job/29343164656)

@mrjbq7
Copy link
Collaborator

mrjbq7 commented Aug 28, 2024

Regarding no-gil, we will have to see what that means for thread-safety. At the very least there are global settings that the C functions use.

@gitctrlx
Copy link
Author

I don't know who maintains the conda-forge package. I'd be happy to do so, if someone wants to work with me to transfer that... two things stick out:

  1. ta-lib-0-4.19 is old (tagged in September 2020, almost 4 years ago)
  2. ta-lib works fine with python 3.13 (https://github.com/TA-Lib/ta-lib-python/actions/runs/10589287581/job/29343164656)

I wanted to provide an update regarding the installation of TA-Lib with Python 3.13. I encountered errors when trying to install TA-Lib using the GIL-free version of Python 3.13 (python3.13t.exe). However, the installation works correctly with the standard version of Python 3.13 (python.exe). Both versions were installed using the Windows installer from the official Python website. Here is the link to the release: Python 3.13.0 Release.

This suggests that there may be compatibility issues specifically with the no-GIL mode. Your attention to this matter would be appreciated.

image
image

@TATAjiHAU
Copy link

@mrjbq7 @gitctrlx ... any update on this... Think could really help in speed things up for large CSV data bases file. Which have like ohlc, wrt to time series, along with hundreds of indicators columns for every row.... 10GB + files i mean. Any way i can help with this?

@mrjbq7
Copy link
Collaborator

mrjbq7 commented Nov 13, 2024

I guess this is waiting for Cython 3.1 to be released?

https://cython.readthedocs.io/en/latest/src/userguide/freethreading.html

And we could start testing this with freethreading feature, but I haven't had time.

@mrjbq7
Copy link
Collaborator

mrjbq7 commented Nov 13, 2024

I released current master branch as 0.5.0, so if you'd like to help with this feature, I'd suggest:

  1. install cython 3.1 alpha release locally
  2. do the freethreading tag like the docs suggest
  3. build it and run local tests, report back?

@Tapanhaz
Copy link
Contributor

Tapanhaz commented Nov 22, 2024

I released current master branch as 0.5.0, so if you'd like to help with this feature, I'd suggest:

1. install cython 3.1 alpha release locally

2. do the freethreading tag like the docs suggest

3. build it and run local tests, report back?

Didn't get the 2nd point. I tried but on import of talib the program exits without showing any error:

Screenshot 2024-11-22 204232

@mrjbq7
Copy link
Collaborator

mrjbq7 commented Nov 22, 2024

Reading through this page:

https://cython.readthedocs.io/en/latest/src/userguide/freethreading.html

The module-level directive # cython: freethreading_compatible = True declares that the module is fully compatible with the free-threaded interpreter. When you specify this directive, importing the module will not cause the interpreter to re-enable the GIL. The directive itself does not do anything to ensure compatibility - it is simply a way for you to indicate that you have tested your module and are confident that it works.

There are perhaps other things on that page to be aware of also, but thats what I meant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants
@mrjbq7 @gitctrlx @Tapanhaz @TATAjiHAU and others