Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schedule Generation Performance Optimization #357

Merged
merged 1 commit into from
Dec 8, 2024

Conversation

jack-of-some-trades
Copy link
Contributor

Small, but notable, performance increase to the _tryholidays() function.

In testing I found that generating market calendars and date ranges comprised a significant portion of what my code was spending time doing. While the majority of the time spent is in panda's date_range function I did find that the _tryholidays function was abnormally slow.

It turns out that some calendars, such as NYSE, use Holiday Calendars for Special opens/closes that behave more like Ad-hoc Days as than Holidays. This is a format that Exchange Calendars follows as well. Currently when making a schedule, these dates are retrieved through Pandas' Holiday Calendar System which is designed around reoccurring holidays making the date retrieval far slower than it needs to be.

I added a check that manually retrieves all the single occurrence holiday dates if it can, and defers to Pandas' Holiday Calendar system if it needs to. This is a pretty niche optimization since it is most noticeable when generating a lot of schedules, but since I found it I figured I'd share it.

Before the Optimization:
image
After the Optimization:
image

Code Executed during profiling:
import pandas_market_calendars as mcal
from timeit import timeit
NYSE = mcal.get_calendar("NYSE")
special_dates = lambda: NYSE.special_dates("market_open", "2025-01-01", "2025-02-01")
print(f"{timeit(special_dates, number=6000)/6000 = }")

@rsheftel
Copy link
Owner

rsheftel commented Dec 8, 2024

Great improvement. thank you

@rsheftel rsheftel merged commit f7c23f5 into rsheftel:dev Dec 8, 2024
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants