Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

possibly speed up save_object #1040

Open
stemangiola opened this issue Nov 20, 2024 · 2 comments
Open

possibly speed up save_object #1040

stemangiola opened this issue Nov 20, 2024 · 2 comments
Labels
feature New feature or request

Comments

@stemangiola
Copy link

stemangiola commented Nov 20, 2024

save_object is slow for big models (~10K parameters)

Possible improvement 1

qs2 is a very fast format for saving and reading data. For huge models, fit$save_object() is slow in the rds format.

It would be great to ass the qs2 (or qs) option to save_object.

Possible improvement 2

allow save_object to save a set of parameters. Would this speed up the function?

Possible improvement 3

If we had the option of saving or reading draws as DuckDB, making the summary of one parameter would not require the loading of the whole posterior draws.

@stemangiola stemangiola added the feature New feature or request label Nov 20, 2024
@stemangiola stemangiola changed the title Add qs2 fomat to save_object possibly speed up save_object Nov 20, 2024
@jgabry
Copy link
Member

jgabry commented Nov 20, 2024

Yeah, it's definitely slow for big models. We do have a section in the vignette (contributed by @wlandau) showing how to do it with qs::qsave(), but there's not a built-in option at the moment:

https://mc-stan.org/cmdstanr/articles/cmdstanr.html#saving-fitted-model-objects

@jgabry
Copy link
Member

jgabry commented Nov 20, 2024

For now #1042 will point to this section of the vignette from the documentation of save_object. But I agree that these are good suggestions!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants