Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make sure params --split and --merge are not specified at same time in gguf-split #9619

Merged
merged 3 commits into from
Oct 2, 2024

Conversation

kylo5aby
Copy link
Contributor

Copy link
Owner

@ggerganov ggerganov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These checks in the loop are hard to follow:

        if (is_op_set) {
            throw std::invalid_argument("error: either --split or --merge can be specified, but not both");
        }
        ...
        if (is_mode_set) {
            throw std::invalid_argument("error: either --split-max-tensors or --split-max-size can be specified, but not both");
        }

Seems something wrong with them already, or at the very least will cause trouble in the future if we add more args. Better to fix this properly.

@ngxson
Copy link
Collaborator

ngxson commented Sep 24, 2024

I'd suggest adding SPLIT_OP_NONE and enum split_mode for it (default to SPLIT_MODE_NONE).

will cause trouble in the future if we add more args.

For gguf-split, I don't think there will be many args added in the future, as this example is quite basic. This check is pretty much optional though, it's there for better user experience. One way is to remove it altogether if it causes trouble.

@kylo5aby
Copy link
Contributor Author

I'd suggest adding SPLIT_OP_NONE and enum split_mode for it (default to SPLIT_MODE_NONE).

will cause trouble in the future if we add more args.

For gguf-split, I don't think there will be many args added in the future, as this example is quite basic. This check is pretty much optional though, it's there for better user experience. One way is to remove it altogether if it causes trouble.

yes, I think these enums will be helpful for handling repeated parameters, which is better for user experience

examples/gguf-split/gguf-split.cpp Outdated Show resolved Hide resolved
@ggerganov ggerganov merged commit 76b37d1 into ggerganov:master Oct 2, 2024
53 checks passed
dsx1986 pushed a commit to dsx1986/llama.cpp that referenced this pull request Oct 29, 2024
* make sure params --split and --merge are not specified at same time

* update gguf-split params parse logic

* Update examples/gguf-split/gguf-split.cpp

Co-authored-by: slaren <[email protected]>

---------

Co-authored-by: Xuan Son Nguyen <[email protected]>
Co-authored-by: slaren <[email protected]>
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Nov 15, 2024
* make sure params --split and --merge are not specified at same time

* update gguf-split params parse logic

* Update examples/gguf-split/gguf-split.cpp

Co-authored-by: slaren <[email protected]>

---------

Co-authored-by: Xuan Son Nguyen <[email protected]>
Co-authored-by: slaren <[email protected]>
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Nov 18, 2024
* make sure params --split and --merge are not specified at same time

* update gguf-split params parse logic

* Update examples/gguf-split/gguf-split.cpp

Co-authored-by: slaren <[email protected]>

---------

Co-authored-by: Xuan Son Nguyen <[email protected]>
Co-authored-by: slaren <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants