Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BabyLlama with CPP backend #2544

Closed
wants to merge 25 commits into from
Closed

BabyLlama with CPP backend #2544

wants to merge 25 commits into from

Commits on Aug 25, 2023

  1. Baby Llama - Porting run.c for integration and fixed clang type conve… 

    …rsion errors.

Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Aug 25, 2023
    Configuration menu
    Copy the full SHA
    641a708 View commit details
    Browse the repository at this point in the history

  2. Custom preprocess implementation 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Aug 25, 2023
    Configuration menu
    Copy the full SHA
    016e4f1 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2023

  1. Free memory only after the inference is done 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    38d3e93 View commit details
    Browse the repository at this point in the history

  2. Implement Postprocess 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    52a7927 View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2023

  1. Setting Fast compiler option 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    c675664 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2023

  1. Reading checkpoint path and tokenizer path from config file using folly 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    374a2e8 View commit details
    Browse the repository at this point in the history

  2. Removing run.c from cmake 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    48f522c View commit details
    Browse the repository at this point in the history

  3. Replace auto with appropriate data type 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    49a3015 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2023

  1. Using smartpointers and initializing the vector with appropriate size… 

    … upfront

Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    aeb1bb0 View commit details
    Browse the repository at this point in the history

  2. Using smartpointers 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    ee20424 View commit details
    Browse the repository at this point in the history

  3. Directly converting the tensor values to prompt token ids 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    f5d9799 View commit details
    Browse the repository at this point in the history

  4. Moving run.c and common variables to .cc file 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    9b3de26 View commit details
    Browse the repository at this point in the history

  5. Moving run.c to a separate folder 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    3e0e2c3 View commit details
    Browse the repository at this point in the history

  6. Uncommenting the original run.c main method 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    5c0495e View commit details
    Browse the repository at this point in the history

  7. Implemented destructor to free up resources 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    e75a5ae View commit details
    Browse the repository at this point in the history

  8. Supporting files for unit test 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    9afce52 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2023

  1. Processing all the batch inputs 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    0d12619 View commit details
    Browse the repository at this point in the history

  2. Setting InferenceMode guard 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    bd03fd8 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2023

  1. Updating InferenceMode to use torch::InferenceMode 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    d2dc632 View commit details
    Browse the repository at this point in the history

  2. Updating class name to BabyLlamaHandler 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    67b46aa View commit details
    Browse the repository at this point in the history

  3. Renaming llm_handler target to babyllama_handler 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    f30aab2 View commit details
    Browse the repository at this point in the history

  4. Adding dummy pt file 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    7174cde View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2023

  1. Typo Fix 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    6dc025b View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2023

  1. Calculate tokens/per second for batch input 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    450b85d View commit details
    Browse the repository at this point in the history

  2. Adding README.md for babyllama example 

    Signed-off-by: Shrinath Suresh <[email protected]>
    @shrinath-suresh
    shrinath-suresh committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    8d279be View commit details
    Browse the repository at this point in the history