Welcome to simple-local-rag Discussions!

[mrdbourke]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[annamalaiarunachalam]

Thanks Bourke. that's an excellent video tutorial on the RAG. quite comprehensive and informative. thank you. Just a couple of things - I tried it in my laptop which doesn't have any GPU. the embeddings model downloaded to the HF cache (in the local machine), and it worked fine. however the LLM model (I changed it to GEMMA-2B-IT), though it downloaded to the cache, the code that runs the model did not work for me. It threw gemmaTokenizer not available error. [ValueError: Tokenizer class GemmaTokenizer does not exist or is not currently imported.].
if you happen to read this comment, please help me how to resolve this. Thank you.

About Me:
I am learning genAI. I am keen to build a RAG pipeline. I want to try both the local pipeline and the API based. I want to load a bunch of documents of various formats, process them, chunk them, embed them, store it in a vector DB. embed the query, retrieve relevant chunks, augment the prompt with the base_prompt, query, and the context, and generate a suitable response.

[josmithiii]

This is great, thanks! For my application, I need to also retrieve metadata with each vector retrieved (it will effectively be a pointer to where that chunk came from, so that perplexity.ai style results can be compiled). Any tips on that direction? Thanks again!

[josmithiii]

I just looked at text_chunks_and_embeddings_df.csv, and I see how simple it will be to add a URL as a new field.
I also see how sentences are being combined and split.
In my case I have access to HTML versions of the documents, so I can generate a .csv file with only complete sentences and/or paragraphs for each chunk. That's got to be better than what I see in the PDF extraction, right?

[EffectiveAgileDev]

I get this error when trying to load the google/gemma-2b-it model from Huggingface.co...

OSError: You are trying to access a gated repo.
Make sure to have access to it at https://huggingface.co/google/gemma-2b-it.
401 Client Error. (Request ID: Root=1-663ead2b-74db3e991e085e83278024fb;6e8160cf-4892-4349-9bfc-8d17b5d8a932)

Cannot access gated repo for url https://huggingface.co/google/gemma-2b-it/resolve/main/config.json.
Access to model google/gemma-2b-it is restricted. You must be authenticated to access it.

Does this mean that they have added a level of authorization since you published the video? I am logged into Huggingface and the model card says "Gated model
You have been granted access to this model".

How do I fix it?