Skip to content

Commit

Permalink
Update README.md after linting
Browse files Browse the repository at this point in the history
  • Loading branch information
SichengStevenLi authored Aug 6, 2024
1 parent fa11ff3 commit bb47ef5
Showing 1 changed file with 23 additions and 19 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -64,36 +64,40 @@ python more_data_type.py -m <path_to_model> -t <path_to_tokenizer> -l <low_bit_f
> <=4.34.
### Text2SQL Example
This example [text2sql](./text2sql.py) demonstrates how to use LlamaIndex with `ipex-llm` to run a text-to-SQL model on Intel hardware. This example shows how to create a database, define a schema, and run SQL queries using low-bit model optimized with `ipex-llm`.

This example [text2sql](./text2sql.py) demonstrates how to use LlamaIndex with `ipex-llm` to run a text-to-SQL model on Intel hardware. This example shows how to create a database, define a schema, and run SQL queries using low-bit model optimized with `ipex-llm`.
### Setup

It requires `llama-index-embeddings-ipex-llm` package as it uses `ipex-llm` embedding.
>

> ```bash
> pip install llama-index-embeddings-ipex-llm
> ```
#### Runtime Configurations
For optimal performance, it is recommended to set several environment variables based on your device:
- For Windows Users with Intel Core Ultra integrated GPU
In Anaconda Prompt:
>>```
>>set SYCL_CACHE_PERSISTENT=1
>>set BIGDL_LLM_XMX_DISABLED=1
>>```
In Anaconda Prompt:
> >```
> >set SYCL_CACHE_PERSISTENT=1
> >set BIGDL_LLM_XMX_DISABLED=1
> >```
- For Linux Users with Intel Arc A-Series GPU:
>>```
>># Configure oneAPI environment variables. Required step for APT or offline installed oneAPI.
>># Skip this step for PIP-installed oneAPI since the environment has already been configured in LD_LIBRARY_PATH.
>>source /opt/intel/oneapi/setvars.sh
>>
>># Recommended Environment Variables for optimal performance
>>export USE_XETLA=OFF
>>export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
>>export SYCL_CACHE_PERSISTENT=1
>>```
> >```
> ># Configure oneAPI environment variables. Required step for APT or offline installed oneAPI.
> ># Skip this step for PIP-installed oneAPI since the environment has already been configured in LD_LIBRARY_PATH.
> >source /opt/intel/oneapi/setvars.sh
> >
> ># Recommended Environment Variables for optimal performance
> >export USE_XETLA=OFF
> >export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
> >export SYCL_CACHE_PERSISTENT=1
> >```
>**NOTE** For the first time that each model runs on Intel iGPU/Intel Arc A300-Series or Pro A60, it may take several minutes to compile.
Expand All @@ -105,6 +109,6 @@ Then, run the example as following:
python text2sql.py -m <path_to_model> -d <device> -e <path_to_embedding_model> -q <query_to_LLM> -n <num_predict>
```
>Please note that in this example we'll use [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) model for demonstration, as well as [bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) for our embedding model. It requires updating transformers and tokenizers packages. But you are also welcomed to use other models.
> Please note that in this example we'll use [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) model for demonstration, as well as [bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) for our embedding model. It requires updating transformers and tokenizers packages. But you are also welcomed to use other models.
>If you use other LLMs and encounter output issues, please try changing it.
> If you use other LLMs and encounter output issues, please try changing it.

0 comments on commit bb47ef5

Please sign in to comment.