Compatibility Issue when using v0.2.0 with transformers and tensorflow

[aws-tianquaw]

I recently encountered some compatibility issue when using sentencepiece v0.2.0 together with latest transformers and tensorflow packages. When I ran some Python script that imports AutoProcessor class from transformers, the Python kernel was killed with the following error message:

ERROR: Flag 'minloglevel' was defined more than once (in files 'src/error.cc' and 'home/conda/feedstock_root/build_artifacts/abseil-split_1720857154496/work/absl/log/flags.cc').

I tried to downgrade sentencepiece to v0.1.99 and then the issue was solved.

Commands to reproduce the issue:

conda create -n "py311" python=3.11
source activate py311
conda install -c conda-forge transformers==4.45 tensorflow==2.17 sentencepiece==0.2
python -c "from transformers import AutoProcessor"

[Genjidegaulle]

Running into a similar issue, with the same temporary resolution of downgrading sentencepiece to 0.1.99.