-
Notifications
You must be signed in to change notification settings - Fork 238
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
KoAlpaca polyglot 12.8b Fine-tuning 시 에러문의 드립니다. #107
Comments
혹시
명령어로 두 패키지 버전을 최신으로 맞추고 한번 다시 실행해서 동일한 에러가 나는지 확인해주시겠어요? |
먼저 빠른 답변감사합니다. 두 패키지들을 업데이트 한 뒤 다시 실행해도 에러가 나는데요.. 다른 서버 (gpu 16장, 8장, 4장) 에서 실행해봐도 같은 에러가 나네요. Traceback (most recent call last):
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
안녕하세요,
12.8b 모델을 https://github.com/Beomi/KoAlpaca/blob/main/train_v1.1b/run_clm.py 코드로 A100 40G 8장에서 파인튜닝 하는중에 다음과 같이 에러가 납니다. (학습 스크립트는 https://github.com/Beomi/KoAlpaca/blob/main/train_v1.1b/train.sh 사용하였습니다.)
Traceback (most recent call last):
File "run_clm_2.py", line 636, in
main()
File "run_clm_2.py", line 412, in main
model = AutoModelForCausalLM.from_pretrained(
File "/usr/local/lib/python3.8/dist-packages/transformers/models/auto/auto_factory.py", line 467, in from_pretrained
return model_class.from_pretrained(
File "/usr/local/lib/python3.8/dist-packages/transformers/modeling_utils.py", line 2172, in from_pretrained
raise ValueError("Passing along a
device_map
requireslow_cpu_mem_usage=True
")ValueError: Passing along a
device_map
requireslow_cpu_mem_usage=True
그래서 모델 불러올때 low_cpu_mem_usage=True 옵션을 주었더니 아래와 같은 에러가 납니다.
Traceback (most recent call last):
File "run_clm_2.py", line 636, in
main()
File "run_clm_2.py", line 412, in main
model = AutoModelForCausalLM.from_pretrained(
File "/usr/local/lib/python3.8/dist-packages/transformers/models/auto/auto_factory.py", line 467, in from_pretrained
return model_class.from_pretrained(
File "/usr/local/lib/python3.8/dist-packages/transformers/modeling_utils.py", line 2180, in from_pretrained
raise ValueError(
ValueError: DeepSpeed Zero-3 is not compatible with
low_cpu_mem_usage=True
or with passing adevice_map
.깃헙에 공유된 코드 그대로, gpu 개수만 변경하여 진행해봤는데 에러가 나는데요, 혹시 이부분 도움주실 수 있으신지 문의드립니다.
The text was updated successfully, but these errors were encountered: