From d6426b4a033939a402753a49210af81048db65f5 Mon Sep 17 00:00:00 2001 From: wonderingabout <38690718+wonderingabout@users.noreply.github.com> Date: Wed, 12 Dec 2018 12:59:19 +0100 Subject: [PATCH 1/7] add FAQ questions 7 and 8 and their answers See this comment for details : https://github.com/Tencent/PhoenixGo/issues/23#issuecomment-444374444 --- README.md | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/README.md b/README.md index 6385d28..aae6d3c 100644 --- a/README.md +++ b/README.md @@ -229,3 +229,44 @@ time_control { `c_denom` and `c_maxply` are parameters for deciding how to use the "main time". `reserved_time` is how many seconds should reserved (for network latency) in "byo-yomi time". + +**7. Syntax error (Windows)** + +For windows, +- in config file, you need to write path with `/` and not `\` in the config file .conf +for example : + +``` +model_config { + train_dir: "c:/users/amd2018/Downloads/PhoenixGo/ckpt" +``` + +- However, in cmd.exe you need to write paths with `\` and not `/` +Also in cmd.exe command format on windows needs a space, +for example : + +`mcts_main.exe --gtp --config_path C:\Users\amd2018\Downloads\PhoenixGo\etc\mcts_1gpu_notensorrt.conf` + +See point 8. below + + +**8. '"ckpt/zero.ckpt-20b-v1.FP32.PLAN"' error: No such file or directory** + +This fix works for all systems : Linux, Mac, Windows, only the name of the ckpt file changes +Modify your config file and write the full path of your ckpt folder : + +for example, for linux, if username is amd2018 : + +``` +model_config { + train_dir: "/home/amd2018/PhoenixGo/ckpt" +``` + +for example, for windows, + +``` +model_config { + train_dir: "c:/users/amd2018/Downloads/PhoenixGo/ckpt" +``` + + From 4ded8de97818bdb45e53ccc74998ab62ff7d1e4d Mon Sep 17 00:00:00 2001 From: wonderingabout <38690718+wonderingabout@users.noreply.github.com> Date: Wed, 12 Dec 2018 15:02:08 +0100 Subject: [PATCH 2/7] for example layout --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index aae6d3c..dd89528 100644 --- a/README.md +++ b/README.md @@ -233,7 +233,8 @@ time_control { **7. Syntax error (Windows)** For windows, -- in config file, you need to write path with `/` and not `\` in the config file .conf +- in config file, you need to write path with `/` and not `\` in the config file .conf, + for example : ``` @@ -243,6 +244,7 @@ model_config { - However, in cmd.exe you need to write paths with `\` and not `/` Also in cmd.exe command format on windows needs a space, + for example : `mcts_main.exe --gtp --config_path C:\Users\amd2018\Downloads\PhoenixGo\etc\mcts_1gpu_notensorrt.conf` @@ -255,7 +257,7 @@ See point 8. below This fix works for all systems : Linux, Mac, Windows, only the name of the ckpt file changes Modify your config file and write the full path of your ckpt folder : -for example, for linux, if username is amd2018 : +for example : ``` model_config { From b9ad48ec4a940dc8bad1db35219e5d733d60c92a Mon Sep 17 00:00:00 2001 From: wonderingabout <38690718+wonderingabout@users.noreply.github.com> Date: Wed, 12 Dec 2018 15:58:21 +0100 Subject: [PATCH 3/7] pre version 1 --- README.md | 84 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 81 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index dd89528..1057223 100644 --- a/README.md +++ b/README.md @@ -249,8 +249,7 @@ for example : `mcts_main.exe --gtp --config_path C:\Users\amd2018\Downloads\PhoenixGo\etc\mcts_1gpu_notensorrt.conf` -See point 8. below - +See point 8. below : **8. '"ckpt/zero.ckpt-20b-v1.FP32.PLAN"' error: No such file or directory** @@ -264,11 +263,90 @@ model_config { train_dir: "/home/amd2018/PhoenixGo/ckpt" ``` -for example, for windows, +for example, for windows : ``` model_config { train_dir: "c:/users/amd2018/Downloads/PhoenixGo/ckpt" ``` +**9. Most common path errors during bazel configure** + +### First of all, please make sure you did the post install path exports + +in the example below, we are using ubuntu 16.04 LTS is used with deb install of cuda 9.0 , +cudnn 7.1.4, and tensorrt 3.0.4, but other linux distributions with tar install are possible too + +The settings below have been tested to be working and to fix most common path issues, and +are shown as an interactive help : + + +- a) after cuda 9.0 deb install, it is needed to do : + +`export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}} && sudo reboot` + +for ubuntu users, it is also recommended to add path in /etc/environment + +`sudo nano /etc/environment` + +add this part in the paths (with the `:`) `:/usr/local/cuda/bin` , then save and exit and reboot to finalize + +- b) and after cudnn 7.1.4 deb install : + +check if cudnn works by compiling and runing a cudnn code sample: + +``` +cp -r /usr/src/cudnn_samples_v7/ ~ && cd ~/cudnn_samples_v7/mnistCUDNN && make clean && make && ./mnistCUDNN + +``` +should display this : `Test passed!` + +Reboot to finalize + +- c) post install for cudnn and cuda + +you can export path to bashrc : +`export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/extras/CUPTI/lib64" && source ~/.bashrc && sudo reboot` + +after reboot check if cuda installation is a success : + +`cat /proc/driver/nvidia/version` + +should display something like this : + +``` +NVRM version: NVIDIA UNIX x86_64 Kernel Module 384.130 Wed Mar 21 03:37:26 PDT 2018 +GCC version: gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10) + +``` +check if nvcc path works : + +`nvcc --version` + +should display something like this : + +``` +nvcc: NVIDIA (R) Cuda compiler driver +Copyright (c) 2005-2017 NVIDIA Corporation +Built on Fri_Sep__1_21:08:03_CDT_2017 +Cuda compilation tools, release 9.0, V9.0.176 + +``` + + + +Secondly, locate cuda and cudnn paths, and update database if not here +you will see something like this +Reboot your computer to finalize + +Thirdly, during bazel compile, this is the paths you need to put + + +Do you wish to build TensorFlow with CUDA support? [y/N]: y +CUDA support will be enabled for TensorFlow. + + +credits : +- [nvidia pdf instal guide for cuda 9.0](http://developer.download.nvidia.com/compute/cuda/9.0/Prod/docs/sidebar/CUDA_Installation_Guide_Linux.pdf) +- [medium.com/@zhanwenchen/](https://medium.com/@zhanwenchen/install-cuda-and-cudnn-for-tensorflow-gpu-on-ubuntu-79306e4ac04e) From 572dab939fa1cb4c7d02d5b23b8b8ec3163444f0 Mon Sep 17 00:00:00 2001 From: wonderingabout <38690718+wonderingabout@users.noreply.github.com> Date: Wed, 12 Dec 2018 16:17:45 +0100 Subject: [PATCH 4/7] release of question 9 : most common paths errors --- README.md | 53 ++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 46 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 1057223..22daaad 100644 --- a/README.md +++ b/README.md @@ -275,7 +275,7 @@ model_config { ### First of all, please make sure you did the post install path exports in the example below, we are using ubuntu 16.04 LTS is used with deb install of cuda 9.0 , -cudnn 7.1.4, and tensorrt 3.0.4, but other linux distributions with tar install are possible too +cudnn 7.1.4, tensorrt 3.0.4, as well as bazel 0.17.2 installed with a .run, but other linux distributions with nvidia tar install are possible too The settings below have been tested to be working and to fix most common path issues, and are shown as an interactive help : @@ -297,7 +297,6 @@ check if cudnn works by compiling and runing a cudnn code sample: ``` cp -r /usr/src/cudnn_samples_v7/ ~ && cd ~/cudnn_samples_v7/mnistCUDNN && make clean && make && ./mnistCUDNN - ``` should display this : `Test passed!` @@ -317,8 +316,8 @@ should display something like this : ``` NVRM version: NVIDIA UNIX x86_64 Kernel Module 384.130 Wed Mar 21 03:37:26 PDT 2018 GCC version: gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10) - ``` + check if nvcc path works : `nvcc --version` @@ -330,21 +329,61 @@ nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2017 NVIDIA Corporation Built on Fri_Sep__1_21:08:03_CDT_2017 Cuda compilation tools, release 9.0, V9.0.176 +``` + + +### Secondly, locate cuda and cudnn paths, and update database if not here + +Run this command : `locate libcudart.so && locate libcudnn.so.7` + +you need to see something like this : ``` +/usr/local/cuda-9.0/doc/man/man7/libcudart.so.7 +/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so +/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so.9.0 +/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so.9.0.176 +/usr/lib/x86_64-linux-gnu/libcudnn.so.7 +/usr/lib/x86_64-linux-gnu/libcudnn.so.7.1.4 +``` +If you don't see this, run this command : + +`sudo updatedb && locate libcudart.so && locate libcudnn.so.7` +It should now display all the cuda and cudnn paths same as above. +Reboot your computer to finalize. -Secondly, locate cuda and cudnn paths, and update database if not here -you will see something like this -Reboot your computer to finalize +### Thirdly, during bazel compile, this is the paths you need to put -Thirdly, during bazel compile, this is the paths you need to put +Press ENTER for every prompt to choose default settings, except for these : +- CUDA : choose `y` , version `9.0`, and custom cuda path `/usr/local/cuda-9.0/` +- cudnn : choose version `7.1` and custom cudnn path `/usr/lib/x86_64-linux-gnu/` +- if you use tensorrt do `y` and press enter to keep default path +same as below : + +``` Do you wish to build TensorFlow with CUDA support? [y/N]: y CUDA support will be enabled for TensorFlow. +Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 9.0]: 9.0 + +Please specify the location where CUDA 9.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: /usr/local/cuda-9.0/ + +Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7.0]: 7.1 + +Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda-9.0/]: /usr/lib/x86_64-linux-gnu/ + +Do you wish to build TensorFlow with TensorRT support? [y/N]: y +TensorRT support will be enabled for TensorFlow. + +Please specify the location where TensorRT is installed. [Default is /usr/lib/x86_64-linux-gnu]: +``` +these settings are just an example, but they have been tested to successfully work on ubuntu 16.04 LTS with deb install of cuda 9.0, deb install of cudnn 7.1.4, deb install of tensorrt 3.0.4, as well as .run install of bazel 0.17.2 + +they are provided as a general help for linux compile and run, they are not an obligatory method to use credits : - [nvidia pdf instal guide for cuda 9.0](http://developer.download.nvidia.com/compute/cuda/9.0/Prod/docs/sidebar/CUDA_Installation_Guide_Linux.pdf) From 4a8589fa282f9a25e2c805aff94b3ab5e73c9686 Mon Sep 17 00:00:00 2001 From: wonderingabout <38690718+wonderingabout@users.noreply.github.com> Date: Wed, 12 Dec 2018 18:40:37 +0100 Subject: [PATCH 5/7] minor typo fixes --- README.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 22daaad..7d39092 100644 --- a/README.md +++ b/README.md @@ -233,7 +233,9 @@ time_control { **7. Syntax error (Windows)** For windows, -- in config file, you need to write path with `/` and not `\` in the config file .conf, +- in config file, + +you need to write path with `/` and not `\` in the config file .conf, for example : @@ -242,8 +244,11 @@ model_config { train_dir: "c:/users/amd2018/Downloads/PhoenixGo/ckpt" ``` -- However, in cmd.exe you need to write paths with `\` and not `/` -Also in cmd.exe command format on windows needs a space, +- in cmd.exe, + +Here you need to write paths with `\` and not `/` + +Also command format on windows needs a space and not a `=` for example : @@ -254,6 +259,7 @@ See point 8. below : **8. '"ckpt/zero.ckpt-20b-v1.FP32.PLAN"' error: No such file or directory** This fix works for all systems : Linux, Mac, Windows, only the name of the ckpt file changes + Modify your config file and write the full path of your ckpt folder : for example : @@ -274,8 +280,8 @@ model_config { ### First of all, please make sure you did the post install path exports -in the example below, we are using ubuntu 16.04 LTS is used with deb install of cuda 9.0 , -cudnn 7.1.4, tensorrt 3.0.4, as well as bazel 0.17.2 installed with a .run, but other linux distributions with nvidia tar install are possible too +In the example below, ubuntu 16.04 LTS is used with cuda 9.0 (deb install) , +cudnn 7.1.4 (deb install), tensorrt 3.0.4 (deb install), as well as bazel 0.17.2 (.sh file install), but other linux distributions with nvidia tar install are possible too The settings below have been tested to be working and to fix most common path issues, and are shown as an interactive help : @@ -381,7 +387,7 @@ TensorRT support will be enabled for TensorFlow. Please specify the location where TensorRT is installed. [Default is /usr/lib/x86_64-linux-gnu]: ``` -these settings are just an example, but they have been tested to successfully work on ubuntu 16.04 LTS with deb install of cuda 9.0, deb install of cudnn 7.1.4, deb install of tensorrt 3.0.4, as well as .run install of bazel 0.17.2 +these settings are just an example, other settings or package versions or linux distributions are possible too, but this example has been tested to successfully work on ubuntu 16.04 LTS with deb install of cuda 9.0, deb install of cudnn 7.1.4, deb install of tensorrt 3.0.4, as well as .sh file install of bazel 0.17.2 they are provided as a general help for linux compile and run, they are not an obligatory method to use From 304bb75704f9af1bf17cedbf4fbc2db1da2c974f Mon Sep 17 00:00:00 2001 From: wonderingabout <38690718+wonderingabout@users.noreply.github.com> Date: Wed, 12 Dec 2018 18:44:07 +0100 Subject: [PATCH 6/7] minor typo fixes 2 --- README.md | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 7d39092..cdacc8d 100644 --- a/README.md +++ b/README.md @@ -246,11 +246,7 @@ model_config { - in cmd.exe, -Here you need to write paths with `\` and not `/` - -Also command format on windows needs a space and not a `=` - -for example : +Here you need to write paths with `\` and not `/`. Also command format on windows needs a space and not a `=`, for example : `mcts_main.exe --gtp --config_path C:\Users\amd2018\Downloads\PhoenixGo\etc\mcts_1gpu_notensorrt.conf` @@ -258,11 +254,7 @@ See point 8. below : **8. '"ckpt/zero.ckpt-20b-v1.FP32.PLAN"' error: No such file or directory** -This fix works for all systems : Linux, Mac, Windows, only the name of the ckpt file changes - -Modify your config file and write the full path of your ckpt folder : - -for example : +This fix works for all systems : Linux, Mac, Windows, only the name of the ckpt file changes. Modify your config file and write the full path of your ckpt folder, for example for linux : ``` model_config { @@ -313,9 +305,7 @@ Reboot to finalize you can export path to bashrc : `export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/extras/CUPTI/lib64" && source ~/.bashrc && sudo reboot` -after reboot check if cuda installation is a success : - -`cat /proc/driver/nvidia/version` +after reboot check if cuda installation is a success : `cat /proc/driver/nvidia/version` should display something like this : From 66c8af634126452fe784af5363ea55c29060b661 Mon Sep 17 00:00:00 2001 From: Daniel Zhang Date: Thu, 13 Dec 2018 15:33:39 +0800 Subject: [PATCH 7/7] move faq-9 to wiki since it was too long. --- README.md | 118 +----------------------------------------------------- 1 file changed, 2 insertions(+), 116 deletions(-) diff --git a/README.md b/README.md index cdacc8d..4dc132f 100644 --- a/README.md +++ b/README.md @@ -268,120 +268,6 @@ model_config { train_dir: "c:/users/amd2018/Downloads/PhoenixGo/ckpt" ``` -**9. Most common path errors during bazel configure** - -### First of all, please make sure you did the post install path exports - -In the example below, ubuntu 16.04 LTS is used with cuda 9.0 (deb install) , -cudnn 7.1.4 (deb install), tensorrt 3.0.4 (deb install), as well as bazel 0.17.2 (.sh file install), but other linux distributions with nvidia tar install are possible too - -The settings below have been tested to be working and to fix most common path issues, and -are shown as an interactive help : - - -- a) after cuda 9.0 deb install, it is needed to do : - -`export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}} && sudo reboot` - -for ubuntu users, it is also recommended to add path in /etc/environment - -`sudo nano /etc/environment` - -add this part in the paths (with the `:`) `:/usr/local/cuda/bin` , then save and exit and reboot to finalize - -- b) and after cudnn 7.1.4 deb install : - -check if cudnn works by compiling and runing a cudnn code sample: - -``` -cp -r /usr/src/cudnn_samples_v7/ ~ && cd ~/cudnn_samples_v7/mnistCUDNN && make clean && make && ./mnistCUDNN -``` -should display this : `Test passed!` - -Reboot to finalize - -- c) post install for cudnn and cuda - -you can export path to bashrc : -`export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/extras/CUPTI/lib64" && source ~/.bashrc && sudo reboot` - -after reboot check if cuda installation is a success : `cat /proc/driver/nvidia/version` - -should display something like this : - -``` -NVRM version: NVIDIA UNIX x86_64 Kernel Module 384.130 Wed Mar 21 03:37:26 PDT 2018 -GCC version: gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10) -``` - -check if nvcc path works : - -`nvcc --version` - -should display something like this : - -``` -nvcc: NVIDIA (R) Cuda compiler driver -Copyright (c) 2005-2017 NVIDIA Corporation -Built on Fri_Sep__1_21:08:03_CDT_2017 -Cuda compilation tools, release 9.0, V9.0.176 -``` - - -### Secondly, locate cuda and cudnn paths, and update database if not here - -Run this command : `locate libcudart.so && locate libcudnn.so.7` - -you need to see something like this : - -``` -/usr/local/cuda-9.0/doc/man/man7/libcudart.so.7 -/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so -/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so.9.0 -/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so.9.0.176 -/usr/lib/x86_64-linux-gnu/libcudnn.so.7 -/usr/lib/x86_64-linux-gnu/libcudnn.so.7.1.4 -``` -If you don't see this, run this command : - -`sudo updatedb && locate libcudart.so && locate libcudnn.so.7` - -It should now display all the cuda and cudnn paths same as above. -Reboot your computer to finalize. - - -### Thirdly, during bazel compile, this is the paths you need to put - -Press ENTER for every prompt to choose default settings, except for these : - -- CUDA : choose `y` , version `9.0`, and custom cuda path `/usr/local/cuda-9.0/` -- cudnn : choose version `7.1` and custom cudnn path `/usr/lib/x86_64-linux-gnu/` -- if you use tensorrt do `y` and press enter to keep default path - -same as below : - -``` -Do you wish to build TensorFlow with CUDA support? [y/N]: y -CUDA support will be enabled for TensorFlow. - -Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 9.0]: 9.0 - -Please specify the location where CUDA 9.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: /usr/local/cuda-9.0/ - -Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7.0]: 7.1 - -Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda-9.0/]: /usr/lib/x86_64-linux-gnu/ - -Do you wish to build TensorFlow with TensorRT support? [y/N]: y -TensorRT support will be enabled for TensorFlow. - -Please specify the location where TensorRT is installed. [Default is /usr/lib/x86_64-linux-gnu]: -``` -these settings are just an example, other settings or package versions or linux distributions are possible too, but this example has been tested to successfully work on ubuntu 16.04 LTS with deb install of cuda 9.0, deb install of cudnn 7.1.4, deb install of tensorrt 3.0.4, as well as .sh file install of bazel 0.17.2 - -they are provided as a general help for linux compile and run, they are not an obligatory method to use - -credits : -- [nvidia pdf instal guide for cuda 9.0](http://developer.download.nvidia.com/compute/cuda/9.0/Prod/docs/sidebar/CUDA_Installation_Guide_Linux.pdf) -- [medium.com/@zhanwenchen/](https://medium.com/@zhanwenchen/install-cuda-and-cudnn-for-tensorflow-gpu-on-ubuntu-79306e4ac04e) +**9. Path errors during bazel configure** +See https://github.com/Tencent/PhoenixGo/wiki/Install-cuda-and-do-bazel-configuration