forked from edooley7/dsp
-
Notifications
You must be signed in to change notification settings - Fork 0
/
README.crh
363 lines (313 loc) · 16.8 KB
/
README.crh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
HackerRank Info:
Cohort Enrolled: NYC Summer 2017
Cohort ID: nyc17_ds12
00a-markdown.md - done
00b-fork_repo.md - done
01a-install.md - done on fedora (revisit on mac for completeness)
01b-install_jupyter.md - todo!
02-editors.md - done (although revisit PyCharm and atom and Rodeo)
03-command_line.md - done
04-git.md - done
next: read more of http://git-scm.com/book/en/v2
05a-python.md - hackerrank - done
05b-python_advanced.md - hackerrank - done
05c-python_pandas.md - hackerrank - https://www.hackerrank.com/tests/beg202nchad/a3ae8be11d8345e83400e68ea9fa10e5 - done
06-linear_algebra.md - harckerrank - http://thisismetis.us3.list-manage2.com/track/click?u=a13fe84fe221defe011ff0d93&id=f9bace37cf&e=2dc3f53bdb - done
07-statistics.md -
- https://github.com/thisismetis/dsp/blob/master/07-statistics.md?mc_cid=ac2b0f9662&mc_eid=2dc3f53bdb
- cloned.
- add to ../ThinkStats2/code/
08-more_resources.md
1/ Using Linux fedora 25 on Dell XPS.
2/ docker
https://fedoraproject.org/wiki/Docker
sudo dnf install docker
sudo systemctl status docker
sudo docker run -i -t fedora /bin/bash
sudo groupadd docker
sudo usermod -aG docker crh
# logout/in to pickup group
[crh@xps dsp]$ sudo docker run fedora /bin/echo hello world
hello world
[crh@xps dsp]$ sudo docker info
Containers: 7
Running: 0
Paused: 0
Stopped: 7
Images: 6
Server Version: 1.12.6
Storage Driver: devicemapper
Pool Name: fedora_new--host--4-docker--pool
Pool Blocksize: 524.3 kB
Base Device Size: 10.74 GB
Backing Filesystem: xfs
Data file:
Metadata file:
Data Space Used: 1.219 GB
Data Space Total: 13.21 GB
Data Space Available: 11.99 GB
Metadata Space Used: 393.2 kB
Metadata Space Total: 146.8 MB
Metadata Space Available: 146.4 MB
Thin Pool Minimum Free Space: 1.321 GB
Udev Sync Supported: true
Deferred Removal Enabled: true
Deferred Deletion Enabled: false
Deferred Deleted Device Count: 0
Library Version: 1.02.136 (2016-11-05)
Logging Driver: journald
Cgroup Driver: systemd
Plugins:
Volume: local
Network: overlay bridge null host
Swarm: inactive
Runtimes: oci runc
Default Runtime: oci
Security Options: seccomp selinux
Kernel Version: 4.10.14-200.fc25.x86_64
Operating System: Fedora 25 (Workstation Edition)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 2
CPUs: 4
Total Memory: 7.702 GiB
Name: xps
ID: UXWO:LPDU:FS57:MNOR:2IPR:QKRY:D57B:65NG:7RVJ:JRPC:FRIO:OJCG
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
172.30.0.0/16
127.0.0.0/8
Registries: docker.io (secure)
[crh@xps dsp]$ sudo docker run busybox /bin/echo this is an echo
Unable to find image 'busybox:latest' locally
Trying to pull repository docker.io/library/busybox ...
sha256:32f093055929dbc23dec4d03e09dfe971f5973a9ca5cf059cbfb644c206aa83f: Pulling from docker.io/library/busybox
7520415ce762: Pull complete
Digest: sha256:32f093055929dbc23dec4d03e09dfe971f5973a9ca5cf059cbfb644c206aa83f
Status: Downloaded newer image for docker.io/busybox:latest
this is an echo
[crh@xps dsp]$ sudo dnf -y install fedora-dockerfiles
Tue Jun 6 15:35:15 EDT 2017
updated to use clone of fork hosted on github.com/chris-r-harwell
added ssh and gpg
git config --global user.signingkey B8B7C4916E5210CB
test ssh:
[crh@xps dsp]$ ssh -T [email protected]
Hi chris-r-harwell! You've successfully authenticated, but GitHub does not provide shell access.
Tue Jun 6 15:37:34 EDT 2017
read 00a-markdown.md
read 00b-fork_repo.md, edit and commit and push
git --version
git version 2.9.4
python3
quit()
[crh@xps dsp]$ sudo dnf install python3-ipython\*
ipython3
ipython3 notebook
sudo dnf search spyder
sudo dnf install python3-spyder
spyder3
loaded the Generic plugin
[crh@xps dsp]$ python3
Python 3.5.3 (default, May 10 2017, 15:05:55)
[GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> import scipy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'scipy'
>>> import matplotlib
>>> import pandas
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'pandas'
>>> import statsmodel
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'statsmodel'
>>> import sklearn
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'sklearn'
[crh@xps dsp]$ sudo dnf install -y python3-scipy python3-pandas\* python3-statsmodels\* python3-scikit\*
[crh@xps dsp]$ python3
Python 3.5.3 (default, May 10 2017, 15:05:55)
[GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> import scipy
>>> import matplotlib
>>> import pandas
>>> import statsmodels
>>> import sklearn
>>> quit()
sudo dnf install -y python3-pyflakes python3-flake8 python3-pytest-flakes python3-pep8
[crh@xpc dsp]$ anaconda --version
/usr/lib64/python3.5/site-packages/pyanaconda/startup_utils.py:20: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
anaconda 25.20.9-1
- jupyter?
sudo dnf install python3-jupyter\* python3-ipykernel python3-jupyroot python3-nbformat
https://ask.fedoraproject.org/en/question/105166/difference-between-jupyter-and-ipython-notebooks/
sudo dnf remove python3-jupyter\* python3-ipykernel python3-jupyroot python3-nbformat
python-jupyter_core.noarch
hmm. still need more work/info to install/launch jupyter somehow
Tue Jun 13 17:38:50 EDT 2017
Try this on xpc
:
[crh@xpc dsp]$ pip3 install --user jupyter
Collecting jupyter
Using cached jupyter-1.0.0-py2.py3-none-any.whl
Collecting nbconvert (from jupyter)
Using cached nbconvert-5.2.1-py2.py3-none-any.whl
Collecting jupyter-console (from jupyter)
Using cached jupyter_console-5.1.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): ipykernel in /usr/lib/python3.5/site-packages (from jupyter)
Collecting qtconsole (from jupyter)
Using cached qtconsole-4.3.0-py2.py3-none-any.whl
Collecting ipywidgets (from jupyter)
Using cached ipywidgets-6.0.0-py2.py3-none-any.whl
Collecting notebook (from jupyter)
Using cached notebook-5.0.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): jinja2 in /usr/lib/python3.5/site-packages (from nbconvert->jupyter)
Requirement already satisfied (use --upgrade to upgrade): mistune!=0.6 in /usr/lib64/python3.5/site-packages (from nbconvert->jupyter)
Requirement already satisfied (use --upgrade to upgrade): jupyter-core in /usr/lib/python3.5/site-packages (from nbconvert->jupyter)
Collecting pandocfilters>=1.4.1 (from nbconvert->jupyter)
Using cached pandocfilters-1.4.1.tar.gz
Collecting testpath (from nbconvert->jupyter)
Using cached testpath-0.3.1-py2.py3-none-any.whl
Collecting bleach (from nbconvert->jupyter)
Using cached bleach-2.0.0-py2.py3-none-any.whl
Collecting entrypoints>=0.2.2 (from nbconvert->jupyter)
Using cached entrypoints-0.2.3-py2.py3-none-any.whl
Collecting traitlets>=4.2 (from nbconvert->jupyter)
Using cached traitlets-4.3.2-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): pygments in /usr/lib/python3.5/site-packages (from nbconvert->jupyter)
Requirement already satisfied (use --upgrade to upgrade): nbformat in /usr/lib/python3.5/site-packages (from nbconvert->jupyter)
Collecting prompt-toolkit<2.0.0,>=1.0.0 (from jupyter-console->jupyter)
Using cached prompt_toolkit-1.0.14-py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): ipython in /usr/lib/python3.5/site-packages (from jupyter-console->jupyter)
Requirement already satisfied (use --upgrade to upgrade): jupyter-client in /usr/lib/python3.5/site-packages (from jupyter-console->jupyter)
Requirement already satisfied (use --upgrade to upgrade): ipython-genutils in /usr/lib/python3.5/site-packages (from qtconsole->jupyter)
Collecting widgetsnbextension~=2.0.0 (from ipywidgets->jupyter)
Using cached widgetsnbextension-2.0.0-py2.py3-none-any.whl
Collecting terminado>=0.3.3; sys_platform != "win32" (from notebook->jupyter)
Using cached terminado-0.6.tar.gz
Requirement already satisfied (use --upgrade to upgrade): tornado>=4 in /usr/lib64/python3.5/site-packages (from notebook->jupyter)
Requirement already satisfied (use --upgrade to upgrade): MarkupSafe in /usr/lib64/python3.5/site-packages (from jinja2->nbconvert->jupyter)
Collecting html5lib>=0.99999999 (from bleach->nbconvert->jupyter)
Using cached html5lib-0.999999999-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python3.5/site-packages (from bleach->nbconvert->jupyter)
Requirement already satisfied (use --upgrade to upgrade): decorator in /usr/lib/python3.5/site-packages (from traitlets>=4.2->nbconvert->jupyter)
Collecting wcwidth (from prompt-toolkit<2.0.0,>=1.0.0->jupyter-console->jupyter)
Using cached wcwidth-0.1.7-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): ptyprocess in /usr/lib/python3.5/site-packages (from terminado>=0.3.3; sys_platform != "win32"->notebook->jupyter)
Requirement already satisfied (use --upgrade to upgrade): setuptools>=18.5 in /usr/lib/python3.5/site-packages (from html5lib>=0.99999999->bleach->nbconvert->jupyter)
Collecting webencodings (from html5lib>=0.99999999->bleach->nbconvert->jupyter)
Using cached webencodings-0.5.1-py2.py3-none-any.whl
Installing collected packages: pandocfilters, testpath, webencodings, html5lib, bleach, entrypoints, traitlets, nbconvert, wcwidth, prompt-toolkit, jupyter-console, qtconsole, terminado, notebook, widgetsnbextension, ipywidgets, jupyter
Running setup.py install for pandocfilters ... done
Running setup.py install for terminado ... done
Successfully installed bleach entrypoints html5lib-0.999 ipywidgets jupyter jupyter-console nbconvert notebook pandocfilters prompt-toolkit qtconsole terminado testpath traitlets-4.0.0 wcwidth webencodings widgetsnbextension
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[crh@xpc dsp]$
And now this seems to work:
[crh@xpc dsp]$ python3.5 ~/.local/lib/python3.5/site-packages/jupyter.py notebook
[I 17:41:52.794 NotebookApp] Copying /home/crh/.ipython/profile_default/security/notebook_cookie_secret -> /home/crh/.local/share/jupyter/notebook_cookie_secret
[I 17:41:52.803 NotebookApp] Writing notebook server cookie secret to /run/user/1000/jupyter/notebook_cookie_secret
[I 17:41:52.858 NotebookApp] Serving notebooks from local directory: /home/crh/git/dsp
[I 17:41:52.858 NotebookApp] 0 active kernels
[I 17:41:52.858 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/?token=4ee9023144ad2479bea46eb5d2dd52ba1f99c0dbffe64ede
[I 17:41:52.858 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 17:41:52.859 NotebookApp]
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://localhost:8888/?token=4ee9023144ad2479bea46eb5d2dd52ba1f99c0dbffe64ede
OR this?
[crh@xpc dsp]$ sudo dnf clean packages
27 files removed
[crh@xpc dsp]$
[crh@xpc dsp]$ sudo dnf install python3-nbformat.noarch python3-ipykernel.noarch python3-qtconsole.noarch python3-jupyter-client
Last metadata expiration check: 1:42:22 ago on Tue Jun 13 16:17:25 2017.
Package python3-ipykernel-4.5.2-3.fc25.noarch is already installed, skipping.
Dependencies resolved.
=========================================================================================================
Package Arch Version Repository Size
=========================================================================================================
Installing:
python3-ipython_genutils noarch 0.1.0-5.fc25 fedora 52 k
python3-jupyter-client noarch 4.4.0-1.fc25 updates 169 k
python3-jupyter-core noarch 4.1.0-5.fc25 fedora 77 k
python3-nbformat noarch 4.0.0-5.fc25 fedora 150 k
python3-qtconsole noarch 4.2.1-6.fc25 updates 181 k
python3-traitlets noarch 4.0.0-4.fc25 fedora 123 k
Transaction Summary
=========================================================================================================
Install 6 Packages
Total download size: 752 k
Installed size: 3.5 M
Is this ok [y/N]: y
Downloading Packages:
(1/6): python3-jupyter-client-4.4.0-1.fc25.noarch.rpm 1.2 MB/s | 169 kB 00:00
(2/6): python3-qtconsole-4.2.1-6.fc25.noarch.rpm 1.0 MB/s | 181 kB 00:00
(3/6): python3-nbformat-4.0.0-5.fc25.noarch.rpm 513 kB/s | 150 kB 00:00
(4/6): python3-ipython_genutils-0.1.0-5.fc25.noarch.rpm 280 kB/s | 52 kB 00:00
(5/6): python3-traitlets-4.0.0-4.fc25.noarch.rpm 542 kB/s | 123 kB 00:00
(6/6): python3-jupyter-core-4.1.0-5.fc25.noarch.rpm 419 kB/s | 77 kB 00:00
---------------------------------------------------------------------------------------------------------
Total 519 kB/s | 752 kB 00:01
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Installing : python3-ipython_genutils-0.1.0-5.fc25.noarch 1/6
Installing : python3-traitlets-4.0.0-4.fc25.noarch 2/6
Installing : python3-jupyter-core-4.1.0-5.fc25.noarch 3/6
Installing : python3-jupyter-client-4.4.0-1.fc25.noarch 4/6
Installing : python3-qtconsole-4.2.1-6.fc25.noarch 5/6
Installing : python3-nbformat-4.0.0-5.fc25.noarch 6/6
Verifying : python3-nbformat-4.0.0-5.fc25.noarch 1/6
Verifying : python3-qtconsole-4.2.1-6.fc25.noarch 2/6
Verifying : python3-jupyter-client-4.4.0-1.fc25.noarch 3/6
Verifying : python3-ipython_genutils-0.1.0-5.fc25.noarch 4/6
Verifying : python3-traitlets-4.0.0-4.fc25.noarch 5/6
Verifying : python3-jupyter-core-4.1.0-5.fc25.noarch 6/6
Installed:
python3-ipython_genutils.noarch 0.1.0-5.fc25 python3-jupyter-client.noarch 4.4.0-1.fc25
python3-jupyter-core.noarch 4.1.0-5.fc25 python3-nbformat.noarch 4.0.0-5.fc25
python3-qtconsole.noarch 4.2.1-6.fc25 python3-traitlets.noarch 4.0.0-4.fc25
Complete!
[crh@xpc dsp]$ jupyter-
jupyter-3 jupyter-kernelspec-3 jupyter-nbextension jupyter-serverextension
jupyter-3.5 jupyter-kernelspec-3.5 jupyter-notebook jupyter-trust
jupyter-bundlerextension jupyter-migrate-3 jupyter-qtconsole
jupyter-console jupyter-migrate-3.5 jupyter-qtconsole-3
jupyter-kernelspec jupyter-nbconvert jupyter-qtconsole-3.5
[crh@xpc dsp]$ jupyter-3.5 notebook
[I 18:00:11.632 NotebookApp] The port 8888 is already in use, trying another port.
[I 18:00:11.665 NotebookApp] Serving notebooks from local directory: /home/crh/git/dsp
[I 18:00:11.665 NotebookApp] 0 active kernels
[I 18:00:11.665 NotebookApp] The Jupyter Notebook is running at: http://localhost:8889/?token=1aebb80f2f4de5de16fa463c17fd2907b86d95fb0d01e1a5
[I 18:00:11.665 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 18:00:11.666 NotebookApp]
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://localhost:8889/?token=1aebb80f2f4de5de16fa463c17fd2907b86d95fb0d01e1a5
but then
[W 18:00:43.870 NotebookApp] Native kernel (python3) is not available
Mon Jun 19 07:40:47 EDT 2017
cd ..
[crh@xps git]$ git clone [email protected]:AllenDowney/ThinkStats2.git
Cloning into 'ThinkStats2'...
remote: Counting objects: 1731, done.
remote: Total 1731 (delta 0), reused 0 (delta 0), pack-reused 1731
Receiving objects: 100% (1731/1731), 141.56 MiB | 5.44 MiB/s, done.
Resolving deltas: 100% (1062/1062), done.
Checking connectivity... done.
git config --global credentials.helper cache