Code_IPythonInstallation.md

#Steps to install ipython notebook on CDH 5.4

Environment

• Cloudera quick start VM 5.4/5.5 ps. Thanks to Bryan's shell script

Steps

Install packages you need later

sudo yum -y groupinstall "Development tools" && 
sudo yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel

Download py2.7.6

cd /opt
sudo wget --no-check-certificate https://www.python.org/ftp/python/2.7.6/Python-2.7.6.tar.xz
sudo tar xf Python-2.7.6.tar.xz
cd Python-2.7.6
sudo ./configure --prefix=/usr/local
sudo make && sudo make altinstall

Modify python's link

sudo ln -s /usr/local/bin/python2.7 /usr/local/bin/python

Echo python version

/usr/bin/python --version

Install pip

sudo wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py
sudo /usr/local/bin/python2.7 ez_setup.py
sudo /usr/local/bin/easy_install-2.7 pip

Install IPython notebook

sudo pip install ipython[all]

Test

IPYTHON_OPTS="notebook --ip=0.0.0.0" ~/spark-1.4.1-bin-hadoop2.6/bin/pyspark

or in your spark standalone cluster

IPYTHON_OPTS="notebook --ip=0.0.0.0" ~/spark-1.4.1-bin-hadoop2.6/bin/pyspark --master spark://ip:7077

or initial ipython notebook with specific config setting,

IPYTHON_OPTS="notebook --ip=0.0.0.0" ~/spark-1.4.1-bin-hadoop2.6/bin/pyspark --master spark://ip:7077 --executor-memory 3G --total-executor-cores 20 

In IPython notebook

Try it.