theano Getting started with theano
Remarks
Theano is a python library, which handles defining and evaluating symbolic expressions over tensor variables. Has various application, but most popular is deep learning.
Installation or Setup
Detailed instructions on getting theano set up or installed.
Installing Theano and configuring the GPU on Ubuntu 14.04
You can use the following instructions to install Theano and configure the GPU (assume a freshly installed Ubuntu 14.04):
# Install Theano sudo apt-get install python-numpy python-scipy python-dev python-pip python-nose g++ libopenblas-dev git sudo pip install Theano # Install Nvidia drivers, CUDA and CUDA toolkit, following some instructions from http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html wget http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb # Got the link at https://developer.nvidia.com/cuda-downloads sudo dpkg -i cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb sudo apt-get update sudo apt-get install cuda sudo reboot At that point, running nvidia-smi should work, but running nvcc won't work.
# Execute in console, or (add in ~/.bash_profile then run "source ~/.bash_profile"): export PATH=/usr/local/cuda-7.5/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-7.5/lib64:$LD_LIBRARY_PATH At that point, both nvidia-smi and nvcc should work.
To test whether Theano is able to use the GPU:
Copy-paste the following in gpu_test.py :
# Start gpu_test.py # From http://deeplearning.net/software/theano/tutorial/using_gpu.html#using-gpu from theano import function, config, shared, sandbox import theano.tensor as T import numpy import time vlen = 10 * 30 * 768 # 10 x #cores x # threads per core iters = 1000 rng = numpy.random.RandomState(22) x = shared(numpy.asarray(rng.rand(vlen), config.floatX)) f = function([], T.exp(x)) print(f.maker.fgraph.toposort()) t0 = time.time() for i in xrange(iters): r = f() t1 = time.time() print("Looping %d times took %f seconds" % (iters, t1 - t0)) print("Result is %s" % (r,)) if numpy.any([isinstance(x.op, T.Elemwise) for x in f.maker.fgraph.toposort()]): print('Used the cpu') else: print('Used the gpu') # End gpu_test.py and run it:
THEANO_FLAGS='mode=FAST_RUN,device=gpu,floatX=float32' python gpu_test.py which should return:
f@f-Aurora-R4:~$ THEANO_FLAGS='mode=FAST_RUN,device=gpu,floatX=float32' python gpu_test.py Using gpu device 0: GeForce GTX 690 [GpuElemwise{exp,no_inplace}(<CudaNdarrayType(float32, vector)>), HostFromGpu(GpuElemwise{exp,no_inplace}.0)] Looping 1000 times took 0.658292 seconds Result is [ 1.23178029 1.61879349 1.52278066 ..., 2.20771813 2.29967761 1.62323296] Used the gpu To know your CUDA version:
nvcc -V Example:
username@server:~$ nvcc -V nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2015 NVIDIA Corporation Built on Tue_Aug_11_14:27:32_CDT_2015 Cuda compilation tools, release 7.5, V7.5.17 Adding cuDNN
To add cuDNN (instructions from http://deeplearning.net/software/theano/library/sandbox/cuda/dnn.html):
- Download cuDNN from https://developer.nvidia.com/rdp/cudnn-download (need registration, which is free)
tar -xvf cudnn-7.0-linux-x64-v3.0-prod.tgz- Do one of the following
Option 1: Copy the *.h files to CUDA_ROOT/include and the *.so* files to CUDA_ROOT/lib64 (by default, CUDA_ROOT is /usr/local/cuda on Linux).
sudo cp cuda/lib64/* /usr/local/cuda/lib64/ sudo cp cuda/include/cudnn.h /usr/local/cuda/include/ Option 2:
export LD_LIBRARY_PATH=/home/user/path_to_CUDNN_folder/lib64:$LD_LIBRARY_PATH export CPATH=/home/user/path_to_CUDNN_folder/include:$CPATH export LIBRARY_PATH=/home/user/path_to_CUDNN_folder/lib64:$LD_LIBRARY_PATH By default, Theano will detect if it can use cuDNN. If so, it will use it. If not, Theano optimizations will not introduce cuDNN ops. So Theano will still work if the user did not introduce them manually.
To get an error if Theano can not use cuDNN, use this Theano flag: optimizer_including=cudnn .
Example:
THEANO_FLAGS='mode=FAST_RUN,device=gpu,floatX=float32,optimizer_including=cudnn' python gpu_test.py To know your cuDNN version:
cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2 Adding CNMeM
The CNMeM library is a "Simple library to help the Deep Learning frameworks manage CUDA memory.".
# Build CNMeM without the unit tests git clone https://github.com/NVIDIA/cnmem.git cnmem cd cnmem mkdir build cd build sudo apt-get install -y cmake cmake .. make # Copy files to proper location sudo cp ../include/cnmem.h /usr/local/cuda/include sudo cp *.so /usr/local/cuda/lib64/ cd ../.. To use with Theano, you need to add the lib.cnmem flag. Example:
THEANO_FLAGS='mode=FAST_RUN,device=gpu,floatX=float32,lib.cnmem=0.8,optimizer_including=cudnn' python gpu_test.py The first output of the script should be:
Using gpu device 0: GeForce GTX TITAN X (CNMeM is enabled with initial size: 80.0% of memory, cuDNN 5005) lib.cnmem=0.8 means that it can use up to 80% of the GPU.
CNMeM has been reported to give some interesting speed improvements, and is supported by Theano, Torch, and Caffee.
The speed up depend of many factor, like the shapes and the model itself. The speed up go from 0 to 2x faster.
If you don't change the Theano flag allow_gc, you can expect 20% speed up on the GPU. In some case (small models), we saw a 50% speed up.
Common issues:
Running Theano on multiple CPU cores
You can run Theano on multiple CPU cores with the OMP_NUM_THREADS=[number_of_cpu_cores] flag.
Example:
OMP_NUM_THREADS=4 python gpu_test.py The script theano/misc/check_blas.py outputs information regarding which BLAS is used:
cd [theano_git_directory] OMP_NUM_THREADS=4 python theano/misc/check_blas.py Your first theano program
In this example, we will compile functions that computes sum and difference given two real number.
from __future__ import print_function import theano import theano.tensor as T #define two symbolic scalar s_x = T.fscalar() s_y = T.fscalar() #compute something s_sum = s_x + s_y s_diff = s_x - s_y #compile a function that adds two number #theano will call system compiler at here fn_add = theano.function(inputs=[s_x, s_y], outputs=s_sum) fn_diff = theano.function(inputs=[s_x, s_y], outputs=s_diff) #call the compiled functions print(fn_add(2., 2.)) #4. print(fn_diff(2., 2.)) #0. 
