Python and Pytorch tutorial and walkthrough

Python & Pytorch Tutorial Adapted from CS224N (Stanford University)

Topics 1. Why Python? 2. Setup 3. Language Basics 4. Common Data Structures 5. Introduction to NumPy 6. Practical Python Tips 7. Other Great References

Why Python? ● Python is a widely used, general purpose programming language. ● Easy to start working with. ● Scientific computation functionality similar to Matlab and Octave. ● Used by major deep learning frameworks such as PyTorch and TensorFlow.

Topics 1. Why Python? 2. Setup 3. Language Basics 4. Common Data Structures 5. Introduction to NumPy 6. Practical Python Tips 7. Other Great References Code is in Courier New. Command line input is prefixed with ‘$’. Output is prefixed with ‘>>’.

Environment Management ● Problem ○ Different versions of Python ○ Countless Python packages and their dependencies ○ Different projects require different packages ■ Even worse, different versions of the same package! ● Solution ○ Keep multiple Python environments that are isolated from each other ○ Each environment ■ Can use different Python version ■ Keeps its own set of packages (can specify package versions) ■ Can be easily replicated

Create a new environment $ conda create –n <environment_name> $ conda create -n <environment_name> python=3.7 $ conda env create -f <environment.yml> Activate/deactivate environment $ conda activate <environment_name> <...do stuff...> $ conda deactivate Export environment $ conda activate <environment_name> $ conda env export > environment.yml Anaconda ● Anaconda is a popular Python environment/package manager ○ Install from https://www.anaconda.com/download/ ○ Supports Windows, Linux, MacOS ○ Can create and manage different isolated environments With specific Python version From environment file Basic workflow

IDEs / text editors for Python ● PyCharm ● Visual Studio Code ● Sublime Text ● Atom ● Vim (for Linux or Mac) Write a Python program in your IDE or text editor of choice. In terminal, activate conda environment and run program with command: $ python <filename.py>

Jupyter Notebook / Google Colab ● Jupyter Notebook ○ A Jupyter notebook lets you write and execute Python code locally in your web browser ○ Interactive, code re-execution, result storage, can interleave text, equations, and images ○ Can add conda environments to Jupyter notebook ● Google Colab ○ https://colab.research.google.com/ ○ Google’s hosted Jupyter notebook service, runs in the cloud, requires no setup to use, provides free access to computing resources including GPUs ○ Comes with many Python libraries pre-installed

Common Operations x = 10 # Declaring two integer variables y = 3 # Comments start with hash x + y >> 13 # Arithmetic operations x ** y >> 1000 # Exponentiation x / y >> 3 # Dividing two integers x / float(y) >> 3.333… # Type casting for float division str(x) + “+” + str(y) >> “10 + 3” # Casting integer as string and string concatenation

Built-in Values True, False None x = None array = [1, 2, None] def func(): return None # Usual true/false values # Represents the absence of something # Variables can be assigned None # Lists can contain None # Functions can return None

Built-in Values and or not if [] != [None]: print(“Not equal”) # Boolean operators in Python written as plain English, as opposed to &&, ||, ! in C++ # Comparison operators == and != check for equality/inequality, return true/false values

Brackets → Indents ● Code blocks are created using indents, instead of brackets like in C++ ● Indents can be 2 or 4 spaces, but should be consistent throughout file ● If using Vim, set this value to be consistent in your .vimrc def sign(num): # Indent level 1: function body if num == 0: # Indent level 2: if statement body print(“Zero”) elif num > 0: # Indent level 2: else if statement body print(“Positive”) else: # Indent level 2: else statement body print(“Negative”)

Language Basics Python is a strongly-typed and dynamically-typed language. Strongly-typed: Interpreter always “respects” the types of each variable. [1] Dynamically-typed: “A variable is simply a value bound to a name.” [1] Execution: Python is first interpreted into bytecode (.pyc) and then compiled by a VM implementation into machine instructions. (Most commonly using C.) [1] https://wiki.python.org/moin/Why%20is%20Python%20a%20dynamic%20language%20and%20also%20a%20strongly%20typed%20language

Language Basics Python is a strongly-typed and dynamically-typed language. Strongly-typed: Interpreter always “respects” the types of each variable. [1] Dynamically-typed: “A variable is simply a value bound to a name.” [1] Execution: Python is first interpreted into bytecode (.pyc) and then compiled by a VM implementation into machine instructions. (Most commonly using C.) What does this mean for me? [1] https://wiki.python.org/moin/Why%20is%20Python%20a%20dynamic%20language%20and%20also%20a%20strongly%20typed%20language

Language Basics Python is a strongly-typed and dynamically-typed language. Strongly-typed: Types will not be coerced silently like in JavaScript. Dynamically-typed: Variables are names for values or object references. Variables can be reassigned to values of a different type. Execution: Python is “slower”, but it can run highly optimized C/C++ subroutines which make scientific computing (e.g. matrix multiplication) really fast. [1] https://wiki.python.org/moin/Why%20is%20Python%20a%20dynamic%20language%20and%20also%20a%20strongly%20typed%20language

Language Basics Python is a strongly-typed and dynamically-typed language. Strongly-typed: 1 + ‘1’ → Error! Dynamically-typed: foo = [1,2,3] ...later... foo = ‘hello!’ Execution: np.dot(x, W) + b → Fast! [1] https://wiki.python.org/moin/Why%20is%20Python%20a%20dynamic%20language%20and%20also%20a%20strongly%20typed%20language

Broadcasting (visually) 1 2 3 4 5 6 7 8 9 10 11 12 1 1 1 1 2 2 2 2 3 3 3 3 2 3 4 5 7 8 9 10 12 13 14 15 x + y 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 5 12 21 32 9 30 33 48 x * z

Topics 1. Why Python? 2. Setup 3. Language Basics 4. Introduction to NumPy 5. Practical Python Tips 6. Other Great References

List Comprehensions ● Similar to map() from functional programming languages. ● Can improve readability & make the code succinct. ● Format: [func(x) for x insome_list] Following are equivalent: squares = [] for i in range(10): squares.append(i**2) — squares = [i**2 for i in range(10)] Can be conditional: odds = [i**2 for i in range(10) if i%2 == 1]

Convenient Syntax Multiple assignment / unpacking iterables age, name, pets = 20, ‘Joy’, [‘cat’] x, y, z = (‘Tensorflow’, ‘PyTorch’, ‘Chainer’) Returning multiple items from a function def some_func(): return 10, 1 ten, one = some_func() Joining list of strings with a delimiter “, ”.join([1, 2, 3]) == ‘1, 2, 3’ String literals with both single and double quotes message = ‘I like “single” quotes.’ reply = “I prefer ‘double’ quotes.”

Debugging Tips Python has an interactive shell where you can execute arbitrary code. ● Great replacement for TI-84 (no integer overflow!) ● Can import any module (even custom ones in the current directory) ● Try out syntax you’re unsure about and small test cases (especially helpful for matrix operations) $ python Python 3.9.7 (default, Sep 16 2021, 08:50:36) [Clang 10.0.0 ] :: Anaconda, Inc. on darwin >> import numpy as np >> A = np.array([[1, 2], [3, 4]]) >> B = np.array([[3, 3], [3, 3]]) >> A * B [[3 6] [9 12]] >> np.matmul(A, B) [[9 9] [21 21]]

Debugging Tools Code What it does array.shape Get shape of NumPy array array.dtype Check data type of array (for precision, for weird behavior) type(stuff) Get type of variable import pdb; pdb.set_trace() Set a breakpoint [1] print(f’My name is {name}’) Easy way to construct a string to print [1] https://docs.python.org/3/library/pdb.html

Common Errors ValueError(s) are often caused by mismatch of dimensions in broadcasting or matrix multiplication If you get this type of error, a good first step to debugging the issue is to print out the shape of relevant arrays to see if they match what you expect. array.shape

NLTK Tokenization https://realpython.com/nltk-nlp-python/

Python and Pytorch tutorial and walkthrough

More Related Content

Similar to Python and Pytorch tutorial and walkthrough

Recently uploaded

Python and Pytorch tutorial and walkthrough