Programming in Python on Steroid

Programming in Python Haim Michael February 25st , 2021 All logos, trade marks and brand names used in this presentation belong to the respective owners. life michae l on Steroids XXL www.lifemichael.com

© 2008 Haim Michael 20150805 What is Python?  Python is an open source free portable powerful and a remarkable easy to learn scripting based programming language.  Python is used for the development of server side applications as well as for the development of stand alone ones.  Python is named after Monty Python.

© 2008 Haim Michael 20150805 Why Python?  The Python programming language focuses on readability. Being readable, the source code written in Python is reusable, maintainable and of an higher quality.  A variety of integration mechanisms allow us to develop code that can easily communicate with other parts of the application, even if written in another software programming language, such as C, C++, Java and C#. The other way around is available as well.

© 2008 Haim Michael 20150805 Why Python?  Python is known for its productivity. Code written in Python is shorter than the equivalent written in Java or C++. In addition, Python development cycle is simpler. There is no need in any lengthy compile and linking phases.  Python has a large collection of ready to use functionality, known as the standard library. That library can be extended by adding more libraries, as well as libraries developed by third party developers.

© 2008 Haim Michael 20150805 Why Python?  Variety of third party development tools for the Python programming language allow us using that language for various tasks, such as web sites development, games development, Matlab equivalent programming and others.  Code written in Python usually doesn't require any change in order to execute it on another computer platform. Porting from one platform to another is straight forward.

© 2008 Haim Michael 20150805 Why Python?  Python has excellent support for most of the common object oriented programming mechanisms.  Python is a popular and enjoyable programming language, already been used by more than 1 million of developers from all over the world. Python is been used by a huge growing number of big companies, such as Google and others.

© 2008 Haim Michael 20150805 Why Python?  Python is free. It is an open source programming language you are free to use and distribute.  Python memory management is automatic. Garbage collector tracks all memories allocations.

© 2008 Haim Michael 20150805 Why Python?  Python is a dynamic typing programming language. It keeps tracking after all objects the program uses when it executes. There is no need to declare the variables with a specific type and a specific size. There is no such thing a type or a variable declaration.

© 2008 Haim Michael 20150805 Real World Samples  The google web search systems was largely developed in Python.  The youtube video sharing service was largely developed in Python.  The famous BitTorrent peer-to-peer files sharing system was developed in Python.  The Google Apps web development framework uses Python extensively.

© 2008 Haim Michael 20150805 Real World Samples  Maya, a 3D modeling and animation system provides a scripting API in Python.  Big financial companies usually use Python in the development of financial applications.

© 2008 Haim Michael 20150805 Python History  Python first implementation was introduced in 1989 by Guido van Rossum at CWI as a successor to the ABC programming language.  Python 2.0 was released in October 2000. This release introduced a garbage collection and built in support for Unicode.

© 2008 Haim Michael 20150805 Python History  Python 3.0 was released in December 2008. The changes in this version are so big that it includes a unique tool the converts Python code written in prior version into a Python 3.0 compatible one.

© 2008 Haim Michael 20150805 The Python Software Foundation  The Python Software Foundation (PSF) is a formal nonprofit organization. PSF is responsible for organizing conferences and it also handles various legal issues related to Python.

© 2008 Haim Michael 20150805 The First Program  Writing a program in Python is relatively simple. The following code prints out “hello students” to the screen. print(“hello students”);  The source file is saved with the “.py” extension.  In order to execute the above code the simplest would be installing the Python environment on your machine and use an IDE, such as PyCharm.  We can turn a Python script into an executable program.

© 2008 Haim Michael 20150805 IronPython  IronPython is a .NET implementation of Python. Code written in IronPython is translated into CLR code, that is executed on the same virtual machine that executes other .NET programming languages. www.ironpython.net

© 2008 Haim Michael 20150805 SL4A  SL4A is an open source project that allows us to execute code in various scripting programming languages, including Python, Perl, Jruby, Lua and JavaScript, on the android platform. https://github.com/damonkohler/sl4a

© 2008 Haim Michael 20150805 Comments  We write comments using the '#' mark. Placing the '#' mark, all code that follows it to the end of the line is considered to be a comment and is ignored. numA = 4 #assigning numA with the value 4 numB = 3 #assigning numB with the value 3 numC = numA + numB #assigning numC with the sum of #numA and numB

© 2008 Haim Michael 20150805 Introduction  The data in Python is in the form of objects, either objects of new types we define or objects of built-in types that Python provides.  An object in Python, as in other OOP languages, is just a piece of memory with values and associated operations.  As we shall see, there are no type declarations in Python. The syntax of the executed expression determines the types of the objects we create and use.

© 2008 Haim Michael 20150805 The Program Structure  Programs developed in Python share a similar structure. Each program is composed of modules. Modules contain statements. Statements contain expressions. Expressions create and process objects.  Everything we process in Python is actually a kind of an object.

© 2008 Haim Michael 20150805 Samples for Built-in Types Type Examples float 12.4 str 'abc', “abc”, “ab'c”, “0xA12” list [12, [2,3,4], 'a'] dict {'one':'The One', 'two': 'Two Files'} tuple (1, 'abc', 23, “A”) set {'a', 'b', 'c'}

© 2008 Haim Michael 20150805 Dynamically Typed  Python is dynamic type programming language. It keeps tracking the types automatically. Python doesn't require us to specify the types.  At the same time, Python is also a strongly typed language. We can perform on a given object those operations that are valid for its type only.

© 2008 Haim Michael 20150805 The Numbers Category  This category includes the following types: int, float, long, decimal and complex.  Each type of this category is expected to support addition, multiplication etc.

© 2008 Haim Michael 20150805 The Sequences Category  This category includes string, list, bytearray, buffer and tuple.  Each type of this category are expected to support indexing, slicing and concatenation.

© 2008 Haim Michael 20150805 Numeric Types  Python's numeric types include the following main types: Integer, Floating Point Numbers, Complex Numbers, Fixed Precision Decimal Numbers, Rational Fraction Numbers, Sets, Booleans, and Unlimited Integer Precision.

© 2008 Haim Michael 20150805 Numeric Literals  Python supports the following basic numeric literals. Literal Interpretation 129, -4.5, 9000000000000000000000 Integers (unlimited size) 2.5, 55.3, 1.22e21 Floating Point Numbers 0o345, 0x98a, 0b1001100 Octal, Hexadecimal and Binary 3+2j, 2.0+4.2j, 2J Complex Numbers

© 2008 Haim Michael 20150805 Variables  Variables are created when they are first assigned with a value. num = 12 #There is no need to define a variable in advance  When been used within an expression they are replaced with their values. numA = 2 numB = 3 total = numA + numB

© 2008 Haim Michael 20150805 Variables  Variables we use in expressions must be assigned with a value before we use them.  Each variable refers an object. There is no need to create that object in advance. These objects are created automatically behind the scene.

© 2008 Haim Michael 20150805 Booleans  Python has the explicit Boolean data type called bool. Its possible values are 'True' and 'False'. These two objects are instances of bool, a subclass of the built-in integer type int.

© 2008 Haim Michael 20150805 Numeric Extensions  There is a large library of third party open source extensions we can use to perform advance operations related to the numeric values we work on. www.scipy.org www.numpy.org

© 2008 Haim Michael 20150805 Variables, Objects & References  A variable is created when the code first assigns it a value. Other assignments that take place after the variable was already created will change the value the variable holds.  A variable doesn't have any type information or constraints regarding the value it can hold. The type information as well as the constraints are associated with the objects their references are held by variables.

© 2008 Haim Michael 20150805 Variables, Objects & References  The value we can assign a variable is a reference for a specific object.  When a variable is been used within an expression it is immediately replaced with the object the variable holds its reference. num FA24B an object that represents the numeric value 242 num = 242

© 2008 Haim Michael 20150805 Objects are Garbage Collected  Whenever a variable is assigned with a referent for a new object the space held by the prior object is reclaimed (unless that object is still referenced by another name or object).  This automatic reclamation of objects' space is known as garbage collection.

© 2008 Haim Michael 20150805 Objects are Garbage Collected  Each object has a counter through which it keeps tracking after the number of references currently pointing at it. When that counter reaches zero the object's memory is automatically reclaimed

© 2008 Haim Michael 20151020 Introduction  Each function is a collection of statements that can be executed more than once in a program.  Functions can receive arguments and they can calculate and return a value back to the caller.

© 2008 Haim Michael 20151020 The def Statement  We create a function by calling the def statement. Each function we create is assigned with a name. We can later use that name in order to call it. def function_name (param1, param2, param3,... paramN): statements

© 2008 Haim Michael 20151020 The def Statement  The execution of 'def' takes place in run-time. Only then the object function is created.  The definition of our function is a statement. We can place a function definition wherever we can place a statement.

© 2008 Haim Michael 20151020 The def Statement  We can place different definitions for the same function and using a simple if statement choosing which of those versions will be defined. ... if test: def func(): ... else: def func(): ... ...

© 2008 Haim Michael 20151020 Function Attributes  Because a function is an object we can add new attributes we choose. def sum(a,b): c = a + b return c sum.version = 101 sum.author = "haim michael" sum.priority = 3 print(sum.author)

© 2008 Haim Michael 20151020 Returned Functions  We can define a function that its returned value is another function.  When one function returns a function it includes its definition.  The returned function is capable of referring variables that belong to the scope of the outer one.

© 2008 Haim Michael 20151020 Arguments  When calling a function passing over names, we actually pass the references held by these names.  Assigning new references to the parameter names within the function scope doesn't effect the names the caller passed.  Changing a mutable object from within the function the caller code should feel that as well.

© 2008 Haim Michael 20151020 Sequence Returned Value  We can define a function that returns a tuple, or any other sequence type. #dmo def f(a,b): numA = 2 * a numB = 2 * b return [numA,numB] x = f(3,5) print(x)

© 2008 Haim Michael 20151020 The func(name) Syntax  By default, the arguments we pass must match by position, left to right, and we must pass exactly as many arguments as required. def f(a,b): sum = a+b print(sum) f(2,3)

© 2008 Haim Michael 20151020 Indirect Function Call  When assigning a function to one of our variables we can append () to that variable and use it in order to call that function. def factorial(a): if a==0: return 1 else: return a * factorial(a-1) f = factorial print(f(5))

© 2008 Haim Michael 20151020 Function Annotations  When we define a function we can optionally specify the types of its parameters and the type of the returned value. def f(country: str, food: str = 'eggs') -> int: print("something")  The function annotations hold metadata information about the function, and specifically about the types of its parameters and the type of the returned value.

© 2008 Haim Michael 20151020 Anonymous Functions (Lambda)  Using the lambda keyword we can define an anonymous function. lambda param1, param2, param3...paramN : expression  Unlike using def, when using lambda we get an expression. Not a statement.

© 2008 Haim Michael 20151026 Functional Programming  Functional programming is a programming paradigm that emphasizes the use of expressions and their evaluation and especially through the definition of functions that are treated as expressions. In addition, it avoids the complexity involved with state changes as the one when objects and variables.

© 2008 Haim Michael 20151026 Functional Programming  The use of functions as expressions enable us getting more expressive code. In many cases we will exploit the power of recursion in order to get expressive succinct (expressed in few words) code.  Python is not a pure functional programming language. Nevertheless, it has more than a few functional programming capabilities.

© 2008 Haim Michael 20151026 Pure Functions  When we define a function that always returns the same value for the very same arguments, and it doesn't depend on any hidden information or state and its evaluation of the result does not cause any observable side effects nor output then it is a pure function.  Pure functions are usually simpler and much easier to test and are very popular in Python programming.

© 2008 Haim Michael 20151026 Pure Functions  In order to write a pure function we should make sure that we write local only code. We should make sure we don't use neither the global statement nor the nonlocal one.  Writing a lambda expression as a pure function is the common approach.

© 2008 Haim Michael 20151026 Lambda Expression  Using lambda expressions we can define a recursive function that feels much more as an expression than a function we define using the def keyword. total = lambda numbers: 0 if len(numbers)==0 else numbers[0] + total(numbers[1:]) print(total([5,2,3,6]))

© 2008 Haim Michael 20151026 Higher Order Functions  When the function we define receives another function (or functions) as an argument(s) or when its returned value is another function it will called an higher order function.  We can use higher order functions for creating new functions in our code.

© 2008 Haim Michael 20151026 Higher Order Functions data = [(13225324,"daniel",54), (3452344,"ronen",92), (98234234,"moshe",80), (65354435,"yael",70)] beststudent = lambda dat: max(dat, key=lambda ob:ob[2]) print(beststudent(data))

© 2008 Haim Michael 20151026 Currying Functions  Currying is the technique of breaking down the evaluation of a function that takes multiple arguments into evaluating a sequence of singe argument functions.

© 2008 Haim Michael 20151020 The func(name=value) Syntax  Calling a function we can specify which parameters should receive a value by using the argument's name in the name=value syntax. #dmo def f(a,b): numA = 2 * a numB = 2 * b return [numA,numB] x = f(a=3,b=5) print(x)

© 2008 Haim Michael 20151020 The func(*name) Syntax  Adding * to the sequence we pass over to the function, the function will be capable of unpacking the passed argument into discrete separated parameters. def f(x1,y1,x2,y2): return (y2-y1)*(y2-y1)+(x2-x1)*(x2-x1) ob = [0,0,4,3] num = f(*ob) print(num)

© 2008 Haim Michael 20151020 The func(**name) Function  Adding ** to the argument name, when calling the function a collection of key/value pairs in the form of a dictionary will be expected to be passed over to the function as individual keyword arguments. def f(a,b): print(a) print(b) ob = {'a':1,'b':2} f(**ob)

© 2008 Haim Michael 20151020 The def func(name=value) Syntax  Defining a function we can use the argument's name in the name=value syntax in order to specify default values for specific arguments. def f(a=4,b=6): numA = 2 * a numB = 2 * b return [numA,numB] x = f() print(x)

© 2008 Haim Michael 20151020 The def func(*name) Syntax  Adding * to the parameter name in the function definition collects unmatched positional arguments into a tuple. #dmo def sum(*tpl): sum = 0 for num in tpl: sum = sum + num return sum print(sum(3,4,6,2,3,6))

© 2008 Haim Michael 20151020 The def func(**name) Syntax  Adding ** to the parameter name in the function definition collects unmatched positional arguments into a dictionary. def f(**args): print(args) f(a=10,b=20)

© 2008 Haim Michael 20151026 Generators  One of the functional programming characteristics that improves its performance is the deferred computation till it is required, also known as lazy evaluation and also known as generators.

© 2020 life michael Introduction  The decorator allows us to add new functionality to existing object without modifying its structure.  When decorating a function, the decoration is invoked before the definition of the function takes place. @decorator def decorated(): pass is indirectly changed into decorator(decorated)

© 2020 life michael Jump Start  The simples way to develop a decorator would be to develop a function that has one parameter to which a function is passed over.  Whenever our decorator is invoked (enough to have in our code @ourdecorator to invoke the decorator) the decorated function will be passed over to our decorator.

© 2020 life michael Jump Start  The decorator function should include the definition of another function, the decorator function returns. That other inner function usually invokes the decorated function.  The function returned by our decorator will take the place of the decorated function. Every call to the decorated function in our code will be replaced by a call to the function returned by our decorator.

© 2020 life michael Jump Start def uppercase(func): print("2") def inner(): print("4") data = func() return data.upper() return inner print("9") @uppercase def getGreeting(): print("13") return "good morning" print("16") print(getGreeting())

© 2020 life michael Decorating Function with Parameters  In order to decorate a function with parameter(s) we just need make sure the function returned by the decorator has the same parameter(s) accordingly.

© 2020 life michael Decorating Function with Parameters def uppercase(func): print("2") def inner(nickname): print("4") data = func(nickname) return data.upper() return inner print("9") @uppercase def getGreeting(nickname): print("13") return "good morning " + nickname print("16") print(getGreeting("dave"))

© 2020 life michael Using a Class as Decorator class uppercase: def __init__(self, f): self.f = f def __call__(self, nickname): return self.f(nickname).upper() @uppercase def greet(nickname): return "good morning " + nickname print(greet("danny"))

© 2020 life michael Decorators Nesting  We can place together more than one decorator. Doing so, we should imagine the decorators execution one after the other in the order in which they are listed (top to bottom).  The output of each decorator execution is the input for the decorator above.

© 2020 life michael Decorators Nesting def uppercase(func): def inner(nickname): data = func(nickname) return data.upper() return inner def stars(func): def inner(text): input = func(text) return "*** "+input+" ***" return inner @stars @uppercase def getGreeting(nickname): return "good morning " + nickname print(getGreeting("dave"))

© 2008 Haim Michael 20151025 Introduction  Using the class statement we create a class object and assign it with a name.  The class object is kind of a factory we can use to create objects in accordance with the template our class object represents.

© 2008 Haim Michael 20151025 Introduction  Whenever we instantiate the class we get a new object on which we can invoke each one of the functions that were defined in the class with the self parameter.  When calling a function, that was defined in the class with the self parameter, the self parameter is assigned with the reference for the object on which the function is invoked.  It is possible to dynamically add new attributes to every object.

© 2008 Haim Michael 20151025 The __init__ Function  The __init__ method is Python's replacement for the constructor. When we create a new object this function will be invoked.  It is possible to define this function with more parameters (in addition to self mandatory parameter) and get the support for instantiating the class passing over multiple arguments.  It is common to add the attributes to the new created object within the scope of the __init__ function.

© 2008 Haim Michael 20151025 The __init__ Function class Rectangle: def __init__(self,w,h): self.width = w self.height = h def area(self): return self.width*self.height a = Rectangle(3,4) b = Rectangle(5,6) print("area of a is %d and area of b is %d " % (a.area(),b.area()))

© 2008 Haim Michael 20151025 The super() Function  We can use the super() function for calling a function's overridden version.  When we define a class that extends another class we can include within the first class' definition for __init__ a call for invoking __init__ in the base class.  Doing so, each __init__ function in each one of the classes will be responsible for building the relevant parts of the object.

© 2008 Haim Michael 20151025 The super() Function class Person: def __init__(self,id,name): self.id = id self.name = name def details(self): return "id=%d name=%s" % (self.id,self.name) class Student(Person): def __init__(self,id,name,average): self.average = average super().__init__(id,name) def details(self): return super().details() + " average=%d" % self.average ob = Student(123123,"danidin",98) print(ob.details())

© 2008 Haim Michael 20151025 Polymorphism  The functions in Python are dynamic ones, which is even better than virtual ones. The function we invoke in our implementation for polymorphism in Python doesn't have to be defined in the base class. We don't even need a base class.

© 2008 Haim Michael 20151025 Polymorphism class Dog: def __init__(self,str): self.name = str def hello(self): print("hau hau") class Cat: def __init__(self,str): self.name = str def hello(self): print("miau miau") class Cow: def __init__(self,str): self.name = str def hello(self): print("moo moo") animals = [Cow("Metilda"), Dog("Doberman"), Cat("Mitzi"), Cow("Shula")] for animal in animals: animal.hello()

© 2008 Haim Michael 20151025 Overloading  Python doesn't allow us to define the same function in more than one version. Python doesn't support overloading.  Nevertheless, the possibility to specify default values for some of the parameters can be used for getting the same result.

© 2008 Haim Michael 20151025 Overloading class Rectangle: def set(self,w=10,h=10): self.width = w self.height = h def area(self): return self.width*self.height ob = Rectangle() ob.set() print(ob.area())

© 2008 Haim Michael 20151025 Private Variables  Python allows us to define private attributes in our class (private instance variables). Outside the scope of the class it won't be possible to access them.  In order to create private attributes we should prefix their names with two underscores ('__').

© 2008 Haim Michael 20151025 Private Variables class Rectangle: def __init__(self,w=10,h=10): self.__width = w self.__height = h def area(self): return self.__width*self.__height ob = Rectangle(3,4) print(ob.area()) ob.__width = 10 print(ob.area())

© 2008 Haim Michael 20151025 Properties class Rectangle: def __init__(self,w_val,h_val): self._w_val = 10 self._h_val = 10 self.width = w_val self.height = h_val def area(self): return self.width*self.height @property def width(self): return self._w_val @property def height(self): return self._h_val @width.setter def width(self,number): if number > 0 : self._w_val = number @height.setter def height(self,number): if number>0 : self._h_val = number

© 2008 Haim Michael 20151025 Magic Methods  The so-called magic methods are the methods with this clumsy syntax of double underscores at the beginning and the end.  The __init__ is just one of these magic methods. There are many others.

© 2008 Haim Michael 20151025 The __str__ Magic Method import math class Circle: def __init__(self, r): self.radius = r def perimeter(self): return 2*math.pi*self.radius def area(self): return math.pi*math.pow(self.radius,2) def __str__(self): return "circle ... radius = " + str(self.radius) ob = Circle(4) print(ob)

© 2008 Haim Michael 20151025 Binary Operators Overloading  Hereto the magic functions we should define in order to overload the binary operators. Operator Method + object.__add__(self, other) - object.__sub__(self, other) * object.__mul__(self, other) // object.__floordiv__(self, other) / object.__truediv__(self, other) % object.__mod__(self, other) ** object.__pow__(self, other[, modulo]) << object.__lshift__(self, other) >> object.__rshift__(self, other) & object.__and__(self, other) ^ object.__xor__(self, other) | object.__or__(self, other)

© 2008 Haim Michael 20151025 Assignment Operators Overloading  Hereto the magic functions we should define in order to overload the extended assignment operators. Operator Method += object.__iadd__(self, other) -= object.__isub__(self, other) *= object.__imul__(self, other) /= object.__idiv__(self, other) //= object.__ifloordiv__(self, other) %= object.__imod__(self, other) **= object.__ipow__(self, other[, modulo]) <<= object.__ilshift__(self, other) >>= object.__irshift__(self, other) &= object.__iand__(self, other) ^= object.__ixor__(self, other) |= object.__ior__(self, other)

© 2008 Haim Michael 20151025 Unary Operators Overloading  Hereto the magic functions we should define in order to overload the unary operators. Operator Method - object.__neg__(self) + object.__pos__(self) abs() object.__abs__(self) ~ object.__invert__(self) complex() object.__complex__(self) int() object.__int__(self) long() object.__long__(self) float() object.__float__(self) oct() object.__oct__(self) hex() object.__hex__(self

© 2008 Haim Michael 20151025 Comparison Operators Overloading  Hereto the magic functions we should define in order to overload the comparison operators. Operator Method < object.__lt__(self, other) <= object.__le__(self, other) == object.__eq__(self, other) != object.__ne__(self, other) >= object.__ge__(self, other) > object.__gt__(self, other)

© 2008 Haim Michael 20151025 Data Classes  Using the @dataclass decorator we can mark a class we define as a data class.  Doing so, auto generated methods will be added to the definition of our class, including the __init__ method for initializing the class, and the __repr__, __ne__, __lt__, __le__, __gt__ and __ge__ methods.

© 2008 Haim Michael 20151025 Data Classes  We will usually use the @dataclass decorator without any parameter. However, it is certainly possible to use parameters in order to configure the data class we get. @dataclass (init=True, repr=True, eq=True) class Something

© 2008 Haim Michael 20151123 Introduction  Each thread is the execution flow of a series of commands one after the other.  We can write code that creates new threads in addition to the main one. Using additional threads we can overcome performance problems due to blocking for threads that already run in our program.  Unlike forked processes, the new threads run all together with the ones that already exist in the same process.

© 2008 Haim Michael 20151123 The _thread Module  The _thread basic module provides us with the same programming interface no matter on which operating system our code runs.  When calling the start_new_thread function on this module we indirectly start a new separated thread. The function we pass over to this method will be the main function of this new thread.

© 2008 Haim Michael 20151123 The _thread Module import _thread def do_something(thread_id): i=0 d=0 while i<10: d=0 print('doing something... thread id is ', thread_id) while d<9999999: d+=1 i+=1 def start_program(): i= 0 while True: i += 1 _thread.start_new_thread(do_something, (i,)) if input() == 'q': break start_program()

© 2008 Haim Michael 20151123 The Thread Class  The threading module includes the Thread class. We can easily create new thread by defining a new class that extends it, overriding the run function and make sure that our __init__ in the new class calls the __init__ function it overrides.

© 2008 Haim Michael 20151123 The Thread Class import threading import time class MyThread(threading.Thread): def __init__(self,str): threading.Thread.__init__(self) self.name = str def run(self): i = 1 while(i<10): time.sleep(5) print(self.name) i=i+1 a = MyThread("gaga") a.start() b = MyThread("dada") b.start()

© 2008 Haim Michael 20151123 The join Method  When calling the join() method, the calling thread will be blocked until the thread, on which the join() method was called, terminates.  When the timeout argument is not passed over or its value is None, the operation will block until the thread terminates.

© 2008 Haim Michael 20151123 The join Method  When a trying to call join() on a thread that is already waiting due to calling the join() method on a third thread then a RuntimeError is raised. This way a dead lock is prevented.

© 2008 Haim Michael 20151123 The join Method import threading import time class MyThread(threading.Thread): def __init__(self,str): threading.Thread.__init__(self) self.str = str def run(self): i = 1 while(i<4): time.sleep(2) print(self.str) i=i+1

© 2008 Haim Michael 20151123 The join Method a = MyThread("gaga") a.start() b = MyThread("mama") b.start() for i in [1,2,3]: print("main thread") time.sleep(1) print("a.join()") a.join() print("b.join()") b.join() for tav in ['a','b','c','d']: time.sleep(1) print(tav)

© 2008 Haim Michael 20151123 Daemon Threads  The application ends when all non daemon threads end. If there is a daemon thread that is still running it will be immediately stopped.  We can control each and every thread and set them to be daemon or not daemon by calling the setDaemon method. Calling the isDaemon method we can get an answer to the question whether a given thread is daemon or not.

© 2008 Haim Michael 20151123 Daemon Threads import threading import time class MyThread(threading.Thread): def __init__(self,str,number,daemon): threading.Thread.__init__(self) self.str = str self.number = number self.daemon = daemon if daemon: self.setDaemon(daemon) def run(self): i = 1 while(i<self.number): time.sleep(2) print(self.str) i=i+1 a = MyThread("gaga not daemon",4,False) a.start() b = MyThread("mama daemon",100,True) b.start()

© 2008 Haim Michael 20151123 The Condition Class  When instantiating Condition we get an object that represents a specific condition.  Calling the wait() method on our condition object releases the lock, and then blocks until another thread awakens it by calling notify() or notify_all().  Once awakened, the wait() function re-acquires the lock and returns. It is also possible to specify a timeout.

© 2008 Haim Michael 20151123 The Condition Class  Calling the notify() method wakes up one of the threads waiting for the condition, if any are waiting. If no thread is waiting then nothing happens.  Calling the notify_all() method wakes up all threads waiting for the condition.

© 2008 Haim Michael 20151123 The Condition Class  Calling the notify() and the notify_all() functions don’t release the lock. When calling these functions none of the waiting threads returns from its wait() call immediately.  The waiting threads will reacquire the ownership of the lock when the thread that called notify or notify_all() relinquish the lock ownership only.

© 2008 Haim Michael 20151123 The Condition Class import threading import time class MyStack: def __init__(self): self.list = [] self.index = 0 self.empty = threading.Condition() def push(self,number): self.empty.acquire() print("inside push ... calling notify()") self.empty.notify() self.list.append(number) print("inside push, number was added ... ",self.list) self.index += 1 self.empty.release()

© 2008 Haim Michael 20151123 The Condition Class def pop(self): self.empty.acquire() print("inside pop ... checking if index==0") while self.index==0: self.empty.wait() print("inside pop ... calling wait()") self.index -= 1 print("inside pop, number was removed ... ",self.list) self.empty.release() return self.list.pop(self.index) class Producer(threading.Thread): def __init__(self,stack): threading.Thread.__init__(self) self.stack = stack def run(self): for num in [12,33,52,54,56,15,200]: self.stack.push(num) time.sleep(4)

© 2008 Haim Michael 20151123 The Condition Class class Consumer(threading.Thread): def __init__(self,stack): threading.Thread.__init__(self) self.stack = stack def run(self): while True: time.sleep(6) print(self.stack.pop()) mystack = MyStack() consumer = Consumer(mystack) producer = Producer(mystack) consumer.start() producer.start()

© 2008 Haim Michael 20151123 The with Statement import threading import time class MyStack: def __init__(self): self.list = [] self.index = 0 self.empty = threading.Condition() def push(self,number): with self.empty: print("inside push ... calling notify()") self.empty.notify() self.list.append(number) print("inside push, number was added ... ",self.list) self.index += 1

© 2008 Haim Michael 20151123 The with Statement def pop(self): with self.empty: print("inside pop ... checking if index==0") while self.index==0: self.empty.wait() print("inside pop ... calling wait()") self.index -= 1 print("inside pop, number was removed ... ",self.list) return self.list.pop(self.index) class Producer(threading.Thread): def __init__(self,stack): threading.Thread.__init__(self) self.stack = stack def run(self): for num in [12,33,52,54,56,15,200]: self.stack.push(num) time.sleep(4)

© 2008 Haim Michael 20151123 The with Statement class Consumer(threading.Thread): def __init__(self,stack): threading.Thread.__init__(self) self.stack = stack def run(self): while True: time.sleep(6) print(self.stack.pop()) mystack = MyStack() consumer = Consumer(mystack) producer = Producer(mystack) consumer.start() producer.start()

© 2008 Haim Michael 20151123 The Semaphore Class  Using the object we instantiate from Semaphore class we can manage an internal counter decremented by each call to the acquire() function and incremented by each call to the release() function.  The counter can never be smaller than zero. When calling the acquire() function and the counter is zero then the thread blocks.

© 2008 Haim Michael 20151123 The Semaphore Class  When instantiating Semaphore we can specify the initial number we want the counter to be assigned with. We do so by using the value parameter. ob = Semaphore(value=1)

© 2008 Haim Michael 20151123 The Semaphore Class import threading import time def worker(text, sema): sema.acquire() i = 1 while i<10: print(threading.current_thread().getName(),text) time.sleep(2) i += 1 sema.release() semaphore = threading.Semaphore(2) number_of_threads = 10 for str in ['haifa','tel-aviv','ramat-gan','eilat','jerusalem']: thread = threading.Thread(target=worker, args=(str, semaphore)) thread.start()

© 2008 Haim Michael 20151123 The Timer Class  Instantiating this class, we shall get an object that represents a specific invocation of a specific action after a certain amount of time.  Timer is a subclass of Thread. When calling start() on a Timer object, the timer starts. Calling cancel() will stop the timer.  The time interval specified when instantiating Timer may be different than the time interval that takes place.

© 2008 Haim Michael 20151123 The Barrier Class  Instantiating this class, we shall get an object that can be used by a fixed number of threads that need to wait for each other.  Each thread will try to pass the barrier by calling the wait() method, and will block until all of the threads have made their wait() calls. When that happens, the threads will be automatically released.

© 2008 Haim Michael 20151020 Introduction  Python module is the highest level program unit. Each module packages code and data in a way that allows us to reuse it.  In most cases, each module is matched with a specific Python source code file or with a specific code file developed in another language, such as C, Java or C#.  Modules can import other modules in order to use the variables, functions and classes they define.

© 2008 Haim Michael 20151020 Introduction  Using modules we enjoy code reuse, system namespaces mechanism and the possibility to implement shared services or data as a module.  Developing a program in Python, we get multiple source code files containing Python statements. Each one of them is a module.  The main file is a top level file that structure all files together.

© 2008 Haim Michael 20151020 The import Statement  When importing another module there is a search for the module file. When found, it is compiled into byte code (if required) and then it is executed in order to build the objects it defines.  The second time a module is imported none of that happens. The module is already available.

© 2008 Haim Michael 20151020 The import Statement  When Python searches for a module and more than one file with the same name (and different extension) exists Python prioritize its search in accordance with the following list: 1. source code file (*.py) 2. byte code file (*.pyc) 3. compiled extension written in another language (e.g. *.so, *.dll, *.pyd etc) 4. compiled built-in module coded in C and statically linked into Python 5. zip file component, that is automatically extracted when imported 6. java class (when using Jython) 7. net component (when using IronPython)

© 2008 Haim Michael 20151123 Introduction  Unlike multi-threading, when executing multi processes, each process has its own memory allocation. Code executed in one process cannot access variables and objects in the other process.

© 2008 Haim Michael 20151123 The multiprocessing Module  This module provides us with the capability to spawn processes using an API similar to the threading module.  This module allows the programmer to fully leverage multiple processors on a given machine. It runs on both Unix and Windows.

© 2008 Haim Michael 20151123 The multiprocessing Module  Apart of providing us with an API similar to the one we enjoy when using the threading module, this module also provides us with unique APIs, such as the Pool class.

© 2008 Haim Michael 20151123 The Process Class  Instantiating this class, and calling the start() method on the new created object will initiate a new separated process that executes the method we specified.

© 2008 Haim Michael 20151123 The Process Class from multiprocessing import Process import time def f(name): index = 1 while index<10: print('hello', name) time.sleep(10) index += 1 p1 = Process(target=f, args=('haim',)) p1.start() p2 = Process(target=f, args=('daniella',)) p2.start() p1.join() p2.join() print("end")

© 2008 Haim Michael 20151123 The Pool Class  This class allows us to parallelize the execution of a function we specify across multiple input values, by having the input data distributed across processes (data parallelism).

© 2008 Haim Michael 20151123 The Pool Class from multiprocessing import Pool import time def f(x): time.sleep(10) return x*x start = time.perf_counter() with Pool(5) as p: list2 = p.map(f, [1, 2, 3, 4, 5, 6, 7]) #list2 = map(f,[1,2,3,4,5,6,7]) for num in list2: print(num) end = time.perf_counter() print(end-start)

© 2008 Haim Michael 20150805 What are Coroutines?  Python supports coroutines. The word coroutine is composed of two words. The word 'Co' ('Cooperative') and the word 'routine' ('Function'). Coroutine is a cooperative functions.  Cooperative functions work in a cooperative way with each other instead of working synchronously.  Coroutines can also be considered as light-weight threads.

© 2008 Haim Michael 20150805 The asyncio Library  This library helps us with writing concurrent code using the async / await syntax.  In addition syncio provides a set of high-level APIs to: run Python coroutines concurrently, and have full control over their execution.

© 2008 Haim Michael 20150805 Simple Demo import asyncio import datetime @asyncio.coroutine def f1(loop): end_time = loop.time() + 5.0 while True: print(datetime.datetime.now()) if (loop.time() + 1.0) >= end_time: break yield from asyncio.sleep(1) def f2(): loop = asyncio.get_event_loop() # blocking call till coroutine is done loop.run_until_complete(f1(loop)) loop.close() f2()

© 2009 Haim Michael All Rights Reserved 196 Learning Resources  Downloading Python https://www.python.org/downloads/release/python-390/  Downloading PyCharm https://www.jetbrains.com/pycharm/download/  Israeli Guide to Python http://www.pythonbook.co.il (check the ‫סרטונים‬ section)  Python Programming Course http://python.course.lifemichael.com  Python Programming Seminars http://lifemichael.com/corporate/index.php/seminars/  Introduction to Programming in Python https://www.udemy.com/course/introduction-to-programming-in-python life michael

Programming in Python on Steroid

More Related Content

What's hot

Similar to Programming in Python on Steroid

More from Haim Michael

Recently uploaded

Programming in Python on Steroid