Patterns in Python

1.
Design patterns inPython Glenn Ramsey Kiwi Pycon 2011

2.
Who I amGlenn Ramsey

3.
ME (1993, Uof Auckland) mechanical engineering – control systems

4.
Freelance developer (mainlyC++, now Python)

5.
Based at Hikutaia

6.
Row Pro (digitalrowing.com)since 2001

7.
Interest in softwareis primarily for modelling and simulation

8.
PhD candidate atU of A (FEA horse's hoof, FORTRAN, Perl) thesis submitted Hikutaia

9.
Outline Motivation

10.
General software design/ pattern concepts (brief)

11.
Specific examples of“Gang of Four” patterns in Python

12.
Motivation “ 16of 23 [GoF] patterns have a qualitatively simpler implementation in Lisp or Dylan than in C++, for at least some uses of each pattern” Peter Norvig (http://norvig.com/design-patterns/) “ Patterns are not needed in Python because design patterns are a sign of a deficiency of a language” … ... for the purpose that the design pattern addresses. How is observer implemented in Python? (equivalent to Boost.Signals, Qt Slot/Signals, .NET events) Coming from C++ or Java, if you already know the GoF patterns then it would be informative to see how they are implemented in Python. This talk documents part of my journey from C++ to Ptyhon.

13.
Software vs Engineeringdesign Physical construction – E.g. building a house Software construction The code is the design! “ Software may be cheap to build, but it is incredibly expensive to design” J W Reeves, What Is Software Design?, www.developerdotstar.com Design stage output

14.
Software design Howdoes one design software, compared to physical engineering design? Data + algorithms? - only a part of the solution Structure and Interpretation of Computer Programs. H Abelson, G Sussman, J Sussman. http://mitpress.mit.edu/sicp/full-text/book/book.html Software Design Concepts (wikipedia) Abstraction – categorize and group concepts Refinement – convert high level to program statements Modularity – isolate independent features Software architecture – overall structure of the software Control Hierarchy – program structure Structural partitioning – horizontal vs vertical ? Data structure – logical relationship among elements Software procedure – an operation within a module Information hiding - information contained within a module is inaccessible to others Not especially helpful - too abstract!

15.
Object Oriented Designprinciples Open Close Principle Software entities like classes, modules and functions should be open for extension but closed for modifications.

16.
Encapsulation – informationhiding Dependency Inversion Principle High-level modules should not depend on low-level modules. Both should depend on abstractions.

17.
Abstractions should notdepend on details. Details should depend on abstractions.

18.
Loose coupling InterfaceSegregation Principle Clients should not be forced to depend upon interfaces that they don't use. Single Responsibility Principle A class should have only one job. Liskov's Substitution Principle Derived types must be completely substitutable for their base types. Prefer composition over inheritance http://www.oodesign.com/design-principles.html

19.
What is adesign pattern? Wikipedia: In software engineering, a design pattern is a general reusable solution to a commonly occurring problem within a given context in software design. A design pattern is not a finished design that can be transformed directly into code.

20.
Christopher Alexander -Architect

21.
Patterns are discovered– not invented

22.
Patterns are notindependent from the programming language. Example: subroutines in assembler. Gamma, Helm, Johnson, Vlissades (1995): Design patterns: elements of reusable object oriented software. Addison Wesley.

23.
Pattern classes PurposeCreational Structural Behavioural Scope Class Factory Method Adapter (class) Interpreter Template Method Object Abstract factory Builder Prototype Singleton Adapter (object) Bridge Composite Decorator Facade Flyweight Proxy Chain of responsibility Command Iterator Mediator Memento Observer State Strategy Visitor Gamma, Helm, Johnson, Vlissades (1995): Design patterns: elements of reusable object oriented software. Addison Wesley. Invisible or simplified in Python due to: first class types first class functions other Fixed at compile time Can change at runtime Object creation Compostion of classes or objects Class and object interactions

24.
Why are theyinvisible/ simplified? Some patterns are work-arounds for static typing

25.
Python has Firstclass* types

26.
First class* functionsAn object* is first-class when it: can be stored in variables and data structures

27.
can be passedas a parameter to a subroutine

28.
can be returnedas the result of a subroutine

29.
can be constructedat run-time

30.
has intrinsic identity(independent of any given name) *The term "object" is used loosely here, not necessarily referring to objects in object-oriented programming. The simplest scalar data types, such as integer and floating-point numbers, are nearly always first-class. Source:wikipedia

31.
Why are theyinvisible/simplified? (2) Python has duck typing

32.
Wikipedia: In computerprogramming with object-oriented programming languages, duck typing is a style of dynamic typing in which an object's current set of methods and properties determines the valid semantics, rather than its inheritance from a particular class or implementation of a specific interface. An object only has to have a method with the right name

33.
This means thata base class is not always needed

34.
Therefore a lotof infrastructure code can be avoided

35.
Why are theyinvisible/simplified? (3) Override special methods Automatic delegation Methods that a class does not know about can be passed on to a another class

36.
When to usea class Use a class only: if you need to inherit from it

37.
If you needto do something special. E.g. # Put in const.py...: class _const : class ConstError (TypeError): pass def __setattr__ ( self ,name,value): if self .__dict__.has_key(name): raise self .ConstError, "Can't rebind const (%s)" %name self .__dict__[name]=value import sys sys.modules[__name__]=_const() # that's all -- now any client-code can import const # and bind an attribute ONCE: const.magic = 23 # but NOT re-bind it: const.magic = 88 # raises const.ConstError # you may also want to add the obvious __delattr__ Alex Martelli http://code.activestate.com/recipes/65207-constants-in-python/

38.
Iterator – builtin Provide a way to access the elements of an aggregate object sequentially without exposing its underlying representation class Sequence : def __init__ ( self , size): self .list = [x for x in xrange(size)] self .index = 0 def __iter__ ( self ): return self def next ( self ): if len( self .list) == self .index: raise StopIteration current = self .list[ self .index] self .index += 1 return current >>> a = Sequence( 3 ) >>> for x in a: print x 0 1 2 >>> http://www.dofactory.com/Patterns/PatternIterator.aspx

39.
Command Encapsulate arequest as an object, thereby letting you parameterize clients with different requests, queue or log requests, and support undoable operations.

40.
Known uses: undo/redo.

41.
OO replacement forcallbacks.

42.
Specify, queue andexecute requests at different times. http://www.cs.mcgill.ca/~hv/classes/CS400/01.hchen/doc/command/command.html

43.
Command – GoFstyle Rahul Verma, Chetan Giridhar. Design Patterns in Python. www.testingperspective.com class Command : """The Command Abstract class""" def __init__ ( self ): pass #Make changes def execute ( self ): #OVERRIDE raise NotImplementedError class FlipUpCommand (Command): """The Command class for turning on the light""" def __init__ ( self , light): self .__light = light def execute ( self ): self .__light.turnOn()

44.
Command in Pythondef greet (who): print "Hello %s" % who greet_command = lambda : greet( "World" ) # pass the callable around, and invoke it later greet_command() class MoveFileCommand (object): def __init__ ( self , src, dest): self .src = src self .dest = dest self () def __call__ ( self ): os.rename( self .src, self .dest) def undo ( self ): os.rename( self .dest, self .src) undo_stack = [] undo_stack.append(MoveFileCommand( 'foo.txt' , 'bar.txt' )) undo_stack.append(MoveFileCommand( 'bar.txt' , 'baz.txt' )) # foo.txt is now renamed to baz.txt undo_stack.pop().undo() # Now it's bar.txt undo_stack.pop().undo() # and back to foo.txt http://stackoverflow.com/questions/1494442/general-command-pattern-and-command-dispatch-pattern-in-python (Ants Aasma) Simple case: Just use a callable More complex case: Use a command object but no need for a base class

45.
Singleton Ensure aclass has only one instance, and provide a global point of access to it. Excessive consumption may be harmful because: it overloads your liver

46.
makes you seemstupid

47.
it's global

48.
creates very strongcoupling with client classes http://en.csharp-online.net

49.
Singleton GoF stylehttp://code.activestate.com/recipes/52558-the-singleton-pattern-implemented-with-python/ class Singleton : class __impl : """ Implementation of the singleton interface """ def spam ( self ): """ Test method, return singleton id """ return id( self ) # storage for the instance reference __instance = None def __init__ ( self ): """ Create singleton instance """ # Check whether we already have an instance if Singleton.__instance is None : # Create and remember instance Singleton.__instance = Singleton.__impl() # Store instance reference as the only member in the handle self .__dict__[ '_Singleton__instance' ] = Singleton.__instance def __getattr__ ( self , attr): """ Delegate access to implementation """ return getattr( self .__instance, attr) def __setattr__ ( self , attr, value): return setattr( self .__instance, attr, value)

50.
Singleton in PythonUse a module (Alex Martelli - 99% of cases) Modules are objects too

51.
Allows you tocreate Fake objects for testing Just create one instance (99% of the rest) You could assign that to a module variable If that doesn't work also see the Borg pattern Shares common state among objects

52.
Strategy Define afamily of algorithms, encapsulate each one and make them interchangeable. Known uses: line breaking algorithms http://java-x.blogspot.com/2006/12/implementing-strategy-pattern-in-java.html

53.
Strategy - staticallytyped class Bisection (FindMinima): def algorithm ( self ,line): Return ( 5.5 , 6.6 ) class ConjugateGradient (FindMinima): def algorithm ( self ,line): Return ( 3.3 , 4.4 ) class MinimaSolver : # context class strategy= '' def __init__ ( self ,strategy): self .strategy=strategy def minima ( self ,line): return self .strategy.algorithm(line) def changeAlgorithm ( self , newAlgorithm): self .strategy = newAlgorithm def test (): solver=MinimaSolver(ConjugateGradient()) print solver.minima(( 5.5 , 5.5 )) solver.changeAlgorithm(Bisection()) print solver.minima(( 5.5 , 5.5 )) From J Gregorio http://assets.en.oreilly.com/1/event/12/_The%20Lack%20of_%20Design%20Patterns%20in%20Python%20Presentation.pdf

54.
Strategy in Pythondef bisection (line): Return 5.5 , 6.6 def conjugate_gradient (line): Return 3.3 , 4.4 def test (): solver = conjugate_gradient print solver(( 5.5 , 5.5 )) solver = bisection print solver(( 5.5 , 5.5 )) From J Gregorio http://assets.en.oreilly.com/1/event/12/_The%20Lack%20of_%20Design%20Patterns%20in%20Python%20Presentation.pdf Invisible because Python has first class functions

55.
Observer Define aone to many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically

56.
Known uses: model– view – controller, Qt Signals/Slots, Boost.Signals, most GUI toolkits http://en.wikipedia.org/wiki/File:Observer.svg

57.
Observer – GoFstyle class Observer (): def update ( self ): raise NotImplementedError class ConcreteObserver (Observer): def __init__ ( self , subject): self .subject = subject def update ( self ): data = self .subject.data # do something with data class Subject (): def __init__ ( self ): self .observers = [] def attach ( self , observer): self .observers.append(observer) def detach ( self , observer): self .observers.remove(observer) def notify ( self ): for o in self .observers: o.update() class ConcreteSubject (Subject): def __init__ ( self , data): self .data = data def do_something ( self ): self .data += 1 self .notify() Issues Deleted observers

58.
Detach during notify()

59.
Pass parameters tothe observer e.g. http://code.activestate.com/recipes/131499-observer-pattern/

60.
Observer in PythonSimplified - Observer base class is not required – use a callable E.g. PyDispatcher ( http://pydispatcher.sourceforge.net ) Use composition instead of inheritance from pydispatch import dispatcher # define the observer function def something_was_updated (data, signal, sender): print "data:" , data, "signal:" , signal, "sender:" ,sender class A_Model (): def __init__ ( self ): self .data = 100 # an object to identify the signal self .data_changed= "data" def do_something ( self ): self .data *= 1.34 # args are: signal, sender, args dispatcher.send( self .data_changed, self , self .data) # create an object to be an identifier for a signal sender = A_Model() # args are: receiver , signal, sender dispatcher.connect(something_was_updated, sender.data_changed, sender) sender.do_something()

61.
Decorator Attach additionalresponsibilities or functions to an object dynamically. Decorators provide a flexible alternative to subclassing for extending functionality.

62.
Not the sameas Python decorators Objects enclose other objects that share similar interfaces. The decorating object appears to mask or modify or annotate the enclosed object. http://en.wikipedia.org/wiki/File:Decorator_UML_class_diagram.svg

63.
Decorator GoF styleclass Writer (object): def write ( self , s): print s class WriterDecorator (object): def __init__ ( self , wrappee): self .wrappee = wrappee def write ( self , s): self .wrappee.write(s) class UpperWriter (WriterDecorator): def write ( self , s): self .wrappee.write(s.upper()) class ShoutWriter (WriterDecorator): def write ( self , s): self .wrappee.write( '!' .join( \ [t for t in s.split( ' ' ) if t]) + '!' ) Magnus Therning http://therning.org/magnus/archives/301 w = Writer() w.write( 'hello' ) uw = UpperWriter(w) uw.write( 'hello' ) wd = WriterDecorator(w) wd.write( 'hello' ) sw1 = ShoutWriter(w) sw1.write( 'hello again' ) sw2 = ShoutWriter(uw) sw2.write( 'hello again' ) >>> hello HELLO hello hello!again! HELLO!AGAIN!

64.
Decorator using functiondecorators def uppercase (f): def wrapper (*args, **kwargs): orig = f(*args, **kwargs) return orig.upper() return wrapper def shout (f): def wrapper (*args, **kwargs): orig = f(*args, **kwargs) return '!' .join( \ [t for t in orig.split( ' ' ) if t]\ ) + '!' return wrapper @shout def s_writer (s): return s print s_writer( "hello again" ) @shout @uppercase def su_writer (s): return s print su_writer( "hello again" ) >>> HELLO AGAIN HELLO!AGAIN!

65.
Decorator using delegationimport string class UpSeq : def __init__ ( self , seqobj): self .seqobj = seqobj def __str__ ( self ): return string.upper( self .seqobj.seq) def __getattr__ ( self ,attr): return getattr( self .seqobj, attr) class DNA (): def __init__ ( self , name, seq): self .seq = seq self .name = name def __getitem__ ( self , item): return self .seq.__getitem__(item) def first ( self ): return self .seq[ 0 ] s=UpSeq(DNA(name= '1' , seq= ' atcgctgtc ' )) >>> print s ATCGCTGTC >>> print s[ 0 : 3 ] atc >>> print s.first() a Adapted from http://www.pasteur.fr/formation/infobio/python/ UpSeq delegates to DNA

66.
State Allow anobject to alter its behaviour when its internal state changes. The object will appear to change its class

67.
Very common

68.
Known uses: TCP,GUIs http://en.wikipedia.org/wiki/File:State_Design_Pattern_UML_Class_Diagram.svg

69.
State GoF styleclass State(object): """Base state. This is to share functionality""" def scan( self ): """Scan the dial to the next station""" self .pos += 1 if self .pos == len( self .stations): self .pos = 0 print "Scanning… Station is" , \ self .stations[ self .pos], self .name class AmState(State): def __init__( self , radio): self .radio = radio self .stations = [ "1250" , "1380" , "1510" ] self .pos = 0 self .name = "AM" def toggle_amfm( self ): print "Switching to FM" self .radio.state = self .radio.fmstate class FmState(State): def __init__( self , radio): self .radio = radio self .stations = [ "81.3" , "89.1" , "103.9" ] self .pos = 0 self .name = "FM" def toggle_amfm( self ): print "Switching to AM" self .radio.state = self .radio.amstate class Radio(object): """A radio. It has a scan button, and an AM/FM toggle switch.""" def __init__( self ): """We have an AM state and an FM state""" self .amstate = AmState( self ) self .fmstate = FmState( self ) self .state = self .amstate def toggle_amfm( self ): self .state.toggle_amfm() def scan( self ): self .state.scan() # Test radio = Radio() actions = [radio.scan] * 2 + [radio.toggle_amfm] + [radio.scan] * 2 actions = actions * 2 for action in actions: action() Jeff ? http://ginstrom.com/scribbles/2007/10/08/design-patterns-python-style/ Scanning... Station is 1380 AM Scanning... Station is 1510 AM Switching to FM Scanning... Station is 89.1 FM Scanning... Station is 103.9 FM Scanning... Station is 81.3 FM Scanning... Station is 89.1 FM Switching to AM Scanning... Station is 1250 AM Scanning... Station is 1380 AM Note lack of state methods “Abstract” state Context Concrete states

70.
State in PythonSwitch classes or methods From: Alex Martelli http://www.aleax.it/goo_pydp.pdf class RingBuffer (object): def __init__ ( self ): self .d = list() def tolist ( self ): return list( self .d) def append ( self , item): self .d.append(item) if len( self .d) == MAX: self .c = 0 self .__class__ = _FullBuffer class _FullBuffer (object): def append ( self , item): self .d[ self .c] = item self .c = ( 1 + self .c) % MAX def tolist ( self ): return ( self .d[ self .c:] + self .d[: self .c] ) class RingBuffer (object): def __init__ ( self ): self .d = list() def append ( self , item): self .d.append(item) if len( self .d) == MAX: self .c = 0 self .append = self .append_full def append_full ( self , item): self .d.append(item) self .d.pop( 0 ) def tolist ( self ): return list( self .d) Irreversible state change Initial state. Use this until the buffer gets full. Method change implementation state change

71.
State in Python(method) class Sequencer(): def __init__( self ): self ._action_impl = self .action1 self .count = 1 def action( self ): self ._action_impl() def next( self ): self .count += 1 if self .count > 3 : self .count = 1 self ._action_impl = \ getattr( self , "action" +str( self .count)) def action1( self ): print "1" def action2( self ): print "2" def action3( self ): print "3" s = Sequencer() actions = [s.action] + [s.next] actions = actions * 3 for f in actions: f() >>> 1 2 3 >>> Switch methods output Use Bridge so that the binding of Sequencer.action doesn't change

72.
State in Python(class) >>> First 1 Second 2 Third 3 First 4 First 1 Second 2 Second 3 class Base (): def __init__ ( self ): self .state = 0 def action ( self ): self .state += 1 print self .__class__.__name__, self .state def change_state ( self , next_class): self .__class__ = next_class class Third (Base): def transition ( self ): self .change_state( First ) class Second (Base): def transition ( self ): self .change_state( Third ) class First (Base): def transition ( self ): self .change_state( Second ) state = First() state.action() state.transition() state.action() state.transition() state.action() state.transition() state.action() state = First() actions = [state.action] + [state.transition] actions = actions * 3 for action in actions: action() output This doesn't work because state is always First

73.
Bridge Decouple anabstraction from its implementation so that the two can vary independently Similar to strategy but isn't simplified in the same way

74.
Strategy is behavioural– interchange algorithms

75.
Bridge is structural– implementation varies independently from abstraction

76.
C++ pimpl http://atlas.kennesaw.edu/~dbraun/csis4650/A&D/GoF_Patterns

77.
Factory method Definean interface for creating an object, but let subclasses decide which class to instantiate. This method lets a class defer instantiation to subclasses http://en.wikipedia.org/wiki/File:FactoryMethod.svg

78.
Factory Method GOFstyle class Person : def __init__ ( self ): self .name = None self .gender = None def getName ( self ): return self .name def getGender ( self ): return self .gender class Male (Person): def __init__ ( self , name): print "Hello Mr." + name class Female (Person): def __init__ ( self , name): print "Hello Miss." + name class Factory : def getPerson ( self , name, gender): if gender == 'M' : return Male(name) if gender == 'F' : return Female(name) From: dpip.testingperspective.com factory = Factory() person = factory.getPerson( " Chetan " , "M" ) person = factory.getPerson( " Money " , "F" ) >>> Hello Mr.Chetan Hello Miss.Money

79.
Factory method inPython class Male (object): def __init__ ( self , name): print "Hello Mr." + name class Female (object): def __init__ ( self , name): print "Hello Ms." + name factory = dict(F=Female, M=Male) if __name__ == '__main__' : person = factory[ "F" ]( "Money" ) person = factory[ "M" ]( " Powers " ) >>> Hello Ms.Money Hello Mr.Powers Adapted from: http://www.rmi.net/~lutz/talk.html # variable length arg lists def factory (aClass, *args, **kwargs): return aClass(*args, **kwargs) class Spam : def __init__ ( self ): print self .__class__.__name__ def doit ( self , message): print message class Person : def __init__ ( self , name, job): self .name = name self .job = job print self .__class__.__name__, name, job object1 = factory(Spam) object2 = factory(Person, " Guido " , "guru" ) >>> Spam Person Guido guru

80.
Abstract factory Providean interface for creating families of related or dependent objects without specifying their concrete classes http://en.wikipedia.org/wiki/File:Abstract_factory.svg

81.
Abstract Factory GoFstyle class PetShop : def __init__ ( self , animal_factory= None ): """pet_factory is our abstract factory. We can set it at will.""" self .pet_factory = animal_factory def show_pet ( self ): """Creates and shows a pet using the abstract factory""" pet = self .pet_factory.get_pet() print "This is a lovely" , pet print "It says" , pet.speak() print "It eats" , self .pet_factory.get_food() class Dog : def speak ( self ): return "woof" def __str__ ( self ): return "Dog" class Cat : def speak ( self ): return " meow " def __str__ ( self ): return "Cat" http://ginstrom.com/scribbles/2007/10/08/design-patterns-python-style/ class DogFactory : def get_pet ( self ): return Dog() def get_food ( self ): return "dog food" class CatFactory : def get_pet ( self ): return Cat() def get_food ( self ): return "cat food" # Create the proper family def get_factory (): return random.choice([DogFactory, CatFactory])() # Show pets with various factories shop = PetShop() for i in range( 3 ): shop.pet_factory = get_factory() shop.show_pet() print "=" * 10 >>> This is a lovely Dog It says woof It eats dog food ========== This is a lovely Cat It says meow It eats cat food ========== This is a lovely Dog It says woof It eats dog food ==========

82.
Abstract factory inPython Use a module – for example similar to the os module # cat.py class Animal (): def __init__ ( self ): print "Cat" def speak (): return " meow " class Food : def acquire ( self ): print "go hunting" def serve ( self ): print "eat your catch, leave " \ "the entrails on the doorstep" #dog.py class Animal (): def __init__ ( self ): print "Dog" def speak (): return "woof" class Food : def acquire ( self ): print "wait for cat to catch" def serve ( self ): print "eat what the cat left" import dog as factory_1 import cat as factory_2 make_animal(factory_1) identify(factory_1) feed(factory_1) make_animal(factory_2) identify(factory_2) feed(factory_2) def make_animal (factory): return factory.Animal() def identify (factory): print factory.speak() def feed (factory): food = factory.Food() food.acquire() food.serve() If inheritance is not needed

83.
Flyweight Use sharingto support large numbers of fine grained objects efficiently http://www.lepus.org.uk/ref/companion/Flyweight.xml

84.
Flyweight in Pythonhttp://codesnipers.com/?q=python-flyweights import weakref class Card (object): _CardPool = weakref.WeakValueDictionary() def __new__ (cls, value, suit): obj = Card._CardPool.get(value + suit, None ) if not obj: obj = object.__new__(cls) Card._CardPool[value + suit] = obj obj.value, obj.suit = value, suit return obj c1 = Card( '9' , 'h' ) c2 = Card( '9' , 'h' ) c3 = Card( '2' , 's' ) print c1 == c2, c1 == c3, c2 == c3 print id(c1), id(c2), id(c3) >>> True False False 38958096 38958096 38958128

85.
Visible Patterns Coveredby Alex Martelli Facade

86.
Template method

87.
Adapter Others Mediator

88.
Bridge

89.
Composite

90.
Memento

91.
Summary Patterns aresimplified because Python has: First class objects and types

92.
Duck typing –base classes may be optional

93.
Ability to overridespecial methods

94.
References Alex Martelli,Design Patterns in Python, Google TechTalks, March 14, 2007. (find them on youtube)

95.
Joe Gregorio –The (lack of) design patterns in Python, Pycon 2009 ( http://bitworking.org/news/428/the-lack-of-design-patterns-in-python-pycon-2009 ) Peter Norvig - Design Patterns in Dynamic Programming, Object World, 1996. http://norvig.com/design-patterns/ppframe.htm Alan Shalloway, James Trott. Design Patterns Explained.

96.
Thank you http://dl.dropbox.com/u/10287301/Ramsey-KiwiPycon-2011.pdf

Patterns in Python

In this document