Introduction to python programming 1

Management Analytics Python Giovanni Della Lunga giovanni.dellalunga@gmail.com MASTER BIG DATA, ANALYTICS AND TECHNOLOGIES FOR MANAGEMENT

Python ABC A Concise Introduction

4 Major Versions of Python »“Python” is written in C/C++ - Version 2.7 came out in mid-2010 - Version 3.1.2 came out in early 2010 »“Jython” is written in Java for the JVM »“IronPython” is (was!) written in C# for the .Net environment

Development Environments what IDE to use? http://stackoverflow.com/questions/81584 1. PyDev with Eclipse 2. Komodo 3. Emacs 4. Vim 5. TextMate 6. Gedit 7. Idle 8. PIDA (Linux)(VIM Based) 9. NotePad++ (Windows) 10.BlueFish (Linux)

Setup »Anaconda  http://docs.continuum.io/conda/index.html  Installs:  Python env (including IPython)  Several packages »Eclipse (pre-requisite: Java)  http://www.eclipse.org/downloads/ »PyDev (requires Java 7)  Install: http://pydev.org/manual_101_install.html  Setup Interpreter

Python Interactive Shell % python Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> You can type things directly into a running Python session >>> 2+3*4 14 >>> name = "Andrew" >>> name 'Andrew' >>> print "Hello", name Hello Andrew >>>

The Python Interpreter • Python is an interpreted language • The interpreter provides an interactive environment to play with the language • Results of expressions are printed on the screen >>> 3 + 7 10 >>> 3 < 15 True >>> 'print me' 'print me' >>> print 'print me' print me >>>

The print Statement >>> print 'hello' hello >>> print 'hello', 'there' hello there • Elements separated by commas print with a space between them • A comma at the end of the statement (print ‘hello’,) will not print a newline character

No Braces, only Spaces! »Python uses indentation instead of braces to determine the scope of expressions »All lines must be indented the same amount to be part of the scope (or indented more if part of an inner scope) »This forces the programmer to use proper indentation since the indenting is part of the program!

Variables »Are not declared, just assigned »The variable is created the first time you assign it a value »Are references to objects »Type information is with the object, not the reference »Everything in Python is an object

Numbers: Integers »Integer – the equivalent of a C long »Long Integer – an unbounded integer value. >>> 132224 132224 >>> 132323 ** 2 17509376329L >>>

Numbers: Floating Point »int(x) converts x to an integer »float(x) converts x to a floating point »The interpreter shows a lot of digits >>> 1.23232 1.2323200000000001 >>> print 1.23232 1.23232 >>> 1.3E7 13000000.0 >>> int(2.0) 2 >>> float(2) 2.0

Numbers are immutable >>> x = 4.5 >>> y = x >>> y += 3 >>> x 4.5 >>> y 7.5 x 4.5 y x 4.5 y 7.5

Basic operations »Assignment:  size = 40  a = b = c = 3 »Numbers  integer, float  complex numbers: 1j+3, abs(z) »Strings  'hello world', 'it's hot'  "bye world"  continuation via or use """ long text """"

Date » import datetime » now = datetime.datetime.now() » print » print "Current date and time using str method of datetime object:" » print str(now) » print » print "Current date and time using instance attributes:" » print "Current year: %d" % now.year » print "Current month: %d" % now.month » print "Current day: %d" % now.day » print "Current hour: %d" % now.hour » print "Current minute: %d" % now.minute » print "Current second: %d" % now.second » print "Current microsecond: %d" % now.microsecond » print » print "Current date and time using strftime:" » print now.strftime("%Y-%m-%d %H:%M")

Lists »lists can be heterogeneous  a = ['spam', 'eggs', 100, 1234, 2*2] »Lists can be indexed and sliced:  a[0]  spam  a[:2]  ['spam', 'eggs'] »Lists can be manipulated  a[2] = a[2] + 23  a[0:2] = [1,12]  a[0:0] = []  len(a)  5

List methods »append(x) »extend(L)  append all items in list (like Tcl lappend) »insert(i,x) »remove(x) »pop([i]), pop()  create stack (FIFO), or queue (LIFO)  pop(0) »index(x)  return the index for value x

List methods »count(x)  how many times x appears in list »sort()  sort items in place »reverse()  reverse list

String Literals »Strings are immutable »There is no char type like in C++ or Java »+ is overloaded to do concatenation >>> x = 'hello' >>> x = x + ' there' >>> x 'hello there'

Strings share many features with lists >>> smiles = "C(=N)(N)N.C(=O)(O)O" >>> smiles[0] 'C' >>> smiles[1] '(' >>> smiles[-1] 'O' >>> smiles[1:5] '(=N)' >>> smiles[10:-4] 'C(=O)' Use “slice” notation to get a substring

String operations »concatenate with + or neighbors  word = 'Help' + x  word = 'Help' 'a' »subscripting of strings  'Hello'[2]  'l'  slice: 'Hello'[1:2]  'el'  word[-1]  last character  len(word)  5  immutable: cannot assign to subscript

String Methods: find, split smiles = "C(=N)(N)N.C(=O)(O)O" >>> smiles.find("(O)") 15 >>> smiles.find(".") 9 >>> smiles.find(".", 10) -1 >>> smiles.split(".") ['C(=N)(N)N', 'C(=O)(O)O'] >>> Use “find” to find the start of a substring. Start looking at position 10. Find returns -1 if it couldn’t find a match. Split the string into parts with “.” as the delimiter

String operators: in, not in if "Br" in “Brother”: print "contains brother“ email_address = “clin” if "@" not in email_address: email_address += "@brandeis.edu“

String Method: “strip”, “rstrip”, “lstrip” are ways to remove whitespace or selected characters >>> line = " # This is a comment line n" >>> line.strip() '# This is a comment line' >>> line.rstrip() ' # This is a comment line' >>> line.rstrip("n") ' # This is a comment line ' >>>

More String methods email.startswith(“c") endswith(“u”) True/False >>> "%s@brandeis.edu" % "clin" 'clin@brandeis.edu' >>> names = [“Ben", “Chen", “Yaqin"] >>> ", ".join(names) ‘Ben, Chen, Yaqin‘ >>> “chen".upper() ‘CHEN'

Control flow: if x = int(raw_input("Please enter #:")) if x < 0: x = 0 print 'Negative changed to zero' elif x == 0: print 'Zero' elif x == 1: print 'Single' else: print 'More' » no case statement

Control flow: for a = ['cat', 'window', 'defenestrate'] for x in a: print x, len(x) »no arithmetic progression, but  range(10)  [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]  for i in range(len(a)): print i, a[i] »do not modify the sequence being iterated over

Loop Control Statements break Jumps out of the closest enclosing loop continue Jumps to the top of the closest enclosing loop pass Does nothing, empty statement placeholder

Loops: break, continue, else »break and continue like C »else after loop exhaustion for n in range(2,10): for x in range(2,n): if n % x == 0: print n, 'equals', x, '*', n/x break else: # loop fell through without finding a factor print n, 'is prime'

Loop example » import fnmatch » import os » images = ['*.jpg', '*.jpeg', '*.png', '*.tif', '*.tiff'] » matches = [] » for root, dirnames, filenames in os.walk('C:'): » for extensions in images: » for filename in fnmatch.filter(filenames, extensions): » print filename » matches.append(os.path.join(root, filename)) Simple Matching fnmatch() compares a single file name against a pattern and returns a Boolean indicating whether or not they match. Filtering To test a sequence of filenames, you can use filter(). It returns a list of the names that match the pattern argument.

Loop example » # Python program to check if the input number is prime or not » num = 407 » # take input from the user » # num = int(input("Enter a number: ")) » # prime numbers are greater than 1 » if num > 1: » # check for factors » for i in range(2,num): » if (num % i) == 0: » print(num,"is not a prime number") » print(i,"times",num//i,"is",num) » break » else: » print(num,"is a prime number") » » # if input number is less than » # or equal to 1, it is not prime » else: » print(num,"is not a prime number") To understand this example, you should have the knowledge of following Python programming topics: • Python if...else Statement • Python for Loop • Python break and continue A positive integer greater than 1 which has no other factors except 1 and the number itself is called a prime number. 2, 3, 5, 7 etc. are prime numbers as they do not have any other factors. But 6 is not prime (it is composite) since, 2 x 3 = 6.

Python Structures Storing Structured Information

List comprehensions »Create lists without map(), filter(), lambda »= expression followed by for clause + zero or more for or of clauses >>> vec = [2,4,6] >>> [3*x for x in vec] [6, 12, 18] >>> [{x: x**2} for x in vec} [{2: 4}, {4: 16}, {6: 36}]

List comprehensions »cross products: >>> vec1 = [2,4,6] >>> vec2 = [4,3,-9] >>> [x*y for x in vec1 for y in vec2] [8,6,-18, 16,12,-36, 24,18,-54] >>> [x+y for x in vec1 and y in vec2] [6,5,-7,8,7,-5,10,9,-3] >>> [vec1[i]*vec2[i] for i in range(len(vec1))] [8,12,-54]

List comprehensions »can also use if: >>> [3*x for x in vec if x > 3] [12, 18] >>> [3*x for x in vec if x < 2] []

Tuples: sort of an immutable list >>> yellow = (255, 255, 0) # r, g, b >>> one = (1,) >>> yellow[0] >>> yellow[1:] (255, 0) >>> yellow[0] = 0 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'tuple' object does not support item assignment Very common in string interpolation: >>> "%s lives in %s at latitude %.1f" % ("Andrew", "Sweden", 57.7056) 'Andrew lives in Sweden at latitude 57.7'

Tuples and sequences »lists, strings, tuples: examples of sequence type »tuple = values separated by commas >>> t = 123, 543, 'bar' >>> t[0] 123 >>> t (123, 543, 'bar')

Tuples »Tuples may be nested >>> u = t, (1,2) >>> u ((123, 542, 'bar'), (1,2)) »like strings, immutable  can't assign to individual items

Tuples »Empty tuples: () >>> empty = () >>> len(empty) 0 »one item  trailing comma >>> singleton = 'foo',

Tuples »sequence unpacking  distribute elements across variables >>> t = 123, 543, 'bar' >>> x, y, z = t >>> x 123 »packing always creates tuple »unpacking works for any sequence

Dictionaries »associative arrays »indexed by keys »keys are any immutable type: e.g., tuples »but not lists (mutable!) »uses 'key: value' notation >>> tel = {'hgs' : 7042, 'lennox': 7018} >>> tel['cs'] = 7000 >>> tel

Dictionaries »no particular order »delete elements with del >>> del tel['foo'] »keys() method  unsorted list of keys >>> tel.keys() ['cs', 'lennox', 'hgs'] »use has_key() to check for existence >>> tel.has_key('foo') 0

Dictionaries » prices = {'apple': 0.40, 'banana': 0.50} » my_purchase = { » 'apple': 1, » 'banana': 6} » grocery_bill = sum(prices[fruit] * my_purchase[fruit] » for fruit in my_purchase) » print 'I owe the grocer $%.2f' % grocery_bill

Defining functions def fib(n): """Print a Fibonacci series up to n.""" a, b = 0, 1 while b < n: print b, a, b = b, a+b >>> fib(2000) » First line is docstring » first look for variables in local, then global » need global to assign global variables

Importing and Modules » Use classes & functions defined in another file » A Python module is a file with the same name (plus the .py extension) » Like Java import, C++ include » Three formats of the command: import somefile from somefile import * from somefile import className » The difference? What gets imported from the file and what name refers to it after importing

import … import somefile » Everything in somefile.py gets imported. » To refer to something in the file, append the text “somefile.” to the front of its name: somefile.className.method(“abc”) somefile.myFunction(34)

from … import * from somefile import * » Everything in somefile.py gets imported » To refer to anything in the module, just use its name. Everything in the module is now in the current namespace. » Take care! Using this import command can easily overwrite the definition of an existing function or variable! className.method(“abc”) myFunction(34)

from … import … from somefile import className » Only the item className in somefile.py gets imported. » After importing className, you can just use it without a module prefix. It’s brought into the current namespace. » Take care! Overwrites the definition of this name if already defined in the current namespace! className.method(“abc”) imported myFunction(34)  Not imported

Module search path »current directory »list of directories specified in PYTHONPATH environment variable »uses installation-default if not defined, e.g., .:/usr/local/lib/python »uses sys.path >>> import sys >>> sys.path ['', 'C:PROGRA~1Python2.2', 'C:Program FilesPython2.2DLLs', 'C:Program FilesPython2.2lib', 'C:Program FilesPython2.2liblib-tk', 'C:Program FilesPython2.2', 'C:Program FilesPython2.2libsite- packages']

Target »Web Scraping  The need and importance of extracting data from the web is becoming increasingly loud and clear.  There are several ways to extract information from the web.  Use of APIs being probably the best way to extract data from a website. If you can get what you need through an API, it is almost always preferred approach over web scrapping.

Target »Web Scraping  Sadly, not all websites provide an API.  Some do it because they do not want the readers to extract huge information in structured way, while others don’t provide APIs due to lack of technical knowledge. What do you do in these cases?  Well, we need to scrape the website to fetch the information.

Target »Ok, but what is Web Scraping?  Web scraping is a computer software technique of extracting information from websites. This technique mostly focuses on the transformation of unstructured data (HTML format) on the web into structured data (database or spreadsheet).  You can perform web scrapping in various ways…  We’ll resort to Python because of its ease and rich ecosystem. It has a library known as ‘BeautifulSoup’ which assists this task.  In this lesson, I’ll show you the easiest way to learn web scraping using python programming.

Downloading Files from the Web with the requests Module » The requests module lets you easily download files from the Web without having to worry about complicated issues such as network errors, connection problems, and data compression. » The requests module doesn’t come with Python, so you’ll have to install it first. From the command line, run pip install requests. » Next, do a simple test to make sure the requests module installed itself correctly. Enter the following into the interactive shell: >>> import requests » If no error messages show up, then the requests module has been successfully installed.

Downloading a Web Page with the requests.get() Function »The requests.get() function takes a string of a URL to download. »By calling type() on requests.get()’s return value, you can see that it returns a Response object, which contains the response that the web server gave for your request…

Downloading a Web Page with the requests.get() Function The Complete Works of William Shakespeare The Tragedy of Romeo and Juliet The Library of the Future Complete Works of William Shakespeare Library of the Future is a TradeMark (TM) of World Library Inc. <<THIS ELECTRONIC VERSION OF THE COMPLETE WORKS OF WILLIAM SHAKESPEARE IS COPYRIGHT 1990-1993 BY WORLD LIBRARY, INC., AND IS PROVIDED BY PROJECT GUTENBERG ETEXT OF CARNEGIE MELLON UNIVERSITY WITH PERMISSION. ELECTRONIC AND MACHINE READABLE COPIES MAY BE DISTRIBUTED SO LONG AS SUCH COPIES (1) ARE FOR YOUR OR OTHERS PERSONAL USE ONLY, AND (2) ARE NOT DISTRIBUTED OR USED COMMERCIALLY. PROHIBITED COMMERCIAL DISTRIBUTION INCLUDES BY ANY SERVICE THAT CHARGES FOR DOWNLOAD TIME OR FOR MEMBERSHIP.>> 1595 THE TRAGEDY OF ROMEO AND JULIET by William Shakespeare

Libraries required for web scraping » Urllib2: It is a Python module which can be used for fetching URLs.  It defines functions and classes to help with URL actions (basic and digest authentication, redirections, cookies, etc).  For more detail refer to the documentation page. » BeautifulSoup: It is an incredible tool for pulling out information from a webpage.  You can use it to extract tables, lists, paragraph and you can also put filters to extract information from web pages.  You can look at the installation instruction in its documentation page.

Basics – Get familiar with HTML (Tags) <!DOCTYPE html> : HTML documents must start with a type declaration HTML document is contained between <html> and </html> The visible part of the HTML document is between <body> and </body> HTML headings are defined with the <h1> to <h6> tags HTML paragraphs are defined with the <p> tag

Basics – Get familiar with HTML (Tags)

Basics – Get familiar with HTML (Tags) »Other useful HTML tags are:  HTML links are defined with the <a> tag, “<a href=“http://www.test.com” >This is a link for test.com</a>”  HTML tables are defined with<Table>, row as <tr> and rows are divided into data as <td>

Scrapping a web Page using BeautifulSoup »Here, I am scraping data from a Wikipedia page.  Our final goal is to extract list of state, union territory capitals in India. And some basic detail like establishment, former capital and others form this wikipedia page.  Let’s learn with doing this project step wise step…

Defining Classes Object Oriented Programming in Python

It’s all objects… »Everything in Python is really an object.  We’ve seen hints of this already… “hello”.upper() list3.append(‘a’) dict2.keys()  These look like Java or C++ method calls.  New object classes can easily be defined in addition to these built- in data-types. »In fact, programming in Python is typically done in an object oriented fashion.

Defining a Class »A class is a special data type which defines how to build a certain kind of object. »The class also stores some data items that are shared by all the instances of this class »Instances are objects that are created which follow the definition given inside of the class »Python doesn’t use separate class interface definitions as in some languages »You just define the class and then use it

Methods in Classes »Define a method in a class by including function definitions within the scope of the class block »There must be a special first argument self in all of method definitions which gets bound to the calling instance »There is usually a special method called __init__ in most classes »We’ll talk about both later…

A simple class def: student class student: “““A class representing a student ””” def __init__(self,n,a): self.full_name = n self.age = a def get_age(self): return self.age

Instantiating Objects » There is no “new” keyword as in Java. » Just use the class name with ( ) notation and assign the result to a variable » __init__ serves as a constructor for the class. Usually does some initialization work » The arguments passed to the class name are given to its __init__() method » So, the __init__ method for student is passed “Bob” and 21 and the new class instance is bound to b: b = student(“Bob”, 21)

Constructor: __init__ »An __init__ method can take any number of arguments. »Like other functions or methods, the arguments can be defined with default values, making them optional to the caller. »However, the first argument self in the definition of __init__ is special…

Self »The first argument of every method is a reference to the current instance of the class »By convention, we name this argument self »In __init__, self refers to the object currently being created; so, in other class methods, it refers to the instance whose method was called »Similar to the keyword this in Java or C++ »But Python uses self more often than Java uses this

Self »Although you must specify self explicitly when defining the method, you don’t include it when calling the method. »Python passes it for you automatically Defining a method: Calling a method: (this code inside a class definition.) def set_age(self, num): >>> x.set_age(23) self.age = num

Deleting instances: No Need to “free” »When you are done with an object, you don’t have to delete or free it explicitly. »Python has automatic garbage collection. »Python will automatically detect when all of the references to a piece of memory have gone out of scope. Automatically frees that memory. »Generally works well, few memory leaks »There’s also no “destructor” method for classes

Definition of student class student: “““A class representing a student ””” def __init__(self,n,a): self.full_name = n self.age = a def get_age(self): return self.age

Traditional Syntax for Access >>> f = student(“Bob Smith”, 23) >>> f.full_name # Access attribute “Bob Smith” >>> f.get_age() # Access a method 23

Two Kinds of Attributes » The non-method data stored by objects are called attributes » Data attributes  Variable owned by a particular instance of a class  Each instance has its own value for it  These are the most common kind of attribute » Class attributes  Owned by the class as a whole  All class instances share the same value for it  Called “static” variables in some languages  Good for (1) class-wide constants and (2) building counter of how many instances of the class have been made

Data Attributes »Data attributes are created and initialized by an __init__() method.  Simply assigning to a name creates the attribute  Inside the class, refer to data attributes using self  for example, self.full_name class teacher: “A class representing teachers.” def __init__(self,n): self.full_name = n def print_name(self): print self.full_name

Class Attributes » Because all instances of a class share one copy of a class attribute, when any instance changes it, the value is changed for all instances » Class attributes are defined within a class definition and outside of any method » Since there is one of these attributes per class and not one per instance, they’re accessed via a different notation:  Access class attributes using self.__class__.name notation -- This is just one way to do this & the safest in general. class sample: >>> a = sample() x = 23 >>> a.increment() def increment(self): >>> a.__class__.x self.__class__.x += 1 24

Data vs. Class Attributes class counter: overall_total = 0 # class attribute def __init__(self): self.my_total = 0 # data attribute def increment(self): counter.overall_total = counter.overall_total + 1 self.my_total = self.my_total + 1 >>> a = counter() >>> b = counter() >>> a.increment() >>> b.increment() >>> b.increment() >>> a.my_total 1 >>> a.__class__.overall_total 3 >>> b.my_total 2 >>> b.__class__.overall_total 3

Subclasses »A class can extend the definition of another class  Allows use (or extension ) of methods and attributes already defined in the previous one.  New class: subclass. Original: parent, ancestor or superclass »To define a subclass, put the name of the superclass in parentheses after the subclass’s name on the first line of the definition. Class Cs_student(student):  Python has no ‘extends’ keyword like Java.  Multiple inheritance is supported.

Redefining Methods »To redefine a method of the parent class, include a new definition using the same name in the subclass.  The old code won’t get executed. »To execute the method in the parent class in addition to new code for some method, explicitly call the parent’s version of the method. parentClass.methodName(self, a, b, c)  The only time you ever explicitly pass ‘self’ as an argument is when calling a method of an ancestor.

Definition of a class extending student Class Student: “A class representing a student.” def __init__(self,n,a): self.full_name = n self.age = a def get_age(self): return self.age Class Cs_student (student): “A class extending student.” def __init__(self,n,a,s): student.__init__(self,n,a) #Call __init__ for student self.section_num = s def get_age(): #Redefines get_age method entirely print “Age: ” + str(self.age)

Introduction to python programming 1

More Related Content

What's hot

Similar to Introduction to python programming 1

More from Giovanni Della Lunga

Recently uploaded

Introduction to python programming 1