Core Python Programming

Praise for Core Python Programming

"The long-awaited second edition of Wesley Chun's Core Python Programming proves to be well worth the wait–its deep and broad coverage and useful exercises will help readers learn and practice good Python."

–Alex Martelli, author of Python in a Nutshell and editor of Python Cookbook

"There has been lot of good buzz around Wesley Chun's Core Python Programming. It turns out that all the buzz is well earned. I think this is the best book currently available for learning Python. I would recommend Chun's book over Learning Python (O'Reilly), Programming Python (O'Reilly), or The Quick Python Book (Manning)."

–David Mertz, Ph.D., IBM DeveloperWorks®

"I have been doing a lot of research [on] Python for the past year and have seen a number of positive reviews of your book. The sentiment expressed confirms the opinion that Core Python Programming is now considered the standard introductory text."

–Richard Ozaki, Lockheed Martin

"Finally, a book good enough to be both a textbook and a reference on the Python language now exists."

–Michael Baxter, Linux Journal

"Very well written. It is the clearest, friendliest book I have come across yet for explaining Python, and putting it in a wider context. It does not presume a large amount of other experience. It does go into some important Python topics carefully and in depth. Unlike too many beginner books, it never condescends or tortures the reader with childish hide-and-seek prose games. [It] sticks to gaining a solid grasp of Python syntax and structure."

–http://python.org bookstore Web site

"[If ] I could only own one Python book, it would be Core Python Programming by Wesley Chun. This book manages to cover more topics in more depth than Learning Python but includes it all in one book that also more than adequately covers the core language. [If] you are in the market for just one book about Python, I recommend this book. You will enjoy reading it, including its wry programmer's wit. More importantly, you will learn Python. Even more importantly, you will find it invaluable in helping you in your day-to-day Python programming life. Well done, Mr. Chun!"

–Ron Stephens, Python Learning Foundation

"I think the best language for beginners is Python, without a doubt. My favorite book is Core Python Programming."

–s003apr, MP3Car.com Forums

"Personally, I really like Python. It's simple to learn, completely intuitive, amazingly flexible, and pretty darned fast. Python has only just started to claim mindshare in the Windows world, but look for it to start gaining lots of support as people discover it. To learn Python, I'd start with Core Python Programming by Wesley Chun."

–Bill Boswell, MCSE, Microsoft Certified Professional Magazine Online

"If you learn well from books, I suggest Core Python Programming. It is by far the best I've found. I'm a Python newbie as well and in three months time I've been able to implement Python in projects at work (automating MSOffice, SQL DB stuff, etc.)."

–ptonman, Dev Shed Forums

"Python is simply a beautiful language. It's easy to learn, it's cross-platform, and it works. It has achieved many of the technical goals that Java strives for. A one-sentence description of Python would be: 'All other languages appear to have evolved over time--but Python was designed.' And it was designed well. Unfortunately, there aren't a large number of books for Python. The best one I've run across so far is Core Python Programming."

–Chris Timmons, C. R. Timmons Consulting

"If you like the Prentice Hall Core series, another good full-blown treatment to consider would be Core Python Programming. It addresses in elaborate concrete detail many practical topics that get little, if any, coverage in other books."

–Mitchell L Model, MLM Consulting

"Core Python Programming is an amazingly easy read! The liberal use of examples helps clarify some of the more subtle points of the language. And the comparisons to languages with which I'm already familiar (C/C++/Java) get you programming in record speed."

–Michael Santos, Ph.D., Green Hills Software

• New to Python? The definitive guide to Python development for experienced programmers
• Covers core language features thoroughly, including those found in the latest Python releases–learn more than just the syntax!
• Learn advanced topics such as regular expressions, networking, multithreading, GUI, Web/CGI, and Python extensions
• Includes brand-new material on databases, Internet clients, Java/Jython, and Microsoft Office, plus Python 2.6 and 3
• Presents hundreds of code snippets, interactive examples, and practical exercises to strengthen your Python skills

Python is an agile, robust, expressive, fully object-oriented, extensible, and scalable programming language. It combines the power of compiled languages with the simplicity and rapid development of scripting languages. In Core Python Programming, Second Edition, leading Python developer and trainer Wesley Chun helps you learn Python quickly and comprehensively so that you can immediately succeed with any Python project.

Using practical code examples, Chun introduces all the fundamentals of Python programming: syntax, objects and memory management, data types, operators, files and I/O, functions, generators, error handling and exceptions, loops, iterators, functional programming, object-oriented programming and more. After you learn the core fundamentals of Python, he shows you what you can do with your new skills, delving into advanced topics, such as regular expressions, networking programming with sockets, multithreading, GUI development, Web/CGI programming and extending Python in C.

This edition reflects major enhancements in the Python 2.x series, including 2.6 and tips for migrating to 3. It contains new chapters on database and Internet client programming, plus coverage of many new topics, including new-style classes, Java and Jython, Microsoft Office (Win32 COM Client) programming, and much more.

• Learn professional Python style, best practices, and good programming habits
• Gain a deep understanding of Python's objects and memory model as well as its OOP features, including those found in Python's new-style classes
• Build more effective Web, CGI, Internet, and network and other client/server applications
• Learn how to develop your own GUI applications using Tkinter and other toolkits available for Python
• Improve the performance of your Python applications by writing extensions in C and other languages, or enhance I/O-bound applications by using multithreading
• Learn about Python's database API and how to use a variety of database systems with Python, including MySQL, Postgres, and SQLite
• Features appendices on Python 2.6 & 3, including tips on migrating to the next generation!

Core Python Programming delivers

• Systematic, expert coverage of Python's core features
• Powerful insights for developing complex applications
• Easy-to-use tables and charts detailing Python modules, operators, functions, and methods
• Dozens of professional-quality code examples, from quick snippets to full-fledged applications

Welcome to Core Python Programming!

We are delighted that you have engaged us to help you learn Python as quickly and as in-depth as possible. Learning the syntax is one goal of this book; however, we also believe that if you learn how Python works under the covers, you won't just be able to program in Python, but you will write more effective Python applications even as a beginner to the language. As you know, just because you learn a language's syntax does not make you competent in it right away.

Throughout the book, you will find many examples that you can try right in front of your computer. To hammer the concepts home, you will also find fun and challenging exercises at the end of every chapter. These easy and intermediate exercises are meant to test your learning and push your Python skills. There simply is no substitute for experience. We believe you should not only pick up Python programming skills but also be able to master them in as short a time period as possible.

About This Book

This book differs from other Python books on the market by presenting a broad range of topics, providing numerous examples, and going in-depth where necessary. This book does not require a specific background such as prior knowledge of C or object-oriented programming. It is also not a large case study book that does not facilitate picking up the language quickly. Finally, this book is not a pure reference nor is it meant to be a quick "dive" into Python. What we have is an extremely comprehensive introduction to the core features of the language (Part I) followed by a set of chapters that delve into specific areas of intermediate Python programming.

This book is 40 percent introductory, 40 percent intermediate to advanced, and 20 percent reference. It is targeted toward technical professionals who are already familiar with programming in one other high-level language, as well as university/college and secondary students. Because Python is used in larger solutions such as Zope, Plone, MailMan, and Django, this book may be used by principals developing, managing, maintaining, or integrating with those systems.

With regard to the code in this book, about a third of the first edition readers sent in complaints that there were not enough large, full-fledged applications in the book, or that the code examples were not long or comprehensive enough. Everyone else wrote that they loved the short, easy-to-understand examples and were not bored of page after page of mind-numbing code. The philosophy behind more short examples is to give you the ability to look at a piece of code and grasp its entirety. These turn into building blocks to understanding and then can be incorporated into larger applications as well. There are line-by-line explanations for most of the larger programs in the book. The abundant interpreter code snippets scattered throughout the book are there for you to try on your computer as you are learning Python—use the interactive interpreter as much as possible. You not only learn and improve your Python from using it, but you can also benefit from working out bugs in your code before you paste it into your source file.

Because you cannot learn Python well without practice, you will find the exercises at the end of every chapter to be one of the greatest strengths of this book. They will test your knowledge of chapter topics and definitions, as well as get you to code as much as possible. There is no substitute to learning a programming language faster and more effectively than by building applications. You will find easy, intermediate, and difficult problems to solve. It is also here that you may have to write one of those "large" applications that many readers wanted to see in the book, but rather than having me do it, you gain the most from such exercises. Appendix A features answers to selected problems from each chapter.

Another set of first edition readers remarked how useful the reference tables were throughout the book, and how they meticulously copied them for reference. Well, instead of flipping through each chapter looking for the tables, we have summarized the most highly used ones in Appendix B. Thanks for all of your feedback. I encourage you to keep talking to us and help us make a third edition possible and better than its predecessors!

Finally, both the "Other References" appendix and the CD-ROM from the first edition are not included with this edition. You would not believe how quickly Web links can become obsolete in six months much less six years! The most up-to-date source code and Python interpreters can easily be downloaded for offline use at the book's Web site, so there really is no reason to include a CD-ROM.

About the Reader

This book is meant for you if you are a programmer completely new to Python or already know some Python but want to know more and improve your Python skillset. Python is used in many fields, including engineering, informa*tion technology, science, business, entertainment, and so on. This means that the list of Python users (and readers of this book) includes but is not limited to:

• Software engineers

• Hardware design/CAD engineers

• QA/testing and automation framework developers

• IS/IT/system and network administrators

• Scientists and mathematicians

• Technical or project management staff

• Multimedia or audio/visual engineers

• SCM or release engineers

• Web masters and content management staff

• Customer/technical support engineers

• Database engineers and administrators

• Research and development engineers

• Software integration and professional services staff

• Collegiate and secondary educators

• Web service engineers

• Financial software engineers

• And many others

Some of the most famous companies using Python include Google, Yahoo!, NASA, Lucasfilm/Industrial Light and Magic, Red Hat, Zope, Disney, Pixar, and Dreamworks.

The Author's Experience with Python

I discovered Python over a decade ago at a company called Four11. At the time, the company had one major product, the Four11.com White Page directory service. Python was being used to design our next product: the Rocketmail Web-based e-mail service that would eventually evolve into what today is Yahoo!Mail.

It was fun learning Python and being on the original Yahoo!Mail engineering team. I helped rearchitect the address book and spell checker. At the time, Python also made its way as part of a number of other Yahoo! sites, including People Search, Yellow Pages, and Maps and Driving Directions, just to name a few. I was the lead engineer for People Search.

Although Python was new to me then, it was fairly easy to pick up—much simpler than other languages I had learned in the past. The scarcity of textbooks at the time led me to primarily use the Library Reference and Quick Reference Guide as my tools in learning, and also led to the motivation for the book you are reading right now.

Since my days at Yahoo!, I have been able to use Python in all sorts of interesting ways at the jobs that followed. In each case, I was able to harness the power of Python in solving the problems at hand and in a timely manner. I have also developed several Python courses and have used this book to teach those classes, truly eating my own dogfood.

Not only is Core Python Programming a great book to learn Python from, but it is also the best book to teach Python with! As an engineer, I know what it takes to learn, understand, and apply a new technology. As a professional instructor, I also know what is needed to deliver the most effective sessions for clients. This provides the experience necessary to be able to give you real-world analogies and tips that you cannot get from someone who is "just a trainer" or "just a book author."

About the Author's Writing Style: Technical, Yet Easy Reading

Rather than strictly a "beginners" book or a pure, hard-core computer science reference book, my instructional experience indicates that an easy-to-read, yet technically oriented book serves our purpose the best, which is to get you up to speed on Python as quickly as possible so that you can apply it to your tasks posthaste. We will introduce concepts coupled with appropriate examples to expedite the learning process. At the end of each chapter you will find numerous exercises to reinforce some of the concepts and ideas acquired in your reading.

We are thrilled and humbled to be compared with Bruce Eckel's writing style (see the reviews to the first edition at the book's Web site (http://www.corepython.com). This is not a dry college textbook. As the author, I am having a conversation with you, as if you were attending one of my well-received Python training courses. As a lifelong student, I constantly put myself in my student's shoes and tell you what you need to hear in order to learn the concepts as quickly and as thoroughly as possible. You will find reading this book fast and easy, without losing sight of the technical details.

As an engineer, I know what I need to tell you in order to teach you a concept in Python. As a teacher, I can take technical details and boil them down into language that is easy to understand and grasp right away. You are getting the best of both worlds with my writing and teaching styles, but you will enjoy programming in Python even more.

About This Second Edition

At the time the first edition was published, Python was entering its second era with the release of version 2.0. Since then, the language has seen significant improvements contributing to the overall continuing success and acceptance of the language. Deficiencies have been removed and new features added that bring a new level of power and sophistication to Python developers worldwide. We are thrilled to be able to update this book yet still deliver easy reading along with comprehensive coverage of the exciting new features. This book includes changes to Python 2.5, released in the fall of 2006, and even some pre-announced features of 2.6 and beyond. As in the first edition, we aim to keep all of the topics relevant for readers regardless of the Python version you are using, extending the lifetime of this book, retarding its obsolescence.

Python is slowly going to be transitioning to the next big version change with a release affectionately called "Python 3000" by its creator, Guido van Rossum. This is just the marketing name for Python 3.0, or "Py3K" for short. It will be developed in parallel with the remaining 2.x releases. There will be some incompatibilities with older versions of Python; however, the core team will work hard to ensure that code will be backwards-compatible for the most part. (This is in tradition with any new Python release.) Look mostly for interesting additions to the language as well as the disappearance of old design flaws and deprecated features.

We will continue to update the book's Web site with white papers, updates, and other related articles to keep Core Python Programming as contemporary as possible, regardless of which new release of Python you have migrated to.

The new topics we have added to this edition include:

Boolean and set types (Chapters 5 and 7)

New-style classes (Chapter 13)

• Subclassing built-in types
• Static methods and class methods
• Slots
• Properties
• Descriptors
• Metaclasses

Functions (Chapter 11)

• Generators
• Function (and method) decorators
• Statically nested scoping
• Inner functions
• Closures
• Currying and partial function application

Looping constructs (Chapter 8)

• Iterators
• List comprehensions
• Generator expressions

Extended import syntax (Chapter 12)

• as keyword
• Multi-line import
• Absolute importing
• Relative importing

Improved exception handling features (Chapter 10)

• with statement
• try-except-finally statement

In addition, we are proud to introduce three new chapters to the book: "Internet Client Programming" (Chapter 17), "Database Programming" (Chapter 21), and "Miscellaneous" (Chapter 23). These are a few intermediate areas where Python is used quite often. All existing chapters have been refreshed and updated to the latest versions of Python. Please see the chapter guide that follows for more details.

Chapter Guide

This book is divided into two main sections. The first part, taking up about two-thirds of the text, gives you treatment of the "core" part of the language, and the second part provides a set of various advanced topics to show what you can build using Python.

Python is everywhere—sometimes it is amazing to discover who is using Python and what they are doing with it—and although we would have loved to produce additional chapters on such topics as Java/Jython, Win32 programming, CGI processing with HTMLgen, GUI programming with third-party toolkits (wxWidgets, GTK+, Qt, etc.), XML processing, numerical and scientific processing, visual and graphics image manipulation, and Web services and application frameworks (Zope, Plone, Django, TurboGears, and so on), there simply wasn't enough time to develop these topics into their own chapters. However, we are certainly glad that we were at least able to provide you with a good introduction to many of the key areas of Python development including some of the topics mentioned reviously.

Here is a chapter-by-chapter guide.

Part I: Core Python

Chapter 1, Welcome to Python!

We begin by introducing Python to you, its history, features, benefits, and so on, as well as how to obtain and install Python on your system.

Chapter 2, Getting Started

If you are an experienced programmer and just want to see "how it's done" in Python, this is the right place to go. We introduce the basic Python concepts and statements, and because many of these will be familiar to you, you can simply learn the proper syntax in Python and get started right away on your projects without sacrificing too much reading time.

Chapter 3, Syntax and Style

This section gives you a good overview of Python's syntax as well as style hints. You will also be exposed to Python's keywords and its memory management ability. Your first Python application will be presented at the end of the chapter to give you an idea of what real Python code looks like.

Chapter 4, Python Objects

This chapter introduces Python objects. In addition to generic object attributes, we will show you all of Python's data types and operators, as well as show you different ways to categorize the standard types. Built-in functions that apply to most Python objects will also be covered.

Chapter 5, Numbers

In this chapter, we discuss Python's main numeric types: integers, floating point numbers, and complex numbers. We look at operators and built-in and factory functions which apply to all numbers, and we also briefly discuss a few other related types.

Chapter 6, Sequences: Strings, Lists, and Tuples

Your first meaty chapter will expose you to all of Python's powerful sequence types: strings, lists, and tuples. We will show you all the built-in functions, methods, and special features, which apply to each type as well as all their operators.

Chapter 7, Mapping and Set Types

Dictionaries are Python's mapping or hashing type. Like other data types, dictionaries also have operators and applicable built-in functions and methods. We also cover Python's set types in this chapter, discussing their operators, built-in and factory functions, and built-in methods.

Chapter 8, Conditionals and Loops

Like many other high-level languages, Python supports loops such as for and while, as well as if statements (and related). Python also has a built-in function called range() which enables Python's for loop to behave more like a traditional counting loop rather than the "foreach" iterative type loop that it is. Also included is coverage of auxiliary statements such as break, continue, and pass, as well as a discussion of newer constructs like iterators, list comprehensions, and generator expressions.

Chapter 9, Files and Input/Output

In addition to standard file objects and input/output, this chapter introduces you to file system access, file execution, and persistent storage.

Chapter 10, Errors and Exceptions

One of Python's most powerful constructs is its exception handling ability. You can see a full treatment of it here, instruction on how to raise or throw exceptions, and more importantly, how to create your own exception classes.

Chapter 11, Functions and Functional Programming

Creating and calling functions are relatively straightforward, but Python has many other features that you will find useful, such as default arguments, named or keyword arguments, variable-length arguments, and some functional programming constructs. We also dip into variable scope and recursion briefly. We will also discuss some advanced features such as generators, decorators, inner functions, closures, and partial function application (a more generalized form of currying).

Chapter 12, Modules

One of Python's key strengths is its ability to be extended. This feature allows for "plug-and-play" access as well as promotes code reuse. Applications written as modules can be imported for use by other Python modules with a single line of code. Furthermore, multiple module software distribution can be simplified by using packages.

Chapter 13, Object-Oriented Programming

Python is a fully object-oriented programming language and was designed that way from the beginning. However, Python does not require you to program in such a manner—you may continue to develop structural/procedural code as you like, and can transition to OO programming anytime you are ready to take advantage of its benefits. Likewise, this chapter is here to guide you through the concepts as well as advanced topics, such as operator overloading, customization, and delegation. Also included is coverage of new features specific to new-style classes, including slots, properties, descriptors, and metaclasses.

Chapter 14, Execution Environment

The term "execution" can mean many different things, from callable and executable objects to running other programs (Python or otherwise). We discuss these topics in this chapter, as well as controlling execution via the operating system interface and different ways of terminating execution.

Part II: Advanced Topics

Chapter 15, Regular Expressions

Regular expressions are a powerful tool used for pattern matching, extracting, and search-and-replace functionality. Learn about them here.

Chapter 16, Network Programming

So many applications today need to be network-oriented. You have to start somewhere. In this chapter, you will learn to create clients and servers, using TCP/IP and UDP/IP, as well as get an introduction to SocketServer and Twisted.

Chapter 17, Internet Client Programming

In Chapter 16, we introduced network programming using sockets. Most Internet protocols in use today were developed using sockets. In this chapter, we explore some of these higher-level libraries, which are used to build clients of such Internet protocols. In particular, we focus on FTP, NNTP, SMTP, and POP3 clients.

Chapter 18, Multithreaded Programming

Multithreaded programming is a powerful way to improve the execution performance of many types of application. This chapter ends the drought of written documentation on how to do threads in Python by explaining the concepts and showing you how to correctly build a Python multithreaded application.

Chapter 19, GUI Programming

Based on the Tk graphical toolkit, Tkinter is Python's default GUI development module. We introduce Tkinter to you by showing you how to build simple sample GUI applications (say that ten times, real fast!). One of the best ways to learn is to copy, and by building on top of some of these applications, you will be on your way in no time. We conclude the chapter by presenting a more complex example, as well as take a brief look at Tix, Pmw, wxPython, and PyGTK.

Chapter 20, Web Programming

Web programming using Python takes three main forms: Web clients, Web servers, and the popular Common Gateway Interface applications that help Web servers deliver dynamically-generated Web pages. We will cover them all in this chapter: simple and advanced Web clients and CGI applications, as well as how to build your own Web server.

Chapter 21, Database Programming

What Python does for application programming carries to database programming as well. It is simplified, and you will find it fun! We first review basic database concepts, then introduce you to the Python database application programmer's interface (API). We then show you how you can connect to a relational database and perform queries and operations with Python. Finally, if you want hands-off using the Structured Query Language (SQL) and want to just work with objects without having to worry about the underlying database layer, we will introduce you to a few object-relational managers (ORMs), which simplify database programming to yet another level.

Chapter 22 Extending Python

We mentioned earlier how powerful it is to be able to reuse code and extend the language. In pure Python, these extensions are modules, but you can also develop lower-level code in C, C++, or Java, and interface those with Python in a seamless fashion. Writing your extensions in a lower-level programming language gives you added performance and some security (because the source code does not have to be revealed). This chapter walks you step-by-step through the extension building process.

Chapter 23, Miscellaneous

This new chapter consists of bonus material that we would like to develop into full, individual chapters in the next edition. Topics covered here include Web Services, Microsoft Office (Win32 COM Client) Programming, and Java/ Jython.