Merge branch 'master' of github.com:mongodb/mongo-python-driver

Author: rozza

doc/api/pymongo/collection.rst

@@ -22,6 +22,8 @@
  .. autoattribute:: database
  .. autoattribute:: slave_okay
  .. autoattribute:: read_preference
+ .. autoattribute:: tag_sets
+ .. autoattribute:: secondary_acceptable_latency_ms
  .. autoattribute:: safe
  .. autoattribute:: uuid_subtype
  .. automethod:: get_lasterror_options

doc/api/pymongo/connection.rst

@@ -17,11 +17,15 @@
 
  .. autoattribute:: host
  .. autoattribute:: port
+ .. autoattribute:: is_primary
+ .. autoattribute:: is_mongos
  .. autoattribute:: nodes
  .. autoattribute:: max_pool_size
  .. autoattribute:: document_class
  .. autoattribute:: tz_aware
  .. autoattribute:: read_preference
+ .. autoattribute:: tag_sets
+ .. autoattribute:: secondary_acceptable_latency_ms
  .. autoattribute:: slave_okay
  .. autoattribute:: safe
  .. autoattribute:: is_locked

doc/api/pymongo/database.rst

@@ -20,6 +20,8 @@
 
  .. autoattribute:: slave_okay
  .. autoattribute:: read_preference
+ .. autoattribute:: tag_sets
+ .. autoattribute:: secondary_acceptable_latency_ms
  .. autoattribute:: safe
  .. automethod:: get_lasterror_options
  .. automethod:: set_lasterror_options

doc/api/pymongo/index.rst

@@ -13,7 +13,7 @@
 
  Alias for :class:`pymongo.replica_set_connection.ReplicaSetConnection`.
 
- .. autoclass:: pymongo.ReadPreference
+ .. autoclass:: pymongo.read_preferences.ReadPreference
  .. autofunction:: has_c
 
 Sub-modules:

doc/api/pymongo/replica_set_connection.rst

@@ -21,6 +21,8 @@
  .. autoattribute:: primary
  .. autoattribute:: secondaries
  .. autoattribute:: read_preference
+ .. autoattribute:: tag_sets
+ .. autoattribute:: secondary_acceptable_latency_ms
  .. autoattribute:: max_pool_size
  .. autoattribute:: document_class
  .. autoattribute:: tz_aware

doc/changelog.rst

@@ -100,7 +100,7 @@ Important New Features:
  automatic failover handling and periodically checks the state of the
  replica set to handle issues like primary stepdown or secondaries
  being removed for backup operations. Read preferences are defined through
- :class:`~pymongo.ReadPreference`.
+ :class:`~pymongo.read_preferences.ReadPreference`.
 - PyMongo supports the new BSON binary subtype 4 for UUIDs. The default
  subtype to use can be set through
  :attr:`~pymongo.collection.Collection.uuid_subtype`

doc/examples/gevent.rst

@@ -53,8 +53,9 @@ Additionally, it will use a background greenlet instead of a background thread
 to monitor the state of the replica set.
 
 Using :meth:`~pymongo.replica_set_connection.ReplicaSetConnection.start_request()`
-with :class:`~pymongo.ReadPreference` PRIMARY ensures that the current greenlet
-uses the same socket for all operations until a call to :meth:`end_request()`.
+with :class:`~pymongo.read_preferences.ReadPreference` PRIMARY ensures that the
+current greenlet uses the same socket for all operations until a call to
+:meth:`end_request()`.
 
 You must `install Gevent <http://gevent.org/>`_ to use
 :class:`~pymongo.replica_set_connection.ReplicaSetConnection`

doc/examples/replica_set.rst

@@ -155,31 +155,113 @@ the operation will succeed::
 ReplicaSetConnection
 --------------------
 
-In Pymongo-2.1 a new ReplicaSetConnection class was added that provides
-some new features not supported in the original Connection class. The most
-important of these is the ability to distribute queries to the secondary
-members of a replica set. To connect using ReplicaSetConnection just
-provide a host:port pair and the name of the replica set::
+Using a :class:`~pymongo.replica_set_connection.ReplicaSetConnection` instead
+of a simple :class:`~pymongo.connection.Connection` offers two key features:
+secondary reads and replica set health monitoring. To connect using
+`ReplicaSetConnection` just provide a host:port pair and the name of the
+replica set::
 
  >>> from pymongo import ReplicaSetConnection
  >>> ReplicaSetConnection("morton.local:27017", replicaSet='foo')
  ReplicaSetConnection([u'morton.local:27019', u'morton.local:27017', u'morton.local:27018'])
 
+Secondary Reads
+'''''''''''''''
+
 By default an instance of ReplicaSetConnection will only send queries to
-the primary member of the replica set. To use secondary members for queries
-we have to change the read preference::
+the primary member of the replica set. To use secondaries for queries
+we have to change the :class:`~pymongo.read_preference.ReadPreference`::
 
  >>> db = ReplicaSetConnection("morton.local:27017", replicaSet='foo').test
- >>> from pymongo import ReadPreference
- >>> db.read_preference = ReadPreference.SECONDARY
+ >>> from pymongo.read_preference import ReadPreference
+ >>> db.read_preference = ReadPreference.SECONDARY_PREFERRED
 
 Now all queries will be sent to the secondary members of the set. If there are
 no secondary members the primary will be used as a fallback. If you have
 queries you would prefer to never send to the primary you can specify that
-using the SECONDARY_ONLY read preference::
+using the ``SECONDARY`` read preference::
 
- >>> db.read_preference = ReadPreference.SECONDARY_ONLY
+ >>> db.read_preference = ReadPreference.SECONDARY
 
 Read preference can be set on a connection, database, collection, or on a
-per-query basis.
-
+per-query basis, e.g.::
+
+ >>> db.collection.find_one(read_preference=ReadPreference.PRIMARY)
+
+Reads are configured using three options: **read_preference**, **tag_sets**,
+and **secondary_acceptable_latency_ms**.
+
+**read_preference**:
+
+- ``PRIMARY``: Read from the primary. This is the default, and provides the
+ strongest consistency. If no primary is available, raise
+ :class:`~pymongo.errors.AutoReconnect`.
+
+- ``PRIMARY_PREFERRED``: Read from the primary if available, or if there is
+ none, read from a secondary matching your choice of ``tag_sets`` and
+ ``secondary_acceptable_latency_ms``.
+
+- ``SECONDARY``: Read from a secondary matching your choice of ``tag_sets`` and
+ ``secondary_acceptable_latency_ms``. If no matching secondary is available,
+ raise :class:`~pymongo.errors.AutoReconnect`.
+
+- ``SECONDARY_PREFERRED``: Read from a secondary matching your choice of
+ ``tag_sets`` and ``secondary_acceptable_latency_ms`` if available, otherwise
+ from primary (regardless of the primary's tags and latency).
+
+- ``NEAREST``: Read from any member matching your choice of ``tag_sets`` and
+ ``secondary_acceptable_latency_ms``.
+
+**tag_sets**:
+
+Replica-set members can be `tagged
+<http://www.mongodb.org/display/DOCS/Data+Center+Awareness>`_ according to any
+criteria you choose. By default, ReplicaSetConnection ignores tags when
+choosing a member to read from, but it can be configured with the ``tag_sets``
+parameter. ``tag_sets`` must be a list of dictionaries, each dict providing tag
+values that the replica set member must match. ReplicaSetConnection tries each
+set of tags in turn until it finds a set of tags with at least one matching
+member. For example, to prefer reads from the New York data center, but fall
+back to the San Francisco data center, tag your replica set members according
+to their location and create a ReplicaSetConnection like so:
+
+ >>> rsc = ReplicaSetConnection(
+ ... "morton.local:27017",
+ ... replicaSet='foo'
+ ... read_preference=ReadPreference.SECONDARY,
+ ... tag_sets=[{'dc': 'ny'}, {'dc': 'sf'}]
+ ... )
+
+ReplicaSetConnection tries to find secondaries in New York, then San Francisco,
+and raises :class:`~pymongo.errors.AutoReconnect` if none are available. As an
+additional fallback, specify a final, empty tag set, ``{}``, which means "read
+from any member that matches the mode, ignoring tags."
+
+**secondary_acceptable_latency_ms**:
+
+If multiple members match the mode and tag sets, ReplicaSetConnection reads
+from among the nearest members, chosen according to ping time. By default,
+only members whose ping times are within 15 milliseconds of the nearest
+are used for queries. You can choose to distribute reads among members with
+higher latencies by setting ``secondary_acceptable_latency_ms`` to a larger
+number. In that case, ReplicaSetConnection distributes reads among matching
+members within ``secondary_acceptable_latency_ms`` of the closest member's
+ping time.
+
+Health Monitoring
+'''''''''''''''''
+
+When ReplicaSetConnection is initialized it launches a background task to
+monitor the replica set for changes in:
+
+* Health: detect when a member goes down or comes up, or if a different member
+ becomes primary
+* Configuration: detect changes in tags
+* Latency: track a moving average of each member's ping time
+
+Replica-set monitoring ensures queries are continually routed to the proper
+members as the state of the replica set changes.
+
+It is critical to call
+:meth:`~pymongo.replica_set_connection.ReplicaSetConnection.close` to terminate
+the monitoring task before your process exits.

gridfs/__init__.py

@@ -54,7 +54,8 @@ def __init__(self, database, collection="fs"):
  self.__collection = database[collection]
  self.__files = self.__collection.files
  self.__chunks = self.__collection.chunks
- if not database.slave_okay and not database.read_preference:
+ connection = database.connection
+ if not hasattr(connection, 'is_primary') or connection.is_primary:
  self.__chunks.ensure_index([("files_id", ASCENDING),
  ("n", ASCENDING)],
  unique=True)
@@ -158,7 +159,7 @@ def get_version(self, filename=None, version=-1, **kwargs):
 
  :Parameters:
  - `filename`: ``"filename"`` of the file to get, or `None`
- - `version` (optional): version of the file to get (defualts
+ - `version` (optional): version of the file to get (defaults
  to -1, the most recent version uploaded)
  - `**kwargs` (optional): find files by custom metadata.
 
@@ -168,8 +169,8 @@ def get_version(self, filename=None, version=-1, **kwargs):
  Accept keyword arguments to find files by custom metadata.
  .. versionadded:: 1.9
  """
- database = self.__database
- if not database.slave_okay and not database.read_preference:
+ connection = self.__database.connection
+ if not hasattr(connection, 'is_primary') or connection.is_primary:
  self.__files.ensure_index([("filename", ASCENDING),
  ("uploadDate", DESCENDING)])
 

pymongo/__init__.py

@@ -48,49 +48,6 @@
 ALL = 2
 """Profile all operations."""
 
-class ReadPreference:
- """An enum that defines the read preferences supported by PyMongo.
-
- +----------------------+--------------------------------------------------+
- | Connection type | Read Preference |
- +======================+================+================+================+
- | |`PRIMARY` |`SECONDARY` |`SECONDARY_ONLY`|
- +----------------------+----------------+----------------+----------------+
- |Connection to a single|Queries are |Queries are |Same as |
- |host. |allowed if the |allowed if the |`SECONDARY` |
- | |connection is to|connection is to| |
- | |the replica set |the replica set | |
- | |primary. |primary or a | |
- | | |secondary. | |
- +----------------------+----------------+----------------+----------------+
- |Connection to a |Queries are sent|Queries are |Same as |
- |mongos. |to the primary |distributed |`SECONDARY` |
- | |of a shard. |among shard | |
- | | |secondaries. | |
- | | |Queries are sent| |
- | | |to the primary | |
- | | |if no | |
- | | |secondaries are | |
- | | |available. | |
- | | | | |
- +----------------------+----------------+----------------+----------------+
- |ReplicaSetConnection |Queries are sent|Queries are |Queries are |
- | |to the primary |distributed |never sent to |
- | |of the replica |among replica |the replica set |
- | |set. |set secondaries.|primary. An |
- | | |Queries are sent|exception is |
- | | |to the primary |raised if no |
- | | |if no |secondary is |
- | | |secondaries are |available. |
- | | |available. | |
- | | | | |
- +----------------------+----------------+----------------+----------------+
- """
-
- PRIMARY = 0
- SECONDARY = 1
- SECONDARY_ONLY = 2
-
 version_tuple = (2, 2, 1, '+')
 
 def get_version_string():
@@ -103,6 +60,7 @@ def get_version_string():
 
 from pymongo.connection import Connection
 from pymongo.replica_set_connection import ReplicaSetConnection
+from pymongo.read_preferences import ReadPreference
 
 def has_c():
  """Is the C extension installed?