Move bulk helpers' docs outside of streaming_bulk's docstring

Author: honzakral

docs/helpers.rst

@@ -7,12 +7,93 @@ Collection of simple helper functions that abstract some specifics or the raw
 API.
 
 
+Bulk helpers
+------------
+
+There are several helpers for the ``bulk`` API since it's requirement for
+specific formatting and other considerations can make it cumbersome if used directly.
+
+All bulk helpers accept an instance of ``Elasticsearch`` class and an iterable
+``actions`` (any iterable, can also be a generator, which is ideal in most
+cases since it will allow you to index large datasets without the need of
+loading them into memory).
+
+The items in the ``action`` iterable should be the documents we wish to index
+in several formats. The most common one is the same as returned by
+:meth:`~elasticsearch.Elasticsearch.search`, for example:
+
+.. code:: python
+
+ {
+ '_index': 'index-name',
+ '_type': 'document',
+ '_id': 42,
+ '_parent': 5,
+ '_ttl': '1d',
+ '_source': {
+ "title": "Hello World!",
+ "body": "..."
+ }
+ }
+
+Alternatively, if `_source` is not present, it will pop all metadata fields
+from the doc and use the rest as the document data:
+
+.. code:: python
+
+ {
+ "_id": 42,
+ "_parent": 5,
+ "title": "Hello World!",
+ "body": "..."
+ }
+
+The :meth:`~elasticsearch.Elasticsearch.bulk` api accepts ``index``, ``create``,
+``delete``, and ``update`` actions. Use the ``_op_type`` field to specify an
+action (``_op_type`` defaults to ``index``):
+
+.. code:: python
+
+ {
+ '_op_type': 'delete',
+ '_index': 'index-name',
+ '_type': 'document',
+ '_id': 42,
+ }
+ {
+ '_op_type': 'update',
+ '_index': 'index-name',
+ '_type': 'document',
+ '_id': 42,
+ 'doc': {'question': 'The life, universe and everything.'}
+ }
+
+
+.. note::
+
+ When reading raw json strings from a file, you can also pass them in
+ directly (without decoding to dicts first). In that case, however, you lose
+ the ability to specify anything (index, type, even id) on a per-record
+ basis, all documents will just be sent to elasticsearch to be indexed
+ as-is.
+
+
 .. py:module:: elasticsearch.helpers
 
 .. autofunction:: streaming_bulk
 
+.. autofunction:: parallel_bulk
+
 .. autofunction:: bulk
 
+
+Scan
+----
+
 .. autofunction:: scan
 
+
+Reindex
+-------
+
 .. autofunction:: reindex

elasticsearch/helpers/__init__.py

@@ -142,45 +142,6 @@ def streaming_bulk(client, actions, chunk_size=500, max_chunk_bytes=100 * 1014 *
  bulk that returns summary information about the bulk operation once the
  entire input is consumed and sent.
 
- This function expects the action to be in the format as returned by
- :meth:`~elasticsearch.Elasticsearch.search`, for example::
-
- {
- '_index': 'index-name',
- '_type': 'document',
- '_id': 42,
- '_parent': 5,
- '_ttl': '1d',
- '_source': {
- ...
- }
- }
-
- Alternatively, if `_source` is not present, it will pop all metadata fields
- from the doc and use the rest as the document data.
-
- When reading raw json strings from a file, you can also pass them in. In
- that case, however, you lose the ability to specify anything (index, type,
- even id) on a per-record basis, all documents will just be sent to
- elasticsearch to be indexed as-is.
-
- The :meth:`~elasticsearch.Elasticsearch.bulk` api accepts `index`, `create`,
- `delete`, and `update` actions. Use the `_op_type` field to specify an
- action (`_op_type` defaults to `index`)::
-
- {
- '_op_type': 'delete',
- '_index': 'index-name',
- '_type': 'document',
- '_id': 42,
- }
- {
- '_op_type': 'update',
- '_index': 'index-name',
- '_type': 'document',
- '_id': 42,
- 'doc': {'question': 'The life, universe and everything.'}
- }
 
  :arg client: instance of :class:`~elasticsearch.Elasticsearch` to use
  :arg actions: iterable containing the actions to be executed
@@ -208,8 +169,8 @@ def bulk(client, actions, stats_only=False, **kwargs):
  information - number of successfully executed actions and either list of
  errors or number of errors if `stats_only` is set to `True`.
 
- See :func:`~elasticsearch.helpers.streaming_bulk` for more information
- and accepted formats.
+ See :func:`~elasticsearch.helpers.streaming_bulk` for more accepted
+ parameters
 
  :arg client: instance of :class:`~elasticsearch.Elasticsearch` to use
  :arg actions: iterator containing the actions
@@ -240,7 +201,7 @@ def parallel_bulk(client, actions, thread_count=4, chunk_size=500,
  max_chunk_bytes=100 * 1014 * 1024,
  expand_action_callback=expand_action, **kwargs):
  """
- Parallel version of the bulk helper.
+ Parallel version of the bulk helper run in multiple threads at once.
 
  :arg client: instance of :class:`~elasticsearch.Elasticsearch` to use
  :arg actions: iterator containing the actions