Reindex documents Generally available; Added in 2.3.0

POST /_reindex
Api key auth Basic auth Bearer auth

Copy documents from a source to a destination. You can copy all documents to the destination index or reindex a subset of the documents. The source can be any existing index, alias, or data stream. The destination must differ from the source. For example, you cannot reindex a data stream into itself.

IMPORTANT: Reindex requires _source to be enabled for all documents in the source. The destination should be configured as wanted before calling the reindex API. Reindex does not copy the settings from the source or its associated template. Mappings, shard counts, and replicas, for example, must be configured ahead of time.

If the Elasticsearch security features are enabled, you must have the following security privileges:

  • The read index privilege for the source data stream, index, or alias.
  • The write index privilege for the destination data stream, index, or index alias.
  • To automatically create a data stream or index with a reindex API request, you must have the auto_configure, create_index, or manage index privilege for the destination data stream, index, or alias.
  • If reindexing from a remote cluster, the source.remote.user must have the monitor cluster privilege and the read index privilege for the source data stream, index, or alias.

If reindexing from a remote cluster, you must explicitly allow the remote host in the reindex.remote.whitelist setting. Automatic data stream creation requires a matching index template with data stream enabled.

The dest element can be configured like the index API to control optimistic concurrency control. Omitting version_type or setting it to internal causes Elasticsearch to blindly dump documents into the destination, overwriting any that happen to have the same ID.

Setting version_type to external causes Elasticsearch to preserve the version from the source, create any documents that are missing, and update any documents that have an older version in the destination than they do in the source.

Setting op_type to create causes the reindex API to create only missing documents in the destination. All existing documents will cause a version conflict.

IMPORTANT: Because data streams are append-only, any reindex request to a destination data stream must have an op_type of create. A reindex can only add new documents to a destination data stream. It cannot update existing documents in a destination data stream.

By default, version conflicts abort the reindex process. To continue reindexing if there are conflicts, set the conflicts request body property to proceed. In this case, the response includes a count of the version conflicts that were encountered. Note that the handling of other error types is unaffected by the conflicts property. Additionally, if you opt to count version conflicts, the operation could attempt to reindex more documents from the source than max_docs until it has successfully indexed max_docs documents into the target or it has gone through every document in the source query.

Refer to the linked documentation for examples of how to reindex documents.

Required authorization

  • Index privileges: read,write
External documentation

Query parameters

  • refresh boolean

    If true, the request refreshes affected shards to make this operation visible to search.

  • requests_per_second number

    The throttle for this request in sub-requests per second. By default, there is no throttle.

  • scroll string

    The period of time that a consistent view of the index should be maintained for scrolled search.

    Values are -1 or 0.

  • slices number | string

    The number of slices this task should be divided into. It defaults to one slice, which means the task isn't sliced into subtasks.

    Reindex supports sliced scroll to parallelize the reindexing process. This parallelization can improve efficiency and provide a convenient way to break the request down into smaller parts.

    NOTE: Reindexing from remote clusters does not support manual or automatic slicing.

    If set to auto, Elasticsearch chooses the number of slices to use. This setting will use one slice per shard, up to a certain limit. If there are multiple sources, it will choose the number of slices based on the index or backing index with the smallest number of shards.

    Value is auto.

  • timeout string

    The period each indexing waits for automatic index creation, dynamic mapping updates, and waiting for active shards. By default, Elasticsearch waits for at least one minute before failing. The actual wait time could be longer, particularly when multiple waits occur.

    Values are -1 or 0.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default value is one, which means it waits for each primary shard to be active.

    Values are all or index-setting.

  • wait_for_completion boolean

    If true, the request blocks until the operation is complete.

  • require_alias boolean

    If true, the destination must be an index alias.

application/json

Body Required

  • conflicts string

    Values are abort or proceed.

  • dest object Required
    Hide dest attributes Show dest attributes object
    • index string Required
    • op_type string

      Values are index or create.

    • pipeline string

      The name of the pipeline to use.

    • routing string
    • version_type string

      Values are internal, external, external_gte, or force.

  • max_docs number

    The maximum number of documents to reindex. By default, all documents are reindexed. If it is a value less then or equal to scroll_size, a scroll will not be used to retrieve the results for the operation.

    If conflicts is set to proceed, the reindex operation could attempt to reindex more documents from the source than max_docs until it has successfully indexed max_docs documents into the target or it has gone through every document in the source query.

  • script object
    Hide script attributes Show script attributes object

    • source string | object

      One of:
      string-1 string SearchRequestBody object
    • id string
    • params object

      Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

      Hide params attribute Show params attribute object
      • * object Additional properties

    • lang string

      Any of:
      string-1 string string-2 string

      Values are painless, expression, mustache, or java.

    • options object
      Hide options attribute Show options attribute object
      • * string Additional properties
  • size number
  • source object Required
    Hide source attributes Show source attributes object
    • index string | array[string] Required
    • query object

      An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

      External documentation
    • remote object
      Hide remote attributes Show remote attributes object
      • connect_timeout string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • headers object

        An object containing the headers of the request.

        Hide headers attribute Show headers attribute object
        • * string Additional properties
      • host string Required
      • username string
      • password string
      • socket_timeout string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • size number

      The number of documents to index per batch. Use it when you are indexing from remote to ensure that the batches fit within the on-heap buffer, which defaults to a maximum size of 100 MB.

      Default value is 1000.

    • slice object
      Hide slice attributes Show slice attributes object
      • field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • id string Required
      • max number Required

    • sort string | object | array[string | object]

      One of:
      Field string SortOptions object array-2 array[string | object]

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      One of:
      Field string SortOptions object

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • _source string | array[string]
    • runtime_mappings object
      Hide runtime_mappings attribute Show runtime_mappings attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_index string
        • script object
          Hide script attributes Show script attributes object

          • source string | object

            One of:
            string-1 string SearchRequestBody object
          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties

          • lang string

            Any of:
            string-1 string string-2 string

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • batches number

      The number of scroll responses that were pulled back by the reindex.

    • created number

      The number of documents that were successfully created.

    • deleted number

      The number of documents that were successfully deleted.

    • failures array[object]

      If there were any unrecoverable errors during the process, it is an array of those failures. If this array is not empty, the request ended because of those failures. Reindex is implemented using batches and any failure causes the entire process to end but all failures in the current batch are collected into the array. You can use the conflicts option to prevent the reindex from ending on version conflicts.

      Hide failures attributes Show failures attributes object
      • cause object Required

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide cause attributes Show cause attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

          One of:
          string-1 string string-2 string | null
        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • id string Required
      • index string Required
      • status number Required
    • noops number

      The number of documents that were ignored because the script used for the reindex returned a noop value for ctx.op.

    • retries object
      Hide retries attributes Show retries attributes object
      • bulk number Required

        The number of bulk actions retried.

      • search number Required

        The number of search actions retried.

    • requests_per_second number

      The number of requests per second effectively run during the reindex.

    • slice_id number

    • task string | number

      One of:
      string-1 string number-2 number
    • throttled_millis number

      Time unit for milliseconds

    • throttled_until_millis number

      Time unit for milliseconds

    • timed_out boolean

      If any of the requests that ran during the reindex timed out, it is true.

    • took number

      Time unit for milliseconds

    • total number

      The number of documents that were successfully processed.

    • updated number

      The number of documents that were successfully updated. That is to say, a document with the same ID already existed before the reindex updated it.

    • version_conflicts number

      The number of version conflicts that occurred.
POST /_reindex 
POST _reindex { "source": { "index": ["my-index-000001", "my-index-000002"] }, "dest": { "index": "my-new-index-000002" } }
resp = client.reindex( source={ "index": [ "my-index-000001", "my-index-000002" ] }, dest={ "index": "my-new-index-000002" }, )
const response = await client.reindex({ source: { index: ["my-index-000001", "my-index-000002"], }, dest: { index: "my-new-index-000002", }, });
response = client.reindex( body: { "source": { "index": [ "my-index-000001", "my-index-000002" ] }, "dest": { "index": "my-new-index-000002" } } )
$resp = $client->reindex([ "body" => [ "source" => [ "index" => array( "my-index-000001", "my-index-000002", ), ], "dest" => [ "index" => "my-new-index-000002", ], ], ]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"source":{"index":["my-index-000001","my-index-000002"]},"dest":{"index":"my-new-index-000002"}}' "$ELASTICSEARCH_URL/_reindex"
client.reindex(r -> r .dest(d -> d .index("my-new-index-000002") ) .source(s -> s .index(List.of("my-index-000001","my-index-000002")) ) );
Request examples
Run `POST _reindex` to reindex from multiple sources. The `index` attribute in source can be a list, which enables you to copy from lots of sources in one request. This example copies documents from the `my-index-000001` and `my-index-000002` indices. 
{ "source": { "index": ["my-index-000001", "my-index-000002"] }, "dest": { "index": "my-new-index-000002" } }
You can use Painless to reindex daily indices to apply a new template to the existing documents. The script extracts the date from the index name and creates a new index with `-1` appended. For example, all data from `metricbeat-2016.05.31` will be reindexed into `metricbeat-2016.05.31-1`. 
{ "source": { "index": "metricbeat-*" }, "dest": { "index": "metricbeat" }, "script": { "lang": "painless", "source": "ctx._index = 'metricbeat-' + (ctx._index.substring('metricbeat-'.length(), ctx._index.length())) + '-1'" } }
Run `POST _reindex` to extract a random subset of the source for testing. You might need to adjust the `min_score` value depending on the relative amount of data extracted from source. 
{ "max_docs": 10, "source": { "index": "my-index-000001", "query": { "function_score" : { "random_score" : {}, "min_score" : 0.9 } } }, "dest": { "index": "my-new-index-000001" } }
Run `POST _reindex` to modify documents during reindexing. This example bumps the version of the source document. 
{ "source": { "index": "my-index-000001" }, "dest": { "index": "my-new-index-000001", "version_type": "external" }, "script": { "source": "if (ctx._source.foo == 'bar') {ctx._version++; ctx._source.remove('foo')}", "lang": "painless" } }
When using Elastic Cloud, you can run `POST _reindex` and authenticate against a remote cluster with an API key. 
{ "source": { "remote": { "host": "http://otherhost:9200", "username": "user", "password": "pass" }, "index": "my-index-000001", "query": { "match": { "test": "data" } } }, "dest": { "index": "my-new-index-000001" } }
Run `POST _reindex` to slice a reindex request manually. Provide a slice ID and total number of slices to each request. 
{ "source": { "index": "my-index-000001", "slice": { "id": 0, "max": 2 } }, "dest": { "index": "my-new-index-000001" } }
Run `POST _reindex?slices=5&refresh` to automatically parallelize using sliced scroll to slice on `_id`. The `slices` parameter specifies the number of slices to use. 
{ "source": { "index": "my-index-000001" }, "dest": { "index": "my-new-index-000001" } }
By default if reindex sees a document with routing then the routing is preserved unless it's changed by the script. You can set `routing` on the `dest` request to change this behavior. In this example, run `POST _reindex` to copy all documents from the `source` with the company name `cat` into the `dest` with routing set to `cat`. 
{ "source": { "index": "source", "query": { "match": { "company": "cat" } } }, "dest": { "index": "dest", "routing": "=cat" } }
Run `POST _reindex` and use the ingest pipelines feature. 
{ "source": { "index": "source" }, "dest": { "index": "dest", "pipeline": "some_ingest_pipeline" } }
Run `POST _reindex` and add a query to the `source` to limit the documents to reindex. For example, this request copies documents into `my-new-index-000001` only if they have a `user.id` of `kimchy`. 
{ "source": { "index": "my-index-000001", "query": { "term": { "user.id": "kimchy" } } }, "dest": { "index": "my-new-index-000001" } }
You can limit the number of processed documents by setting `max_docs`. For example, run `POST _reindex` to copy a single document from `my-index-000001` to `my-new-index-000001`. 
{ "max_docs": 1, "source": { "index": "my-index-000001" }, "dest": { "index": "my-new-index-000001" } }
You can use source filtering to reindex a subset of the fields in the original documents. For example, run `POST _reindex` the reindex only the `user.id` and `_doc` fields of each document. 
{ "source": { "index": "my-index-000001", "_source": ["user.id", "_doc"] }, "dest": { "index": "my-new-index-000001" } }
A reindex operation can build a copy of an index with renamed fields. If your index has documents with `text` and `flag` fields, you can change the latter field name to `tag` during the reindex. 
{ "source": { "index": "my-index-000001" }, "dest": { "index": "my-new-index-000001" }, "script": { "source": "ctx._source.tag = ctx._source.remove(\"flag\")" } }