Create an AlibabaCloud AI Search inference endpoint Generally available; Added in 8.16.0

PUT /_inference/{task_type}/{alibabacloud_inference_id}

Create an inference endpoint to perform an inference task with the alibabacloud-ai-search service.

Required authorization

  • Cluster privileges: manage_inference

Path parameters

  • task_type string

    The type of the inference task that the model will perform.

    Values are completion, rerank, space_embedding, or text_embedding.

  • alibabacloud_inference_id string Required

    The unique identifier of the inference endpoint.

Query parameters

application/json

Body

  • chunking_settings object

    The chunking configuration object.

    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      Default value is 250.

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      Default value is 100.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      Default value is 1.

    • strategy string

      The chunking strategy: sentence or word.

      Default value is sentence.

  • service string Required

    The type of service supported for the specified task type. In this case, alibabacloud-ai-search.

    Value is alibabacloud-ai-search.

  • service_settings object Required

    Settings used to install the inference model. These settings are specific to the alibabacloud-ai-search service.

    Hide service_settings attributes Show service_settings attributes object
    • api_key string Required

      A valid API key for the AlibabaCloud AI Search API.

    • host string Required

      The name of the host address used for the inference task. You can find the host address in the API keys section of the documentation.

      External documentation
    • rate_limit object

      This setting helps to minimize the number of rate limit errors returned from AlibabaCloud AI Search. By default, the alibabacloud-ai-search service sets the number of requests allowed per minute to 1000.

      Hide rate_limit attribute Show rate_limit attribute object
      • requests_per_minute number

        The number of requests allowed per minute.

    • service_id string Required

      The name of the model service to use for the inference task. The following service IDs are available for the completion task:

      • ops-qwen-turbo
      • qwen-turbo
      • qwen-plus
      • qwen-max ÷ qwen-max-longcontext

      The following service ID is available for the rerank task:

      • ops-bge-reranker-larger

      The following service ID is available for the sparse_embedding task:

      • ops-text-sparse-embedding-001

      The following service IDs are available for the text_embedding task:

      ops-text-embedding-001 ops-text-embedding-zh-001 ops-text-embedding-en-001 ops-text-embedding-002

    • workspace string Required

      The name of the workspace used for the inference task.

  • task_settings object

    Settings to configure the inference task. These settings are specific to the task type you specified.

    Hide task_settings attributes Show task_settings attributes object
    • input_type string

      For a sparse_embedding or text_embedding task, specify the type of input passed to the model. Valid values are:

      • ingest for storing document embeddings in a vector database.
      • search for storing embeddings of search queries run against a vector database to find relevant documents.
    • return_token boolean

      For a sparse_embedding task, it affects whether the token name will be returned in the response. It defaults to false, which means only the token ID will be returned in the response.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • chunking_settings object

      Chunking configuration object

      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

        Default value is 250.

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

        Default value is 100.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

        Default value is 1.

      • strategy string

        The chunking strategy: sentence or word.

        Default value is sentence.

    • service string Required

      The service type

    • service_settings object Required

      Settings specific to the service

    • task_settings object

      Task settings specific to the service and task type

    • inference_id string Required

      The inference Id

    • task_type string Required

      The task type

      Values are text_embedding, rerank, completion, or sparse_embedding.

PUT /_inference/{task_type}/{alibabacloud_inference_id}
PUT _inference/completion/alibabacloud_ai_search_completion { "service": "alibabacloud-ai-search", "service_settings": { "host" : "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com", "api_key": "AlibabaCloud-API-Key", "service_id": "ops-qwen-turbo", "workspace" : "default" } }
resp = client.inference.put( task_type="completion", inference_id="alibabacloud_ai_search_completion", inference_config={ "service": "alibabacloud-ai-search", "service_settings": { "host": "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com", "api_key": "AlibabaCloud-API-Key", "service_id": "ops-qwen-turbo", "workspace": "default" } }, )
const response = await client.inference.put({ task_type: "completion", inference_id: "alibabacloud_ai_search_completion", inference_config: { service: "alibabacloud-ai-search", service_settings: { host: "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com", api_key: "AlibabaCloud-API-Key", service_id: "ops-qwen-turbo", workspace: "default", }, }, });
response = client.inference.put( task_type: "completion", inference_id: "alibabacloud_ai_search_completion", body: { "service": "alibabacloud-ai-search", "service_settings": { "host": "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com", "api_key": "AlibabaCloud-API-Key", "service_id": "ops-qwen-turbo", "workspace": "default" } } )
$resp = $client->inference()->put([ "task_type" => "completion", "inference_id" => "alibabacloud_ai_search_completion", "body" => [ "service" => "alibabacloud-ai-search", "service_settings" => [ "host" => "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com", "api_key" => "AlibabaCloud-API-Key", "service_id" => "ops-qwen-turbo", "workspace" => "default", ], ], ]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"service":"alibabacloud-ai-search","service_settings":{"host":"default-j01.platform-cn-shanghai.opensearch.aliyuncs.com","api_key":"AlibabaCloud-API-Key","service_id":"ops-qwen-turbo","workspace":"default"}}' "$ELASTICSEARCH_URL/_inference/completion/alibabacloud_ai_search_completion"
client.inference().put(p -> p .inferenceId("alibabacloud_ai_search_completion") .taskType(TaskType.Completion) .inferenceConfig(i -> i .service("alibabacloud-ai-search") .serviceSettings(JsonData.fromJson("{\"host\":\"default-j01.platform-cn-shanghai.opensearch.aliyuncs.com\",\"api_key\":\"AlibabaCloud-API-Key\",\"service_id\":\"ops-qwen-turbo\",\"workspace\":\"default\"}")) ) ); 
Request examples
Run `PUT _inference/completion/alibabacloud_ai_search_completion` to create an inference endpoint that performs a completion task.
{ "service": "alibabacloud-ai-search", "service_settings": { "host" : "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com", "api_key": "AlibabaCloud-API-Key", "service_id": "ops-qwen-turbo", "workspace" : "default" } }
Run `PUT _inference/rerank/alibabacloud_ai_search_rerank` to create an inference endpoint that performs a rerank task.
{ "service": "alibabacloud-ai-search", "service_settings": { "api_key": "AlibabaCloud-API-Key", "service_id": "ops-bge-reranker-larger", "host": "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com", "workspace": "default" } }
Run `PUT _inference/sparse_embedding/alibabacloud_ai_search_sparse` to create an inference endpoint that performs perform a sparse embedding task.
{ "service": "alibabacloud-ai-search", "service_settings": { "api_key": "AlibabaCloud-API-Key", "service_id": "ops-text-sparse-embedding-001", "host": "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com", "workspace": "default" } }
Run `PUT _inference/text_embedding/alibabacloud_ai_search_embeddings` to create an inference endpoint that performs a text embedding task.
{ "service": "alibabacloud-ai-search", "service_settings": { "api_key": "AlibabaCloud-API-Key", "service_id": "ops-text-embedding-001", "host": "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com", "workspace": "default" } }