Container Service for Kubernetes (ACK) allows you to enable tracing for the NGINX Ingress controller and import the trace data to Managed Service for OpenTelemetry. Managed Service for OpenTelemetry persists the trace data and aggregates and computes the trace data in real time to generate monitoring data, which includes trace details and real-time topology. You can troubleshoot and diagnose issues based on the monitoring data.
Prerequisites
The NGINX Ingress controller is installed. For more information, see Manage the NGINX Ingress controller.
Version description
The support for tracing varies based on the NGINX Ingress controller version. The following table describes the NGINX Ingress controller versions that support and do not support tracing.
NGINX Ingress controller version | OpenTelemetry | OpenTracing |
≥ 1.10.2-aliyun.1 | Supported | Not supported |
v1.9.3-aliyun.1 | Supported | Supported |
v1.8.2-aliyun.1 | Supported | Supported |
< v1.8.2-aliyun.1 | Not supported | Supported |
Procedure
Perform the following steps based on the version of the NGINX Ingress controller installed in your cluster.
OpenTelemetry
Step 1: Obtain an endpoint from Managed Service for OpenTelemetry
New version of the Managed Service for OpenTelemetry console
Log on to the Managed Service for OpenTelemetry console. In the left-side navigation pane, click Integration Center.
In the Open Source Frameworks section, click the OpenTelemetry card.
In the OpenTelemetry panel, select the region from which you want to import trace data.
Record the endpoint used to import data over gRPC.
NoteThe NGINX Ingress controller is deployed on Alibaba Cloud and resides in the same region as the Managed Service for OpenTelemetry agent in this example. Therefore, we recommend that you use a virtual private cloud (VPC) endpoint. If the NGINX Ingress controller and the Managed Service for OpenTelemetry agent are deployed in different regions, use a public endpoint.
Previous version of the Managed Service for OpenTelemetry console
Log on to the Managed Service for OpenTelemetry console.
In the left-side navigation pane, click Cluster Configurations. On the right side of the page that appears, click the Access point information tab.
In the upper part of the page, select the region from which you want to import trace data.
Select Show Token in the Cluster Information section and click OpenTelemetry in the Client section. Then, record the endpoint used to import data over gRPC.
NoteThe NGINX Ingress controller is deployed on Alibaba Cloud and resides in the same region as the Managed Service for OpenTelemetry agent in this example. Therefore, we recommend that you use a VPC endpoint. If the NGINX Ingress controller and the Managed Service for OpenTelemetry agent are deployed in different regions, use a public endpoint.
Step 2: Enable Managed Service for OpenTelemetry for the NGINX Ingress controller
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster you want to manage and click its name. In the left-side pane, choose .
In the upper part of the page, select kube-system from the Namespace drop-down list. Then, enter nginx-ingress-controller in the search box and click the search icon. Find nginx-ingress-controller and click Edit in the Actions column.
In the upper part of the Edit page, select the nginx-ingress-controller container. On the Environments tab, click Add to add the following environment variable.
Type
Variable Key
Value/ValueFrom
Description
Custom
OTEL_EXPORTER_OTLP_HEADERS
authentication=<Authentication token>Specify the authentication token you obtained in Step 1: Obtain an endpoint from Managed Service for OpenTelemetry. Example:
authentication=bfXXXXXXXe@7bXXXXXXX1_bXXXXXe@XXXXXXX1.
After you add the environment variable, click Update on the right side of the Edit page. In the message that appears, click Confirm.
In the left-side navigation pane, choose .
In the upper part of the ConfigMap page, select kube-system from the Namespace drop-down list. Then, enter nginx-configuration in the Name search box and click the search icon. Find nginx-configuration and click Edit in the Actions column.
In the Edit panel, click Add to add the following configurations and click OK.
Name
Description
Valid value
Example
enable-opentelemetry
Specifies whether to enable Managed Service for OpenTelemetry.
true: enables Managed Service for OpenTelemetry.false: does not enable Managed Service for OpenTelemetry.
truemain-snippet
-
env OTEL_EXPORTER_OTLP_HEADERS;env OTEL_EXPORTER_OTLP_HEADERS;otel-service-name
Enter a service name.
Configure a custom value.
nginx-ingress
otlp-collector-host
The domain name used to import data over gRPC.
Delete
http://and the port number from the VPC endpoint you obtained in Step 1: Obtain an endpoint from Managed Service for OpenTelemetry. Example: tracing-analysis-XX-XX-XXXXX.aliyuncs.com.tracing-analysis-XX-XX-XXXXX.aliyuncs.comotlp-collector-port
The port used to import data over gRPC.
Enter a value based on your business requirements.
8090opentelemetry-trust-incoming-span
Specifies whether to trust the call traces of other services or systems.
true: trusts the call traces of other services or systems.
false: does not trust the call traces of other services or systems.
trueopentelemetry-operation-name
The span format of call traces.
HTTP $request_method $service_name $uriHTTP $request_method $service_name $uriotel-sampler
The sampling rule.
For more information, see opentelemetry.
TraceIdRatioBased
otel-sampler-ratio
The sampling rate.
Valid values: 0 to 1. The value can be accurate to two decimal places. A value of 0 indicates that no data is collected. A value of 1 indicates that all data is collected. For more information, see opentelemetry.
0.1
otel-sampler-parent-based
Specifies whether to inherit the upstream sampled flag.
false: does not inherit the upstream sampled flag. This is the default value. In this case, otel-sampler and otel-sampler-ratio take effect.
true: inherits the upstream sampled flag. In this case, otel-sampler and otel-sampler-ratio do not take effect and the upstream sampled flag is reused.
For more information, see opentelemetry.
false
Step 3: View the trace data in the Managed Service for OpenTelemetry console
Log on to the Managed Service for OpenTelemetry console
In the left-side navigation pane, click Applications.
In the upper part of the Applications page, select the region from which you want to import trace data. Then, click nginx-ingress.
On the application details page, you can view the trace data.
On the Application Overview tab, you can view the number of requests and the number of errors.
On the Trace Analysis tab, you can view the trace list and average duration.
On the Trace Analysis tab, you can click a trace ID to view the trace details.
OpenTracing
Step 1: Obtain an endpoint from Managed Service for OpenTelemetry
New version of the Managed Service for OpenTelemetry console
Log on to the Managed Service for OpenTelemetry console. In the left-side navigation pane, click Integration Center.
In the Open Source Frameworks section, click the Zipkin card.
NoteObtain an endpoint of the client used to collect data. In this example, a Zipkin client is used.
In the Zipkin panel, select the region from which you want to import trace data.
Record the endpoint.
NoteThe NGINX Ingress controller is deployed on Alibaba Cloud and resides in the same region as the Managed Service for OpenTelemetry agent in this example. Therefore, we recommend that you use a VPC endpoint. If the NGINX Ingress controller and the Managed Service for OpenTelemetry agent are deployed in different regions, use a public endpoint.
Previous version of the Managed Service for OpenTelemetry console
Log on to the Managed Service for OpenTelemetry console.
In the left-side navigation pane, click Cluster Configurations. On the page that appears, click the Access point information tab.
In the upper part of the page, select the region from which you want to import trace data.
Select Show Token in the Cluster Information section and click Zipkin in the Client section. Then, record the endpoint.
NoteThe NGINX Ingress controller is deployed on Alibaba Cloud and resides in the same region as the Managed Service for OpenTelemetry agent in this example. Therefore, we recommend that you use a VPC endpoint. If the NGINX Ingress controller and the Managed Service for OpenTelemetry agent are deployed in different regions, use a public endpoint.
Step 2: Enable Managed Service for OpenTelemetry for the NGINX Ingress controller
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the one you want to change. In the left-side navigation pane, choose .
In the upper part of the ConfigMap page, select kube-system from the Namespace drop-down list. Then, enter nginx-configuration in the Name search box and click the search icon. Find nginx-configuration and click Edit in the Actions column.
In the Edit panel, click Add to add the following configurations and click OK.
Name
Description
Valid value
Example
enable-opentracing
Specifies whether to enable Tracing Analysis.
true: enables Tracing Analysis.
false: does not enable Tracing Analysis.
true
zipkin-service-name
The service name.
Configure a custom value.
nginx-ingress
zipkin-collector-host
The domain name used to import data over gRPC.
Modifiy the endpoint information obtained in Step 1: Obtain an endpoint from Managed Service for OpenTelemetry. Remove
http://and add a question mark at the end. Example:The original endpoint information is
http://tracing-analysis-dc-hz-internal.aliyuncs.com/adapt_******_******/api/v1/spans.The modified value is
tracing-analysis-dc-hz-internal.aliyuncs.com/adapt_******_******/api/v1/spans?.
tracing-analysis-dc-hz-internal.aliyuncs.com/adapt_******_******/api/v1/spans?opentracing-trust-incoming-span
Specifies whether to trust the call traces of other services or systems.
true: trusts the call traces of other services or systems.
false: does not trust the call traces of other services or systems.
true
zipkin-sample-rate
The sampling rate.
Valid values: 0 to 1. The value can be accurate to two decimal places. A value of 0 indicates that no data is collected. A value of 1 indicates that all data is collected.
0.1
Step 3: View the trace data in the Managed Service for OpenTelemetry console
Log on to the Managed Service for OpenTelemetry console.
In the left-side navigation pane, click Applications.
In the upper part of the Applications page, select the region from which you want to import trace data. Then, click nginx.
In the left-side navigation pane of the details page, click Interface Calls. On the right side of the page, you can view the trace data.
On the Overview tab, you can view the trace topology.
Click the Traces tab to view the top 100 time-consuming traces of the application. For more information, see Interface calls.
On the Traces tab, you can click a trace ID to view the trace details.
(Optional) Change the protocol used by Managed Service for OpenTelemetry to pass trace data
If you use Managed Service for OpenTelemetry to enable tracing for the NGINX Ingress controller, Managed Service for OpenTelemetry passes trace data in the W3C trace context specification to the downstream service. If the frontend and backend applications use other protocols, such as Jaeger or Zipkin, you must change the protocol used by Managed Service for OpenTelemetry to pass trace data. This way, the traces of the frontend application, NGINX Ingress, and backend application can be streamlined. The following example describes how to change the protocol used by Managed Service for OpenTelemetry to pass trace data:
Add the OTEL_PROPAGATORS environment variable in the fourth sub-step in Step 2. Then, save the changes and redeploy nginx-ingress-controller.
Variable key
Value
Description
OTEL_PROPAGATORS
tracecontext,baggage,b3,jaeger
The protocol used to pass trace data. For more information, see Specify the format to pass trace data.
Modify the main-snippet configurations in the seventh sub-step in Step 2 to make the OTEL_PROPAGATORS environment variable take effect.
Name
Value
Description
main-snippet
env OTEL_EXPORTER_OTLP_HEADERS; env OTEL_PROPAGATORS;
Load the OTEL_PROPAGATORS environment variable.
References
For more information about Managed Service for OpenTelemetry, see What is Managed Service for OpenTelemetry?
For more information about ACK, see What is Container Service for Kubernetes?
To use different clients to collect trace data, such as Zipkin, Jaeger, and SkyWalking, you must configure different parameters in the nginx-configuration ConfigMap. For more information about how to import data to Managed Service for OpenTelemetry, see Preparations.