Skip to content

Conversation

@SylvainJuge
Copy link
Member

@SylvainJuge SylvainJuge commented Mar 24, 2023

fixes #94689

The APM agent version 1.33.0 fails to start on JDK 20, which prevents the APM integration to work as expected.
As a consequence, the tracing does not work.

When setting ELASTIC_APM_LOG_LEVEL=debug and ELASTIC_APM_LOG_FILE=/tmp/log.txt, the agent log shows that there is an issue with accessing Unsafe (sorry I don't have the exact stack trace).

There was a few changes in APM agent regarding the security manager (SM) in recent versions, and updating the agent seems to make it work as expected.

However, there is one known caveat so far (elastic/apm-agent-java#3074), keeping the agent with debug log level with ELASTIC_APM_LOG_LEVEL=debug makes it trigger another security exception when trying to establish connection with apm-server because the agent prints few details if a proxy is used or not (which is forbidden by default by the SM and isn't yet wrapped in a privileged call.

@elasticsearchmachine elasticsearchmachine added v8.8.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Mar 24, 2023
@SylvainJuge SylvainJuge requested a review from pugnascotia March 24, 2023 09:57
@SylvainJuge SylvainJuge marked this pull request as ready for review March 24, 2023 10:18
@elasticsearchmachine elasticsearchmachine added the needs:triage Requires assignment of a team area label label Mar 24, 2023
Copy link
Contributor

@pugnascotia pugnascotia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good. Have you tried configuring ES with tracing enabled?

@pugnascotia pugnascotia added >upgrade :Core/Infra/Core Core issues without another label labels Mar 24, 2023
@pugnascotia pugnascotia self-assigned this Mar 24, 2023
@elasticsearchmachine elasticsearchmachine added Team:Core/Infra Meta label for core/infra team and removed needs:triage Requires assignment of a team area label labels Mar 24, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@SylvainJuge
Copy link
Member Author

Changes look good. Have you tried configuring ES with tracing enabled?

Yes, just starting ES locally, so only a single node, if you have the chance to double check with a 2+ node cluster that would probably be safer.

@mark-vieira
Copy link
Contributor

Thanks @SylvainJuge. I've created #94730 to also put a process in place to ensure we're keeping the bundled agent up to date.

@pugnascotia
Copy link
Contributor

@SylvainJuge I built a Docker image from your branch and tested a 3 node ES cluster with Kibana, and it all seems to work, so long as I have agent debug logging disabled. Initially, I didn't notice that I had the agent running with debug logging, and that did indeed seems to break a lot of span recording. But without debugging, all good.

@SylvainJuge
Copy link
Member Author

Thanks for checking that @pugnascotia !

@SylvainJuge
Copy link
Member Author

@elasticmachine run elasticsearch-ci/docs

@pugnascotia pugnascotia merged commit 16c5b7e into elastic:main Mar 24, 2023
@SylvainJuge SylvainJuge deleted the update-apm-agent branch March 27, 2023 07:18
tvernum pushed a commit to tvernum/elasticsearch that referenced this pull request Apr 19, 2023
Fixes elastic#94689. The APM agent version 1.33.0 fails to start on JDK 20, which prevents the APM integration to work as expected. As a consequence, the tracing does not work. When setting `ELASTIC_APM_LOG_LEVEL=debug` and `ELASTIC_APM_LOG_FILE=/tmp/log.txt`, the agent log shows that there is an issue with accessing `Unsafe` (sorry I don't have the exact stack trace). There was a few changes in APM agent regarding the security manager (SM) in recent versions, and updating the agent seems to make it work as expected. However, there is one known caveat so far (elastic/apm-agent-java#3074), keeping the agent with `debug` log level with `ELASTIC_APM_LOG_LEVEL=debug` makes it trigger another security exception when trying to establish connection with apm-server because the agent prints few details if a proxy is used or not (which is forbidden by default by the SM and isn't yet wrapped in a privileged call. Backport of: elastic#94716
tvernum added a commit that referenced this pull request Apr 19, 2023
The APM agent version 1.33.0 fails to start on JDK 20, which prevents the APM integration to work as expected. As a consequence, the tracing does not work. When setting `ELASTIC_APM_LOG_LEVEL=debug` and `ELASTIC_APM_LOG_FILE=/tmp/log.txt`, the agent log shows that there is an issue with accessing `Unsafe` (sorry I don't have the exact stack trace). There was a few changes in APM agent regarding the security manager (SM) in recent versions, and updating the agent seems to make it work as expected. However, there is one known caveat so far (elastic/apm-agent-java#3074), keeping the agent with `debug` log level with `ELASTIC_APM_LOG_LEVEL=debug` makes it trigger another security exception when trying to establish connection with apm-server because the agent prints few details if a proxy is used or not (which is forbidden by default by the SM and isn't yet wrapped in a privileged call. Backport of: #94716 Co-authored-by: SylvainJuge <sylvain.juge@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Core/Infra/Core Core issues without another label external-contributor Pull request authored by a developer outside the Elasticsearch team Team:Core/Infra Meta label for core/infra team >upgrade v8.8.0

4 participants