Skip to content

Conversation

basepi
Copy link
Contributor

@basepi basepi commented May 27, 2020

Previously these would show up as big tracebacks in the logs which
were ugly, considering all sorts of things can go wrong when suddenly
closing the transport (such as when a celery worker terminates)

Fixes #823

Previously these would show up as big tracebacks in the logs which was ugly, considering all sorts of things can go wrong when suddenly closing the transport (such as when a celery worker terminates) Fixes elastic#823
@ghost
Copy link

ghost commented May 27, 2020

💔 Tests Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #838 updated]

  • Start Time: 2020-06-05T15:18:02.757+0000

  • Duration: 28 min 31 sec

Test stats 🧪

Test Results
Failed 1
Passed 9256
Skipped 6796
Total 16053

Test errors

Expand to view the tests failures

  • Name: Test / windows-3.6-none / test_send_remote_failover_sync[validating_httpserver0-sending_elasticapm_client0] – tests.client.client_tests

    • Age: 1
    • Duration: 0.599
    • Error Details: AssertionError

Log output

Expand to view the last 100 lines of log output

[2020-06-05T15:45:13.849Z] You should consider upgrading via the '/opt/pypy/pypy2.7-7.3.1/bin/pypy -m pip install --upgrade pip' command. [2020-06-05T15:45:13.849Z] + built_wheel=(/tmp/built_wheel/*.whl) [2020-06-05T15:45:13.849Z] + rm -rf /tmp/repaired_wheels [2020-06-05T15:45:13.849Z] + mkdir /tmp/repaired_wheels [2020-06-05T15:45:13.849Z] + [[ /tmp/built_wheel/elastic_apm-5.6.0-py2-none-any.whl == *none-any.whl ]] [2020-06-05T15:45:13.849Z] + mv /tmp/built_wheel/elastic_apm-5.6.0-py2-none-any.whl /tmp/repaired_wheels [2020-06-05T15:45:13.849Z] + repaired_wheels=(/tmp/repaired_wheels/*.whl) [2020-06-05T15:45:13.849Z] + '[' '!' -z '' ']' [2020-06-05T15:45:13.849Z] + mv /tmp/repaired_wheels/elastic_apm-5.6.0-py2-none-any.whl /output [2020-06-05T15:45:13.849Z] + for repaired_wheel in '"${repaired_wheels[@]}"' [2020-06-05T15:45:13.849Z] + basename /tmp/repaired_wheels/elastic_apm-5.6.0-py2-none-any.whl [2020-06-05T15:45:13.849Z] + chown 1156:1157 /output/elastic_apm-5.6.0-py2-none-any.whl [2020-06-05T15:45:13.849Z] + docker exec -i cibuildwheel-db177bcb-fb54-436d-bf0b-3366b98445d2 sh -c 'cat > /constraints.txt' [2020-06-05T15:45:14.437Z] + docker exec -i cibuildwheel-db177bcb-fb54-436d-bf0b-3366b98445d2 /bin/bash [2020-06-05T15:45:14.700Z] + mkdir -p /output [2020-06-05T15:45:14.700Z] + cd /project [2020-06-05T15:45:14.700Z] + PYBIN=/opt/python/pp36-pypy36_pp73/bin [2020-06-05T15:45:14.700Z] + export PATH=/opt/python/pp36-pypy36_pp73/bin:/opt/rh/devtoolset-8/root/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin [2020-06-05T15:45:14.700Z] + PATH=/opt/python/pp36-pypy36_pp73/bin:/opt/rh/devtoolset-8/root/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin [2020-06-05T15:45:14.700Z] + which pip [2020-06-05T15:45:14.700Z] + '[' /opt/python/pp36-pypy36_pp73/bin/pip '!=' /opt/python/pp36-pypy36_pp73/bin/pip ']' [2020-06-05T15:45:14.700Z] + which python [2020-06-05T15:45:14.700Z] + '[' /opt/python/pp36-pypy36_pp73/bin/python '!=' /opt/python/pp36-pypy36_pp73/bin/python ']' [2020-06-05T15:45:14.700Z] + '[' '!' -z '' ']' [2020-06-05T15:45:14.700Z] + rm -rf /tmp/built_wheel [2020-06-05T15:45:14.700Z] + mkdir /tmp/built_wheel [2020-06-05T15:45:14.700Z] + pip wheel /project/. -w /tmp/built_wheel --no-deps [2020-06-05T15:45:18.017Z] Processing /project [2020-06-05T15:45:18.017Z] Installing build dependencies: started [2020-06-05T15:45:22.239Z] Installing build dependencies: finished with status 'done' [2020-06-05T15:45:22.239Z] Getting requirements to build wheel: started [2020-06-05T15:45:24.215Z] Getting requirements to build wheel: finished with status 'done' [2020-06-05T15:45:24.215Z] Preparing wheel metadata: started [2020-06-05T15:45:25.240Z] Preparing wheel metadata: finished with status 'done' [2020-06-05T15:45:25.240Z] Building wheels for collected packages: elastic-apm [2020-06-05T15:45:25.240Z] Building wheel for elastic-apm (PEP 517): started [2020-06-05T15:45:27.329Z] Building wheel for elastic-apm (PEP 517): finished with status 'done' [2020-06-05T15:45:27.329Z] Created wheel for elastic-apm: filename=elastic_apm-5.6.0-py3-none-any.whl size=447470 sha256=049b33c530dadd00dcb1acb453a99422ed513364d001ee4e05ce7f2fa43288cd [2020-06-05T15:45:27.329Z] Stored in directory: /tmp/pip-ephem-wheel-cache-bxj3gay_/wheels/de/ab/88/a237fdf750e25f63db6cf5343e3feef89c5f93e985be647379 [2020-06-05T15:45:27.329Z] Successfully built elastic-apm [2020-06-05T15:45:27.633Z] WARNING: You are using pip version 20.0.2; however, version 20.1.1 is available. [2020-06-05T15:45:27.633Z] You should consider upgrading via the '/opt/pypy/pypy3.6-7.3.1/bin/pypy -m pip install --upgrade pip' command. [2020-06-05T15:45:27.633Z] + built_wheel=(/tmp/built_wheel/*.whl) [2020-06-05T15:45:27.633Z] + rm -rf /tmp/repaired_wheels [2020-06-05T15:45:27.633Z] + mkdir /tmp/repaired_wheels [2020-06-05T15:45:27.633Z] + [[ /tmp/built_wheel/elastic_apm-5.6.0-py3-none-any.whl == *none-any.whl ]] [2020-06-05T15:45:27.634Z] + mv /tmp/built_wheel/elastic_apm-5.6.0-py3-none-any.whl /tmp/repaired_wheels [2020-06-05T15:45:27.634Z] + repaired_wheels=(/tmp/repaired_wheels/*.whl) [2020-06-05T15:45:27.634Z] + '[' '!' -z '' ']' [2020-06-05T15:45:27.634Z] + mv /tmp/repaired_wheels/elastic_apm-5.6.0-py3-none-any.whl /output [2020-06-05T15:45:27.634Z] + for repaired_wheel in '"${repaired_wheels[@]}"' [2020-06-05T15:45:27.634Z] + basename /tmp/repaired_wheels/elastic_apm-5.6.0-py3-none-any.whl [2020-06-05T15:45:27.634Z] + chown 1156:1157 /output/elastic_apm-5.6.0-py3-none-any.whl [2020-06-05T15:45:27.634Z] + docker cp cibuildwheel-db177bcb-fb54-436d-bf0b-3366b98445d2:/output/. /var/lib/jenkins/workspace/thon_apm-agent-python-mbp_PR-838/src/github.com/elastic/apm-agent-python/wheelhouse [2020-06-05T15:45:27.634Z] + docker rm --force -v cibuildwheel-db177bcb-fb54-436d-bf0b-3366b98445d2 [2020-06-05T15:45:28.272Z] cibuildwheel-db177bcb-fb54-436d-bf0b-3366b98445d2 [2020-06-05T15:45:28.272Z] + ls -l wheelhouse [2020-06-05T15:45:28.272Z] total 12720 [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 497091 Jun 5 15:43 elastic_apm-5.6.0-cp27-cp27m-manylinux1_i686.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 502926 Jun 5 15:42 elastic_apm-5.6.0-cp27-cp27m-manylinux1_x86_64.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 497093 Jun 5 15:43 elastic_apm-5.6.0-cp27-cp27m-manylinux2010_i686.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 502929 Jun 5 15:42 elastic_apm-5.6.0-cp27-cp27m-manylinux2010_x86_64.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 497092 Jun 5 15:44 elastic_apm-5.6.0-cp27-cp27mu-manylinux1_i686.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 502917 Jun 5 15:42 elastic_apm-5.6.0-cp27-cp27mu-manylinux1_x86_64.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 497093 Jun 5 15:44 elastic_apm-5.6.0-cp27-cp27mu-manylinux2010_i686.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 502919 Jun 5 15:42 elastic_apm-5.6.0-cp27-cp27mu-manylinux2010_x86_64.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 500604 Jun 5 15:44 elastic_apm-5.6.0-cp35-cp35m-manylinux1_i686.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 506314 Jun 5 15:42 elastic_apm-5.6.0-cp35-cp35m-manylinux1_x86_64.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 500606 Jun 5 15:44 elastic_apm-5.6.0-cp35-cp35m-manylinux2010_i686.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 506318 Jun 5 15:42 elastic_apm-5.6.0-cp35-cp35m-manylinux2010_x86_64.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 500854 Jun 5 15:44 elastic_apm-5.6.0-cp36-cp36m-manylinux1_i686.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 506556 Jun 5 15:43 elastic_apm-5.6.0-cp36-cp36m-manylinux1_x86_64.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 500857 Jun 5 15:44 elastic_apm-5.6.0-cp36-cp36m-manylinux2010_i686.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 506558 Jun 5 15:43 elastic_apm-5.6.0-cp36-cp36m-manylinux2010_x86_64.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 501778 Jun 5 15:44 elastic_apm-5.6.0-cp37-cp37m-manylinux1_i686.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 507496 Jun 5 15:43 elastic_apm-5.6.0-cp37-cp37m-manylinux1_x86_64.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 501783 Jun 5 15:44 elastic_apm-5.6.0-cp37-cp37m-manylinux2010_i686.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 507499 Jun 5 15:43 elastic_apm-5.6.0-cp37-cp37m-manylinux2010_x86_64.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 506406 Jun 5 15:44 elastic_apm-5.6.0-cp38-cp38-manylinux1_i686.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 511815 Jun 5 15:43 elastic_apm-5.6.0-cp38-cp38-manylinux1_x86_64.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 506409 Jun 5 15:44 elastic_apm-5.6.0-cp38-cp38-manylinux2010_i686.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 511816 Jun 5 15:43 elastic_apm-5.6.0-cp38-cp38-manylinux2010_x86_64.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 447462 Jun 5 15:45 elastic_apm-5.6.0-py2-none-any.whl [2020-06-05T15:45:28.272Z] -rw-r--r-- 1 jenkins jenkins 447470 Jun 5 15:45 elastic_apm-5.6.0-py3-none-any.whl [2020-06-05T15:45:29.603Z] Stashed 26 file(s) [2020-06-05T15:45:31.389Z] Scheduling project: APM Integration Test MBP Selector » master [2020-06-05T15:45:31.412Z] [INFO] Can not determine redirect link!!! [2020-06-05T15:45:32.455Z] Stage "Benchmarks" skipped due to when conditional [2020-06-05T15:45:32.508Z] Timeout set to expire in 12 hr [2020-06-05T15:45:32.580Z] Stage "Prepare Release" skipped due to when conditional [2020-06-05T15:45:32.603Z] Stage "Prepare Release" skipped due to when conditional [2020-06-05T15:45:32.645Z] Stage "Prepare Release" skipped due to when conditional [2020-06-05T15:45:32.687Z] Stage "Prepare Release" skipped due to when conditional [2020-06-05T15:45:32.984Z] Running on Jenkins in /var/lib/jenkins/workspace/thon_apm-agent-python-mbp_PR-838 [2020-06-05T15:45:33.109Z] [INFO] getVaultSecret: Getting secrets [2020-06-05T15:45:33.191Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID [2020-06-05T15:45:33.866Z] + chmod 755 generate-build-data.sh [2020-06-05T15:45:33.866Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-838/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-838/runs/10 UNSTABLE 1650848 [2020-06-05T15:45:34.417Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-838/runs/10/steps/?limit=10000 -o steps-info.json [2020-06-05T15:45:38.541Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-838/runs/10/tests/?status=FAILED -o tests-errors.json 

basepi added 3 commits May 28, 2020 09:06
I don't know how this happened, I think black added it? But then it didn't re-add it this time when I removed it. Probably more of black's magic comma stuff that has been added recently
Copy link
Contributor

@beniwohli beniwohli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

self.queue("close", None)
if not self._flushed.wait(timeout=self._max_flush_time):
raise ValueError("close timed out")
logger.error("Closing the transport connection timed out.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we additionally also should use a shorter timeout than max_flush_time. Maybe 2s?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm. I can potentially see the reasoning, but let's assume the user has upped the api_request_time because they're on a slow network (or for some other reason). I don't particularly want to lose data due to being impatient at close time.

I think I'm going to leave it, unless you think there's a hole in my reasoning?

@basepi
Copy link
Contributor Author

basepi commented Jun 5, 2020

More flakey tests. Merging.

@basepi basepi merged commit 321a344 into elastic:master Jun 5, 2020
romulorosa pushed a commit to romulorosa/apm-agent-python that referenced this pull request Oct 15, 2020
* Catch and log any exceptions while closing the transport Previously these would show up as big tracebacks in the logs which was ugly, considering all sorts of things can go wrong when suddenly closing the transport (such as when a celery worker terminates) Fixes elastic#823 * Remove trailing comma I don't know how this happened, I think black added it? But then it didn't re-add it this time when I removed it. Probably more of black's magic comma stuff that has been added recently * Add to changelog
romulorosa pushed a commit to romulorosa/apm-agent-python that referenced this pull request Oct 15, 2020
* Catch and log any exceptions while closing the transport Previously these would show up as big tracebacks in the logs which was ugly, considering all sorts of things can go wrong when suddenly closing the transport (such as when a celery worker terminates) Fixes elastic#823 * Remove trailing comma I don't know how this happened, I think black added it? But then it didn't re-add it this time when I removed it. Probably more of black's magic comma stuff that has been added recently * Add to changelog
beniwohli pushed a commit to beniwohli/apm-agent-python that referenced this pull request Sep 14, 2021
* Catch and log any exceptions while closing the transport Previously these would show up as big tracebacks in the logs which was ugly, considering all sorts of things can go wrong when suddenly closing the transport (such as when a celery worker terminates) Fixes elastic#823 * Remove trailing comma I don't know how this happened, I think black added it? But then it didn't re-add it this time when I removed it. Probably more of black's magic comma stuff that has been added recently * Add to changelog
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants