- Notifications
You must be signed in to change notification settings - Fork 314
Upload code coverage to Datadog #9132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upload code coverage to Datadog #9132
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 9 unstable metrics. Startup time reports for petclinicgantt title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d dateFormat X axisFormat %s section tracing Agent [baseline] (996.018 ms) : 0, 996018 Total [baseline] (10.631 s) : 0, 10631295 Agent [candidate] (994.652 ms) : 0, 994652 Total [candidate] (10.664 s) : 0, 10664289 section appsec Agent [baseline] (1.175 s) : 0, 1174619 Total [baseline] (10.778 s) : 0, 10777978 Agent [candidate] (1.174 s) : 0, 1174233 Total [candidate] (10.756 s) : 0, 10756430 section iast Agent [baseline] (1.14 s) : 0, 1140051 Total [baseline] (11.008 s) : 0, 11007520 Agent [candidate] (1.135 s) : 0, 1134859 Total [candidate] (10.868 s) : 0, 10867839 section profiling Agent [baseline] (1.246 s) : 0, 1246040 Total [baseline] (10.989 s) : 0, 10988923 Agent [candidate] (1.245 s) : 0, 1244910 Total [candidate] (10.991 s) : 0, 10990915
gantt title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (688.163 ms) : 0, 688163 BytebuddyAgent [candidate] (687.017 ms) : 0, 687017 GlobalTracer [baseline] (241.837 ms) : 0, 241837 GlobalTracer [candidate] (242.067 ms) : 0, 242067 AppSec [baseline] (30.276 ms) : 0, 30276 AppSec [candidate] (30.044 ms) : 0, 30044 Debugger [baseline] (5.987 ms) : 0, 5987 Debugger [candidate] (5.977 ms) : 0, 5977 Remote Config [baseline] (672.367 µs) : 0, 672 Remote Config [candidate] (673.461 µs) : 0, 673 Telemetry [baseline] (8.224 ms) : 0, 8224 Telemetry [candidate] (8.171 ms) : 0, 8171 section appsec BytebuddyAgent [baseline] (709.495 ms) : 0, 709495 BytebuddyAgent [candidate] (710.165 ms) : 0, 710165 GlobalTracer [baseline] (234.809 ms) : 0, 234809 GlobalTracer [candidate] (235.169 ms) : 0, 235169 AppSec [baseline] (171.421 ms) : 0, 171421 AppSec [candidate] (170.263 ms) : 0, 170263 Debugger [baseline] (5.767 ms) : 0, 5767 Debugger [candidate] (5.729 ms) : 0, 5729 Remote Config [baseline] (615.4 µs) : 0, 615 Remote Config [candidate] (607.022 µs) : 0, 607 Telemetry [baseline] (8.132 ms) : 0, 8132 Telemetry [candidate] (8.129 ms) : 0, 8129 IAST [baseline] (23.546 ms) : 0, 23546 IAST [candidate] (23.297 ms) : 0, 23297 section iast BytebuddyAgent [baseline] (812.776 ms) : 0, 812776 BytebuddyAgent [candidate] (808.739 ms) : 0, 808739 GlobalTracer [baseline] (233.553 ms) : 0, 233553 GlobalTracer [candidate] (233.123 ms) : 0, 233123 AppSec [baseline] (27.555 ms) : 0, 27555 AppSec [candidate] (31.576 ms) : 0, 31576 Debugger [baseline] (6.725 ms) : 0, 6725 Debugger [candidate] (5.777 ms) : 0, 5777 Remote Config [baseline] (589.021 µs) : 0, 589 Remote Config [candidate] (584.756 µs) : 0, 585 Telemetry [baseline] (7.996 ms) : 0, 7996 Telemetry [candidate] (8.062 ms) : 0, 8062 IAST [baseline] (30.045 ms) : 0, 30045 IAST [candidate] (26.194 ms) : 0, 26194 section profiling ProfilingAgent [baseline] (104.15 ms) : 0, 104150 ProfilingAgent [candidate] (103.644 ms) : 0, 103644 BytebuddyAgent [baseline] (678.554 ms) : 0, 678554 BytebuddyAgent [candidate] (677.531 ms) : 0, 677531 GlobalTracer [baseline] (361.731 ms) : 0, 361731 GlobalTracer [candidate] (361.864 ms) : 0, 361864 AppSec [baseline] (33.686 ms) : 0, 33686 AppSec [candidate] (31.032 ms) : 0, 31032 Debugger [baseline] (9.16 ms) : 0, 9160 Debugger [candidate] (10.642 ms) : 0, 10642 Remote Config [baseline] (674.437 µs) : 0, 674 Remote Config [candidate] (673.458 µs) : 0, 673 Telemetry [baseline] (9.449 ms) : 0, 9449 Telemetry [candidate] (10.969 ms) : 0, 10969 Profiling [baseline] (104.174 ms) : 0, 104174 Profiling [candidate] (103.669 ms) : 0, 103669 Startup time reports for insecure-bankgantt title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d dateFormat X axisFormat %s section tracing Agent [baseline] (994.811 ms) : 0, 994811 Total [baseline] (8.569 s) : 0, 8569279 Agent [candidate] (994.367 ms) : 0, 994367 Total [candidate] (8.545 s) : 0, 8544635 section iast Agent [baseline] (1.131 s) : 0, 1130688 Total [baseline] (9.303 s) : 0, 9303453 Agent [candidate] (1.131 s) : 0, 1131268 Total [candidate] (9.253 s) : 0, 9253401
gantt title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (686.79 ms) : 0, 686790 BytebuddyAgent [candidate] (686.773 ms) : 0, 686773 GlobalTracer [baseline] (242.069 ms) : 0, 242069 GlobalTracer [candidate] (241.762 ms) : 0, 241762 AppSec [baseline] (30.287 ms) : 0, 30287 AppSec [candidate] (30.252 ms) : 0, 30252 Debugger [baseline] (6.017 ms) : 0, 6017 Debugger [candidate] (5.956 ms) : 0, 5956 Remote Config [baseline] (682.56 µs) : 0, 683 Remote Config [candidate] (675.581 µs) : 0, 676 Telemetry [baseline] (8.235 ms) : 0, 8235 Telemetry [candidate] (8.211 ms) : 0, 8211 section iast BytebuddyAgent [baseline] (806.145 ms) : 0, 806145 BytebuddyAgent [candidate] (806.627 ms) : 0, 806627 GlobalTracer [baseline] (231.853 ms) : 0, 231853 GlobalTracer [candidate] (232.111 ms) : 0, 232111 IAST [baseline] (26.472 ms) : 0, 26472 IAST [candidate] (26.238 ms) : 0, 26238 AppSec [baseline] (30.485 ms) : 0, 30485 AppSec [candidate] (31.435 ms) : 0, 31435 Debugger [baseline] (6.589 ms) : 0, 6589 Debugger [candidate] (5.729 ms) : 0, 5729 Remote Config [baseline] (576.818 µs) : 0, 577 Remote Config [candidate] (582.83 µs) : 0, 583 Telemetry [baseline] (7.892 ms) : 0, 7892 Telemetry [candidate] (7.919 ms) : 0, 7919 LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 2 performance regressions! Performance is the same for 9 metrics, 12 unstable metrics.
Request duration reports for petclinicgantt title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d dateFormat X axisFormat %s section baseline no_agent (37.248 ms) : 36941, 37555 . : milestone, 37248, appsec (49.259 ms) : 48812, 49706 . : milestone, 49259, code_origins (44.627 ms) : 44238, 45016 . : milestone, 44627, iast (44.863 ms) : 44475, 45251 . : milestone, 44863, profiling (48.183 ms) : 47750, 48616 . : milestone, 48183, tracing (42.559 ms) : 42205, 42912 . : milestone, 42559, section candidate no_agent (36.802 ms) : 36504, 37100 . : milestone, 36802, appsec (46.35 ms) : 45943, 46758 . : milestone, 46350, code_origins (45.191 ms) : 44813, 45570 . : milestone, 45191, iast (43.775 ms) : 43401, 44149 . : milestone, 43775, profiling (49.771 ms) : 49301, 50241 . : milestone, 49771, tracing (45.306 ms) : 44919, 45693 . : milestone, 45306,
Request duration reports for insecure-bankgantt title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d dateFormat X axisFormat %s section baseline no_agent (4.2 ms) : 4150, 4249 . : milestone, 4200, iast (9.316 ms) : 9159, 9472 . : milestone, 9316, iast_FULL (13.824 ms) : 13549, 14099 . : milestone, 13824, iast_GLOBAL (10.287 ms) : 10084, 10490 . : milestone, 10287, profiling (8.626 ms) : 8476, 8776 . : milestone, 8626, tracing (7.612 ms) : 7504, 7719 . : milestone, 7612, section candidate no_agent (4.313 ms) : 4264, 4362 . : milestone, 4313, iast (9.52 ms) : 9358, 9681 . : milestone, 9520, iast_FULL (13.584 ms) : 13311, 13857 . : milestone, 13584, iast_GLOBAL (10.372 ms) : 10191, 10553 . : milestone, 10372, profiling (8.56 ms) : 8426, 8693 . : milestone, 8560, tracing (7.495 ms) : 7382, 7608 . : milestone, 7495,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for biojavagantt title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d dateFormat X axisFormat %s section baseline no_agent (14.966 s) : 14966000, 14966000 . : milestone, 14966000, appsec (14.902 s) : 14902000, 14902000 . : milestone, 14902000, iast (18.691 s) : 18691000, 18691000 . : milestone, 18691000, iast_GLOBAL (18.214 s) : 18214000, 18214000 . : milestone, 18214000, profiling (14.931 s) : 14931000, 14931000 . : milestone, 14931000, tracing (14.761 s) : 14761000, 14761000 . : milestone, 14761000, section candidate no_agent (14.848 s) : 14848000, 14848000 . : milestone, 14848000, appsec (14.862 s) : 14862000, 14862000 . : milestone, 14862000, iast (18.441 s) : 18441000, 18441000 . : milestone, 18441000, iast_GLOBAL (18.233 s) : 18233000, 18233000 . : milestone, 18233000, profiling (14.833 s) : 14833000, 14833000 . : milestone, 14833000, tracing (14.807 s) : 14807000, 14807000 . : milestone, 14807000,
Execution time for tomcatgantt title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d dateFormat X axisFormat %s section baseline no_agent (1.478 ms) : 1466, 1489 . : milestone, 1478, appsec (2.436 ms) : 2385, 2486 . : milestone, 2436, iast (2.218 ms) : 2155, 2281 . : milestone, 2218, iast_GLOBAL (2.255 ms) : 2191, 2318 . : milestone, 2255, profiling (2.044 ms) : 1994, 2094 . : milestone, 2044, tracing (2.024 ms) : 1975, 2072 . : milestone, 2024, section candidate no_agent (1.484 ms) : 1473, 1496 . : milestone, 1484, appsec (2.429 ms) : 2379, 2479 . : milestone, 2429, iast (2.207 ms) : 2145, 2270 . : milestone, 2207, iast_GLOBAL (2.253 ms) : 2191, 2316 . : milestone, 2253, profiling (2.513 ms) : 2338, 2688 . : milestone, 2513, tracing (2.025 ms) : 1977, 2074 . : milestone, 2025,
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a way to test it or let's just give it a try?
# Upload test results to production environment like all other CI jobs | ||
junit_upload "$DATADOG_API_KEY_PROD" | ||
| ||
coverage_upload "$DATADOG_API_KEY_PROD" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should the script return the result of both upload functions now? Not only the coverage one?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Won't it just fail if any of the uploads fails?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, nevermind, I realised the default is to just continue executing if a command fails. Updated the script to consider both statuses
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See my comment here: https://github.com/DataDog/dd-trace-java/pull/9132/files#r2197581837
It was tested by the CI jobs of this PR :) |
# Upload test results to production environment like all other CI jobs | ||
junit_upload "$DATADOG_API_KEY_PROD" | ||
| ||
coverage_upload "$DATADOG_API_KEY_PROD" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See my comment here: https://github.com/DataDog/dd-trace-java/pull/9132/files#r2197581837
What Does This Do
Configures uploading code coverage data to Datadog.
Motivation
Dogfooding new Code Coverage product.
Contributor Checklist
type:
and (comp:
orinst:
) labels in addition to any usefull labelsclose
,fix
or any linking keywords when referencing an issue.Use
solves
instead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]