Integration test #266

twilleit · 2021-12-01T15:13:54Z

Added Integration test class and 2 cobertura-files for the Integration-test, because there where problems with the existing ones.

Created PR for the lesson on Thursday 02.12.

uhafner · 2021-12-04T22:58:06Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+
+ WorkflowJob job2 = createPipelineWithWorkspaceFiles(JACOCO_LESS_FILE_NAME);
+ job2.setDefinition(new CpsFlowDefinition("node {"
+ + "publishCoverage adapters: [jacocoAdapter('**/*.xml')],"


You need to add the reference job finder:

discoverReferenceBuild(referenceJob: 'reference')

uhafner

Please also check your coding style settings.

uhafner · 2021-12-05T11:19:56Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+
+
+ @Rule
+ public JenkinsRule j = new JenkinsRule();


This creates another Jenkins instance per test. Remove it, it is already in the base class.

uhafner · 2021-12-05T11:21:44Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+ WorkflowJob job = createPipelineWithWorkspaceFiles(JACOCO_FILE_NAME, JACOCO_LESS_FILE_NAME);
+ job.setDefinition(new CpsFlowDefinition("node {"
+ + "publishCoverage adapters: [jacocoAdapter('**/*.xml')]"
+ + "}", true));
+


Try to find a way that you do not need to duplicate code in each test

uhafner · 2021-12-05T11:24:32Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+ assertThat(build.getNumber()).isEqualTo(1);
+
+ CoverageBuildAction coverageResult = build.getAction(CoverageBuildAction.class);
+ assertThat(coverageResult.getLineCoverage()).isEqualTo(new Coverage(12166, 12736-12166));


It makes sense to use constants in the expressions for the whole test so this is better understandable. Something like JACOCO_LARGE_MISSED_LINES, JACOCO_SMLL_COVERES_LINE. Then you can let Java sum up these values for you.

uhafner · 2021-12-05T11:25:19Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+ Run<?, ?> build = buildSuccessfully(job);
+ assertThat(build.getNumber()).isEqualTo(1);


uhafner · 2021-12-05T11:25:55Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+ job.setDefinition(new CpsFlowDefinition("node {"
+ + "publishCoverage adapters: [cobertura('**/*.xml')]"
+ + "}", true));
+
+ Run<?, ?> build = buildSuccessfully(job);
+ assertThat(build.getNumber()).isEqualTo(1);


See above, the code is almost identical

uhafner · 2021-12-05T11:26:45Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+ public void CoveragePluginPipeline_1_CoberturaInputFile_1JacocoInputFile() throws IOException {
+ WorkflowJob job = createPipelineWithWorkspaceFiles(JACOCO_FILE_NAME, COBERTURA_FILE_NAME);
+ job.setDefinition(new CpsFlowDefinition("node {"
+ + "publishCoverage adapters: [cobertura('**/*.xml'),jacocoAdapter('**/*.xml')]"


This makes no sense, each adapter should read only files that it can understand.

uhafner · 2021-12-05T11:27:43Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+ private static final String JACOCO_FILE_NAME = "jacoco-analysis-model.xml";
+ private static final String JACOCO_LESS_FILE_NAME = "jacoco-codingstyle.xml";
+ private static final String COBERTURA_FILE_NAME = "cobertura.xml";
+ private static final String COBERTURA_FILE_NAME_2 = "cobertura2.xml";


Use meaningful names

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

uhafner · 2021-12-05T11:34:41Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+ WorkflowJob job2 = createPipelineWithWorkspaceFiles(JACOCO_LESS_FILE_NAME);
+ job2.setDefinition(new CpsFlowDefinition("node {"
+ + "publishCoverage adapters: [jacocoAdapter('**/*.xml')],"
+ //+ "sourceFileResolver: sourceFiles('STORE_LAST_BUILD'),"


See chat on how to specify the reference build

uhafner · 2021-12-05T11:43:02Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+ CoverageBuildAction coverageResult = build.getAction(CoverageBuildAction.class);
+ assertThat(coverageResult.getLineCoverage()).isEqualTo(new Coverage(6083, 6368-6083));
+
+ //TODO: Wie assertet man publish checks???


You need to check for the existence/absence of the string:

[Checks API] No suitable checks publisher found.

uhafner

The tests are in a somehow unfinished state.

uhafner · 2022-01-20T22:30:45Z

pom.xml

+ <dependency>
+ <groupId>io.jenkins.plugins</groupId>
+ <artifactId>warnings-ng</artifactId>
+ <version>9.6.0-SNAPSHOT</version>
+ <scope>compile</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.jenkins-ci</groupId>
+ <artifactId>acceptance-test-harness</artifactId>
+ <version>1.101</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.jenkins-ci</groupId>
+ <artifactId>acceptance-test-harness</artifactId>
+ <version>1.103</version>
+ <scope>test</scope>
+ </dependency>


These make no sense.

uhafner · 2022-01-20T22:31:34Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+/**
+ * Integration test for the coverage API plugin.
+ *
+ * todo: DRY, constants for values, javadoc


uhafner · 2022-01-20T22:32:06Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+ //HealthReportingAction result = build.getAction(HealthReportingAction.class);
+ //assertThat(result.getBuildHealth()).isEqualTo(null); //healthReport is null why???


This should work now?

uhafner · 2022-01-20T22:34:41Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+ @Test
+ public void coveragePluginPipelineFailUnhealthyWithResultSuccess() {
+ WorkflowJob job = createPipelineWithWorkspaceFiles(JACOCO_FILE_WITH_HIGHER_COVERAGE);
+ job.setDefinition(new CpsFlowDefinition("node {"
+ + "publishCoverage adapters: [jacocoAdapter('**/*.xml')],"
+ + "globalThresholds: [[failUnhealthy: true, thresholdTarget: 'Line', unhealthyThreshold: 90.0]]"
+ + "}", true));
+
+ Run<?, ?> build = buildWithResult(job, Result.SUCCESS);
+ assertThat(build.getNumber()).isEqualTo(1);
+ }
+
+
+ @Test
+ public void coveragePluginPipelineFailUnstable() {
+ WorkflowJob job = createPipelineWithWorkspaceFiles(JACOCO_FILE_WITH_HIGHER_COVERAGE);
+ job.setDefinition(new CpsFlowDefinition("node {"
+ + "publishCoverage adapters: [jacocoAdapter('**/*.xml')],"
+ + "failUnstable: true,"
+ + "globalThresholds: [[thresholdTarget: 'Line', unstableThreshold: 96.0]]"
+ + "}", true));
+
+ Run<?, ?> build = buildWithResult(job, Result.FAILURE);
+ assertThat(build.getNumber()).isEqualTo(1);
+ }


It is hard to understand why it fails in one method only. Maybe adding a comment or improving the method names would help (or using some constants).

uhafner · 2022-01-20T22:35:54Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+ copyFilesToWorkspace(job, JACOCO_FILE_WITH_HIGHER_COVERAGE);
+ job.setDefinition(new CpsFlowDefinition("node {"
+ + "publishCoverage adapters: [jacocoAdapter('" + JACOCO_FILE_WITH_HIGHER_COVERAGE + "')]\n"
+ + "discoverReferenceBuild(referenceJob: '" + job.getName() + "')"


This needs to be called before recording.

uhafner · 2022-01-20T22:57:53Z

src/test/java/io/jenkins/plugins/coverage/model/CoveragePluginITest.java

+ /**
+ * Test ignored ... falsche Imports?
+ * java.lang.IllegalArgumentException: Unknown server host key algorithm 'ssh-ed25519'
+ */


The code and pom actually is broken.

uhafner · 2022-01-20T23:01:04Z

I'm closing this PR as we can only merge one of the 4 integration test PRs. There is no need to add additional commits.

twilleit added 3 commits November 11, 2021 01:02

Added QualityGateStatus-logic for code-coverage

95962bd

Added some Integration-Tests

db266c1

Removed new Quality Gates in this branch

bc01289

uhafner reviewed Dec 4, 2021

View reviewed changes

uhafner suggested changes Dec 5, 2021

View reviewed changes

twilleit added 2 commits December 9, 2021 02:29

Added new Integration-Tests, fixed bug in CoverageProcessor

970a94e

Added docker test

63ae7cd

uhafner reviewed Jan 20, 2022

View reviewed changes

uhafner closed this Jan 20, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Integration test #266

Integration test #266

Uh oh!

twilleit commented Dec 1, 2021

uhafner Dec 4, 2021

uhafner left a comment

uhafner Dec 5, 2021

uhafner Dec 5, 2021

uhafner Dec 5, 2021

uhafner Dec 5, 2021

uhafner Dec 5, 2021

uhafner Dec 5, 2021

uhafner Dec 5, 2021

Uh oh!

uhafner Dec 5, 2021

uhafner Dec 5, 2021

uhafner left a comment

uhafner Jan 20, 2022

uhafner Jan 20, 2022

uhafner Jan 20, 2022

uhafner Jan 20, 2022

uhafner Jan 20, 2022

uhafner Jan 20, 2022

uhafner commented Jan 20, 2022

Labels

2 participants

		Run<?, ?> build = buildSuccessfully(job);
		assertThat(build.getNumber()).isEqualTo(1);

		//HealthReportingAction result = build.getAction(HealthReportingAction.class);
		//assertThat(result.getBuildHealth()).isEqualTo(null); //healthReport is null why???

Integration test #266

Integration test #266

Uh oh!

Conversation

twilleit commented Dec 1, 2021

Choose a reason for hiding this comment

uhafner left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

uhafner left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

uhafner commented Jan 20, 2022

Labels

2 participants