Threshold value not working #48
Description
Hi,
I just upgraded to the latest v1.1.0 GA build. I'm using the new /p:Threshold=80 option to the build process yet the test run still returns 'successful' even if the coverage is below the threshold.
This is the command I run to build:
dotnet test --configuration Release --logger 'trx;logfilename=mstest-results.trx' /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:CoverletOutputDirectory=TestResults /p:Threshold=80
This is the output:
Microsoft (R) Test Execution Command Line Tool Version 15.3.0-preview-20170628-02 Copyright (c) Microsoft Corporation. All rights reserved. Starting test execution, please wait... [xUnit.net 00:00:01.5821864] Discovering: UnitTests [xUnit.net 00:00:01.7179983] Discovered: UnitTests [xUnit.net 00:00:01.7245869] Starting: UnitTests [xUnit.net 00:00:01.9276830] AuthResponsesOperationFilter does not add 401 response to operation without AuthoriseAttribute [SKIP] [xUnit.net 00:00:01.9282262] Cannot yet mock ApiDescription objects Skipped AuthResponsesOperationFilter does not add 401 response to operation without AuthoriseAttribute Standard Output Messages: Cannot yet mock ApiDescription objects [xUnit.net 00:00:01.9862873] AuthResponsesOperationFilter Adds 401 response to operation with AuthoriseAttribute [SKIP] [xUnit.net 00:00:01.9863816] Cannot yet mock ApiDescription objects Skipped AuthResponsesOperationFilter Adds 401 response to operation with AuthoriseAttribute Standard Output Messages: Cannot yet mock ApiDescription objects [xUnit.net 00:00:02.3872529] Finished: UnitTests Total tests: 33. Passed: 31. Failed: 0. Skipped: 2. Test Run Successful. Test execution time: 3.8409 Seconds WARNING: Overwriting results file: /home/jenkins/agent/workspace/an-updated-coverage-process-7N7FVRFUDPWSNYEY7B7OE6IM5VJ26KCY5NFVLTNHW4HURJ65KZAQ/tests/UnitTests/TestResults/mstest-results.trx Results File: /home/jenkins/agent/workspace/an-updated-coverage-process-7N7FVRFUDPWSNYEY7B7OE6IM5VJ26KCY5NFVLTNHW4HURJ65KZAQ/tests/UnitTests/TestResults/mstest-results.trx Calculating coverage result... Generating report 'TestResults/coverage.xml' | Module | Coverage | |------------------------|----------| | ApplicationCore | 75% | | Infrastructure | 37.8% | | Nedbank.Authentication | 0% | | Web | 42.9% | Am I missing something or is this not working right now.
Also my build environment is:
Microsoft (R) Build Engine version 15.3.409.57025 for .NET Core
We're running .NET Core 2.0 on Linux. (Install .NET Core SDK 2.1.104 Package)