Continuous code quality in java projects
Igor Suhorukov Continuous code quality in java projects Information from this report is my subjective opinion based on my experience, knowledge, mistakes... ;-) Subjective opinion 6/27/19 2010 DB Blue template 2
Igor Suhorukov Continuous code quality in java projects Subjective opinion 6/27/19 2010 DB Blue template 3 https://youtu.be/mGiDkLgy7IM?t=279
Igor Suhorukov Continuous code quality in java projects Why Java? 6/27/19 2010 DB Blue template 4 https://madnight.github.io/githut/#/pull_requests/2019/1
Igor Suhorukov Continuous code quality in java projects Software functional quality reflects how well it complies with or conforms to a given design, based on functional requirements or specifications. Quality is subjective from end user point of view and is not constant in software development life cycle. ISO/IEC 9126, ISO/IEC 25000:2014, Сonsortium for IT Software Quality(CISQ), Software Quality Assessment based on Lifecycle Expectations(SQALE) Software quality 6/27/19 2010 DB Blue template 5
Igor Suhorukov Continuous code quality in java projects Software development process 6/27/19 2010 DB Blue template 6
Igor Suhorukov Continuous code quality in java projects Software development process constraints 6/27/19 2010 DB Blue template 7 Quality Cost Schedule Scope
Igor Suhorukov Continuous code quality in java projects Metrics measure the quantitative assessment of some property of software or its specification. Metrics usage and holy wars: ● How to choose the right metrics? ● Are metrics set blessed? ● What I need to do with metrics results? Metrics 6/27/19 2010 DB Blue template 8
Igor Suhorukov Continuous code quality in java projects ● Reliability ● Security ● Maintainability ● Duplications ● Complexity ● Issues/Code smell https://docs.sonarqube.org/latest/user-guide/metric-definitions/ Complexity metrics for software development 6/27/19 2010 DB Blue template 9
Igor Suhorukov Continuous code quality in java projects Technical debt / big ball of mud 6/27/19 2010 DB Blue template 10
Igor Suhorukov Continuous code quality in java projects Fragile code and unpredictable application failure after small changes Delayed improvements and miss deadlines Tight coupling code Technical debt 6/27/19 2010 DB Blue template 11
Igor Suhorukov Continuous code quality in java projects Technical debt is related to new code or bug fixes. Examples: increased code complexity, absence of tests for new code, subsystem or code decomposition issues and spaghetti code . Tech debt as violation of SOLID principles (single responsibility, open-closed, Liskov substitution, interface segregation and dependency inversion). Root cause: dev experience, limited time, team player discipline. Technical debt 6/27/19 2010 DB Blue template 12
Igor Suhorukov Continuous code quality in java projects Test-driven development (TDD) Behavior driven development (BDD) Performance Test Driven Development Continuous Code Quality Inspection Is it mandatory or recommended only? Depends on – team size, project complexity, outsourcing/in house project, schedule, management culture, team qualification/experience/velocity. Software quality should be part of SDLC 6/27/19 2010 DB Blue template 13
Igor Suhorukov Continuous code quality in java projects Agile Manifesto Individuals and interactions over processes and tools. Working software over comprehensive documentation. Customer collaboration over contract negotiation. Responding to change over following a plan. Individuals and interactions Software quality should be part of SDLC 6/27/19 2010 DB Blue template 14
Igor Suhorukov Continuous code quality in java projects Based on functional and non functional requirements: Black/White-box testing Manual/Unit/Integration testing/System testing Mutation testing/Fuzzing Load testing/Stress Testing/Performance testing Usability testing Software quality validation approach 6/27/19 2010 DB Blue template 15
Igor Suhorukov Continuous code quality in java projects ● Static code analysis just one tool in the box to reach good quality. Helps team to focus on some issues in large codebase. ● Formal verification of software programs. Too difficult to explain specification and limited usage. ● Running dynamic program analysis of software on emulator or real hardware. Time consuming method. Software quality. White box testing 6/27/19 2010 DB Blue template 16
Igor Suhorukov Continuous code quality in java projects ● Search by template in abstract syntax tree(AST). ● Rice's theorem. Theorem states that all non-trivial, semantic properties of programs are undecidable. ● False positive alerts. ● Nested method invocation. Static analysis constraints 6/27/19 2010 DB Blue template 17
Igor Suhorukov Continuous code quality in java projects ● IntelliJ Idea Community Edition - code inspections ● PVS-Studio Java free for several projects on github. Too many usage constraints. License key may be revoked in any time. ● SonarJava static analyzer for SonarLint & SonarQube Java code static analyzers 6/27/19 2010 DB Blue template 18
Igor Suhorukov Continuous code quality in java projects IntelliJ Idea code inspections 6/27/19 2010 DB Blue template 19
Igor Suhorukov Continuous code quality in java projects IntelliJ Idea code inspections 6/27/19 2010 DB Blue template 20
Igor Suhorukov Continuous code quality in java projects PSV Studio 6/27/19 2010 DB Blue template 21
Igor Suhorukov Continuous code quality in java projects SonarLint 6/27/19 2010 DB Blue template 22
Igor Suhorukov Continuous code quality in java projects https://www.sonarqube.org Community Edition/Developer Edition/Enterprise Edition/Data Center Edition https://sonarcloud.io SonarQube. Сontinuous code quality server 6/27/19 2010 DB Blue template 23
Igor Suhorukov Continuous code quality in java projects From first day SonarQube. New project 6/27/19 2010 DB Blue template 24
Igor Suhorukov Continuous code quality in java projects SonarQube. Code smells 6/27/19 2010 DB Blue template 25
Igor Suhorukov Continuous code quality in java projects SonarQube. Strategy how to use it in legacy project 6/27/19 2010 DB Blue template 26 ● Ignore existing issues, don’t pass new issue in code. QualityGate by default. ● Fix all issue ● Don’t use Sonar ● ?
Igor Suhorukov Continuous code quality in java projects git clone https://github.com/apache/ignite.git mvn sonar:sonar SonarQube. Project dashboard 6/27/19 2010 DB Blue template 27
Igor Suhorukov Continuous code quality in java projects SonarQube. Duplicate code 6/27/19 2010 DB Blue template 28
Igor Suhorukov Continuous code quality in java projects SonarQube. Maintainability 6/27/19 2010 DB Blue template 29
Igor Suhorukov Continuous code quality in java projects git clone https://github.com/apache/ignite.git mvn sonar:sonar SonarQube. Issues 6/27/19 2010 DB Blue template 30
Igor Suhorukov Continuous code quality in java projects SonarQube. New language feature inspection 6/27/19 2010 DB Blue template 31
Igor Suhorukov Continuous code quality in java projects SonarQube. Code complexity example 6/27/19 2010 DB Blue template 32
Igor Suhorukov Continuous code quality in java projects SonarQube. Issue description 6/27/19 2010 DB Blue template 33
Igor Suhorukov Continuous code quality in java projects SonarQube. Rules 6/27/19 2010 DB Blue template 34 https://rules.sonarsource.com/java/
Igor Suhorukov Continuous code quality in java projects ● https://docs.sonarqube.org/display/SCAN/Analyzing+with+Son arQube+Scanner+for+Jenkins ● https://docs.sonarqube.org/latest/analysis/pull-request/ ● https://sonarcloud.io/documentation/analysis/pull-request/ CI/CD integration 6/27/19 2010 DB Blue template 35
Igor Suhorukov Continuous code quality in java projects ● Black Duck Software ● Sonatype Nexus ● Artifactory ● Looks good to me LGTM Alternatives ● https://www.codacy.com ● https://github.com/marketplace/category/code-quality License compatibility/ known library issues 6/27/19 2010 DB Blue template 36
Igor Suhorukov Continuous code quality in java projects ● https://github.com/checkstyle/checkstyle https://github.com/spring-io/spring-javaformat/blob/master/src/checkstyle/checkstyle.xml Code style 6/27/19 2010 DB Blue template 37
Igor Suhorukov Continuous code quality in java projects https://github.com/TNG/ArchUnit-Examples/blob/master/example- junit5/src/test/java/com/tngtech/archunit/exampletest/junit5/DaoRulesTest.java Code structure tests 6/27/19 2010 DB Blue template 38
Igor Suhorukov Continuous code quality in java projects ● javadoc ● Use case(BDD) report - net.masterthought::maven-cucumber-reporting ● SchemaSpy (javadoc for RDBMS) ● PlantUML Is project documentation actual? 6/27/19 2010 DB Blue template 39
Igor Suhorukov Continuous code quality in java projects BDD scenarios reports 6/27/19 2010 DB Blue template 40
Igor Suhorukov Continuous code quality in java projects SchemaSpy 6/27/19 2010 DB Blue template 41
Igor Suhorukov Continuous code quality in java projects PlantUml 6/27/19 2010 DB Blue template 42
Igor Suhorukov Continuous code quality in java projects ● Measured technical debt is good argument to ask management for more resources or change project scope. ● Quick project state assessment. ● Focus team attention on most important issues. ● Helps to find untested code. Continuous Code Quality and enterprise project 6/27/19 2010 DB Blue template 43
Igor Suhorukov Continuous code quality in java projects ● Large open source project can use continuous code quality approach on regular basis or occasionally ● Some projects just looks like community friendly but is not in real interactions – too many bureaucracy. ● ML libraries code from scientists developers are very specific and not so frequently follow common code style. ● I’ve cleaned code and fixed some issues in Spring framework, Spring Boot, Elasticsearch, H2Database Continuous Code Quality and open source 6/27/19 2010 DB Blue template 44
Igor Suhorukov Continuous code quality in java projects Conclusion 6/27/19 2010 DB Blue template 45
Igor Suhorukov Continuous code quality in java projects 6/27/19 2010 DB Blue template 46
Thanks! igor.suhorukov@gmail.com github.com/igor-suhorukov

Continuous code quality_in_java

  • 1.
  • 2.
    Igor Suhorukov Continuous codequality in java projects Information from this report is my subjective opinion based on my experience, knowledge, mistakes... ;-) Subjective opinion 6/27/19 2010 DB Blue template 2
  • 3.
    Igor Suhorukov Continuous codequality in java projects Subjective opinion 6/27/19 2010 DB Blue template 3 https://youtu.be/mGiDkLgy7IM?t=279
  • 4.
    Igor Suhorukov Continuous codequality in java projects Why Java? 6/27/19 2010 DB Blue template 4 https://madnight.github.io/githut/#/pull_requests/2019/1
  • 5.
    Igor Suhorukov Continuous codequality in java projects Software functional quality reflects how well it complies with or conforms to a given design, based on functional requirements or specifications. Quality is subjective from end user point of view and is not constant in software development life cycle. ISO/IEC 9126, ISO/IEC 25000:2014, Сonsortium for IT Software Quality(CISQ), Software Quality Assessment based on Lifecycle Expectations(SQALE) Software quality 6/27/19 2010 DB Blue template 5
  • 6.
    Igor Suhorukov Continuous codequality in java projects Software development process 6/27/19 2010 DB Blue template 6
  • 7.
    Igor Suhorukov Continuous codequality in java projects Software development process constraints 6/27/19 2010 DB Blue template 7 Quality Cost Schedule Scope
  • 8.
    Igor Suhorukov Continuous codequality in java projects Metrics measure the quantitative assessment of some property of software or its specification. Metrics usage and holy wars: ● How to choose the right metrics? ● Are metrics set blessed? ● What I need to do with metrics results? Metrics 6/27/19 2010 DB Blue template 8
  • 9.
    Igor Suhorukov Continuous codequality in java projects ● Reliability ● Security ● Maintainability ● Duplications ● Complexity ● Issues/Code smell https://docs.sonarqube.org/latest/user-guide/metric-definitions/ Complexity metrics for software development 6/27/19 2010 DB Blue template 9
  • 10.
    Igor Suhorukov Continuous codequality in java projects Technical debt / big ball of mud 6/27/19 2010 DB Blue template 10
  • 11.
    Igor Suhorukov Continuous codequality in java projects Fragile code and unpredictable application failure after small changes Delayed improvements and miss deadlines Tight coupling code Technical debt 6/27/19 2010 DB Blue template 11
  • 12.
    Igor Suhorukov Continuous codequality in java projects Technical debt is related to new code or bug fixes. Examples: increased code complexity, absence of tests for new code, subsystem or code decomposition issues and spaghetti code . Tech debt as violation of SOLID principles (single responsibility, open-closed, Liskov substitution, interface segregation and dependency inversion). Root cause: dev experience, limited time, team player discipline. Technical debt 6/27/19 2010 DB Blue template 12
  • 13.
    Igor Suhorukov Continuous codequality in java projects Test-driven development (TDD) Behavior driven development (BDD) Performance Test Driven Development Continuous Code Quality Inspection Is it mandatory or recommended only? Depends on – team size, project complexity, outsourcing/in house project, schedule, management culture, team qualification/experience/velocity. Software quality should be part of SDLC 6/27/19 2010 DB Blue template 13
  • 14.
    Igor Suhorukov Continuous codequality in java projects Agile Manifesto Individuals and interactions over processes and tools. Working software over comprehensive documentation. Customer collaboration over contract negotiation. Responding to change over following a plan. Individuals and interactions Software quality should be part of SDLC 6/27/19 2010 DB Blue template 14
  • 15.
    Igor Suhorukov Continuous codequality in java projects Based on functional and non functional requirements: Black/White-box testing Manual/Unit/Integration testing/System testing Mutation testing/Fuzzing Load testing/Stress Testing/Performance testing Usability testing Software quality validation approach 6/27/19 2010 DB Blue template 15
  • 16.
    Igor Suhorukov Continuous codequality in java projects ● Static code analysis just one tool in the box to reach good quality. Helps team to focus on some issues in large codebase. ● Formal verification of software programs. Too difficult to explain specification and limited usage. ● Running dynamic program analysis of software on emulator or real hardware. Time consuming method. Software quality. White box testing 6/27/19 2010 DB Blue template 16
  • 17.
    Igor Suhorukov Continuous codequality in java projects ● Search by template in abstract syntax tree(AST). ● Rice's theorem. Theorem states that all non-trivial, semantic properties of programs are undecidable. ● False positive alerts. ● Nested method invocation. Static analysis constraints 6/27/19 2010 DB Blue template 17
  • 18.
    Igor Suhorukov Continuous codequality in java projects ● IntelliJ Idea Community Edition - code inspections ● PVS-Studio Java free for several projects on github. Too many usage constraints. License key may be revoked in any time. ● SonarJava static analyzer for SonarLint & SonarQube Java code static analyzers 6/27/19 2010 DB Blue template 18
  • 19.
    Igor Suhorukov Continuous codequality in java projects IntelliJ Idea code inspections 6/27/19 2010 DB Blue template 19
  • 20.
    Igor Suhorukov Continuous codequality in java projects IntelliJ Idea code inspections 6/27/19 2010 DB Blue template 20
  • 21.
    Igor Suhorukov Continuous codequality in java projects PSV Studio 6/27/19 2010 DB Blue template 21
  • 22.
    Igor Suhorukov Continuous codequality in java projects SonarLint 6/27/19 2010 DB Blue template 22
  • 23.
    Igor Suhorukov Continuous codequality in java projects https://www.sonarqube.org Community Edition/Developer Edition/Enterprise Edition/Data Center Edition https://sonarcloud.io SonarQube. Сontinuous code quality server 6/27/19 2010 DB Blue template 23
  • 24.
    Igor Suhorukov Continuous codequality in java projects From first day SonarQube. New project 6/27/19 2010 DB Blue template 24
  • 25.
    Igor Suhorukov Continuous codequality in java projects SonarQube. Code smells 6/27/19 2010 DB Blue template 25
  • 26.
    Igor Suhorukov Continuous codequality in java projects SonarQube. Strategy how to use it in legacy project 6/27/19 2010 DB Blue template 26 ● Ignore existing issues, don’t pass new issue in code. QualityGate by default. ● Fix all issue ● Don’t use Sonar ● ?
  • 27.
    Igor Suhorukov Continuous codequality in java projects git clone https://github.com/apache/ignite.git mvn sonar:sonar SonarQube. Project dashboard 6/27/19 2010 DB Blue template 27
  • 28.
    Igor Suhorukov Continuous codequality in java projects SonarQube. Duplicate code 6/27/19 2010 DB Blue template 28
  • 29.
    Igor Suhorukov Continuous codequality in java projects SonarQube. Maintainability 6/27/19 2010 DB Blue template 29
  • 30.
    Igor Suhorukov Continuous codequality in java projects git clone https://github.com/apache/ignite.git mvn sonar:sonar SonarQube. Issues 6/27/19 2010 DB Blue template 30
  • 31.
    Igor Suhorukov Continuous codequality in java projects SonarQube. New language feature inspection 6/27/19 2010 DB Blue template 31
  • 32.
    Igor Suhorukov Continuous codequality in java projects SonarQube. Code complexity example 6/27/19 2010 DB Blue template 32
  • 33.
    Igor Suhorukov Continuous codequality in java projects SonarQube. Issue description 6/27/19 2010 DB Blue template 33
  • 34.
    Igor Suhorukov Continuous codequality in java projects SonarQube. Rules 6/27/19 2010 DB Blue template 34 https://rules.sonarsource.com/java/
  • 35.
    Igor Suhorukov Continuous codequality in java projects ● https://docs.sonarqube.org/display/SCAN/Analyzing+with+Son arQube+Scanner+for+Jenkins ● https://docs.sonarqube.org/latest/analysis/pull-request/ ● https://sonarcloud.io/documentation/analysis/pull-request/ CI/CD integration 6/27/19 2010 DB Blue template 35
  • 36.
    Igor Suhorukov Continuous codequality in java projects ● Black Duck Software ● Sonatype Nexus ● Artifactory ● Looks good to me LGTM Alternatives ● https://www.codacy.com ● https://github.com/marketplace/category/code-quality License compatibility/ known library issues 6/27/19 2010 DB Blue template 36
  • 37.
    Igor Suhorukov Continuous codequality in java projects ● https://github.com/checkstyle/checkstyle https://github.com/spring-io/spring-javaformat/blob/master/src/checkstyle/checkstyle.xml Code style 6/27/19 2010 DB Blue template 37
  • 38.
    Igor Suhorukov Continuous codequality in java projects https://github.com/TNG/ArchUnit-Examples/blob/master/example- junit5/src/test/java/com/tngtech/archunit/exampletest/junit5/DaoRulesTest.java Code structure tests 6/27/19 2010 DB Blue template 38
  • 39.
    Igor Suhorukov Continuous codequality in java projects ● javadoc ● Use case(BDD) report - net.masterthought::maven-cucumber-reporting ● SchemaSpy (javadoc for RDBMS) ● PlantUML Is project documentation actual? 6/27/19 2010 DB Blue template 39
  • 40.
    Igor Suhorukov Continuous codequality in java projects BDD scenarios reports 6/27/19 2010 DB Blue template 40
  • 41.
    Igor Suhorukov Continuous codequality in java projects SchemaSpy 6/27/19 2010 DB Blue template 41
  • 42.
    Igor Suhorukov Continuous codequality in java projects PlantUml 6/27/19 2010 DB Blue template 42
  • 43.
    Igor Suhorukov Continuous codequality in java projects ● Measured technical debt is good argument to ask management for more resources or change project scope. ● Quick project state assessment. ● Focus team attention on most important issues. ● Helps to find untested code. Continuous Code Quality and enterprise project 6/27/19 2010 DB Blue template 43
  • 44.
    Igor Suhorukov Continuous codequality in java projects ● Large open source project can use continuous code quality approach on regular basis or occasionally ● Some projects just looks like community friendly but is not in real interactions – too many bureaucracy. ● ML libraries code from scientists developers are very specific and not so frequently follow common code style. ● I’ve cleaned code and fixed some issues in Spring framework, Spring Boot, Elasticsearch, H2Database Continuous Code Quality and open source 6/27/19 2010 DB Blue template 44
  • 45.
    Igor Suhorukov Continuous codequality in java projects Conclusion 6/27/19 2010 DB Blue template 45
  • 46.
    Igor Suhorukov Continuous codequality in java projects 6/27/19 2010 DB Blue template 46
  • 47.