Overview

Open Source testing tools are being used aggressively and there are many tools over the web for Java Applications. These tools are great asset to the development teams and provides many features / services over the quality of applications under development. We can use this as the starting point and focus more efforts on the uncovered areas for better quality.

It’s very important to know the context of development and the right tools for quick benefits. However, heavy usages of these tools lead to misleading data and metrics too. So think twice on what is required for the project and which tools might provide the required data / services.

Services Offered

The following services / features are offered from most of the open source tools

• Continuous Integration for the builds
• Unit Testing
• Code Coverage
• Source Code Metrics (like size, Complexity, design, dependency)
• Static Analysis for various bug patterns
• Test Automation
• Performance Testing

List of Open Source Tools

• Functional Testing
• Unit Testing
• Performance Testing
• Test Management
• Security testing
• Bug databases
• Link Checkers
• Open Source Testing Tools from Java Source
• Web Testing Tools

Quick Benefits from Static Analysis

It’s very easy to integrate static analysis tool(s) with builds and there quick benefits. The following issues can be uncovered

• Null Pointer Exceptions
• Other un handled exceptions
• Infinite Loops
• Dead code
• Compliance with Java Coding standards
• Code Coverage data
• Trends  / history on the above checks against previous builds
• Validations against Java coding guidelines from Sun

The following Metrics can be captured

• Source Code Metrics
• Coverage Metrics
• Dependencies with the Design
• Code complexity metrics

The Resources listed below helps in implementing the same.

• Series of articles listed under In Pursuit of Code Quality from IBM Developer works
• Discussion Forum on Improve Your Java Code Quality from IBM Developer works

The term quality might mean different things to different people based on their context of operation and it’s tough to have universal definition for the same.My experiments with software development started over a decade ago & exploring the ways the helps to build & deliver good quality code.The quest towards Code Quality Metrics started with the above objectives.

The discussion around Useful Code Quality Metrics at Linkedin started two months ago. I am excited over the response around the internet to this thread & there are some excellent articles around subject.

• Micheal Boltan has brought up interesting view points and also bloged about the same.
• Nigel Cheshire talks about Static Analysis metrics and expressed views on the right metrics
• Andrew Glover calls it as Code Quality’s singular metric

In the recent past Agitar comes up with CRAP4J as a code quality metric by combining Cyclomatic Complexity and Code Coverage.

In the process, we do use Static Analysis and Dynamic Analysis to derive the above mentioned metrics. Some more good articles around Code Quality are here

• In pursuit of Code Quality series
• How useful are Code Metrics ?

I wish that the info around Code Quality is helpful for my blog readers

Happy Testing…

(more…)

I have been talking about Code Quality through Static Analysis for a while here, here & here. The quality for any application development must begin from it’s gross roots and the Application Code is one of it’s starting point to begin with.Static Analysis techniques help us to identify some metrics over the application code base

• Cyclomatic Complexity
• Application Design & Dependency Metrics
• Exception Handling
• Infinite Loops
• Dead Code
• Performance Issues
• Programming Language guidelines & best practices in their context

The above information will be useful and it’s easy to get the same via Test Framework with a single click.

Code Coverage along with a bunch of unit tests is one widely used technique to help regression testing for the dev & test teams.

Now it’s good to see that Agitar combines Code Coverage & Cyclomatic Complexity to derive risk metrics for Java Code Base. They call it as a code Change Risk Analyzer and Predictor (i.e. CRAP) for Java.

Though it’s a prototype and see how the industry receives the same, i see it as a good initiative on the code quality front and might be a metric going forward for white box testing.

Some useful links for CRAP4J

• Introducing CRAP4J
  
• The Code C.R.A.P. Metric Hits the Fan – Introducing the crap4j Plug-in
• Junit Factory’s Code C.R.A.P metric

Update on Oct 23rd

Now we have a dedicated site for CRAP4J. This contains latest news, forum discussions and many more. I would say that this a good resource for Code Quality lovers & the good news is that they are designed to be open source tools.

Crap4j is a Java implementation of the CRAP (Change Risk Analysis and Predictions) software metric – a mildly offensive metric name to help protect you from truly offensive code.

The CRAP metric combines cyclomatic complexity and code coverage from automated tests (e.g. JUnit tests) to help you identify code that might be particularly difficult to understand, test, or maintain – the kind of code that makes developers say: “This is crap!” or, if they are stuck maintaining it, “Oh, crap!”.

The best way to learn more about CRAP and Crap4j is to check the various articles, newsgroups and blogs about them.

(more…)

The applications become complex as their code base increases. This has challenges for the testers to determine the nonlinear paths in the application.Most of the Static Analysis tools over the application code helps us to identify the cyclomatic complexity (nonlinear paths) at a method level. These might be helpful to validate those methods and to achieve good code coverage over the same.

But the Code coverage at a Unit Level may not be a big help since most of the end user scenarios won’t run after unit level paths. These paths is an integration of the above unit level paths.

Since the Testers focus on simulating the end user scenarios, it will be good to identify all the possible nonlinear paths around the application code base and capture the code coverage based on these paths.

You might want to go through some discussion around this on Linkedin Answers

In case you have similar experiences over white box testing drop me a mail at venkatreddyc@gmail.com

Happy Testing…

Cyclomatic Complexity (CC) is a software metric (mostly code based metric) used to the number of independent path executions in the application. Introduced by Thomas McCabe in 1976, it measures the number of linearly-independent paths through a program module.

It helps the developers to determine the independent path executions and base line unit tests that they need to validate. Using this, the developers can assure that all the paths have been tested atleast once. It’s a great comfort for the developer and their respective managers.

It’s better to write JUnit Tests for all these linearly-independent paths and integrate it with any code coverage tool. These reports help to focus more on the un covered paths and improve the code coverage.

It also helps to evaluate the risk associated with the application. The following are the results published by SEI and they are being followed widely to determine the health of the code base.

Cyclomatic Complexity Risk Evaluation
1-10 A simple program, without much risk
11-20 More complex, moderate risk
21-50 Complex, high risk program
Greater than 50 Un testable program (very high risk)

Explore more at Cyclomatic Complexity in Software Technology Roadmap from SEI.

Further Reading on the topic

• Cyclomatic Complexity from Wikipedia
• McCabe Metrics
• Code Improvement Through Cyclomatic Complexity
• Cyclomatic Complexity from High Integrity Software Systems Assurance
• Project Metrics Help – Cyclomatic Metrics

Use metrics to evaluate the risk early in the cycle & improve your test coverage.

Happy Testting …

(more…)

I have been working on Java Projects for quite some time and interested in exploring the source code metrics like size and complexity across project not specific to perticular package or class and looking after LOC (Line of Code) counters to capture some metrics of the source code.

Explored some tools and found that JavaNCSS is a good match for the context of sizing metrics on Java. JavaNCSS is a command line utility which measures some standard source code metrics for the Java programming language. The metrics are collected globally, for each class and/or for each function.

The following are some of the advantages that i have seen

1. Support for Ant Tasks, so easy to integrate with build process
2. Reports can be in Text, XML, HTML etc
3. Support for Stylsheets and easy to get nice HTML reports
4. Metrics at each level Package / Class / Method
5. Cyclomatic Complexity Number
6. List the number of packages / classes / functions / LOC counter at each level

Further Reading:

1. JavaNCSS Home
2. LOC Counters for C++ / Java on Joel Software Discussion group
3. SLOC on Wikipedia

In this post, I will be sharing my views on Static Analysis and how the same is useful for Software Testing. Using Static Analysis in Testing will also be called or simulated as Static Testing.  Static analysis / testing a method to validate the software without executing the same.

The generic definition for Static Testing is that

Testing of an Application at the specification or implementation level without execution of that Application.

Static Analysis helps a lot for better Code Quality and we can perform the following set activities against the source code.

1. Identify the bug patterns in the source code
2. Capture Metrics
3. Complexity of the System (eg. Cyclomatic Complexity)
4. Compliance against Standards (eg. check for the Java Coding Guidelines)
5. Exception Handling (Captures the code that contains Unhandled exceptions)
6. Infinite Loops
7. Copy Paste Dectors (Most of the developers used to reuse the code via a copy paste and introduce some issues in the process)
8. Duplicate Code
9. Dead Code

We should be able to restrict most of the above said issues at very early stage of SDLC by applying Static Testing techniques and make these part of the build process.

Static Analysis is the technique used to analyze the software without executing the same. It helps to look at issues that are there with in the Source Code even before it gets executed. Find some reading on Static Code Analysis here from Wikipedia.

Static Analysis also used to publish some of the Software Metrics. For example, KLOC, Cyclomatic Complexity will be captured through Static Analysis.

The following are some the links to Static Analysis Tools

1. List of Tools from wikipedia
2. List of Tools from Testing FAQs

(more…)