I have been writing a series of articles on Bug life cycle and some thoughts are still pending. In this article, I will describe my views on effective bug reports and how the same might help to improve the productivity of software development teams. There are many stories around the web on this topic, but still I would like to add my two cents here.
Objective
The bug reports must help to reproduce the problem with minimum efforts and help the developer to debug and fix the problem. The good bug reports contain most of the information required to stake holders to reproduce, analyze, debug and fix the problem in a given context.
Concerns
There are many instances in my experience where in which the bugs were returned to the testers or the bug reporters seeking more information / clarifications / steps etc.
How often the developers says that the issue is non reproducible at their end and seek clear steps to reproduce the bug?
How often the developers require more information from the bug reporters which is not part of Bug report?
How often do we spend more time on investigating the bug even after the bug has been reported?
How often, we were asked to provide test data for the issues
How often, we were asked for test environment details around the issue
And the list goes on …
We are often a hurry or excitement to report the issue and may provide just basic info and ignore. Some of us are also under the impression that we don’t need to spend so much time of the bug reports and might be waste of time too. Most of the information around the issue will be available only when the issue has been found. It would be tough to provide the same data at a later time.
But in reality, we spend more time even after reporting the problem and try to provide the information asked by stake holders.
The problem lies with the quality of the bug report. We often tend to report the bug with minimum details around the problem and tend to ignore context driven information required for other stake holder to save time.
The Bug Summary
Make it simple and point to the actual problem. Keep in mind that its developer’s initial interaction & the same must grab the attention and points to the actual problem.
Steps to Reproduce
Describe the state of application and the pre-requisites. The flow of user actions and data provided is important. All the steps must be logical and we can remove redundant steps. It’s a good idea to have a relook on all the steps and try it on a fresh system.
Expected Behavior & Actual Behavior
The perceptions and views tend to differ always. It’s good the document the expected behavior and actual behavior of the system under test.
Test data
We provide lot of data to the systems under test at specific user actions. It’s important to list down this data and also capture data around the issue. Provide the following as part of bug report
The input data to the application
Screenshots around the Issue
The Log files
Test Environment
Test Environment plays key role in reproducing the issues and it’s the root cause for many of the non reproducible issues. We must take that extra mile to capture all the environment details like OS, databases, build numbers, browsers. There are no best practices here and we must capture information based on the context.
Bug Tracking Systems
We log bugs in bug tracking tool and assumes that all the things have been taken care. But if try to look at history and pattern of these bugs based releases or environments etc via bug databases, all the issues may not be present. We do miss out many attributes of bug databases that might help us to derive patterns, categorize over specific need in the long term.
Final words
The purpose of a bug report is to showcase that there is a problem that exists and let the developers see the failure. It must defend and advocate the reasons around the problem and finally helps to resolve the bug in the system
It’s been a while since i wrote any thing new on my blog. I would like to thank all the readers of my blog for keeping this live as of today. There were no posts for the last two years and still attracts good number of visitors. I have another blog called Tech Bytes and this just contains my reading list over technology and tools. The Tech Bytes now merged into this blog and will be keep the efforts only here. I have recently came to know that this blog has been listed in the MSDN Tester Center Community Blogs. It’s a great feeling to get listed there.
I am back to blogging and will be sharing my views regularly.I will be focusing on the craft of software testing, quality and development. Lot many things happened over the past two years. Quality is the buzz word now every where.
Almost an year ago, I have evaluated some low cost and open source Test Management Tools. Managing the Test Efforts & Test documentation is always an issue for small and medium companies.
The following were my requirements for Test Management Tool
Capture the Requirements
Design & Prepare Test Cases
Map Test Cases with Requirements
Link Bug reports with Test Case ID after the test execution
Test Execution Reports
Version Management for the Test Cases
Search Feature
Testuff comes with all the above features and some additional capabilities to record the test execution and link them with the issue tracker. It comes free for single user. Check out the testuff blog for more updates
Podcasts are useful resources for testing community. These are free mp3 files you can download and listen to at your convenience. It contains various topics on Software Testing, Quality, & Development. I would like to share some of the podcasts that i follow for my blog readers.
The Testability Explorer is an open source project that lets you measure the testability of Java code. This is an interesting idea: a metric not of direct quality, or of testing, or of test coverage, but of ease of testing. Presumably code that is easy to test will get tested, and will therefore be of higher quality, other things being equal.
The following information over Testability Explorer is useful
I have always challenges with the Software Requirements for the products and would be the same for others too. Most of the time, the requirements talk about just the functional needs and ignore on the non functional aspects of the system. Once the requirements are freezed, we rarely go back to them for any updates required. The development teams always have implementation issues and the behavior changes from the drafted requirements.
Jonathan Erickson, Editor in Chief for DDJ talks about requirements in Software Development projects. It’s interesting article on Requirements & Check this out here – Dr. Dobb’s | Requirements Are Required Reading | May 7, 2008
This leads to more challenges to the testers as we go by the drafted requirements into the documents. The understanding on the non functional needs differs from dev teams to test teams. So, tt’s quite common to receive comments like That’s expected behavior and Not a Bugfrom dev teams.
Interesting trends are happening around Static Analysis and Code Quality. The tools vendors closely looking at each of the code quality metrics and merging them to have a better metric. We can see the pattern from Agitar by coming up with CRAP4J. Here is one more vendor coming up with the tools on Code Quality and Static Analysis.
The AppPerfect comes with a suite of products around code quality metrics and improvements. DevTest4J is a suite of products designed to help you build and test high-performance applications during the development phase of the product life-cycle.
I have come across yet another static analysis tool for Java programs. ENERGY is a free static analysis tool to measure code quality & presents overall status on the health of source code. Energy comes with a code quality metric by combining the bug patterns on the source code and rates it between 1 to 10. The tool tries to identify the symptoms via static analysis and then publish a rating. How it works is an interesting insight.
Check the following video to learn more about the tool.
The following articles are helpful to improve code quality in general & more specific to java applications. This is an attempt to list down all the articles in the blog which are around Code Quality.
The term quality might mean different things to different people based on their context of operation and it’s tough to have universal definition for the same.My experiments with software development started over a decade ago & exploring the ways the helps to build & deliver good quality code.The quest towards Code Quality Metrics started with the above objectives.
have been talking about Code Quality through Static Analysis for a while here, here & here. The quality for any application development must begin from it’s gross roots and the Application Code is one of it’s starting point to begin with.Static Analysis techniques help us to identify some metrics over the application code base
The applications become complex as their code base increases. This has challenges for the testers to determine the nonlinear paths in the application.Most of the Static Analysis tools over the application code helps us to identify the cyclomatic complexity (nonlinear paths) at a method level. These might be helpful to validate those methods and to achieve good code coverage over the same.
Cyclomatic Complexity (CC) is a software metric (mostly code based metric) used to the number of independent path executions in the application. Introduced by Thomas McCabe in 1976, it measures the number of linearly-independent paths through a program module. It helps the developers to determine the independent path executions and base line unit tests that they need to validate. Using this, the developers can assure that all the paths have been tested atleast once. It’s a great comfort for the developer and their respective managers.
JavaNCSS is a command line utility which measures some standard source code metrics for the Java programming language. The metrics are collected globally, for each class and/or for each function.
Lines of Code (LOC) is one of the software metric that is used by most of the people for Software Measurement. Thousand lines of code is treated as KLOC. This metric helps us in knowing the size and complexity of the Software Application. Click here for more details on KLOC. This info is from wikipedia
In this post, I will be sharing my views on Static Analysis and how the same is useful for Software Testing. Using Static Analysis in Testing will also be called as Static Testing.
Come across this cool video aboutUsing Static Analysis For Software Defect Detection on Google Tech Talks. William Pugh talk about how can we use Static Analysis for defect detection and his tool FindBugs which is Static Analysis tool for Java Applications.
Static Analysis is the technique used to analyze the software without executing the same. It helps to look at issues that are there with in the Source Code even before it gets executed. Find some reading on Static Code Analysis here from Wikipedia.
