Effective Bug Reports
Objective
The bug reports must help to reproduce the problem with minimum efforts and help the developer to debug and fix the problem. The good bug reports contain most of the information required to stake holders to reproduce, analyze, debug and fix the problem in a given context.
Concerns
There are many instances in my experience where in which the bugs were returned to the testers or the bug reporters seeking more information / clarifications / steps etc.
- How often the developers says that the issue is non reproducible at their end and seek clear steps to reproduce the bug?
- How often the developers require more information from the bug reporters which is not part of Bug report?
- How often do we spend more time on investigating the bug even after the bug has been reported?
- How often, we were asked to provide test data for the issues
- How often, we were asked for test environment details around the issue
- And the list goes on …
We are often a hurry or excitement to report the issue and may provide just basic info and ignore. Some of us are also under the impression that we don’t need to spend so much time of the bug reports and might be waste of time too. Most of the information around the issue will be available only when the issue has been found. It would be tough to provide the same data at a later time.
But in reality, we spend more time even after reporting the problem and try to provide the information asked by stake holders.
The problem lies with the quality of the bug report. We often tend to report the bug with minimum details around the problem and tend to ignore context driven information required for other stake holder to save time.
The Bug Summary
Make it simple and point to the actual problem. Keep in mind that its developer’s initial interaction & the same must grab the attention and points to the actual problem.
Steps to Reproduce
Describe the state of application and the pre-requisites. The flow of user actions and data provided is important. All the steps must be logical and we can remove redundant steps. It’s a good idea to have a relook on all the steps and try it on a fresh system.
Expected Behavior & Actual Behavior
The perceptions and views tend to differ always. It’s good the document the expected behavior and actual behavior of the system under test.
Test data
We provide lot of data to the systems under test at specific user actions. It’s important to list down this data and also capture data around the issue. Provide the following as part of bug report
- The input data to the application
- Screenshots around the Issue
- The Log files
Test Environment
Test Environment plays key role in reproducing the issues and it’s the root cause for many of the non reproducible issues. We must take that extra mile to capture all the environment details like OS, databases, build numbers, browsers. There are no best practices here and we must capture information based on the context.
Bug Tracking Systems
We log bugs in bug tracking tool and assumes that all the things have been taken care. But if try to look at history and pattern of these bugs based releases or environments etc via bug databases, all the issues may not be present. We do miss out many attributes of bug databases that might help us to derive patterns, categorize over specific need in the long term.
Final words
The purpose of a bug report is to showcase that there is a problem that exists and let the developers see the failure. It must defend and advocate the reasons around the problem and finally helps to resolve the bug in the system
Open Source Testing Tools for Java Applications
Overview
Open Source testing tools are being used aggressively and there are many tools over the web for Java Applications. These tools are great asset to the development teams and provides many features / services over the quality of applications under development. We can use this as the starting point and focus more efforts on the uncovered areas for better quality.
It’s very important to know the context of development and the right tools for quick benefits. However, heavy usages of these tools lead to misleading data and metrics too. So think twice on what is required for the project and which tools might provide the required data / services.
Services Offered
The following services / features are offered from most of the open source tools
- Continuous Integration for the builds
- Unit Testing
- Code Coverage
- Source Code Metrics (like size, Complexity, design, dependency)
- Static Analysis for various bug patterns
- Test Automation
- Performance Testing
List of Open Source Tools
- Functional Testing
- Unit Testing
- Performance Testing
- Test Management
- Security testing
- Bug databases
- Link Checkers
- Open Source Testing Tools from Java Source
- Web Testing Tools
Quick Benefits from Static Analysis
It’s very easy to integrate static analysis tool(s) with builds and there quick benefits. The following issues can be uncovered
- Null Pointer Exceptions
- Other un handled exceptions
- Infinite Loops
- Dead code
- Compliance with Java Coding standards
- Code Coverage data
- Trends / history on the above checks against previous builds
- Validations against Java coding guidelines from Sun
The following Metrics can be captured
- Source Code Metrics
- Coverage Metrics
- Dependencies with the Design
- Code complexity metrics
The Resources listed below helps in implementing the same.
- Series of articles listed under In Pursuit of Code Quality from IBM Developer works
- Discussion Forum on Improve Your Java Code Quality from IBM Developer works
Exploring the craft of Software Testing
I am back to blogging and will be sharing my views regularly.I will be focusing on the craft of software testing, quality and development. Lot many things happened over the past two years. Quality is the buzz word now every where.
Testuff – The Test Management Tool for small and medium projects
The following were my requirements for Test Management Tool
- Capture the Requirements
- Design & Prepare Test Cases
- Map Test Cases with Requirements
- Link Bug reports with Test Case ID after the test execution
- Test Execution Reports
- Version Management for the Test Cases
- Search Feature
Testuff comes with all the above features and some additional capabilities to record the test execution and link them with the issue tracker. It comes free for single user. Check out the testuff blog for more updates
Software Quality and Testing Podcasts
- Podcasts from Stickyminds
- Randy Rice’s Podcasts
- Testing & QA Podcasts from SD Times
Explore the Testability aspect for Java Code
The following information over Testability Explorer is useful
Software Requirements are Required Reading by project teams.
I have always challenges with the Software Requirements for the products and would be the same for others too. Most of the time, the requirements talk about just the functional needs and ignore on the non functional aspects of the system. Once the requirements are freezed, we rarely go back to them for any updates required. The development teams always have implementation issues and the behavior changes from the drafted requirements.
Jonathan Erickson, Editor in Chief for DDJ talks about requirements in Software Development projects. It’s interesting article on Requirements & Check this out here – Dr. Dobb’s | Requirements Are Required Reading | May 7, 2008
This leads to more challenges to the testers as we go by the drafted requirements into the documents. The understanding on the non functional needs differs from dev teams to test teams. So, tt’s quite common to receive comments like That’s expected behavior and Not a Bug from dev teams.
Trends in Static Analysis Tools and Code Quality Metrics
The AppPerfect comes with a suite of products around code quality metrics and improvements. DevTest4J is a suite of products designed to help you build and test high-performance applications during the development phase of the product life-cycle.
Trends in Static Analysis – ENERGY for better java code
Check the following video to learn more about the tool.
Technorati Tags: Code Quality Metrics, Java, Static Analysis, Code Quality.
Communicating the Value of Testing Throughout the Organization
Test managers constantly lament that few outside their group understand or care much about the value they provide and consistently deliver. Unfortunately, they are often correct. The lack of visibility and understanding of the test team’s contribution can lead to restricted budgets, fewer resources, tighter timelines, and ultimately, lower group productivity. Join Theresa Lanowitz and Dan Koloski as they highlights ways to move from simply being a tester of software to an advocate for your organization’s customers. Learn how to effectively and concisely communicate with key stakeholders in your organization to ensure that they understand the value and role of the testing group. With effective and concise communication, the testing group will be perceived as more strategically important and integral to the success of every project.
Communicating the Value of Testing Throughout the Organization
