ICS 121: Issue Tracking

Software always has defects. A lot of defects. Too many defects to keep track of without a special tool.
Software testing and repair is a team activity. The tracking must be done with a collaborative tool.
Each issue contains significant information
- There can be debate about what the exact problem really is, and how it should be addressed.
- Evidence (e.g., error messages, debugging output, screen shots) must be gathered to document the problem.
- Later, this information must be available to explain why changes were made.
Software engineering means multiple people working on the product at the same time. There needs to be a clear indication of who is responsible for each unit of work.
Defects are not the only work items. The team must also track enhancement requests and other development tasks.
Managers and developers need to understand everything that must still be done to reach a release.
Changes to a release branch must be motivated by documented needs.
Facilities 1
Facilities 2

An issue tracker is like a client-server database, plus collaborative features
- Central database server with all information about all issues
- Users use clients to access the database. Can be web-based or desktop applications.
- Users become stakeholders in a particular issue and are notified when changes occur
It is where everyone enters issues and tasks
It is where managers assess project status and set priorities on tasks
It is where developers look to see what they should work on each day

Error: v. The mistake in the developer's mind. The mistaken idea that they had when they did the design or code. Often caused by miscommunication or bad assumptions.
Defect: n. The result of the developer's error embodied in the product source code or documents.
Fault: n. The execution of defective code. This is not directly visible.
Failure: n. The user-visible result of the fault. E.g., an error message. This is evidence that can be used in debugging.

Enter new issue
Review/Validate issue (reproduce the problem)
Check to see if this issue is a duplicate
Triage: set milestone, priority, assign to developer, comment on plan of action
Developer provides estimate of effort needed
Developer marks issues as STARTED
Developer comments on progress, requests more information from reporter, gathers evidence
Developer thinks the problem is resolved
QA team verifies that the issue is really solved
QA team implements regression test to automate future verification
Technical docs team summaries verified issues in release notes, updates documentation
Technical support advises users of known issues and work-arounds
Management reviews lists of open issues for each release
Management reviews workload/backlog for each developer
Management reviews individual issues
Management slips issues to a later release, or decides to drop them

Pending issue states:
- Unconfirmed: Entered by outsider, not yet checked for validity
Open issue states:
- New: Valid issue, but no work done yet
- Started: Developer has started work on the issue
- Reopened: The fix failed, work must be redone
Closed issue states:
- Resolved: The developer thinks they have solved the problem
  - Invalid: The reporter was mistake. E.g., user error or misunderstanding.
  - Duplicate: This issue has already been reported before
  - Fixed: The defect was repaired
  - Won't fix: Team agrees not to fix the defect
  - Later: This is already fixed in a later release, user must upgrade
  - Remind: Team cannot fit this into current roadmap, reconsider it later
- Verified: QA has verified that the developer's fix worked
- Closed: All work on this issue has been completed (including docs, release notes, etc.)

Forecast development milestone dates:
- Number of requested changes in a release
- Estimated vs. actual effort
- Rate of defects introduction per requested change
- Defect detection rate: from testing or reviews
- Defect resolution rate
- "Bug charts": projection of date when release has zero know defects
Risk analysis and SPI:
- Total issues / Total code size * 1000 = average defects per KLOC
- Components with many defects are "at risk" and may need redesign
- Identify common root causes of defects, e.g., 10 buffer overflow security holes
- Gather/adjust coefficients for future schedule estimates
Highlighting missed areas:
- Some defects indicate poor requirement or design
- Shipped defects indicate holes in test suite
- Reopened issues indicate mis-communications/misunderstandings

In this release we request 10 new features and 30 enhancements
Total estimated effort on those 40 is 400 hours
Work on those 40 changes introduces 60 defects
20 out of 60 defects were caught in review
The defects are being found in testing at about 10 per day, only 1/2 will be found
The defects are being found in by customers at about 1 per day
The defects are being fixed at about 3 per day
The 60 repairs introduce 12 new defects
Of the 72 repairs, 4 fail and must be redone
Actual effort on the 40 requested changes is 600 hours
Actual effort on the 60 defects is 320 hours