Software Feature Testing

I came to the software test group on a one year loan to assist in a comprehensive analysis of requirements and help out with general regression testing. I was assigned to the voice networking group which had responsibility for station features such as call forward, hold, park, hunt, pickup, conferencing etc, locally and over a mixed network.

A PBX is a feature rich device in which many different devices using different code written at different times, interact with each other in different states. The interactions can vary based on the way devices and features are configured and the same device can be configured many different many ways. It's these feature interactions where defects are commonly found.

I was handed some existing test plans/specifications to execute and assigned a lab schedule, and test quota. It quickly became apparent I was good at finding bugs and isolating them to reproducible scenarios, not just the routine bugs but the flakey, sporadic type bugs.

When performing the tests, I noticed that most defects could not be exactly correlated to a numbered test case. What I would do was take an individual test and before marking it as having passed, try to test as many related usage scenarios as I could dream up before moving on to the next test case. In this manner, I was able to find far more bugs than could be found by simply executing the test as written and then moving on to the next test case.

Defects manifested themselves in many different ways. A tone might be incorrect, distorted or not present. An indicator lamp might or might not light up. The switch might reset. Speech paths might be one way, no way or garbled. A call might not complete or be dropped. An error message might appear on the administration console. A word on a display might be spelled incorrectly.

When I identified an issue I was unsure about, I would make sure I knew how to reproduce it. Then I would makes notes about it in my logbook. Sometime between lab shifts, I would go to the appropriate developer and discuss the issue. Sometimes I would be told it works as designed. Other times, It might be a known bug and there was no need to open an error report. Other times, it would be a previously unknown bug and I would be asked to open an error report. When writing up the error report, I always tried to include a detailed description as to how to reproduce the bug. I also indicated which test failed and which tests were blocked by the bug.

When the bug was finally fixed in source code, I would retest it. If it passed, I would unblock the previously blocked tests, mark the previously failed test as passed and close the error report. Sometimes, an attempted fix failed. In these cases, I would talk to the developer and decide if this was a new bug or part of the old one. If it was part of the old bug, I would leave the error report open. If it was a new bug I would close the old error report and open a new one with updated information.

As my "on loan" year approached an end, I began reflecting upon my prior roles within Siemens. prior to being in the test group, I never really had the opportunity to develop an understanding of telecommunications. In the test group, I learned a lot and enjoyed what I was learning. When the 12 months were up, I elected to remain. There really wasn't much of a problem in my remaining in test since my old development group had been moved away from Boca Raton and all the groups I previously worked in to that point; development, startup and verification had the same manager.