What I have tended to notice is that critical bugs usually come from issues that deal with shared resources, such as memory, disk usage, cpu usage, etc.
Knowing how the program has been designed (what data structure it uses, what design patterns is being used, etc.) helps with understanding how to get to certain “states”. Here I refer to “states” is referring to a condition or a set of conditions.
If a program uses only one process, it follows that path and is fairly easy to understand. It will branch out on certain conditions so you can take a look at those conditions and see what is accounted for, and also take a look at what is not accounted for in the conditions.
When a program uses multiple processes or threads, it becomes more complex to deal with. In order to simplify what is going on, understanding the conditions is important.
A specific example would be for instance with a mobile application such as a web browser.
A condition would be if the device is in portrait or landscape. Another condition would be if the hardware keyboard is out or not. Another condition may be if another application is running in the background such as something that is running music. The accumulation of all these conditions is what I consider a state.
From this state, if the web browser goes and tries to play a video that has audio capabilities, you test to see what happens. What is expected to happen based on what an end user would expect?
This is where understanding and knowing the application well helps.
Other places where critical bugs may occur is the initialization of things such as memory allocations. Having a profile before hand versus having a clean profile. Installation with a previous version and upgrading versus having no previous versions. A new profile versus an old profile. These are all conditions that should be tested.
Having an outline of conditions that can be reached and doing analysis on the conditions to see possible weak spots is important.
Other things that can be looked at by the code is performance. Looking at the code, one could use the big O notation to figure out to prove why something is more efficient or less efficient. The data structure is important to understand as well, as some things are more suitable for arrays than hash tables, etc. due to memory usage, storage, performance, etc. Understanding why a developer used one method over the other can help the QA person figure out what is designed and a limitation versus what is a bug.
Einstein once stated “Insanity is doing the same thing over and over again and expecting different results.”
Most “low hanging fruit” bugs can be found based on the changes that are made to the application whether it be a new feature or a fix for a bug. The change that occurs has a direct impact and an implicit impact. Knowing what the implicit impact a change has is difficult. Multiple methods can be used to determine the implicit impact, one of the most common ways is to do regression testing which in most cases, test case running accomplishes. It’s a breadth width search for bugs. It is not necessarily the most efficient because it tests all tests including areas that may not have been impacted by the change. In some sense, automation can be used for these test cases to help people concentrate on areas that are more likely to change.
Doing session based testing or some modified version of session based testing leads to a higher yield result, but can lead to more duplicate bugs if the root cause of the bug is properly identified by the QA engineer. Session based testing is most effective when the QA engineer has a thorough understanding of how the application/feature is designed.
All in all, QA to me is an understanding of the risk assessment and to properly report the risk with factual data so that the appropriate people can make the decisions of fixing or not fixing the bugs during the allotted scheduled time.
A good manager once told me “there are 2 departments that are the heart of the company; sales (the people that sell the product) and development (the people that make the product). Every other department is there to support those two departments.” The statement does not diminish the importance of the other departments, but reaffirms the roles that they play and they should help support those two departments in being successful.