A wise man once said…

“Just talking does not resolve issues.”

“Less you have to maintain, the more you are free to do other things.”

Exploratory testing continued.

I came to realize for someone that’s starting saying “exploratory testing” is kind of an ominous blank page.

The essence is that you are exploring each combination of things that you can do and map them as you go along… how they work, how they function.  Like any person that gets lost, it’s good to stop and ask for hints in terms of how things are built.  That helps you find bugs faster.

The more you know about how the architecture lies the more you can explore.  Ie if someone hands you a map, don’t you think you’ll be able to get to point B from A quicker?  So of course, asking the dev for more info will help.  Do you necessarily need it?  Eventually you can read the code and figure stuff out on your own, but that will take more time. So yes, I will ask dev questions because time is a limiting factor when you have a target dates… and that’s what I did during work weeks and such.  Sometimes they help me come up with more or better test cases, just from the discussion of their area and something I wasn’t aware of before talking to them.  So a QA doesn’t work in isolation from people, even though they can.

Better testing means breaking testing into areas, and sub areas, esp where things have changed in the code… and being able to go fine grain down the alley ways and hidden passages that the dev might help uncover.

I want to get away now from saying exploratory testing because it’s too vague… I suggest that we start using a more specific type of exploratory testing : Session based testing

I highly recommend please read about session based testing.  It’s a method that works for top bug finding testers.  I use a modified version of it:

http://en.wikipedia.org/wiki/Session-based_testing

Example of Session based testing

see https://bugzilla.mozilla.org/show_bug.cgi?id=830657#c23

30 minutes.  Tested a fix.  found 7 potential bugs.

If you want agile testing… there you go.

Now I just have check and verify they aren’t reported already… and file them if they aren’t…

Walk the walk…

I realize now that I have to prove what I have stated in terms of not just bugs, but critical bugs and showing that I can find them faster…

Otherwise, I’m just blowing smoke.  I get it.  I’ll show the proof within the next few weeks.  (hopefully)

Raising the question:

Per version of the product per run, could you graph the output of how many bugs a test case produces?

Reason I ask is how can we tell the effectiveness of a test case, and what are the best test cases to run?

Personally I would keep all the positive test cases in a need to run basis for each stage of a project and hopefully automate them.  negative test cases… probably the high risk ones.  ie if it breaks then the project would suffer bad news by consumers…  hopefully they’re automated too…

The rest?  not so sure.  In a release cycle you have limited time and one should consider using it effectively if you want to make a deadline.

IMO, this should mean that you want to run as effectively as possible to uncover bugs as quickly as possible.  Risk analysis based on how things are coded, how things work, how things are designed, how long things take to implement… etc… these things should be analyzed to try to hit the high risk areas first.

When running test cases, and they aren’t effective in finding bugs… I would hope they would be effective in determining some thing like a positive result, or you just may be wasting time running it.

From “http://www-archive.mozilla.org/xpfe/issues.html”:

At 2:44 … you can hear this said : http://www.youtube.com/watch?v=vcfaKCWflL4

1. If you see a snake, don’t form a committee, don’t send a memo – kill the snake.
2. Don’t play with dead snakes.
3. All opportunities start out looking like snakes.

 

How to root your Galaxy S to Cynogenmod 10 Nightly…

Disclaimer: there is a chance that you may mess up your phone.  I am warning you ahead of time that you are doing this at your own risk and that I might not be able to help you!

I didn’t bother backing up… you may want to do so :

http://www.theandroidsoul.com/cyanogenmod-10-cm10-samsung-captivate-i897-upgrade-jelly-bean-android-4-1/

1. You will need Android SDK installed (follow the instructions : http://developer.android.com/sdk/index.html )

2. install heimdall ( https://developer.mozilla.org/en-US/docs/Mozilla/Boot_to_Gecko/B2G_build_prerequisites#Installing_heimdall )

3. download Hardcore-speedmod ( http://people.mozilla.com/~nhirata/cynogenmod_galaxy_s/Hardcore-speedmod-i9000.zip ) and unzip it

4. attach the device to the computer, and hold down volume + home button to get to the download

5. You can use some sort of UI for heimdall, I used the command line :

heimdall flash –kernel zImage-i9000

[props to : http://www.mydaily-gadget.com/how-to/9487-install-clockworkmod-recovery-samsung-galaxy-s-heimdall-suite ]

6. once it downloads, reboot the phone if it hasn’t already and hold both volume buttons + power button while still plugged in.

7. once clockwork mod launches, it will allow you to adb into the device.  make sure you have write access to the sdcard by adb shell and then doing a ls sdcard.
If you don’t have write access, chances are that the sdcard is not mounted, and you will have to go through the clockwork mod options to mount the sdcard.

8. download the latest nightly cynogenmod 10 for the galaxy s : http://www.cyanogenmod.com/devices/samsung-galaxy-s

9. push the zip file to the sdcard and rename it update.zip

10. using the clockwork mod select the file to use as the update.

11. the phone will reboot near the beginning of the update process.  don’t panic.  The clockwork mod just got updated in the process.

12. wait til the phone reboots again, and you should be in cynogenmod 10 nightly.
[Note: if you find there needs to be steps added or what not, please feel free to comment]