Tester’s attitude

As Chris Gardner says in the movie ‘Pursuit of Happyness’ during an interview in a stock broker firm:

‘I am a type of person if you ask me a question and I don’t know the answer, I am going tell you that I don’t know. But I bet you, I know how to find an answer. I WILL find the answer.’

That moment I felt that this is how a tester’s attitude should be.

Testing is such a subject that no tester can have a perfect answer.

A tester must possess qualities like exploring ways to find possible answers and one should be eager to learn new things to get to the solution.

All Oracles are model but not all models are oracle

Yesterday I was demonstrating the installation testing to my students. One of the student observed that in the installation process ‘Custom’ option does not provide the ‘delete’ folder facility. He raised this as an issue. After questioning why he thinks that this is an issue, I came to know that actual software behavior was conflicting with his mental model. He modeled the regular windows explorer in which create or delete folder options are given in Windows OS.

I wanted him to think ‘Why is ‘Delete’ option not provided?’ One of the reasons could be that while installing software, default location of any program is C:\’program files’ and there are high chances that a user may delete an important file.

As I said a mental model was conflicting with actual working of software that means it helps a tester to find potential problem. Before we decide whether mental model helped him to raise the issue or not, first let’s look at the definition of Oracle –

|| Oracle – An Oracle is a heuristic principle or mechanism by which you recognize a potential problem. (Definition taken from BBST course by Cem Kaner)

For example your Oracles in your project can be Requirement, Specification or authority document, with the help of these documents tester raises bug against of software under test.

Now as we know that most of the time, requirements are not complete or it changes frequently so they may not help us to raise an issue all the time. This is heuristic nature of oracle.

What is Heuristic?

|| Heuristic is a fallible method which helps us to solve the problem.

It means when you would use any heuristic and with the help of heuristic you finds an issue in your software under test, then your heuristic becomes your Oracle (you may call a reference program).

So it means ‘All Oracles are heuristic but not all heuristics are oracle’.

(If you want to understand in very simple language above statement then please read Michael Bolton’s post on “All Oracles are heuristic”)

What is a model? (These are dictionary meaning of a word ‘model’)

|| a three-dimensional copy of a person or thing.

||something used as an example

||an excellent example of quality

||a particular version of a product

In our computation world external models are Flow Charts, Graphs, Decision flow diagram or UML diagrams.

Internal models can be tester’s mental model. They can be workings of different or similar software.

Models can be incorrect. So think before trusting too much on your models. They might be wrong because they are developed on your assumptions.

Let’s go back to the problem which I have mentioned at the start of the post. My student actually felt that his mental model was correct. We compared with two more different installations of software and observed that none of these manufacturers provided ‘delete’ folder option. We concluded that our mental model might be wrong because we did not get any example where while installation ‘delete’ folder option is provided.

If something does not guarantee to solve the problem is called a heuristic. I can say that models are heuristic in nature because they help us to solve the problem but do not guarantee solution all the time. Model may be incorrect because they are built on assumptions. And we human can’t question everything.

It means models are heuristic in nature. They are fallible but they may help us to solve the problem.

By default every tester does model based testing, comparing her mental model with the actual software workings.

If we can say that “All Oracles are heuristic but not all heuristics are oracle”.

Similarly I would say that “All Oracles are model but not all models are oracle”

Test Technique, Method, Type or Style

Recently I had a discussion with one of my colleague on ‘Boundary analysis is a technique and Function testing is a type’. I knew that there is no point of forcing my view point onto her because she was very strong in her opinion.

I told myself that what would make a difference if I would call a Function testing technique as function testing type.

Let’s see what all these words tell us.

|| Technique: A special way of doing something

||Method: A way of doing something, an idea or ideas that specify behaviour

||Type: a kind or category

||Style: a way of doing something

What does it tell us? Isn’t it that all these words talk about a specific behaviour of doing something? The way of doing something needs to be different from each other. They can’t be identical.

Now if I would call function testing as technique, type or method then that does not make any difference. It does not change the way I do function testing.

However I wonder what makes testers to think that function testing is a type but not a technique. Or Boundary analysis is a technique but not a method or type.

Does technique gives meaning or sense of being technically skilled to do Boundary analysis? What about Security or Usability testing?

I thought of looking into the ISTQB syllabus to see what is given as test technique. When I filtered out techniques from ISTQB Glossary then to my surprise I came to know that Function, Security, Usability, Stress or Load etc testing does not come under test technique (Or type, method or style).

I failed to understand why only 7 test techniques were mentioned in ISTQB syllabus?

Below techniques are from Advance ISTQB Syllabus 2007:

• Equivalence partitioning
• Boundary value analysis
• Decision tables
• State Transition Testing
• Classification Tree method
• Pairwise testing
• Use cases

(Note – There might be few more mentioned in the syllabus which I might have overlooked)

When I look back to my discussion with my colleague then I understood ‘why did not she call Function testing as a technique?’

I think calling function testing a type or technique will not make any difference. It does not change the way I do function testing.

Tester need to question before accepting anything in testing. This helps tester to get more clarity while applying them day to day’s testing.

I strongly feel that clarity brings effectiveness in your actions!

Testing definitions

I have observed that most of the testers look for ‘testing’ related definitions, they look for fixed answers. I think everyone can’t remember definitions. Especially people like me find it difficult to remember it.

I see lot of questions asked by testers to know the testing definitions in groups like LinkedIn “Quality Assurance” group, Interview questions forums and Google groups.

Hold on for a moment.

Before you look out testing terminology and mugging up those definitions, do you really think that you want to learn those definitions which you really can’t use in real in your testing? Do you use those words in your practice? Do you know the original resources of those definitions?

There are two objectives of finding these testing terminologies first to clear certifications and other one to crack job interviews.

I wonder why testers do not want to THINK on definitions that matters to them and something what they understand gives meaning to the terminology and define its meaning. One should know those terms through practice not through mugging from various sources.

Let’s take an example to see how software testing is diverse in nature.

One of the Software testing interviews you must have experienced the questions like “What is Function Testing?”

Actually the confusion starts from: should I call “Functional Testing” or “Function Testing” or “Testing of Function”

Anyways first of all see how many definitions are available in different books or certification syllabus.

|| “Function testing is a process of attempting to find discrepancies between the program and the external specification.” – The art of Software Testing 2^nd Edition (Glenford J Myers)

|| “The term functional testing is also used for behavioral testing. Behavioral testing can, in principle but not in practice, be done in total ignorance of how the object is constructed. Behavioral testing is also called black-box testing.” – Black Box Testing Techniques for Functional Testing of Software and System (Boris Beizer)

|| “Functional Testing is one type of black box testing. Functions are tested by feeding them input and examining the output. Internal program structure is rarely considered.” – Testing Computer Software 2.0 (Cem Kaner, Jack Falk, Hung Quoc Nguyen)

|| “The functions that a system, subsystem or component are to perform may be described in work products such as a requirements specification, use cases, or a functional specification, or they may be undocumented. The functions are “what” the system does.” – ISTQB foundation level syllabus (version 2011)

To understand each definition one should know what are the terms used in those definitions according to the authors understanding.

Let’s not consider all these definitions for a while. You can find out your own meaning for those terms. I think it’s quite easy in software testing rather than in other fields. It requires your willingness to think and passion to learn.

What is definition?

A definition (≝) is a passage that explains the meaning of a term (a word, phrase or other set of symbols), or a type of thing. A term may have many different senses or meanings. For each such specific sense, a definiens is a cluster of words that defines that term. (Source is Wikipedia)

We look for a specific meaning of the term when it has many different meanings. It also matters if you belong to a particular school then you follow one definition which is understood or taught in the school. But when it comes to an interview or working in companies then people from different school work together and obviously they hold different meaning.

Now it is quite difficult to explain the definition which you know to others. That’s ok.

To get out of such dilemma or confusion one should have deep thought process for each testing terms. I feel that everyone belongs to their own “Thought of School”. Think of your own meaning for a while. Do not rely or accept anything unless you agree.

I would like to explain how one can come up with their own understanding of the term.

It’s ok if you would pick up any definition from the above list but think twice before you agree. Generally people assume that you thoroughly understood the meaning of it and you are good at practice.

Let’s break the term ‘function testing’ into ‘function’ and ‘testing’

What is function? (According to Oxford dictionary)

Function is:

1. A purpose or natural activity of a person or thing.
2. Work or operate
3. (Function as) fulfil the purpose of.

What is functional (adj)?

1. Practical and useful
2. Working or operating

What is testing?

Though I like James Bach’s definition “Questioning a product in order to evaluate” but I would prefer here Cem Kaner’s Software Testing definition.

“Software Testing is an empirical technical investigation conducted to provide stakeholders with information about the quality of the product or service under test.”

If I would relate function to the application (product or service under test) then what counts function in the application?

Currently I am writing this post in Microsoft Word so let me take this as a standalone application. All the elements visible to my eyes have some purpose. Menu bar, Copy paste, bullets, font style, font size, mailings, Save, Print, Publish, Page layout, Insert objects, shapes, footer, header or page number. All these are elements or parts of MS word. I would say everything designed in MS word has purpose. Look at human body every part of body has purpose.

Elements or features can work independently or together. So the function can be a feature (element) or collection of features.

I feel testing functions of MS word includes single or multiple features. Testing one function at a time, I would call a function testing.

Functional means working. Testing the ‘working’ of the function is similar to the testing of function.

How I test the function?

Can I just observe function without giving any inputs? I think giving some inputs to see how function works or what it can do.

When I am testing a function, it means that I am testing ‘what product can do’ and basically whether it’s working or not. The scope remains to one function at a time.

That’s why I find software testing field very interesting because it gives complete freedom to think.

Our independence of thoughts is a gift!

18 Software Testing Contest Questions by Santhosh Tuppad

I missed the deadline of submitting my answers to Santhosh Tuppad’s Software Testing quiz contest.

Here is my answers

1. What if you click on something (A hyperlink) and to process or navigate to that webpage you need to be signed in? Currently, you are not signed in. Should you be taken to Sign up form or Sign in form? What is the better solution that you can provide?

This does not provide complete information to decide what and why I suppose to test as a tester. My answer would depend on context in which I will be testing. There could be many scenarios to handle sign in & sign up.

Anyways let’s assume that there are two different scenarios in which user would take two different paths and these two scenarios I will test.

1. Let say I am testing a bank’s website. The scenarios is customer receives his netbanking credentials through which he try to access online his bank’s account. If customer is accessing it first time then system should prompt for new password and if not then sign in screen. As per my understanding Bank’s website does not have sign up form because based on account registration bank provides netbanking facility.

• Now I will test this scenario based on certain assumptions where I receive netbanking system generated auto password. I will test whether system prompts ‘change password’ or not. I will observe system’s state. I will also try to pause while entering a password.
• Other scenario in which I would test sign in form to access account second time.

2. In second case if user try to download songs, movies or books then system prompts user for user authentications.

• In first scenario I will test if user is visiting first time and try to download songs from website then system should prompt for sign in where there should be button available which ask user to do sign up
• In second scenario I will test if user is already registered but did not logged in and she try to download songs then system must prompt for sign in.

The conclusion is I can’t answer you what is better solution unless and until I understand in which context this problem is.

2. Using “Close” naming convention to go back to the homepage is good or it should be named as “Cancel” or it is not really required because there is a “Home” link which is accessible. What are your thoughts?

Without context it is very difficult for me to answer this question. I try real hard to model such requirement on web application because I am assuming that ‘Home’ link will appear on websites not in any desktop applications. I might be wrong.

In spite of my assumptions I would suggest if it is website having ‘home’ link then creating breadcrumb would serve the purpose of SEO and Usability of website.

3. Logout should be placed on top right hand side? What if it is on the top left hand side or in the left hand sidebar which is menu widget like “My Profile”, “Change Password” etc. – Is it a problem or what is your thought process?

As per my understanding most of websites has login and logout facility at right hand side on the top because most users hold mouse in right hand. It’s easy and convenient to keep these facilities at right hand side at top. It will be more visible and handy.

• I would say it’s usability issue if you keep log out (or log in) at left hand side in menu bar.
• Any window you close, you find that cross or close button at right hand side at top because of mouse design as per user’s convenience placed at right hand side.
• Keeping at right hand side at top would remind users to logout from highly secured applications like banks.

4. Current design of forgot password asks for username and security answer and then sends a link to e-mail inbox to set new password. How does “security answer” increase the cost of operations? Also, what questions do you frame for security questions?

I think many possibilities in which security questions would increase the cost of operations internally as well as from user side.

1. Users who forgets thing may not remember the security question or security answer. Because of this they may not get their account back and company may lose one user
2. When user is unable to recall his security answer then he needs further help to extra his password. It may increase company’s cost to assign someone to do this job.
3. When user asks for password then firing a query and extracting results would cost to company.

What security questions I would frame for security. First of all I am not in favour this functionality however below are my framed security questions

• What is your nick name?
• What is your first girlfriend / boyfriend’s name

I am unable to find more than this because criteria of framing security question is quite challenging. You need to make sure that hacker would not be able to crack these questions. Questions need to be very personal but memorable answers. Generic questions would get easily hacked by hackers.

5. If you had to design “Forgot Password” working, how would you do it and why? You are free to give different many functional designs.

I would keep it very simple. I would not user tricky security questions where it would be inconvenient for user to recall those details.

I would design a form in which text box for accepting an email id and action button (name ‘send link’)  and next page appears with a message and if user did not receive password reset link then resending the password link to email address.

I want user should not get irritated by asking security related questions. User might be anxious because he must be worried about his data or security of his account. He may not be in position to answer security questions or other extra stuff.

It is also easy to maintain at operation end. You don’t need to fire extra query to get those details.

6. There is neither account lockout policy nor captcha for the login or security answer forms; what kind of problems do you see with the current implementation and what do you propose?

• I think it’s highly vulnerable to security concerns & threat of spammers.
• Hackers can easily extract data from user’s account & data can be misused
• User might lose her account permanently
• Her account get loaded with spam messages
• User might lose valuable information like her credit card details, policy details or other account details

I would propose to have logout policy rather than Captcha or Security answer. Because system is not forcing user to think on some security answers which she might have forgotten or that catpcha which is takes few seconds or minute to enter the data. I think system should smartly handle logout functionality. It may be that if user is idle for more than 5 minutes then automatically logout from the system. Or when user closes browser without loggin out then system should logout from user’s account.

7. Well, it is about context and there are no best practices in general. What are your thoughts on usage of captcha? Where should they be used and why?

Captcha is used to protect sites from spammer & attackers. The captcha image or text is difficult to interpret to computers but human can easily read.

Practice is something that actually individual does. We call a practice best when you are sure the guaranteed results.

Using Captcha is good in particular context because it saves sites like webmails gmail or yahoo from injecting thousands of account data to break the system.

I think Captcha are vulnerable to attack. Human can easily break this security by gathering all available Captach database & put them in one code.

I will not call Captch as best practice, it is good practice to avoid spammers.

8. If you are the solution architect for a retail website which has to be developed; what kind of questions would you ask with respect to “Scalability” purpose with respect to “Technology” being used for the website?

Scalability is system ability to perform with increasing load of users & their transactions. System must be scalable with minimum cost of hardware or technologies.

I am being an architect for retail website, below questions I will ask:

1. What is hardware requirement?
2. What database is going to use? & Why?
3.  How much is the budget?
4. Is the technology latest?
5. Can we update new definition or patch if required to the technology?
6. Does this scalable solution adds any administrative or maintenance cost?
7. How much load is expected on system every day, every month & yearly?

9. How do you think “Deactivate Account” should work functionally keeping in mind about “Usability” & “Security” quality criteria?

With respect to Usability where user should able to easily use the system. It’s good feature to provide to user if she wants to deactivate her account then system have that option.

What if system does not provide deactivate account feature then user might get irritated & she thinks that it is violating the security because as I do not want to deactivate my account & system does not have that option. User might feel that she is deceived.

10. For every registration, there is an e-mail sent with activation link. Once this activation link is used account is activated and a “Welcome E-mail” is sent to the end-users e-mail inbox. Now, list down the test ideas which could result in spamming if specific tests are not done.

One of the test would be System should not allow duplicate email ids.

11. In what different ways can you use “Tamper Data” add-on from “Mozilla Firefox” web browser? If you have not used it till date then how about exploring it and using it; then you can share your experience here.

I almost spent 4 hours to learn the Tamper Data firefox plug in. I must conclude that it is great to tool to hack websites.

How I used Tamper Data? I experimented with ibibo.com website to try to change my password, I tamper that data & from plug in I submitted the changed information to server. I was extremely happy to see that from Tamper Data you can change the values which you want to send it to server.

I think online game websites are more vulnerable to such plug ins because easily game score or points can be changed.

Using this plug in, hacker can easily change information specially in public internet services if user forgets to logout.

I have not tried but it could be possible to alter information from cookies & send the changed information to get control over account

12. Application is being launched in a month from now and management has decided not to test for “Usability” or there are no testers in the team who can perform it and it is a web application. What is your take on this?

First of all I will insist them to do usability testing atleast for couple of days. However as you said that there are no testers in the team who can perform usability testing.

• If that is the case then I may call a expert to teach my test team some tips of usability.
• I may gather information of competitor’s web application, to convince management to take up usability testing.
• If they do not want to spend time & efforts then I may suggest for crowd source testing for atleast for 2 days
• There is other alternative way to testing the application too, that is in-house testing bug bash
• Or Beta release where end user would give feedback on web application

13. Share your experience wherein; the developer did not accept security vulnerability and you did great bug advocacy to prove that it is a bug and finally it was fixed. Even if it was not fixed then please let me know about what was the bug and how did you do bug advocacy without revealing the application / company details.

I have not got a chance to debate on Security related bugs however I had long day discussions on bug advocacy which help developers to fix those bugs. I learned working developer that

1. They are under high pressure
2. They are tired with monotonous bug fixing tasks
3. They do not want to see their baby is not in shape J

As a tester I used to have lot of discussion to prove that this is a bug. At times I have to involve other stakeholders in discussion. I worked in the project which was in red, so our company’s Vice president used to seat on same floor to monitor project’s progress with our client.

There cases where I have to hear that test team is raising false alarms & in other scenario my team used to receive appreciation for raising critical & complex bugs.

14. What do you have in your tester’s toolkit? Name at least 10 such tools or utilities. Please do not list like QTP, LoadRunner, SilkTest and such things. Something which you have discovered (Example: Process Explorer from SysInternals) on your own or from your colleague. If you can also share how you use it then it would be fantastic.

• Word document – recording & note purpose
• Notepad – recording purpose & note purpose
• Perl Clip by James Bach – helps me to get long strings with various combinations
• Mind Map by Free Mind – design my test, test plan & strategy
• qTrace by QASymphon- screen capturing tool
• BBFlash Player –to record my test
• Rapid Reporter – create session reports
• Software Testing books or non software testing books
• Mspaint – highlighting the bug screen & creating a .jpeg file
• My Brain – it’s most important tool required while testing 
• Google – helps me to search information
• Testing related Articles, blogs & contest
• Puzzles – helps me to produce test ideas
• Wikipedia – find required information
• Heuristics – helps me to find potential problems without any guarantee
• Log files – helps me to track every transactions in detail
• My emotions (frustrations & irritations emotions – helps me to decide usability of application, surprise – helps me to find more details on the features & redesign my test, monotonous- helps me to identify tests for automations)

15. Let us say there is a commenting feature for the blog post; there are 100 comments currently. How would you load / render every comment. Is it one by one or all 100 at once? Justify.

There are many ways to show comments in comment section. Definitely I will not show 1 comment at a time because clicking on ‘Next’ button is monotonous & generally reader will not like that blog anymore.

I think rendering comments 50 or 100 first then if there are more then use the HTML feature of ‘Load more’. If user is willing to see further then load next 50 or 100

Because loading 50 or 100 would help a user to glance comments if anything interesting she may read further.

16. Have you ever done check automation using open-source tools? How did you identify the checks and what value did you add by automating them? Explain.

I have not done so far. I tried doing with selenium however I have not learned fully.

17.  What kind of information do you gather before starting to test a software? (Example: Purpose of this application)

It depends on the context. It depends on when I receive the software to test.

1. First iteration

• What is the purpose of testing?
• How do I claim that this is bug? Do I have any authority documents?
• Do I have any similar kind of application?
• Which is most critical area of the application?
• Which is loosely coded area?
• Who are customers?
• What is purpose of this application? What is it trying to solve?
• What are hardware & software requirements?
• What is durations given to test?
• What are features testing trying to cover?
• How do a test team report the bug? Bug tracking tool
• Gather information on Company’s image in market
• What kind of data is required?
• Who are stakeholders?
• Who is my test team?
• Do I know test environment?
• Is there any training given to learn this application?
• What is the test approach?
• What is the test techniques used?
• What skills are required?
• I will find out if there is similar feature within the project
• Who are end users?
• What are the quality criteria?

2. Second or many times iterations

• I included all the above points with some additions
• I will find out bug history of the product
• I will review the bug fixes of previous cycle
• I compare previous release with latest to declare bugs in latest build
• I will have a look at bugs in bug tracking tool

3. Pre-production

• I included all the above points with some additions
• I will find out if any ‘help’ file is available

18. How do you achieve data coverage (Inputs coverage) for a specific form with text fields like mobile number, date of birth etc? There are so many character sets and how do you achieve the coverage? You could share your past experience. If not any then you can talk about how it could be done.

• I will study the limits given in requirement documents about input data for the form.
• I will create positive data inputs
• I will also create negative data inputs
• I will test with boundary values e.g. if it is allow to enter 0-9 digits then / and ; will be boundary because the ASCII code after 0 and 9 is / and ;
• If the keyboard is locked except some inputs like alphabets then I will enter special characters & digits and combinations of both

Certifications narrow down the scope of talent

I am fitness freak. I was Aerobics instructor in one of the gym for almost two years in Hyderabad. Being in Pune, It motivates me to join some gym as an Aerobic instructor because puneites are extremely cautious about their health and its the most happening place for fitness freaks.

Last evening I inquired for aerobics instructor in one of the gym near to my house. The owner of the gym asked for certification in aerobics (Reebok provides Aerobic certification program). I informed that though I do not have any certification but I am confident about my skills. Even I said that I am ready to demonstrate my aerobic skills. He said that if instructor is not certified then his gym’s reputation is at stake. I was surprised with his statement.

I realized what certification does:

• It narrows down the scope of talent
• You may lose good, sincere & willing to learn candidate
• You ascribe your success with certified trainers
• You think that certification is the only criteria to recruit candidate
• You overlook non-certified candidate’s skills, talent & interest
• You think that certification brings business
• You assume that certification equals to all required skills to perform that job

What if certified instructor fail to perform what she is suppose to perform? What would happen to your company’s reputation? What if she performs the task but your client is not happy? Though she is skilled but not punctual? What if she knows only one kind of aerobics dance?

I am not denying the fact that even certified trainer perform extremely well. My point is that while recruiting candidates, one must look at skills before rejecting. Certification is not the only criteria that one must have to be a good professional. Sachin Tendulkar was never certified to play cricket, it was his skills and performance that took him to places!!!

Asking her to demonstrate skills will help recruiter to know the good qualities like confidence, ready to learn & professionalism a candidate may possess without possessing a certificate.

I wonder if any gym’s client (individual) would have asked Gym owner that if your gym is certified or not else she may refuse to join. Clients join Gym based on ambience, services & facilities available in gym then why not recruit candidates based on their capabilities or skills?

List of factors that I think making assumptions are bad for testing

In my previous posts I have written that tester can’t test without assumptions and assumptions are dangerous for testing.

It’s like you know that sugar is dangerous for health but (most of) human can’t live without sugar (confectionery or sweets or any item of in which sugar is present).

Assumptions:  ‘An unproven statement or proposition, treated as if it were true’

You being a Tester how do you know, that assumptions are dangerous for testing. I came up with 8 factors which inform that though you can’t test without assumptions but beware of those because they are dangerous.

Below is the list of 9 factors I understand that makes assumptions more dangerous.

Here is the list:

1. Create confusion – Suppose requirement document has details about user password acceptance criteria. It accepts combinations of character and numbers. If I assumed based on my prior knowledge, experience that it also accepts Special characters then I might get confused with actual requirement.

I would like to give one more example, if a tester assumes that she is working on a build which is latest but latest    build is not yet deployed by development team then she might be raising a bug which is not acceptable by development team. This might create confusion in test and development team.

2. Time Waster – Suppose I assume that the test data which I received from other team is valid data. After few tests I found few bugs which I report to development team. Programmer claim after some investigation that the data used for test is not valid. This could result in loss of time of test and dev.

3. It stops tester’s ability to think other possibility – Consider that ‘fire fighter light up fire.’ I was unable to think other possibility. I fail to understand why fire fighter would lights up fire? My assumption was that fire fighter extinguishes fire rather than lights fire. When I read about backfire then I came to know that fire fighter could lights up the fire.

When you assume you might not think other view or possibilities.

4. Limit the scope of testing – If a tester assumes that a user will not behave this then tester is limiting the scope of testing. She will not consider other possible scenarios just because of assumptions she makes. Tester creates tests based on some mental models based on their assumptions. This might reduce other possible scenarios.

5. Fail to notice new bugs – Suppose a tester knows that she has already tested this functionality then she might overlook that function. She might misses new bugs (also non obvious bugs)

6. Reporting False Alarm – Suppose tester is testing based on requirement but if she fails to verify her assumptions with stakeholders then she might raise false alarms. Though the actual feature works appropriately but her assumption declares it as bug.

7. Lose stakeholder’s trust – If testers raise false alarms then chances are high that stakeholder loses trust in tester.

Suppose Test lead asks testers to test certain function in product. Tester might think that it’s already tested and she is very sure that there are no bugs. She declares (she assume) that function works. If anything goes wrong in production then management may question the testing ability of a tester. Manager might lose trust in tester. They might refrain from assigning important task. It might affect her appraisal.

• Lose credibility – Raising false alarms, assuming that available build is latest, assuming test data is valid. I think this is similar to point no. 7, losing trust. When someone loses trust in you, it means that you also lose credibility. Stakeholders stop assigning bigger responsibilities, they start taking testers for granted.

8. It costs – Suppose Yahoo has 10 features like weather, cricket, entertainment, RSS etc. Test team assume that RSS feature does not generate revenue and Team convince their stakeholder that not to develop and test RSS feature because it takes more time and efforts. This decision could be expensive because RSS feature may not be profitable for one market but it might be profitable for rest of 30 Yahoo’s markets (I am referring market here as Customer’s customer). Assumption could be costly as well.

Above 8 factors inform tester that though you do not have ability to test without assumptions but many assumptions are bad for testing and we avoid making them.

You can come up with your own list of factors which you think that making assumptions are bad for testing.

Testing is not part of Computer Science

Let me explain why Testing is not part of Computer Science but knowing computer science will be added advantage in your career.

Some of the definitions given by Testing Experts:
- Questioning a product in order to evaluate – James Bach
- It’s empirical technical investigation conducted to provide
stakeholders with information about the quality of the product or
service under test – Cem Kaner

For example you try to ignite your car engine but it fail to start. What you do? Whatever steps you take immediately to start car’s engine those steps are not based on any specialized degree. Your attempts may or may not start car’s engine. Few investigations (choke/jumper etc) you declare that you need to give your car to servicing or you decide to call mechanics. Can you justify this behaviour? Do you really need mechanical knowledge to do this task?

This is one approach of testing which is exploratory. I would say this is very basic nature of exploratory testing. Exploratory testing can be done systematically with skills. I believe even non technical person can perform immediate steps to find the solution of a particular problem. I do not need to be specialized in electrical, electronics or mechanical to find out solution for particular problem. But if car’s engine is damaged then I must contact mechanics to fix the problem because I am incapable to fix such problem.

I won’t deny the fact that Computer science’s knowledge would add value in your career.

Software has purpose to solve human’s problem.

Testing is social science. We need to study the human behaviour and human psychology. Human interact with software and that’s where tester need to understand. Development is separate part.

Explain me, testing a car do you need any mechanical education? Or you need to know how human drives a car? How this car is suitable in all Indian seasons? How is Car AC’s cooling effect? What will happen if car stops at the speed of 100 kmph? How human can survive in car accidents?

Let say pencil manufacturing company’s tester does testing. Does she know how pencil is developed? Or she should understand how kids use this pencil? What are the risks involved while using a pencil? What if kids chew a pencil? Does pencil’s material create any medical reaction to kid’s stomach?

Here is the point. I am again stating and emphasize that Testing is a social science. You should know subjects like psychology, human behaviour which comes under social sciences subject.
Tester should learn subjects like ‘How to decide?’, ‘What is conjectures and refutations?’, ‘What is general system’s law?’, ‘Basics of maths’, ‘ How to solve’, ‘The pleasure of finding things out’, ‘Philosophy’, ‘Critical thinking’, ‘ Lateral thinking’, ‘How to read and do proofs’, and ‘studying  systems’, and lot more!!!

Ask yourself below questions just to understand what I am trying to convey :

1. How do you know that this is a bug?
2. How do you enhance your observation skills?
3. How do you use your problem solving skills to solve testing problem?
4. How do you ensure that your test coverage cover all possible bugs?
5. How can you create your own method to handle this contextual testing problem?
6. How do you ensure that your reported bugs get fixed by development team?
7. How do you decide that these are right combinations of inputs for program to find bugs?
8. What is your project risk model? How do you design it?
9. How do you decide right test strategy?
10. How do you do test planning with available tools and resources?
11. How do you know that you know is the right?
12.  How do you notice, observe and decide?
13. Do you understand human psychology?
14. How do you do search?
15. How do you rapidly learn software, techniques or domain?
16. How do you know that these assumptions are dangerous for testing?

Understand the depth of testing and above things are not taught in schools or colleges!!!

If you know computer science then it will add value in your career. But you must learn the skills of testing. Tester is the one who should rapidly learn anything and everything in given time.

I never mind to teach testing to BA graduates because to test something you need to have right kind of attitude.

Tester needs to be…

After attending VodQA and discussing with Shrini about ‘BDT (Behaviour driven Testing), I realised that soon people will come up with new development approaches, they’ll talk about the new happenings and will share their experiences.

Whether it’s a Water fall model, V Model, Agile or BDT, people would implement either of these with a hope to create better softwares. Industry will move around such theories and will keep people busy and talking to decide what’s good and what’s not. But this will eventually lead to extinction of human interference in terms of Quality.

Jerry Weinberg said in one of the interviews that our industry requires Quality, until we start caring; we are not going to get better. It ultimately turns out that, quality software is cheaper in the end.

The point of writing this excerpt is to emphasize on the fact that human skills are essential especially Testing skills to prepare good quality software in the end. No other things can replace human skills.

Customer and Employer need to pay attention to what they deliver to end user in terms of bug free Software else end customer would keep finding workarounds for the bugs in your software.

A skilled tester shouldn’t be afraid of such things. He must have belief in his capabilities to provide various solutions to customer till it satisfies requirement.

Thought provoking VodQA Pune – A Testing Conference

VodQA’s tag line ‘The testing spirit’, the spirit I witnessed in ThoughtWorks Conference held in Pune.

Hats-off to the spirit of Puneites for the energy, enthusiasm and participation in VodQA. Thoughts and ideas were shared, visions were exchanged and the whole conference was resonating with likeminded people.

I already attended VodQA in Chennai so I knew this time as well I am going to hear more on Automation and Agile Testing. But to my surprise, it was more on Mobile Testing!

I was excited to attend VodQA for two reasons, first, I’ll be a tester among testers sharing ideas on the same platform and secondly I’ll be meeting world’s most inquisitive tester, Shrini Kulkarni. Unfortunately I missed his presentation because I had certain personal chores to attend in morning so made it late to the conference.

Anyways I attended the second half of VodQA which turn out to be more of interaction with people, games, group discussions and few presentations.

I must mention a game of building tallest tower through which Sneha shared very good agile lessons. The team of 5 members were trying to build the tallest tower with the help of Spaghetti sticks, thread, cello tape and marshmallows. My team realised that we spent lot of time on planning and failed to do prototyping.

I would like to add about presentations on BDT and Negative Testing, which were thought provoking indeed. Offline I discussed with Anand to understand how they implemented BDT (Behaviour Driven Testing) in their project. It was quite an interesting discussion we had.

I appreciate for the efforts ThoughtWorks puts in to strengthen Software Testing community.

Sitting in Café Lounge post conference, my mind ran through the points we discussed about BDT, Automation and term called QA.