Archive for the ‘Testing’ Category
I asked one of my students about the homework (assignment) I gave previous day but she said that she could not continue testing the software as the software and the requirements were confusing. She was frustrated with the whole thing that she could not understand what needs to be done next. She was disappointed and she stopped testing the product.
I understand her situation and reasons for frustration because I have worked in projects where everything looked so confusing, there’s always the time crunch and managers expect deliverable because client is sitting on their head. This is a typical situation in testing projects. And this is a same story of many testers who get confused, frustrated with requirements and with products. And no wonder why the poor quality product gets released in market.
How do you motivate yourself when you are in such situation?
It’s a fact in software development that requirements are never written clearly & completely. It is not possible because technology & business keeps changing every second. So, how do you expect requirements will always guide testers and developers in right direction? Secondly, machine and programming languages has limitations and that’s the reason new programming languages and frameworks keep emerging every day. So under such limitations, developer writes program on incomplete requirements then imagine how the quality of the product is going to be like? Your client is not aware of the quality of product. They have invested lot of money to build the product.
If I have to place a tester in such a chaotic situation then she should be the one to provide fact based information about product’s quality. She knows what product can do & what it can’t. She gives direction for further development and helps in making important decisions about the product. Testers help client, manager and developer to improve quality of the product. Without testing, software could turn out to be a nightmare not only for the end user but also for the vendor.
Testers work like a light house because they give timely feedback and appropriate direction. A tester has most important and critical role to play in software development. All your stakeholders look for information which guides them because they cannot see what a tester can see. They are dependent on you like a blind person so if you stop testing or get de-motivated due to the situation then who else will guide them? Who will tell them the truth about their software? So don’t give up so easily because you have not yet come up with your best testing solution. You have not come up with your best test technique which is uniquely designed for this project. You need to strive for better testing that can only happen when you do not lose hope, when you do not lose confidence, when you do not stop trusting your testing skills.
Your service would make a lot of difference to your software’s quality, so keep providing your service honestly and do not stop!
“I slept and dreamt that life was joy. I awoke and saw that life was service. I acted and behold, service was joy” – Rabindranath Tagore
I believe Bug Report is equally important like Test Cases, Test Plan or Test Strategy.
Let’s say, you have a great test plan and strategy for testing. And your testers are also good at writing test cases. You found many important bugs which makes lot of sense to fix them. But what is the point of having excellent test plan if your bug reports are getting rejected? There may be various reasons to reject bug reports but the Number one reason as per my understanding is lack of clear and precise information about bug.
Cem Kaner rightly said that bug report is ultimate product of testing which is like tester needs to sell bug reports to their stakeholders.
I worked in and as a triage team in one of my previous project and realized that it’s extremely difficult to deal with a report which has inadequate information.
Below sentences in your bug report may trigger rejection from your stakeholders:
- It is not working properly: Imagine how receiver of your report would take up the meaning of this statement. I get confused with this sentence. It does not tell, ‘how the feature is not working?’ It does not tell what your oracle is? It does not give any clue about bug. Developer spends lot of time in reproducing. They would then pick their own meaning about the feature which is not working properly. Based on this understanding, developer would fix the bug. Tester again reopens the same bug because something else got fixed. Isn’t it wasting crucial time of your project?
- It is not properly displayed: Sometime testers (including me) find it difficult to describe what they have observed to put it in description. And it soon becomes the main reason to add words like ‘something is not displayed properly or working properly’ which come as a rescue line for most testers but we are missing the importance of bug report. Bug report is for making things easy rather than complicated or time consuming.
- Take any input to reproduce bug: Computer has innumerable options available to give inputs to the software. And every individual’s imagination is different and subjective. She can’t think of same inputs as you entered when you encountered an error. This may lead to assumptions which may not be fruitful with respect to the timeline & criticality of the project. I think one of the reasons of projects being critical is that manager focuses on processes, plans and pay less attention to enhancing employee’s skills which is key thing in project.
- Avoid these kind of statement in your report because they are ambiguous
- Make sure that you write in such a way that developer would understand only what you intend to say about a bug
- Write clear input & output
- Write what you observed
- Do not let your bug sit in stakeholder’s imagination else it will never be fixed.
It’s in your hand to convince your stakeholder to fix your bugs through your clear and precise bug report. So write carefully!
I attended a workshop on BDT. It was great workshop because it helped me to understand what BDT is. Thanks to ThoughtWorks and Anand Bagmar and The Team of ThoughtWorks for conducting this event in Pune.
I came back with lot of thoughts, confusion and disappointment. The confusion is with the implementation of behaviour driven testing.
As per dictionary meaning, Behaviour means ‘The way in which something functions or operates’.
And Beizer has given definition of Behavioral testing –‘tests of visible behaviour, informed by knowledge of the internals where possible –as making for better test design’
I wonder what tester would focus when they implement BDT. Is it on product’s behaviour or user interaction with product? Or does it include both? If we say tester’s focus is on Product behaviour then it can be ‘Function Testing’ in Black Box and ‘Unit Testing’ in White Box. If we say tester’s focus is on user interaction with product then it can be ‘Scenario Testing’. If tester’s focus is on different users then it can be ‘User Testing’. Is BDT different from Scenario, User, Function or Unit Testing?
Another point where I felt disappointed is BDT is too much focussed on how tester needs to write test cases. The intention behind writing business or domain oriented test cases in declarative form is that the Customer can also read and understand what test cases are or what business aspect is covered by software testing team. Now what is more important? Is it whether Customer is able to understand test cases or not OR whether the test case has potential information to find bugs in product? What if Customer does not like test cases but has enough potential to find bug then are we going to change the test case according to Customer’s likes or dislikes? I felt that our focus is too much on how actor should dress up for the scene rather than what acting skills she possesses.
If we would keep focusing on how test case should be written with customer’s focus as centre of discussion then does it really give testing solution what we are expected to give in first place?
Isn’t it tester’s focus need to shift from customer oriented test cases to product, real world and end user and their needs?
Are we saying that testing means just writing test cases? I feel testing involves many other things like understanding & analyzing requirements, generating test ideas which would have potential to find bugs, deciding Oracles, writing clear & convincing bug reports and ensuring that important bugs get fixed. Whereas BDT is one small part of complete testing which focuses on writing effective test cases in declarative statement so that customer would know what test cases the test team will execute in test cycle?
If customer really wants to know what test team is upto then there are many other important and interested things where customer can focus, for example, what bugs test team come up with or what test team’s dashboard says or what skills tester possess.
If we really want to give good solution in given time and with available resources then our focus needs to be on tester’s skills. Tell me if director wants to make his movie hit then if he keep focusing on editing part rather than the actor’s skill or story line or audience then do you think that it’s going to be a hit.
I understand that software testing field has various religions like as we human have. Human have different religion because they speak different language, different culture and they find comfort in the community of their religion. I do not find anything wrong here.
Similarly Testing community speaks different language, thoughts and methods. This leads to many arguments, confusion or debates. I find it perfectly all right because it’s been ages that we humans never stopped discussing about religion so nothing new in software testing field. Tester’s are going to talk different languages because they come from different companies, different work culture. My main concern is you worship your religion but do not forget humanity similarly you have your own methods to solve testing problem but do not forget that a good tester needs testing skills and situation awareness. Whether you are agile tester or Behaviour Driven tester or Context Driven tester or Test driven tester, it does not matter. What matters is your skill because they are going to help you to solve testing problems and find good enough solutions to really satisfy your customers.
I do not find agile testing different from traditional testing because the way I would do function testing will not change whether I am in agile project or a different project. What makes difference as a tester is to keep adding new testing skills while working in different culture.
Bugzilla is an open source Bug reporting tool.
It took me 18 hours to resolve all the issues / problems I came across while it’s installation.
Before you continue reading, let me tell you that this post is not about ‘How to install Bugzilla’. If you are looking for installation steps, then, you may not be on the desired webpage.
I recommend you to go through detailed steps provided for windows installation at Bugzilla Site.
I wanted to configure Bugzilla for my test lab where students can store their work in a centralized location. Two months ago, I tried installing it, however, I was stuck at a problem where I was unable to run ‘checksetup.pl’ file. I searched on net but nothing seemed to work out so I eventually gave up. However I did not gave up completely. Last weekend I made up my mind that no matter what, I am going to install Bugzilla.
I reinstalled everything this time but again I came across the same roadblock which I saw couple of months ago. But this time I was prepared mentally that whatever time it’s going to take, I am not going to stop in middle and will resolve all my issues. I so very much wanted to see how Bugzilla looks like post installation.
I faced a lot of challenges. I did lot of mistakes but corrected them with my questioning, analytical and Heuristics skills.
These are my findings (learning’s) which I would like to share. They may be of help to you while you install Bugzilla on your machine:
- Always download the latest version of Mysql, Apache Server and Bugzilla.
- Restart your system after every installation so that everything gets updated properly in your system.
- The ultimate solution to download Bugzilla is through Bazaar application else you may end up having some problem.
- Mysql has a bug where if you reinstall software in different directory and then remove remaining files from previously installed location, still, Mysql Registry will take previous installation path and not the current path. This is a bug in Mysql. It took me a while to figure out this problem but I managed to resolve it.
- Edit your httpd.conf file carefully. Even small mistake may lead to a problem. Make sure that you update ‘Directory’ field in httpd.conf properly at two locations.
- Change httpd.conf User to ‘apache’ & Group to ‘apache’. Also change Bugzilla’s localconfig file $webservergroup to ‘apache’
- Make sure that once you’ve done final changes, you must rerun your ‘Checksetup.pl’ file
- Grant all admin access to folders like Bugzilla, Apache Server, Temp files and Mysql
- Once you run your ‘Checksetip.pl’ then open your URL address in browser but make sure that you clear History else it may take cached pages.
- All three folders Mysql, Bugzilla & Apache server keep in C:\ instead of default location C:\Program Files
That’s it. Your Bugzilla would be ready once you configure ‘parameters’ & other administrator settings.
If you have fear of learning any open source tools then pick them up because that’s a good opportunity to let your fear go away.
I have seen that many testers stop learning with an excuse of not knowing programming language. Is it really a big deal to learn basics of programming? It requires two important qualities, first, your willingness to learn and second, time devotion. I am very sure that every individual have both in abundance. You keep postponing it either because you are stuck somewhere or you do not want to learn it. It’s in your hand to be a good student or take the easy road.
Why I am saying to pick any open source tools because first, they are free and easily available and second important thing about them is that it’s always a challenge to install them on your PC. You will learn lot of things compared to any licensed version. Licensed version has user friendly features which make you lazy. When there is no challenge then there may be lesser opportunities to learn. It’s my personal experience that if your get everything very easily in life then you would miss fun of ‘being in slumps’ (ups and downs) because those phases of your life will teach you good lessons.
As a tester I find open source tools comes with wonderful learning and painful bugs which teach you many things like
- It teaches you that Bugs means irritation, disliking, confusion and bad experience with product.
- Even uninstallation sometimes leaves few files (undeleted) which may create problems later on. It may change the configuration file forever and that software never gets installed on your PC again. What a mess? It’s like you bought a house for investment purpose and your tenant damage your house completely. Another tenant refuses to take it on rent. You may need to repair it which cost you both time & money.
- It teaches you how to be patient. Sometimes you may not find a solution so easily. You need to keep trying till you reach to any conclusion.
- It builds an attitude of solving problem if you face any challenges while installing or learning open source software then you must keep trying till you reach the end. Look from other end of the finish line, you may gain tremendous confidence.
- You can use those products for testing practice so that you may indirectly help a developer by sharing your bug report.
- It also gives you an idea of how user friendly a software product needs to be. I wish that software should come up with customer service buttons where I just need to click and it resolves all customer issues in a jiffy.
Open source software is great opportunity for testers who wants to learn software testing.
This was a question asked by one of my student who recently completed his education this year. It’s very obvious that when someone wants to try new thing then the person would come up with questions. Like if you want to marry then you may ask questions like ‘whether is it a good option or right time or not?’ you may not get satisfying answers from anyone because they will be sharing their own experiences and different perspective about it.
You being a wise person I would recommend to search for answers within you. First understand what your interest is. If you do not come to any conclusion that’s ok because very few can find their interest so easily. You need to try some career options then only you will come to know what your passion is.
I being a trainer, I would like to write down few aspects of being a tester which may help you to make your decision making stronger. I am assuming that you already decided to be a tester and my view points would add some value. I am not here to advise anyone on their career path. I know people can make decisions even without advice. And keep one thing in mind that whatever decision you make, thing that matters most is how to make it right and that’s in your hand.
Another thing which I experienced is the quality of being a student. This quality of human to stay student forever would help him to grow in the field you selected as a career option. So ask yourself ‘How good are you as a student?’ Though your teacher (I think teacher can be anything or a person who gives some meaningful lessons) is great, he has high regards in that field but what is a point if student is not willing to learn. What is a point if student does not want to try those lessons? That’s where you may find it tough when it comes to demonstration or performing what you learned. You can make your career life easy by learning and practically implementing those lessons.
Anyways hopefully these points would help you to make your decision more firm rather than getting disappointed with this option. Do not worry what world says negative about software testing. You take good things and throw rest because that does not help to grow anyway.
- You will grow intellectually: I agree with you that in any field you would grow intellectually if you would show willingness to learn. Similarly software testing has huge scope to learn intellectually because your mind gets involved in many test ideas, bugs and bug reports.
- You will learn how to be creative: To find bugs you need to be creative. You need to come up with Ideas which would help you to find bugs. Because bugs are not so easy to find.
- You will learn to form your own judgement: I have seen people including me they would accept someone’s statement as it is without even bothering about the truth behind that statement. I stopped it because testing taught me that it’s quite harmful.
- You will be wise in your thoughts: When you learn how to find bugs and how you need to convince stakeholders that this would impact customer or their business then you would start getting real business sense and customers. You would soon find yourself learning customer’s business and get involved in it.
- You will improve your skills: As tester would need communication, observation, judgement, analytical, problem solving and questioning skills. Testing would help you to improve your skills and that becomes your asset for your career.
- You will form good habits: Human is bundle of habits. I know it’s quite tough to form a new habit because it requires lot of practice. When you would start taking interest in this field you will learn that you need to read books, articles and blogs. You make a habit of reading which is good because that would help to learn testing better way. You try to be with likeminded people because you also want to share your thoughts and listen to theirs. You want to be a part of testing contests because you are also confident to solve them.
- You will be a creator: The time would come in your career where you would stop using anyone’s testing methodology and you would create your own. Every new project you would find an opportunity to create new methodology. Isn’t it great?
- You will start accepting challenges: We realize that this generation is completely digital and every minute technology is changing. When you would start learning new domain, technologies and test techniques then this kind of attitude would help you to survive in this ever changing digital generation.
‘Quality is value to some person’ – Jerry Weinberg
I feel tester needs to understand the meaning of quality and how quality is subjective because it changes from person to person.
Testing is nothing but providing quality related information of the product under test to stakeholders.
Imagine how tester’s work needs to be?
Let say there are two testers who are reporting the bugs of the same application. The first tester hardly cares who is going to read or work upon her bug report. She just wants whatever bug she reports, it should get fixed inspite of inadequate information given in the report. And in management eyes, she is more productive compared to another tester because she reports 20-30 bugs a day. But most of her bugs get rejected because of lack of quality information in her bug report.
Now consider another tester, she puts her communication, writing skills into her report. She makes sure that from client to developer everyone can easily understand bug report and its consequences on business. She provides good enough information so that Development Manager can make a right decision whether to fix it immediately or not. She also investigates before she raises a bug whether there is any point in reporting the symptoms of the problem or not, it’s a waste of time and effort for both dev & test. She also makes sure that her important bugs get fixed by convincing the risk involved in it if not fixed. Her reports are also technically sound. Developer is actually willing to fix her reported bug.
You must be wondering why second tester’s bugs are getting fix because
- She understands the Quality
- She knows the importance of billing hours
- She respect other’s time
- She is willing to hear constructive feedback on her report
- She is willing to serve developer because they are the one who is going to work upon her bug reports
- She also realize that whatever efforts she puts into testing if her reports get rejected every time then all her testing is waste
Accepting your bug report to fix in this release is like exam of your skills. If they keep getting rejected that means you need to improve your skills. It means your stakeholders are expecting a good quality bug reports. That means you need to learn more about your customer’s business. That means wear a hat of service oriented thinking.
What is a point of raising bugs just because your manager is looking for a count? Raise it because they are important with respect to your customer’s business. These reports reflect you. They show how much you know about the business. They reflect how much you are aware of quality.
Quality is not related to the product which you are testing. It is also important that testers need to think quality in their own work else it might affect your credibility. Stakeholder may not consider your report inspite of the fact that your bug is very genuine. Lack of good enough information may force the stakeholder to reject your bugs.
Imagine if your reported bug is very critical to business but it gets rejected because the way you reported it was not very convincing as per your manager. And then you get surprises when customer reports those production bugs.
Testers need to think and work upon their bug report because lot of business decisions are taken based on reported bugs. That means your information would help managers to improve quality of product only when you demonstrate your understanding of quality through your bug report, through your communication and through your testing skills.