Beautiful Testing

Tim Riley, Adam Goucher

Mentioned 3

"Beautiful Testing offers 23 essays from 27 leading testers and developers that illustrate the qualities and techniques that make testing an art. Through personal anecdotes, you'll learn how each of these professionals developed ideas of beauty in testing a wide range of products - valuable knowledge that you can apply to your own projects." --Book Jacket.

More on

Mentioned in questions and answers.

I have to return a random entry from my database.

I wrote a function, and since I'm using the random module in Python, it's probably unless I used it in a stupid way.

Now, how can I write a unit test that check that this function works? After all, if it's a good random value, you can never know.

I'm not paranoid, my function is not that complex and the Python standard library is 1000 x time good enough for my purpose. I'm not doing cryptography or something critical. I'm just curious to know if there is a way.

There are several statistical tests listed on RANDOM.ORG for testing randomness. See the last two sections of the linked article.

Also, if you can get a copy of Beautiful Testing there's a whole chapter by John D. Cook called Testing a Random Number Generator. He explains a lot of the statistical methods listed in the article above. If you really want to learn about RNGs, that chapter is a really good starting point. I've written about the subject myself, but John does a much better job of explaining it.

I have developed a basic Chat application in Java. It consists of a server and multiple client. The server continually monitors for incoming messages and broadcasts them to all the clients. The client is made up of a Swing GUI with a text area (for messages sent by the server and other clients), a text field (to send Text messages) and a button (SEND). The client also continually monitors for incoming messages from other clients (via the Server). This is achieved with Threads and Event Listeners and the application works as expected.

But, how do I go about unit testing my chat application? As the methods involve establishing a connection with the server and sending/receiving messages from the server, I am not sure if these methods should be unit tested. As per my understanding, Unit Testing shouldn't be done for tasks like connecting to a database or network.

The few test cases that I could come up with are:
1) The max limit of the text field
2) Client can connect to the Server
3) Server can connect to the Client
4) Client can send message
5) Client can receive message
6) Server can send message
7) Server can receive message
8) Server can accept connections from multiple clients

But, since most of the above methods involve some kind of network communication, I cannot perform unit testing. How should I go about unit testing my chat application?

Chapter 7 of Beautiful Testing describes testing an XMPP chat client. I recommend reading the chapter. The conclusion is illustrative and may provide some pointers for your chat application:

In our quest to create beautiful tests for checking XMPP protocol implementations, we started out by testing simple request-response protocols at the lowest level: the data sent of the network stream. After discovering that this form of testing does not really scale well, we abstracted out the protocol to a higher level, up to the point where the tests used only high-level data structures. By testing protocol behavior on a high level, we were able to write tests for more complex protocols without compromising the clarity of the tests. For the most complex protocols, writing scenarios helped to cover all of the possible situations that can arise in a protocol session. Finally, since XMPP is an open protocol with many different implementations, it's very important to test an XMPP application on the real network, to ensure interoperability with other implementations. By running small test programs regularly, we were able to test the system in its entirety, and check whether our implementation of the protocol plays together nicely with other entities on the network.

I'm college grad, soon to be graduating. SO i have started my job search. So my question is for New Grad who is interested in QA (API Level testing in particular). What is level of knowledge in JAVA required? I'm not that great in Java but can rate myself 5/10.

So do companies expect candidate to have extensive coding knowledge in java or just basic java

Again if its just basic java, what concepts should i focus more on ?


This really depends on what level of job you expect to land for your first position. If you are happy writing very simple test cases over and over, basic Java with a good understanding of refactoring (e.g., reusing code by extracting common code into methods, rather than copy-and-pasting), then basic Java is fine. The better your coding skills, the more you can do on your own without relying on developers to create fixtures for you. I personally really enjoy coding, and try to get SDET positions where I am writing lots of test tools - so I try to write code almost as well as a full-time developer; I do allow myself to lag a year or so behind "state of the art" so I can focus on testing as well.

Even more important than programming skills are your test skills. You won't land any test job unless you work on these.

Have you read Cem Kaner's Testing Computer Software or another basic testing book? (If you've found another good one - please write a comment to me! I'm looking for good intro test books to recommend, Kaner's still seems to be the best but is getting a little outdated) And followed it up with something a bit more advanced and thorough, like Alan Page's How we Test Software at Microsoft (again, comments on better books appreciated)? If you still want more reading, Beautiful Testing is a great book for more advanced professionals.

Have you tried to "test" common objects around you, to get used to thinking about how things can fail? Can you determine which areas of testing (functional, security, performance, safety, etc.) are most important for a given object or program, and then come up with a list of tests you could write to test that aspect of it, including boundary and error testing? Can you do this in an orderly fashion in an interview? If it is a program, can you then implement those tests? Once you land a job, can you advocate for the importance of a bug without upsetting the developer who wrote that code? And, can you work with developers to introduce quality into the product before the bugs get written?

These kinds of questions are great for The Software Testing Club, btw. This seems to be the site that is getting the most credibility as a resource for QA professionals asking these kinds of meta questions. I'd still look to Stack Overflow for specific, objective "how-to" questions.