Software Testing Education

Software Testing Education
(maintained by Tao Xie) [Feedback]

Besides software testing research, I am so interested in software testing education, developing materials and educational tools for teaching software testing or software engineering in general. In my view, computer science education needs to reduce the gap between what we produce and what the real world needs. As one example, software testing has become increasingly important and popular in practice for improving software reliability. I believe software testing needs to be put into the early phase of the introductory computing curriculum so that it can benefit students throughout their programming projects in later courses as well as their future software development careers.

I maintain this web to provide useful resources for educators in teaching software testing. In the near future, I plan to make the software testing tools that I developed for research into tools that I (as well as other instructors) can use in teaching software testing or data structures. Different from the existing data structures visualization tools, my tools are not bound to any specific data structure but accept any Java program. I expect that the new tools are not only useful for teachers to teach data structures or testing data structures, but also useful for students to test, debug, and understand the dynamic behavior of their own solutions for programming assignments.

Teaching Materials by Topics
Software Testing or Verification Tools (that can be used for teaching purposes)
Educational Tools
Textbooks
Courses (with course materials)
Other Resource Webs

Teaching Materials by Topics (recommended by Tao Xie):

Teaching JUnit. JUnit has become a popular topic in undergraduate programming courses. Although most courses cover sufficiently how to use JUnit as a convenient tool for writing and running test cases, few courses could cover sufficient materials on why we want to use the JUnit framework and how we use the JUnit framework in a "right" way. I found the following two book chapters very useful for addressing the last two questions.
- Testing with JUnit chapter from the book Java Development with Ant (focusing on Pages 85-91). The material addresses why we want to use the JUnit framework instead of Java main() testing.
- Testing: Cactus and JUnit chapter from the book Jakarta Pitfalls. Slides from a testing class (ignoring the parts about Cactus). The material addresses how we use the JUnit framework in a "right" way.
Why is software testing important to the society?
- U. Maryland Atif Memon's slides
- Stanford U. Monica Lam's slides
Teaching testing data structures
- I have given a guest lecture at CSE 373 on this topic. Here are the slides and the homework assignment that I developed.
Teaching black-box testing
- Georgia Tech Alex Orso's slides for illustrating category-partition method through testing the grep program. I found that the grep example is helpful but I suspect that this topic would be more appropriate for graduate students.
Teaching white-box testing [soon to be posted]
Teaching testing skills
- James Bach's slides for illustrating testing skills through an example of input constraint testing. I found that the example of input constraint testing is very interesting and useful for illustrating how to produce extreme input values with some clever ideas.

Software Testing and Verification Tools:

Available dynamic analysis tools: Daikon@MIT Eclat@MIT CUTE/jCUTE@UIUC CnC@Gerogia Tech JCrasher@Gerogia Tech DIDUCE@Stanford UTJML@UTEP eToc@ITC Quest@Saarland Lever@UIUC Terracotta@U. Virignia kBehavior@UNIMIB Objectflattener@UNIMIB CowSuite@ISTI UMLTesting@CoroladoStateU TorX@Wwente MVT@HUT Bug Isolation@UWisconsin GUITAR@U. Maryland Delta Debugging@Saarland U. muJava@GMU TestCon@U. Queensland InsECT@Gerogia Tech GERT@NCSU
Available static analysis tools: Prospector@UCBerkeley CP-Miner@UIUC JML@Iowa Stae U. ESC/Java 2 SQLGuard@OSU STG@IRISA ArCats@NTNU UCTSystem@IRISA FindBugs@Maryland Blast@UCBerkeley MOPS@UCDavis MAGIC@CMU Focuscheck@Iowa State U. BOOP Splint@Virginia Margrave@Brown Uppaal TRON@AalborgU. Atasyn@UCLA Avrora@UCLA Blade@ASU Repair@MIT JSA@Brics Charon@UPenn MIST@UPenn LTSA-MSC@Imperial College
Generic program analysis tools: Soot@McGill BCEL bddbddb@Stanford Idus@KSU (slicing) JSA@Brics Valgrind Pin Tracer@SUNY SB
Available open source tools (opensourcetesting): NASA Java PathFinder SAL-atg@SRI Dejagnu@GNU
Available executables: Spec Explorer@MSR FSE Spec#@MSR FSE Zing@MSR AETG@Telcordia ConTest@IBM Research Agedis@IBM Research et al. IBM Alphaworks-Eclipse IBM Alphaworks-Java UBET@Bell Labs
Commercial tools: Parasoft Jtest Agitar Agitator Silvermark Test Mentor Reactive Reactis
Available tools collected by Jonathan Aldrich as tool suggestions for his Analysis of Software Artifacts course

Educational Tools (not limited to software testing):

Bug Hunt: A web-based tutorial to instill good testing practices at UNL
Data Structures Library in Java (JDSL) Tools based on JDSL (open source)
MVT - Matrix Visual Tester Matrix - Algorithm Simulation Framework
Testers and visualizers for teaching data structures (paper) JDSL Visualizer (open source)
Opsis - Teaching Binary Search Tree Algorithms through Programming, Proof, and Animation (open source)
Gawain algorithm animation system (open source)
UTA's applets for teaching data structures (open source)
Visualization of Splay Tree (open source)
A Tool for Teaching Advanced Data Structures to Computer Science Students: An Overview of the BDP System (paper)

Textbooks:

Introduction to Software Testing (Amazon) by Paul Ammann and Jeff Offutt. published in Feb 2008 by Cambridge
Foundations of Software Testing by Aditya P. Mathur. published in Oct 2007 by Pearson Education
Model-based Software Testing and Analysis with C# by Jonathan Jacky, Margus Veanes, Colin Campbell, and Wolfram Schulte, published in 2008 by Cambridge
Software Test and Analysis: Process, Principles, and Techniques (Amazon) by Mauro Pezze` and Michal Young, published in 2007 by Wiley
Why does my program fail? - A Guide to Systematic Debugging by Andreas Zeller. published in 2005 by Morgan Kaufmann
Practical Model-Based Testing: A Tools Approach by Mark Utting and Bruno Legeard. published in 2006 by Morgan-Kaufmann
Fuzzing: Brute Force Vulnerability Discovery (Amazon) by Michael Sutton et al. published in 2007 by Addison-Wesley

Testing Commercial-off-the-shelf Components And Systems by Sami Beydeda, Volker Gruhn. 2004
Testing and Quality Assurance for Component-Based Software by Jerry Zeyu Gao, H.-S. Jacob Tsao, and Ye Wu. 2003. Course
Black-Box Testing : Techniques for Functional Testing of Software and Systems by Boris Beizer. 1995
Software Testing Techniques by Boris Beizer 1990

Courses (with course materials):

Other Resource Webs: