Testing Intro

CSC-430

Phillip Wright

Testing

No matter how good you are, you will make mistakes. Probably more mistakes than you will ever know.

Testing II

Accordingly, it is necessary to not only do what you can to not make mistakes, but also to rigorously test your code to confirm that you were successful.

Testing III

You will still have bugs, though…

Unit Tests

Our first line of defense will be unit tests: small, directed tests aimed to check whether or not the smallest units of our code are working correctly.

Unit Tests II

Ideally, our development process would be driven by our unit tests.

This concept is referred to as Test Driven Development (TDD).

Test Driven Development

  • First Law: Don’t write production code until you have failing tests
  • Second Law: You may not write more test code than is necessary to create a failure
  • Third Law: You may not write more code than is necessary to fix the failures.

Test Driven Development II

Alternatively, we can restate these laws as the following list of steps:

  1. Write a test that fails
  2. Fix the failure with minimal changes
  3. Refactor and repeat

Test Driven Development III

If we do this well, then we know that all of the desired functionality has been tested, because, otherwise, it would not exist.

Also, we now have a safety net to protect us from regressions in the code base. If someone breaks our code, we should immediately know because of failing tests.

Test Driven Development IV

Unfortunately, it doesn’t always go so smoothly. For instance, we have to write our tests correctly, we have to test for unintended results as well as intended results (which is more difficult to do!), and, in some cases, we can’t write unit tests!

Swiss Cheese Model

In many fields, safety relies on the “Swiss Cheese” model:

There will always be holes in any layer of safety that you implement, but if you have enough layers in places, it greatly decreases the odds that a hole will exist in the entire system.

Swiss Cheese Model II

So, we try to use our brains to make sure we right good code, and we try to write good tests. We will fail at both, but will hopefully come closer to eliminating bugs than if we didn’t test at all.

Static Typing

Another simple layer to add, which a lot of people overlook, is using the type system to model your problems.

Static Typing II

Strong Static Typing can often give you similar benefits as unit testing.

  • The more you model your problem with types, the fewer invalid states will compile
  • …and the easier it is to refactor code with confidence

Static Typing III

In languages like Agda, Idris, Coq, you can model problems so completely that compilation “guarantees” correctness (sort of…), because invalid states can not be compiled.

In these languages, you are basically writing proofs.

Swiss Cheese Model III

This gives us:

  • Our coding skills
  • The type system/compiler
  • Unit tests

To combat bugs in our code

Unit Tests III

So, our unit tests will allow us to ensure code performs as expected, to modify code without causing regressions, and empowers us to refactor our and improve our code without fear.

But how do we write unit tests…