Test Doubles- Terminology, Definitions and Illustrations -with Examples - Part 1

Test Doubles
Terminology, Definitions and Illustrations
with Examples
Part 1
Test Doubles
@philip_schwarz
slides by https://fpilluminated.org/
Examples

This deck is about a subset of the test automation patterns in Gerard Meszaros’ great book, xUnit Test Patterns – Refactoring Test Code.
The subset in question consists of the patterns relating to the concept of Test Doubles.
The deck is inspired by the patterns, and heavily reliant on extracts from the book.
The motivation for the deck is my belief that it is quite beneficial, when using and discussing Test Doubles, to rely on standardised
terminology and patterns.
Later on we’ll see how Gerard Meszaros describes the benefits of such standardisation.
@gerardmes
Gerard Meszaros
@philip_schwarz
This deck was designed using resources from Flaticon.com

While this deck may serve as an introduction to test doubles, or help reinforce the topic, it
is aimed at spelling out in sufficient detail the context and terminology necessary for a firm
understanding of the different types (variations) of Test Double, and their motivation.
Here is a diagram showing the different types. It is based on a diagram found in xUnit Test
Patterns, but I have annotated it with icons and added two types (variations) of Test Stub.

One of the small challenges faced when presenting the material covered in this slide deck is that some terms are
better defined only after introducing certain topics which:
• provide essential context for understanding the terms’ definitions…
• …and yet contain a few forward references to those as yet undefined terms!
In upcoming slides for example, you’ll come across two or three forward references to both of the following terms:
In practice however, it is quite feasible to make notes (mental or otherwise) of such forward references, so that
when the definitions of the terms are reached, the sections containing the forward references can then be
revisited in order to fully understand them.
Test
Stub
Mock
Object

In xUnit Test Patterns, each of the test doubles highlighted in green is a test pattern,
while the ones highlighted in orange are variations of the Test Stub pattern.
Before we look at the patterns, we are going to familiarise ourselves with the following:
1. some essential terminology
2. two other test patterns - State Verification and Behaviour Verification.

The next four slides consist of the the definition, from the
glossary of xUnit Test Patterns, of the following terms :
• system under test
• depended-on component
• interaction points:
• control point
• observation point
• input:
• direct
• indirect
• output:
• direct
• indirect

system under test (SUT)
Whatever thing we are testing. The SUT is always defined from the perspective of the test. When we are
writing unit tests, the SUT is whatever class (also known as CUT), object (also known as OUT), or method
(also known as MUT) we are testing; when we are writing customer tests, the SUT is probably the entire
application (also known as AUT) or at least a major subsystem of it. The parts of the application that we
are not verifying in this particular test may still be involved as a depended-on component (DOC).
depended-on component (DOC)
An individual class or a large-grained component on which the system under test (SUT) depends. The
dependency is usually one of delegation via method calls. In test automation, the DOC is primarily of interest
in that we need to be able to observe and control its interactions with the SUT to get complete test coverage.
SUT DOC

interaction point
A point at which a test interacts with the system under test (SUT).
An interaction point can be either a control point or an observation point.
control point
How the test asks the system under test (SUT) to do something for it. A control point could be
created for the purpose of setting up or tearing down the fixture or it could be used during
the exercise SUT phase of the test.
It is a kind of interaction point. Some control points are provided strictly for testing purposes;
they should not be used by the production code because they bypass input validation or short-
circuit the normal life cycle of the SUT or some object on which it depends.
observation point
The means by which the test observes the behavior of the system under test (SUT). This kind
of interaction point can be used to inspect the post-exercise state of the SUT or to monitor
interactions between the SUT and its depended-on components.
Some observation points are provided strictly for the tests; they should not be used by
the production code because they may expose private implementation details of the SUT that
cannot be depended on not to change.

direct input
A test may interact with the system under test (SUT) directly via its "front door" or public
application programming interface (API) or indirectly via its "back door."
The stimuli injected by the test into the SUT via its front door are direct inputs of the SUT. Direct
inputs may consist of method or function calls to another component or messages sent on a
message channel (e.g., MQ or JMS) and the arguments or contents thereof.
indirect input
When the behavior of the system under test (SUT) is affected by the values returned by another
component whose services it uses, we call those values the indirect inputs of the SUT.
Indirect inputs may consist of actual return values of functions, updated (out) parameters of
procedures or subroutines, and any errors or exceptions raised by the depended-on component
(DOC).
Testing of the SUT behavior with indirect inputs requires the appropriate control point on the
"back side" of the SUT.
We often use a Test Stub to inject the indirect inputs into the SUT.
SUT
Test
DOC
SUT
front door
back door

indirect output
When the behavior of the system under test (SUT) includes actions that cannot be observed
through the public application programming interface (API) of the SUT but that are seen or
experienced by other systems or application components, we call those actions the indirect outputs
of the SUT.
Indirect outputs may consist of method or function calls to another component, messages sent on a
message channel (e.g., MQ or JMS), and records inserted into a database or written to a file.
Verification of the indirect output behaviors of the SUT requires the use of
appropriate observation points on the "back side" of the SUT.
Mock Objects are often used to implement the observation point by intercepting the indirect
outputs of the SUT and comparing them to the expected values.
direct output
A test may interact with the system under test (SUT) directly via its "front door" or public
application programming interface (API) or indirectly via its "back door."
The responses received by the test from the SUT via its front door are direct outputs of the SUT.
Direct outputs may consist of the return values of method or function calls, updated arguments
passed by reference, exceptions raised by the SUT, or messages received on a message channel
(e.g., MQ or JMS) from the SUT.
SUT
Test
DOC
SUT
front door
back door

The next three slides contain diagrams aimed
at reinforcing our understanding of the terms
whose definitions we have just seen.

output
DOC
SUT
Test
input
output
input
output
input
direct input
direct output
indirect output
indirect input
Setup
Exercise
Verify
Teardown
Test
input
output

output
DOC
SUT
Test
input
output
input
output
input
direct input
direct output
indirect output
indirect input
Setup
Exercise
Verify
Teardown
Test
input
output
A test may interact with the system under test (SUT) directly via its "front door" or public application programming interface (API) or indirectly via its "back door."
front door back door

output
DOC
SUT
Test
input
output
input
output
input
direct input
direct output
indirect output
indirect input
potential observation points
Setup
Exercise
Verify
Teardown
Test
input
output
Verification of the indirect output behaviors of the SUT requires the use of appropriate observation points on the "back side" of the SUT.

output
DOC
SUT
Test
input
output
input
output
input
direct input
direct output
indirect output
indirect input
potential observation points
Setup
Exercise
Verify
Teardown
Test
input
output
Verification of the indirect output behaviors of the SUT requires the use of appropriate observation points on the "back side" of the SUT.
Testing of the SUT behavior with indirect inputs requires the appropriate control point on the "back side" of the SUT.
potential control points

Having familiarised ourselves with that essential terminology,
it is time to look at the State Verification pattern.
Let’s see some excerpts from xUnit Test Patterns.

State Verification
How do we make tests self-checking when there is state to be verified?
We inspect the state of the SUT after it has been exercised and
compare it to the expected state.
A Self-Checking Test (see page 26) must verify that the expected outcome has occurred without
manual intervention by whoever is running the test.
But what do we mean by "expected outcome"? The SUT may or may not be "stateful"; if it is
stateful, it may or may not have a different state after it has been exercised.
As test automaters, it is our job to determine whether our expected outcome is a change of final
state or whether we need to be more specific about what occurs while the SUT is being exercised.
State Verification involves inspecting the state of the SUT after it has been exercised.
Also known as: State-Based Testing
Fixture
DOC
Setup
Exercise
Verify
Teardown
SUT
Get State
Exercise
B
A C
Behaviour
(Indirect
Outputs)
Note that because we don’t intend to verify indirect output behaviours of the
SUT, we are not setting up observation points on the "back side" of the SUT.
State Verification
no observation points
observation points

How It Works
We exercise the SUT by invoking the methods of interest. Then, as a separate step, we interact with the SUT to
retrieve its post-exercise state and compare it with the expected end state by calling Assertion Methods (page 362).
Normally, we can access the state of the SUT simply by calling methods or functions that return its state. This is
especially true when we are doing test-driven development because the tests will have ensured that the state is
easily accessible. When we are retrofitting tests, however, we may find it more challenging to access the relevant
state information. In these cases, we may need to use a Test-Specific Subclass (page 579) or some other technique to
expose the state without introducing Test Logic in Production (page 217).
A related question is "Where is the state of the SUT stored?" Sometimes, the state is stored within the actual SUT; in
other cases, the state may be stored in another component such as a database. In the latter case, State
Verification may involve accessing the state within the other component (essentially a layer-crossing test). By
contrast, Behavior Verification (page 468) would involve verifying the interactions between the SUT and the other
component.
Fixture
DOC
Setup
Exercise
Verify
Teardown
SUT
Get State
Exercise
B
A C
Behaviour
(Indirect
Outputs)
When to Use It
We should use State Verification when we care about only the end state of the
SUT—not how the SUT got there. Taking such a limited view helps us maintain
encapsulation of the implementation of the SUT.
State Verification comes naturally when we are building the software inside
out. That is, we build the innermost objects first and then build the next layer of
objects on top of them. Of course, we may need to use Test Stubs (page 529) to
control the indirect inputs of the SUT to avoid Production Bugs (page 268)
caused by untested code paths. Even then, we are choosing not to verify the
indirect outputs of the SUT.
When we do care about the side effects of exercising the SUT that are not visible
in its end state (its indirect outputs), we can use Behavior Verification to
observe the behavior directly. We must be careful, however, not to create Fragile
Tests (page 239) by overspecifying the software.
see
next
slide
State Verification
observation points

extract from smell Production Bugs
Production Bugs
We find too many bugs during formal tests or in production.
Symptoms
We have put a lot of effort into writing automated tests, yet
the number of bugs showing up in formal (i.e., system) testing
or production remains too high.
Impact
…
Causes
…
Cause: Infrequently Run Tests …
Cause: Lost Test …
Cause: Missing Unit Test …
Cause: Untested Code
Cause: Untested Requirement …
Cause: Neverfail Test …
Symptoms
We may just "know" that some piece of code in the SUT is not being exercised by
any tests. Perhaps we have never seen that code execute, or perhaps we used code
coverage tools to prove this fact beyond a doubt. In the following example, how can
we test that when timeProvider throws an exception, this exception is handled
correctly?
…
Root Cause
The most common cause of Untested Code is that the SUT includes code paths that
react to particular ways that a depended-on component (DOC) behaves and we
haven't found a way to exercise those paths. Typically, the DOC is being called
synchronously and either returns certain values or throws exceptions. During
normal testing, only a subset of the possible equivalence classes of indirect inputs
are actually encountered.
Another common cause of Untested Code is incompleteness of the test suite caused
by incomplete characterization of the functionality exposed via the SUT's interface.
Possible Solution
If the Untested Code is caused by an inability to control the indirect inputs of the
SUT, the most common solution is to use a Test Stub (page 529) to feed the various
kinds of indirect inputs into the SUT to cover all the code paths. Otherwise, it may
be sufficient to configure the DOC to cause it to return the various indirect inputs
required to fully test the SUT.
see
previous
slide

Now that we have seen the State Verification pattern,
let’s turn to the Behaviour Verification pattern.
Again, let’s see some excerpts from xUnit Test Patterns.

Behaviour Verification
How do we make tests self-checking when there is no state to verify?
We capture the indirect outputs of the SUT as they occur and
compare them to the expected behavior.
A Self-Checking Test (see page 26) must verify that the expected outcome has occurred without
manual intervention by whoever is running the test.
But what do we mean by "expected outcome"? The SUT may or may not be "stateful"; if it is
stateful, it may or may not be expected to end up in a different state after it has been exercised.
The SUT may also be expected to invoke methods on other objects or components.
Behavior Verification involves verifying the indirect outputs of the SUT as it is being exercised.
Also known as: Interaction Testing
Note that because we intend to verify indirect output behaviours of the
SUT, we are setting up observation points on the "back side" of the SUT.
Setup
Exercise
Verify
Teardown
Fixture
DOC
Setup
Exercise
Verify
Teardown
SUT
Exercise
B
A C
Behaviour
(Indirect
Outputs)
Verify
observation points

When to Use It
Behavior Verification is primarily a technique for unit tests and component tests. We can use Behavior Verification whenever the SUT calls
methods on other objects or components. We must use Behavior Verification whenever the expected outputs of the SUT are transient and cannot
be determined simply by looking at the post-exercise state of the SUT or the DOC. This forces us to monitor these indirect outputs as they occur.
A common application of Behavior Verification is when we are writing our code in an "outside-in" manner. This approach, which is often
called need-driven development, involves writing the client code before we write the DOC. It is a good way to find out exactly what the interface
provided by the DOC needs to be based on real, concrete examples rather than on speculation. The main objection to this approach is that we need
to use a lot of Test Doubles (page 522) to write these tests. That could result in Fragile Tests (page 239) because each test knows so much about how
the SUT is implemented. Because the tests specify the behavior of the SUT in terms of its interactions with the DOC, a change in the implementation
of the SUT could break a lot of tests. This kind of Overspecified Software (see Fragile Test) could lead to High Test Maintenance Cost (page 265).
The jury is still out on whether Behavior Verification is a better approach than State Verification. In most cases, State Verification is clearly
necessary; in some cases, Behavior Verification is clearly necessary. What has yet to be determined is whether Behavior Verification should be
used in all cases or whether we should use State Verification most of the time and resort to Behavior Verification only when State
Verification falls short of full test coverage.
How It Works
Each test specifies not only how the client of the SUT interacts with it during the exercise
SUT phase of the test, but also how the SUT interacts with the components on which it
should depend. This ensures that the SUT really is behaving as specified rather than
just ending up in the correct post-exercise state.
Behavior Verification almost always involves interacting with or replacing a
depended-on component (DOC) with which the SUT interacts at runtime.
The line between Behavior Verification and State Verification (page 462) can get a bit
blurry when the SUT stores its state in the DOC because both forms of verification
involve layer-crossing tests. We can distinguish between the two cases based on whether
we are verifying the post-test state in the DOC (State Verification) or whether we are
verifying the method calls made by the SUT on the DOC (Behavior Verification).
Setup
Exercise
Verify
Teardown
Setup
Exercise
Verify
Teardown
Fixture
DOC
Setup
Exercise
Verify
Teardown
SUT
Exercise
B
A C
Behaviour
(Indirect
Outputs)
Verify
See next slide for an example of when to use Behaviour Verification
observation points

extract from smell Production Bugs
Production Bugs
We find too many bugs during formal tests or in production.
Symptoms
We have put a lot of effort into writing automated tests, yet
the number of bugs showing up in formal (i.e., system) testing
or production remains too high.
Impact
…
Causes
…
Cause: Infrequently Run Tests …
Cause: Lost Test …
Cause: Missing Unit Test …
Cause: Untested Requirement
Cause: Neverfail Test …
Cause: Untested Requirement
Symptoms
We may just "know" that some piece of functionality is not being tested. Alternatively, we
may be trying to test a piece of software but cannot see any visible functionality that can be
tested via the public interface of the software. All the tests we have written pass, however.
When doing test-driven development, we know we need to add some code to handle a
requirement. However, we cannot find a way to express the need for code to …
…
Root Cause
The most common cause of Untested Requirements is that the SUT includes behavior that
is not visible through its public interface. It may have expected "side effects" that cannot be
observed directly by the test (such as writing out a file or record or calling a method on
another object or component)—in other words, it may have indirect outputs.
When the SUT is an entire application, the Untested Requirement may be a result of not
having a full suite of customer tests that verify all aspects of the visible behavior of the SUT.
Possible Solution
If the problem is missing customer tests, we need to write at least enough customer tests to
ensure that all components are integrated properly. This may require improving the
design-for-testability of the application by separating the presentation layer from the
business logic layer.
When we have indirect outputs that we need to verify, we can do Behavior
Verification (page 468) through the use of Mock Objects (page 544). Testing of indirect
outputs is covered in Chapter 11, Using Test Doubles.

I find it is useful to compare the diagrams for State Verification
and Behaviour Verification, which is what the next slide does.

Fixture
DOC
Setup
Exercise
Verify
Teardown
SUT
Get State
Exercise
B
A C
Behaviour
(Indirect
Outputs)
Fixture
DOC
Setup
Exercise
Verify
Teardown
SUT
Exercise
B
A C
Behaviour
(Indirect
Outputs)
Verify
State Verification
observation points
observation points

Let’s finally turn to the test double patterns.
Let’s start with Test Double.

Fixture
DOC
Setup
Exercise
Verify
Teardown
SUT
Test
Double
Test Double
How can we verify logic independently when code it depends on
is unusable?
How can we avoid Slow Tests?
We replace a component on which the SUT depends with a
"test-specific equivalent."
Sometimes it is just plain hard to test the SUT because it depends on other components that cannot be used in the test environment.
Such a situation may arise because those components aren't available, because they will not return the results needed for the test, or
because executing them would have undesirable side effects.
In other cases, our test strategy requires us to have more control over or visibility of the internal behavior of the SUT.
When we are writing a test in which we cannot (or choose not to) use a real depended-on component (DOC), we can replace it with
a Test Double. The Test Double doesn't have to behave exactly like the real DOC; it merely has to provide the same API as the real
DOC so that the SUT thinks it is the real one!
Also known as: Imposter
indirect output
indirect input
Test Double

How It Works
When the producers of a movie want to film something that is potentially risky or dangerous for the leading actor to carry out, they hire a
"stunt double" to take the place of the actor in the scene. The stunt double is a highly trained individual who is capable of meeting
the specific requirements of the scene. The stunt double may not be able to act, but he or she knows how to fall from great heights, crash
a car, or do whatever the scene calls for. How closely the stunt double needs to resemble the actor depends on the nature of the
scene. Usually, things can be arranged such that someone who vaguely resembles the actor in stature can take the actor's place.
For testing purposes, we can replace the real DOC (not the SUT!) with our equivalent of the "stunt double": the Test Double. During the
fixture setup phase of our Four-Phase Test (page 358), we replace the real DOC with our Test Double. Depending on the kind of test we
are executing, we may hard-code the behavior of the Test Double or we may configure it during the setup phase. When the SUT interacts
with the Test Double, it won't be aware that it isn't talking to the real McCoy, but we will have achieved our goal of making impossible
tests possible.
Regardless of which variation of Test Double we choose to use, we must keep in mind that we don't need to implement the entire interface
of the DOC. Instead, we provide only the functionality needed for our particular test. We can even build different Test Doubles for
different tests that involve the same DOC.
Fixture
DOC
Setup
Exercise
Verify
Teardown
SUT
Test
Double
Test Double

As an aside, the icon chosen for Test Double is not great, but
that’s not going to be a big issue because we won’t be using the
icon as much as the ones for the various Test Double variations.
For what it is worth, here is how I arrived at it.
Stunt Double Test Double
“For testing purposes, we can replace the real DOC (not the SUT!) with our equivalent of the "stunt double": the Test Double.“

When to Use it
We might want to use some sort of Test Double during our tests in the following circumstances:
• If we have an Untested Requirement (see Production Bugs on page 268) because neither the SUT nor its DOCs provide
an observation point for the SUT's indirect output that we need to verify using Behavior Verification (page 468)
• If we have Untested Code (see Production Bugs) and a DOC does not provide the control point to allow us to exercise the
SUT with the necessary indirect inputs
• If we have Slow Tests (page 253) and we want to be able to run our tests more quickly and hence more often
Each of these scenarios can be addressed in some way by using a Test Double. Of course, we have to be careful when
using Test Doubles because we are testing the SUT in a different configuration from the one that will be used in
production. For this reason, we really should have at least one test that verifies the SUT works without a Test Double. We
need to be careful that we don't replace the parts of the SUT that we are trying to verify because that practice can result in
tests that test the wrong software! Also, excessive use of Test Doubles can result in Fragile Tests (page 239) as a result
of Overspecified Software.
Test Doubles come in several major flavors, as summarized in Figure 23.1. The implementation variations of these
patterns are described in more detail in the corresponding pattern write-ups.
Figure 23.1. Types of Test Doubles. Dummy Objects are really an alternative to the value patterns. Test Stubs are used to
verify indirect inputs; Test Spies and Mock Objects are used to verify indirect outputs. Fake objects provide an alternative
implementation.
Test Double
These variations are classified based on how/why
we use the Test Double. We will deal with variations
around how we build the Test Doubles in the
"Implementation" section.

Our coverage of the Test Double pattern continues on the next slide with
an introductory look at Test Double usage pattern types (variations).

We use a Test Stub (page 529) to replace a real component on which the SUT depends so that the test has a control point for the indirect
inputs of the SUT. Its inclusion allows the test to force the SUT down paths it might not otherwise execute. We can further classify Test
Stubs by the kind of indirect inputs they are used to inject into the SUT. A Responder (see Test Stub) injects valid values, while
a Saboteur (see Test Stub) injects errors or exceptions.
Some people use the term "test stub" to mean a temporary implementation that is used only until the real object or procedure becomes
available. I prefer to call this usage a Temporary Test Stub (see Test Stub) to avoid confusion.
We can use a more capable version of a Test Stub, the Test Spy (page 538), as an observation point for the indirect outputs of the SUT.
Like a Test Stub, a Test Spy may need to provide values to the SUT in response to method calls. The Test Spy, however, also captures the
indirect outputs of the SUT as it is exercised and saves them for later verification by the test. Thus, in many ways, the Test Spy is "just
a" Test Stub with some recording capability. While a Test Spy is used for the same fundamental purpose as a Mock Object (page 544),
the style of test we write using a Test Spy looks much more like a test written with a Test Stub.
We can use a Mock Object as an observation point to verify the indirect outputs of the SUT as it is exercised. Typically, the Mock
Object also includes the functionality of a Test Stub in that it must return values to the SUT if it hasn't already failed the tests but the
emphasis1 is on the verification of the indirect outputs. Therefore, a Mock Object is a lot more than just a Test Stub plus assertions: It is
used in a fundamentally different way.
We use a Fake Object (page 551) to replace the functionality of a real DOC in a test for reasons other than verification of indirect inputs
and outputs of the SUT. Typically, a Fake Object implements the same functionality as the real DOC but in a much simpler way. While
a Fake Object is typically built specifically for testing, the test does not use it as either a control point or an observation point.
The most common reason for using a Fake Object is that the real DOC is not available yet, is too slow, or cannot be used in the test
environment because of deleterious side effects. The sidebar "Faster Tests Without Shared Fixtures" (page 319) describes how we
encapsulated all database access behind a persistence layer interface and then replaced the database with in-memory hash tables and
made our tests run 50 times faster. Chapter 6, Test Automation Strategy, and Chapter 11, Using Test Doubles, provide an overview of the
various techniques available for making our SUT easier to test.
Some method signatures of the SUT may require objects as parameters. If neither the test nor the SUT cares about these objects, we may
choose to pass in a Dummy Object (page 728), which may be as simple as a null object reference, an instance of the Object class, or an
instance of a Pseudo-Object (see Hard-Coded Test Double on page 568). In this sense, a Dummy Object isn't really a Test Double per se
but rather an alternative to the value patterns Literal Value (page 714), Derived Value (page 718), and Generated Value (page 723).
Test
Stub
Test
Spy
Mock
Object
Fake
Object
Dummmy
Object
Responder
Saboteur
Test Double Usage Pattern Variations

After that introductory look at Test Double usage pattern
types (variations), it is time to see how Gerard Meszaros
describes the benefits of standardising terminology.

Why Standardize Testing Patterns?
…I think it is important for us to standardize the names of the test automation patterns, especially those related
to Test Stubs (page 529) and Mock Objects (page 544). The key issue here relates to succinctness of
communication.
When someone tells you, "Put a mock in it" (pun intended!), what advice is that person giving you?
Depending on what the person means by a "mock," he or she could be suggesting that you control the indirect
inputs of your SUT using a Test Stub or that you replace your database with a Fake Database (see Fake
Object on page 551) that will reduce test interactions and speed up your tests by a factor of 50. …
Or perhaps the person is suggesting that you verify that your SUT calls the correct methods by installing
an Eager Mock Object (see Mock Object) preconfigured with the Expected Behavior (see Behavior
Verification on page 468).
If everyone used "mock" to mean a Mock Object—no more or less—then the advice would be pretty clear.
As I write this, the advice is very murky because we have taken to calling just about any Test Double (page 522)
a "mock object" (despite the objections of the authors of the original paper on Mock Objects [ET]).

The next slide introduces two Test Double
implementation (construction) pattern variations.

Next, here is Pseudo-Object, a variation
of the Hard-Coded Test Double pattern.
Variation: Pseudo-Object
One challenge facing writers of Hard-Coded Test Doubles is that we must implement all the methods in the interface that the
SUT might call. In statically typed languages such as Java and C#, we must at least implement all methods declared in the interface
implied by the class or type associated with however we access the DOC. This often "forces" us to subclass from the real DOC to avoid
providing dummy implementations for these methods.
One way of reducing the programming effort is to provide a default class that implements all the interface methods and throws a
unique error. We can then implement a Hard-Coded Test Double by subclassing this concrete class and overriding just the one
method we expect the SUT to call while we are exercising it. If the SUT calls any other methods, the Pseudo-Object throws an error,
thereby failing the test.
PSEUDO-OBJECT

We are now going to go through each Test Double usage pattern type (variation) in turn, beginning with Test Stub.
As we work our way through the different types, we shall also look at code examples of their usage.

In many circumstances, the environment or context in which the SUT operates very much influences the
behavior of the SUT. To get adequate control over the indirect inputs of the SUT, we may have to replace some of
the context with something we can control —namely, a Test Stub.
How It Works
First, we define a test-specific implementation of an interface on which the SUT depends. This implementation is
configured to respond to calls from the SUT with the values (or exceptions) that will exercise the Untested
Code (see Production Bugs on page 268) within the SUT. Before exercising the SUT, we install the Test Stub so
that the SUT uses it instead of the real implementation. When called by the SUT during test execution, the Test
Stub returns the previously defined values. The test can then verify the expected outcome in the normal way.
Fixture
DOC
Setup
Exercise
Verify
Teardown
SUT
Test
Stub
Test Stub
How can we verify logic independently when it depends on
indirect inputs from other software components?
We replace a real object with a test-specific object that
feeds the desired indirect inputs into the SUT.
Also known as: Stub
indirect
input
indirect output
indirect input
Test Stub
Return
Values
Installation
Creation
front
door
back
door
observation
point
control
point

When to Use it
A key indication for using a Test Stub is having Untested Code caused by our inability to control the indirect inputs of the SUT.
We can use a Test Stub as a control point that allows us to control the behavior of the SUT with various indirect inputs and we
have no need to verify the indirect outputs. We can also use a Test Stub to inject values that allow us to get past a particular
point in the software where the SUT calls software that is unavailable in our test environment.
If we do need an observation point that allows us to verify the indirect outputs of the SUT, we should consider using a Mock
Object (page 544) or a Test Spy (page 538). Of course, we must have a way of installing a Test Double (page 522) into the SUT to
be able to use any form of Test Double.
Test Stub
A Test Stub that is used to inject invalid indirect inputs into the SUT is often called
a Saboteur because its purpose is to derail whatever the SUT is trying to do so that we
can see how the SUT copes under these circumstances. The "derailment" might be
caused by returning unexpected values or objects, or it might result from raising an
exception or causing a runtime error. Each test may be either a Simple Success Test or
an Expected Exception Test (see Test Method), depending on how the SUT is expected
to behave in response to the indirect input.
Fixture
DOC
Setup
Exercise
Verify
Teardown
SUT
Test
Stub
indirect
input
Return
Values
Installation
Creation
front
door
back
door
observation
point
control
point
A Test Stub that is used to inject valid indirect inputs into the SUT so that it can go
about its business is called a Responder. Responders are commonly used in "happy
path" testing when the real component is uncontrollable, is not yet available, or is
unusable in the development environment. The tests will invariably be Simple Success
Tests (see Test Method on page 348).
Variation: Responder
Variation: Saboteur

Now that we have seen the Test Stub pattern, we
are going to look at examples of its usage.
Among the Configurable Test Double variations in
xUnit Test Patterns there are the Hand-Built Test
Double, and the Dynamically Generated Test
Double. The first one can be seen below. The second
one will be covered in part two of this deck.
The Test Stub examples that we are about to see are
all either hand-built or hard-coded. We will look at
dynamically generated Test Stub examples in part 2.
Variation: Hand-Built Test Double
A Hand-Built Test Double is one that was defined by the test automater for one or more specific tests. A Hard-Coded Test
Double is inherently a Hand-Built Test Double, while a Configurable Test Double can be either hand-built or generated.
This book uses Hand-Built Test Doubles in a lot of the examples because it is easier to see what is going on when we have
actual, simple, concrete code to look at. This is the main advantage of using a Hand-Built Test Double; indeed, some people
consider this benefit to be so important that they use Hand-Built Test Doubles exclusively. We may also use a Hand-Built Test
Double when no third-party toolkits are available or if we are prevented from using those tools by project or corporate policy.
Hand
Built

Configurable
Hard-Coded
Test
Stub
Responder
Saboteur
Test
Stub
Responder Configurable
Configurable Typelevel Rite
of Passage
Test
Stub
Responder
To conclude part 1, we are going to pick, from each of the
following books, one example of using the Test Stub pattern.
Hard-Coded
Test
Stub
Responder
Hard-Coded
Test
Stub
Responder
PSEUDO-OBJECT
!
1
2
!
1
2

The first example of using the Test Stub pattern is from the book Scala Programming Projects.
It is somewhat similar to the pattern’s motivating example in xUnit Test Patterns (see next slide).
That’s because in both examples, the system clock is a DOC that gets replaced by a Test Stub.
Hard-Coded
Test
Stub
Responder
PSEUDO-OBJECT
!
1
2
!
1
2
https://github.com/PacktPublishing/Scala-Programming-Projects/blob/master/Chapter10-11/bitcoin-analyser

Test Stub
…
Motivating Example
The following test verifies the basic functionality of a component that formats an HTML string containing the current time. Unfortunately, it
depends on the real system clock so it rarely ever passes!
…
Refactoring Notes
We can achieve proper verification of the indirect inputs by getting control of the time. To do so, we use the Replace Dependency with Test Double
(page 522) refactoring to replace the real system clock (represented here by TimeProvider) with a Virtual Clock [VCTP]. We then implement it as
a Test Stub that is configured by the test with the time we want to use as the indirect input to the SUT.
[VCTP]
The Virtual Clock Test Pattern
http://www.nusco.org/docs/virtual_clock.pdf
By: Paolo Perrotta
This paper describes a common example of a Responder called Virtual Clock [VCTP]. The author uses the Virtual Clock Test Pattern as a Decorator
[GOF] for the real system clock, which allows the time to be "frozen" or resumed. One could use a Hard-Coded Test Stub or a Configurable Test
Stub just as easily for most tests. Paolo Perrotta summarizes the thrust of his article:
We can have a hard time unit-testing code that depends on the system clock. This paper describes both the problem and a common, reusable solution.

Example #1
DOC
SparkSession
Timer[IO]
BatchProducer SUT
Timer[F]
DOC DOC
SparkSession
FakeTimer
BatchProducer SUT
Timer[F]
BatchProducerIT
Test
Stub
Production Integration Test
Responder
PSEUDO-OBJECT
!
1
2 !
1
2

class AppContext(val transactionStorePath: URI)
(implicit val spark: SparkSession,
implicit val timer: Timer[IO])
object BatchProducer {
val WaitTime: FiniteDuration = 59.minute
/** Number of seconds required by the API to make a transaction visible */
val ApiLag: FiniteDuration = 5.seconds
…
def processOneBatch(fetchNextTransactions: IO[Dataset[Transaction]],
transactions: Dataset[Transaction],
saveStart: Instant,
saveEnd: Instant)(implicit appCtx: AppContext)
: IO[(Dataset[Transaction], Instant, Instant)] = {
import appCtx._
val transactionsToSave = filterTxs(transactions, saveStart, saveEnd)
for {
_ <- BatchProducer.save(transactionsToSave, appCtx.transactionStorePath)
_ <- IO.sleep(WaitTime)
beforeRead <- currentInstant
// We are sure that lastTransactions contain all transactions until end
end = beforeRead.minusSeconds(ApiLag.toSeconds)
nextTransactions <- fetchNextTransactions
} yield (nextTransactions, saveEnd, end)
}
…
def currentInstant(implicit timer: Timer[IO]): IO[Instant] =
timer.clockRealTime(TimeUnit.SECONDS) map Instant.ofEpochSecond
DOC
SparkSession
FakeTimer
BatchProducer SUT
Timer[F]
BatchProducerIT
Test
Stub
Responder
PSEUDO-OBJECT
!
1
2 !
1
2
def sleep(duration: FiniteDuration)(implicit timer: Timer[IO]): IO[Unit] =
timer.sleep(duration)
cats.effect.IO
Example #1
Test Stub
• FakeTimer
Control Point
• clockRealTime function of FakeTimer
Indirect Input
• time returned by clockRealTime function

implicit object FakeTimer extends Timer[IO] {
private var clockRealTimeInMillis: Long =
Instant.parse("2018-08-02T01:00:00Z").toEpochMilli
def clockRealTime(unit: TimeUnit): IO[Long] =
IO(unit.convert(clockRealTimeInMillis, TimeUnit.MILLISECONDS))
def sleep(duration: FiniteDuration): IO[Unit] = IO {
clockRealTimeInMillis = clockRealTimeInMillis + duration.toMillis
}
def shift: IO[Unit] = ???
def clockMonotonic(unit: TimeUnit): IO[Long] = ???
}
DOC
SparkSession
FakeTimer
BatchProducer SUT
Timer[F]
BatchProducerIT
Test
Stub
Responder
PSEUDO-OBJECT
Hard-Coded
!
1
2
!
1
2
Example #1
In the usual way of implementing the Pseudo-Object pattern, we would have (1) a base class providing undefined implementations (???
expressions) of all Timer’s functions, and (2) a concrete subclass that overrides only the functions we expect the SUT to call. Instead, here we
have a concrete Timer that provides (1) real implementations of the functions we expect the SUT to call, and (2) undefined implementations of
functions that the SUT is NOT expected to call.
By the way, note that while the term `fake` is included in the name of the timer implementation, it is not being used in a way that is consistent
with the terminology described in this deck series.
Test Stub
• FakeTimer
Control Point
Indirect Input

class BatchProducerIT extends WordSpec with Matchers with SharedSparkSession {
…
"BatchProducer.processOneBatch" should {
"filter and save a batch of transaction, wait 59 mn, fetch the next batch" in withTempDir { tmpDir =>
implicit object FakeTimer extends Timer[IO] { … }
implicit val appContext: AppContext = new AppContext(transactionStorePath = tmpDir.toURI)
implicit def toTimestamp(str: String): Timestamp = Timestamp.from(Instant.parse(str))
val tx1 = Transaction("2018-08-01T23:00:00Z", 1, 7657.58, true, 0.021762)
val tx2 = Transaction("2018-08-02T01:00:00Z", 2, 7663.85, false, 0.01385517)
val tx5 = Transaction("2018-08-02T02:30:00Z", 5, 7661.49, true, 0.1)
// Start at 01:00, tx 2 ignored (too soon)
val txs0 = Seq(tx1)
// Fetch at 01:59, get nb 2 and 3, but will miss nb 4 because of Api lag
val txs1 = Seq(tx2, tx3)
// Fetch at 02:58, get nb 3, 4, 5
val txs2 = Seq(tx3, tx4, tx5)
// Fetch at 03:57, get nothing
val txs3 = Seq.empty[Transaction]
val start0 = Instant.parse("2018-08-02T00:00:00Z")
val end0 = Instant.parse("2018-08-02T00:59:55Z")
val threeBatchesIO = …
val (ds1, start1, end1, ds2, start2, end2) = threeBatchesIO.unsafeRunSync()
ds1.collect() should contain theSameElementsAs txs1
start1 should ===(end0)
// initialClock + 1mn - 15s - 5s
end1 should ===(Instant.parse("2018-08-02T01:58:55Z"))
ds2.collect() should contain theSameElementsAs txs2
start2 should ===(end1)
// initialClock + 1mn -15s + 1mn -15s -5s = end1 + 45s
end2 should ===(Instant.parse("2018-08-02T02:57:55Z"))
val lastClock = Instant.ofEpochMilli(FakeTimer.clockRealTime(TimeUnit.MILLISECONDS).unsafeRunSync())
lastClock should === (Instant.parse("2018-08-02T03:57:00Z"))
…
DOC
SparkSession
FakeTimer
BatchProducer SUT
Timer[F]
BatchProducerIT
Test
Stub
val threeBatchesIO =
for {
tuple1 <- BatchProducer.processOneBatch(IO(txs1.toDS()), txs0.toDS(), start0, end0) // end - Api lag
(ds1, start1, end1) = tuple1
tuple2 <- BatchProducer.processOneBatch(IO(txs2.toDS()), ds1, start1, end1)
(ds2, start2, end2) = tuple2
_ <- BatchProducer.processOneBatch(IO(txs3.toDS()), ds2, start2, end2)
} yield (ds1, start1, end1, ds2, start2, end2)
Example #1
Test Stub
• FakeTimer
Control Point
Indirect Input

The second example is from the book Pure functional HTTP APIs in Scala.
Configurable
Test
Stub
Responder
Hand
Built
https://github.com/jan0sch/pfhais/blob/main/pure/src/test/scala/com/wegtam/books/pfhais/pure/api/
Example #2

TestRepository
ProductRoutes
ProductRoutesTest SUT
DoobieRepository
ProductRoutes SUT
DOC
Production Unit Test
Repository Repository
Test
Stub
Responder
Example #2
DOC

final class ProductRoutes[F[_]: Sync](repo: Repository[F]) extends Http4sDsl[F] {
implicit def decodeProduct: EntityDecoder[F, Product] = jsonOf
implicit def encodeProduct[A[_]: Applicative]: EntityEncoder[A, Product] = jsonEncoderOf
val routes: HttpRoutes[F] = HttpRoutes.of[F] {
case GET -> Root / "product" / UUIDVar(id) =>
for {
rows <- repo.loadProduct(id)
resp <- Product.fromDatabase(rows).fold(NotFound())(p => Ok(p))
} yield resp
case req @ PUT -> Root / "product" / UUIDVar(id) =>
…
}
}
class TestRepository[F[_]: Effect](data: Seq[Product]) extends Repository[F] {
override def loadProduct(id: ProductId): F[Seq[(ProductId, LanguageCode, ProductName)]] =
data.find(_.id === id) match {
case None => Seq.empty.pure[F]
case Some(p) =>
val ns = p.names.toNonEmptyList.toList.to[Seq]
ns.map(n => (p.id, n.lang, n.name)).pure[F]
}
…
}
trait Repository[F[_]] {
/**
* Load a product from the database repository.
*
* @param id The unique ID of the product.
* @return A list of database rows for a single product which you'll need to combine.
*/
def loadProduct(id: ProductId): F[Seq[(ProductId, LanguageCode, ProductName)]]
…
}
TestRepository
ProductRoutes
Repository
Test Stub
• TestRepository
Control Point
• loadProduct function of TestRepository
Indirect Input
• sequence of tuples returned by loadProduct function
Test
Stub
Responder
Configurable

final class ProductRoutesTest extends BaseSpec {
implicit def decodeProduct: EntityDecoder[IO, Product] = jsonOf
implicit def encodeProduct[A[_]: Applicative]: EntityEncoder[A, Product] = jsonEncoderOf
private val emptyRepository: Repository[IO] = new TestRepository[IO](Seq.empty)
"ProductRoutes" when {
"GET /product/ID" when {
"product does not exist" must {
val expectedStatusCode = Status.NotFound
s"return $expectedStatusCode" in {
forAll("id") { id: ProductId =>
Uri.fromString("/product/" + id.toString) match {
case Left(_) => fail("Could not generate valid URI!")
case Right(u) =>
def service: HttpRoutes[IO] =
Router("/" -> new ProductRoutes(emptyRepository).routes)
val response: IO[Response[IO]] = service.orNotFound.run(
Request(method = Method.GET, uri = u)
)
val result = response.unsafeRunSync
result.status must be(expectedStatusCode)
result.body.compile.toVector.unsafeRunSync must be(empty) }}}}
"product exists" must {
val expectedStatusCode = Status.Ok
s"return $expectedStatusCode and the product" in {
forAll("product") { p: Product =>
Uri.fromString("/product/" + p.id.toString) match {
case Left(_) => fail("Could not generate valid URI!")
case Right(u) =>
val repo: Repository[IO] = new TestRepository[IO](Seq(p))
def service: HttpRoutes[IO] =
Router("/" -> new ProductRoutes(repo).routes)
val response: IO[Response[IO]] = service.orNotFound.run(
Request(method = Method.GET, uri = u)
)
val result = response.unsafeRunSync
result.status must be(expectedStatusCode)
result.as[Product].unsafeRunSync must be(p)}}}}
}
TestRepository
ProductRoutes
Repository
Test
Stub
Responder
Test Stub
• TestRepository
Control Point
• loadProduct function of TestRepository
Indirect Input
• sequence of tuples returned by loadProduct function

The third example is from the book Practical FP in Scala.
Saboteur
Test
Stub
Responder Configurable
Configurable
Hand
Built
https://github.com/gvolpe/pfps-shopping-cart/tree/second-edition/modules/tests/src/test/scala/shop

Items
ItemRoutes SUT
DOC
Items
TestItems
ItemRoutes
Items
ItemRoutesSuite SUT
dataItems failingItems
Test
Stub
Responder Saboteur

def dataItems(items: List[Item]) = new TestItems {
override def findAll: IO[List[Item]] =
IO.pure(items)
override def findBy(brand: BrandName): IO[List[Item]] =
IO.pure(items.find(_.brand.name === brand).toList)
}
def failingItems(items: List[Item]) = new TestItems {
override def findAll: IO[List[Item]] =
IO.raiseError(DummyError) *> IO.pure(items)
override def findBy(brand: BrandName): IO[List[Item]] =
findAll
}
protected class TestItems extends Items[IO] {
def findAll: IO[List[Item]] = IO.pure(List.empty)
def findBy(brand: BrandName): IO[List[Item]] = IO.pure(List.empty)
def findById(itemId: ItemId): IO[Option[Item]] = IO.pure(none[Item])
def create(item: CreateItem): IO[ItemId] = ID.make[IO, ItemId]
def update(item: UpdateItem): IO[Unit] = IO.unit
}
final case class ItemRoutes[F[_]: Monad](
items: Items[F]
) extends Http4sDsl[F] {
private[routes] val prefixPath = "/items"
object BrandQueryParam extends OptionalQueryParamDecoderMatcher[BrandParam]("brand")
private val httpRoutes: HttpRoutes[F] = HttpRoutes.of[F] {
case GET -> Root :? BrandQueryParam(brand) =>
Ok(brand.fold(items.findAll)(b => items.findBy(b.toDomain)))
}
val routes: HttpRoutes[F] = Router(prefixPath -> httpRoutes)
}
trait Items[F[_]] {
def findAll: F[List[Item]]
def findBy(brand: BrandName): F[List[Item]]
def findById(itemId: ItemId): F[Option[Item]]
def create(item: CreateItem): F[ItemId]
def update(item: UpdateItem): F[Unit]
}
TestItems
ItemRoutes
Items
ItemRoutesSuite SUT
Test
Stub
Responder Saboteur
Test Stubs
• dataItems, failingItems
Control Point
• findAll function of dataItems and failingItems
Indirect Input
• list of items
Configurable
Configurable

object ItemRoutesSuite extends HttpSuite {
…
test("GET items succeeds") {
forall(Gen.listOf(itemGen)) { it =>
val req = GET(uri"/items")
val routes = ItemRoutes[IO](dataItems(it)).routes
expectHttpBodyAndStatus(routes, req)(it, Status.Ok)
}
}
…
test("GET items fails") {
forall(Gen.listOf(itemGen)) { it =>
val req = GET(uri"/items")
val routes = ItemRoutes[IO](failingItems(it)).routes
expectHttpFailure(routes, req)
}
…
}
Test Stubs
• dataItems, failingItems
Control Point
• findAll function of dataItems and failingItems
Indirect Input
• list of items
TestItems
ItemRoutes
Items
ItemRoutesSuite SUT
Test
Stub
Responder Saboteur

The fourth and final example is from the course Typelevel Rite of Passage.
Typelevel Rite
of Passage
Hard-Coded
Test
Stub
Responder
Hard-Coded
Test
Stub
Responder
https://rockthejvm.com/courses/typelevel-rite-of-passage
https://github.com/rockthejvm/typelevel-rite-of-passage

AuthRoutes
AuthRoutesSpec
mockedAuthenticator
mockedAuth
AuthRoutes SUT
Auth
DOC
Users Tokens Emails
JWTAuthenticator
LiveAuth
AuthRoutes SUT
Auth
Backing
Store
Identity
Store
DOC DOC
DOC
Test
Stub
Responder
Identity
Store
idStore
Test
Stub
Responder
Typelevel Rite
of Passage

Note that while the term `mocked` is
included in the names of the highlighted
items on the right, it is not being used in
a way that is consistent with the
terminology described in this deck series.
AuthRoutes
AuthRoutesSpec
mockedAuthenticator
mockedAuth
SUT
Auth DOC
DOC
Test
Stub
Responder
Identity
Store
idStore
Test
Stub
Responder
Unit Test
Typelevel Rite
of Passage

type Authenticator[F[_]] = JWTAuthenticator[F, String, User, Crypto]
class AuthRoutes[F[_]: Concurrent: Logger: SecuredHandler] private (
auth: Auth[F],
authenticator: Authenticator[F]
) extends HttpValidationDsl[F] {
// POST /auth/login { LoginInfo } => 200 Ok with Authorization Bearer {jwt}
private val loginRoute: HttpRoutes[F] = HttpRoutes.of[F] {
case req@POST -> Root / "login" =>
req.validate[LoginInfo] { loginInfo =>
val maybeJwtToken =
for
maybeUser <- auth.login(loginInfo.email, loginInfo.password)
_ <- Logger[F].info(s"User logging in: ${loginInfo.email} ")
maybeToken <- maybeUser.traverse(user => authenticator.create(user.email))
yield maybeToken
maybeJwtToken.map {
case Some(token) => authenticator.embed(Response(Status.Ok), token)
case None => Response(Status.Unauthorized)
}
}
}
…
}
AuthRoutes
AuthRoutesSpec
mockedAuthenticator
mockedAuth
AuthRoutes SUT
Auth DOC
DOC
Test
Stub
Responder
idStore
Test
Stub
Responder
Typelevel Rite
of Passage

val mockedAuth: Auth[IO] = probedAuth(None)
def probedAuth(maybeRefEmailToTokenMap: Option[Ref[IO, Map[String, String]]]): Auth[IO] = new Auth[IO]:
override def login(email: String, password: String): IO[Option[User]] =
if email == danielEmail && password == danielPassword
then IO(Some(Daniel))
else IO.pure(None)
override def signup(newPasswordInfo: NewUserInfo): IO[Option[User]] = …
override def changePassword(email: String, newPasswordInfo: NewPasswordInfo): IO[Either[String, Option[User]]] = …
override def delete(email: String): IO[Boolean] = IO.pure(true)
override def sendPasswordRecoveryToken(email: String): IO[Unit] = …
override def recoverPasswordFromToken(email: String, token: String, newPassword: String): IO[Boolean] = …
type Authenticator[F[_]] = JWTAuthenticator[F, String, User, Crypto]
trait SecuredRouteFixture extends UserFixture:
val mockedAuthenticator: Authenticator[IO] = {
val key = HMACSHA256.unsafeGenerateKey
val idStore: IdentityStore[IO, String, User] = (email: String) =>
if email == danielEmail then OptionT.pure(Daniel)
else if email == riccardoEmail then OptionT.pure(Riccardo)
else OptionT.none[IO, User]
JWTAuthenticator.unbacked.inBearerToken(
expiryDuration = 1.day,
maxIdle = None,
identityStore = idStore,
signingKey = key
)
}
Test
Stub
Responder
Test
Stub
Responder Hard-Coded Typelevel Rite
of Passage
Hard-Coded

class AuthRoutesSpec extends AsyncFreeSpec with AsyncIOSpec with Matchers with Http4sDsl[IO] with SecuredRouteFixture {
…
val authRoutes: HttpRoutes[IO] =
AuthRoutes[IO](mockedAuth, mockedAuthenticator).routes
given Logger[IO] = Slf4jLogger.getLogger[IO]
"AuthRoutes" - {
"should return a 401 Unauthorised if login fails" in {
for
response <- authRoutes.orNotFound.run(
Request(method = Method.POST, uri"/auth/login")
.withEntity(
LoginInfo(email = danielEmail, password = "wrongpassword")))
yield response.status shouldBe Status.Unauthorized
}
"should return a 200 Ok + a JWT if login is successful" in {
for
response <- authRoutes.orNotFound.run(
Request(method = Method.POST, uri"/auth/login")
.withEntity(
LoginInfo(email = danielEmail, password = danielPassword)))
yield {
response.status shouldBe Status.Ok
response.headers.get(ci"Authorization") shouldBe defined
}
}
…
}
}
AuthRoutes
mockedAuthenticator
mockedAuth
AuthRoutes SUT
Auth
Test
Stub
Responder
idStore
Test
Stub
Responder
Typelevel Rite
of Passage
AuthRoutesSpec

That’s all for part 1.
I hope you found it useful.
See you in part 2.

Test Doubles - Terminology, Definitions and Illustrations - with Examples - Part 1

More Related Content

Similar to Test Doubles - Terminology, Definitions and Illustrations - with Examples - Part 1

More from Philip Schwarz

Recently uploaded