How does it work?
First, you need to write a concurrency unit test. These tests allow (and encourage) the use of concurrency and non-determinism, which you would normally avoid in traditional unit tests due to flakiness. By writing such a test, Coyote allows you to exercise what happens, for example, when two requests execute concurrently in your service, as if it was deployed in production.
To use Coyote, you do not need to change a single line in your production code! Using binary
coyote rewrite tool will instrument your
program-under-test with hooks and stubs that allow Coyote to take control of
Task objects and
related concurrency types from the .NET Task Parallel Library. This is where the magic happens.
Coyote controls the execution of each
Task so that it can explore various different interleavings
to find and deterministically reproduce concurrency bugs.
Although Coyote supports testing unmodified task-based programs, it also gives you the option to use
the Coyote in-memory actor and state machine types from the
Microsoft.Coyote.Actors library. This is a more advanced asynchronous reactive programming model.
This approach requires you to change the design of your application, but gives you powerful (and
battle-tested inside Azure) constructs for building highly-reliable applications.
Regardless if you use Coyote binary rewriting on your unmodified program or wrote your code using
Coyote actors, the next step is to use Coyote to systematically test your code! The
tool takes over the scheduling of a Coyote program. It is able to do this reliably because it deeply
understands the concurrency in .NET applications. The tool knows all concurrent operations in a test
as well as sources of synchronization between them. By controlling the program schedule, Coyote can
reliably explore different interleavings of operations.
coyote test tool uses several state-of-the-art exploration strategies that have been known to
find very deep bugs easily. The tool can run a portfolio of available strategies in parallel,
maximizing chances of revealing bugs. Coyote refers to these exploration strategies as scheduling
strategies and makes it easy to incorporate new strategies, as they come out of
research. New scheduling strategies are being developed in Microsoft
Research based on a wealth of experience gathered from the Microsoft product groups that are using
During testing, Coyote will repeatedly run a test from start to completion, each time exercising different scheduling choices. This methodology has proven to be very effective at providing high coverage, especially concurrency coverage, in a short amount of time. In a similar way, the tool can also take over other sources of non-determinism, such as the delivery of timeouts or injection of failures.
If a bug is found, the
coyote test tool reports a reproducible bug trace that provides the
global order of all scheduling decisions and nondeterministic choices made during the execution of a
test. The trace can be replayed reliably over and over again, until the bug is identified. This
makes a bug reported by the tool significantly easier to debug than regular unit-/integration-tests
and logs from production or stress tests, which are typically nondeterministic. No more
See animating state machine demo for a visual explanation of what Coyote does when it is looking for bugs.