Taming event-driven software via formal verification

TAMING EVENT-DRIVEN SOFTWARE
VIA FORMAL VERIFICATION
Dr Tom Gibson-Robinson
CTO, Cocotec Limited

2 | © Cocotec Ltd 2022
EVENT-DRIVEN SOFTWARE
Event-driven software is any software that consists
of a number of components that communicate by
sending messages.
It’s often:
• Stateful: sometimes handled by an “interesting”
hand-written state machine implementation.
• Asynchronous: requests start and complete later.
• Concurrent: requests overlap – two peripherals
might be sent commands that complete in either
order.
It can be found all over the software stack:
• Drivers are often event driven.
• User interfaces are event driven.
• The core of hardware-centric systems often need
to coordinate the other components via events.
Hardware
Drivers
Algorithms
User Interface
Coordination
Software
Data Storage
Communication

CHALLENGES WITH EVENT-DRIVEN SOFTWARE
Imagine there are three processes:
• One producer who produces a sequence of n messages;
• One producer who produces a sequence of m messages;
• One consumer who reads these messages.
In total, there are (n + m) C n possible orderings the
consumer can receive the messages in.
n m Interleavings
7 3 120
17 13 119 × 106
70 30 3 x 1025
Testing struggles to adequately cover the possible behaviours. Issues are also hard to diagnose if
they happen in the field, since it requires fine-grained logging to know the entire history of events.

Testing Formal Verification
An alternative approach is formal verification
where the system is instead analysed
mathematically and proven correct.
FORMAL VERIFICATION

An alternative approach is formal verification
where the system is instead analysed
mathematically and proven correct.
To apply formal verification we have to:
• Formally describe what we want to prove;
• Formally describe what we can assume.
Most verification tools for conventional programs
use per-function contracts based on pre- and
post- conditions to formalise this.
FORMAL VERIFICATION
/**
* pre size > 0
* post size' = size – 1
&& return_value == contents[size - 1]
*/
template <typename T>
T stack<T>::pop_front() {
…
}

EVENT-DRIVEN CONTRACTS
For event-driven software, this means we need to
be able to describe:
• The events, including any data in them;
• The order that events will be seen in.
State machines are good techniques for describing
such properties. State machines can either be
described graphically, or textually.
State machines can act as either the specification
or the assumption, depending on the context.
port Executable {
/// Starts the action
function start() : Nil
/// If the action is running, cancels it.
function cancel() : Nil
/// Sent when the action finishes.
outgoing signal done()
}
This shows Coco, a language we have developed
for developing event-driven software.
• It facilitates verification-driven development.
• Once correct, C and C++ code can be
generated.

EVENT-DRIVEN CONTRACTS
For event-driven software, this means we need to
be able to describe:
• The events, including any data in them;
• The order that events will be seen in.
State machines are good techniques for describing
such properties. State machines can either be
described graphically, or textually.
State machines can act as either the specification
or the assumption, depending on the context.
port Executable {
outgoing signal done()
machine {
state Idle {
start() = setNextState(Running)
}
state Running {
cancel() = setNextState(Idle)
spontaneous = {
done();
setNextState(Idle);
}
}
}
}

EVENT-DRIVEN IMPLEMENTATIONS
Executable is a convenient interface to use, but
low-level hardware devices might not be able to
send events directly.
@runtime(.MultiThreaded)
component PollerImpl {
val client : Provided<Executable>
val pollable : Required<Pollable>
machine {
state Idle {
client.start() = {
pollable.start();
setNextState(Running);
}
}
state Running {
client.cancel() = { … }
periodic(milliseconds(100)) = {
if (!pollable.isRunning()) {
client.done();
setNextState(Idle);
}
}
}
}
}
port Pollable {
function isRunning() : Bool
machine {
state Idle {
start() = setNextState(Running)
}
state Running {
cancel() = setNextState(Idle)
isRunning() = nondet {
{ return true; },
@eventually
{
setNextState(Idle);
return false;
},
}
}
}
}
The component must implement
the Executable state machine
The component can
assume pollable behaves as
per the state machine

TYPICAL ERRORS: DESYNCHRONISATION
barrier[1] detects an
obstruction
Component handles the
obstruction and
considers the request
cancelled
When a new open()
request starts, the
component calls close
on the still-closing
barrier[0]

TYPICAL ERRORS: MULTIPLE USERS
Another user starts
maintenance on the lift
The bridge tries to start
opening the lift, whilst
the lift is still under
maintenance.
One user starts a request
to open the bridge

CHALLENGES

CHALLENGE: LIVENESS
Liveness properties are generally difficult to handle
in formal verification tools.
• Safety properties say nothing bad will happen.
• Liveness properties say something good will
eventually happen.
port Pollable {
function isRunning() : Bool
machine {
state Running {
isRunning() = nondet {
{ return true; },
@eventually
{
setNextState(Idle);
return false;
},
}
}
}
}

CHALLENGE: DATA
Data can cause verification time to increase.
• Often the exact data values are not relevant to the
stateful behaviour of the system.
What we need is a way of abstracting the data so
that only the the parts relevant to the verification
remain.
external val kThresholdPressure : Pressure =
Pressure{ value = .OutOfRange }
component PumpControllerImpl {
val client : Provided<PumpController>
val gauge : Required<PressureGauge>
val pump : Required<Pump>
machine {
state Monitoring {
if (gauge.measure() >= kThresholdPressure) {
pump.stop();
client.emergencyStop();
setNextState(Stopped);
}
}
}
…
}
}

CHALLENGE: DATA
enum AbstractPressure {
case WithinRange
case OutOfRange
}
@CPP.mapToType("uint32_t", .Value)
external type Pressure {
var value : AbstractPressure
}
port PressureGauge {
function measure() : Pressure
machine {
measure() = Pressure{
value = nondet { .WithinRange, .OutOfRange }
}
}
}
external val kThresholdPressure : Pressure =
Pressure{ value = .OutOfRange }
component PumpControllerImpl {
val client : Provided<PumpController>
val gauge : Required<PressureGauge>
val pump : Required<Pump>
machine {
state Monitoring {
if (gauge.measure() >= kThresholdPressure) {
pump.stop();
client.emergencyStop();
setNextState(Stopped);
}
}
}
…
}
}
External types have two
relevant types:
• The type at runtime
• The type during verification
Ports are specifications, so
operate on the abstract
verification types

CHALLENGE: SCALING
Large systems can be efficiently verified due to
compositional verification. This means that each
component is verified in isolation, using only the
local ports:
• This naturally corresponds to other formal
verification techniques based around pre/post
conditions.
For Coco, the soundness of this is guaranteed by
the underlying theory (which is based on CSP).
There isn’t a limit to the scaling offered by this. The
largest Coco system (we know of) has ~700
components, and is around 300k lines of code.

LESSONS LEARNT
• On applying formal verification:
• Different verification tools have different capabilities, so choose the right tool for the right job!
• Identifying a clear boundary is hard:
• Existing software has often evolved away from a clean design.
• Sometimes easier on new projects.
• Scaling is less of an issue than thought: in practice humans fail to scale first!
• On building Coco:
• Language has to be defined around formal verification rather than bolted on.
• Simulation and programmer-like debugging tools are vital for adoption beyond formal verification
fanatics.
• Liveness properties are hard, both for users and for the verification.
• Open challenges:
• Better ways of making assumptions about behaviour over multiple components.
• Better ways of specifying bespoke properties.

Taming event-driven software via formal verification

More Related Content

Similar to Taming event-driven software via formal verification

More from AdaCore

Recently uploaded

Taming event-driven software via formal verification