The Design of the Scalaz 8 Effect System

TheDesignoftheScalaz8
EffectSystem
Scale By The Bay - San Francisco
John A. De Goes
@jdegoes - http://degoes.net

Agenda
· Intro
· Tour
· Versus
· Wrap

AboutMe
· I program with functions
· I contribute types & functions to FLOSS
· I start companies powered by functions

MostScalaProgrammersDon't
ProgramFunctionally
!

4MonsterPains
1. Asynchronous
2. Concurrent
3. Resource-Safe
4. Performant

Scalaz8Effectimport scalaz.effect._
Scalaz 8 eﬀect system is a small, composable collection of data
types and type classes that help developers build principled,
performant, and pragmatic I/O applications that don't leak
resources, don't block, and scale across cores.

Scalaz8IO
TheHeartofScalaz8
IO[A] is an immutable value that describes an eﬀectful program
that either produces an A, fails with a Throwable, or runs forever.

TLDR
Scalaz 8 IO helps you quickly build
asynchronous, concurrent, leak-free, performant
applications.2
2
Which coincidentally happen to be type-safe, purely functional, composable, and easy to reason about.

Main
SafeApp
object MyApp extends SafeApp {
def run(args: List[String]): IO[Unit] =
for {
_ <- putStrLn("Hello! What is your name?")
n <- getStrLn
_ <- putStrLn("Hello, " + n + ", good to meet you!")
} yield ()
}

Core
PureValues
object IO {
...
def apply[A](a: => A): IO[A] = ???
...
}
...
val answer: IO[Int] = IO(42)

Core
Mapping
trait IO[A] {
...
def map[B](f: A => IO[B]): IO[B] = ???
}
...
IO(2).map(_ * 3) // IO(6)

Core
Chaining
trait IO[A] {
...
def flatMap[B](f: A => IO[B]): IO[B] = ???
}
...
IO(2).flatMap(x => IO(3).flatMap(y => IO(x * y)) // IO(6)

Core
Failure
object IO {
...
def fail[A](t: Throwable): IO[A] = ???
...
}
...
val failure = IO.fail(new Error("Oh noes!"))

Core
Recovery
trait IO[A] {
...
def attempt: IO[Throwable / A] = ???
...
}
...
action.attempt.flatMap {
case -/ (error) => IO("Uh oh!")
case /-(value) => IO("Yay!")
}

Core
DerivingAbsolve
object IO {
...
def absolve[A](io: IO[Throwable / A]): IO[A] = io.flatMap {
case -/ (error) => IO.fail(error)
case /-(value) => IO(value)
}
...
}
...
IO.absolve(action.attempt)

Core
DerivingAlternative
trait IO[A] {
...
def orElse(that: => IO[A]): IO[A] =
self.attempt.flatMap(_.fold(_ => that)(IO(_)))
...
}
...
val openAnything = openFile("primary.data").orElse(openFile("secondary.data"))

Synchronous
ImportingEffects
object IO {
...
def sync[A](a: => A): IO[A] = ???
...
}

Synchronous
ImportingExample
def putStrLn(line: String): IO[Unit] =
IO.sync(scala.Console.println(line))
def getStrLn: IO[String] =
IO.sync(scala.io.StdIn.readLine())

Synchronous
EffectExample
val program: IO[Unit] =
for {
_ <- putStrLn("Hello. What is your name?")
name <- getStrLn
_ <- putStrLn("Hello, " + name + ", good to meet you!")
} yield ()

Asynchronous
EffectImport:Definition
object IO {
...
def async0[A](k: (Throwable / A => Unit) => AsyncReturn[A]): IO[A] = ???
...
}
...
sealed trait AsyncReturn[+A]
object AsyncReturn {
final case object Later extends AsyncReturn[Nothing]
final case class Now[A](value: A) extends AsyncReturn[A]
final case class MaybeLater[A](canceler: Throwable => Unit) extends AsyncReturn[A]
}

Asynchronous
ImportingEffects
def spawn[A](a: => A): IO[A] =
IO.async0 { (callback: Throwable / A => Unit) =>
java.util.concurrent.Executors.defaultThreadFactory.newThread(new Runnable() {
def run(): Unit = callback(/-(a))
})
AsyncReturn.Later
}
def never[A]: IO[A] =
IO.async0 { (callback: Throwable / A => Unit) =>
AsyncReturn.Later
}

Asynchronous
EffectExample
for {
response1 <- client.get("http://e.com")
limit = parseResponse(response1).limit
response2 <- client.get("http://e.com?limit=" + limit)
} yield parseResponse(response2)

Asynchronous
Sleep
IO {
...
def sleep(duration: Duration): IO[Unit] = ???
...
}

Asynchronous
SleepExample
for {
_ <- putStrLn("Time to sleep...")
_ <- IO.sleep(10.seconds)
_ <- putStrLn("Time to wake up!")
} yield ()

Asynchronous
DerivingDelay
trait IO[A] {
...
def delay(duration: Duration): IO[A] =
IO.sleep(duration).flatMap(_ => self)
...
}
...
putStrLn("Time to wake up!").delay(10.seconds)

Concurrency
Models
1. Threads — Java
· OS-level
· Heavyweight
· Dangerous interruption
2. Green Threads — Haskell
· Language-level
· Lightweight
· Eﬀicient
3. Fibers — Scalaz 8
· Application-level
· Lightweight
· Zero-cost for pure FP
· User-defined semantics

Concurrency
Fork/Join
trait IO[A] {
...
def fork: IO[Fiber[A]] = ???
def fork0(h: Throwable => IO[Unit]): IO[Fiber[A]] = ???
...
}
trait Fiber[A] {
def join: IO[A]
def interrupt(t: Throwable): IO[Unit]
}

Concurrency
Fork/JoinExample
def fib(n: Int): IO[BigInt] =
if (n <= 1) IO(n)
else for {
fiberA <- fib(n-1).fork
fiberB <- fib(n-2).fork
a <- fiberA.join
b <- fiberB.join
} yield a + b

Concurrency
raceWith
trait IO[A] {
...
def raceWith[B, C](that: IO[B])(
finish: (A, Fiber[B]) / (B, Fiber[A]) => IO[C]): IO[C] = ???
...
}

Concurrency
DerivingRace
trait IO[A] {
...
def race(that: IO[A]): IO[A] = raceWith(that) {
case -/ ((a, fiber)) => fiber.interrupt(Errors.LostRace( /-(fiber))).const(a)
case /-((a, fiber)) => fiber.interrupt(Errors.LostRace(-/ (fiber))).const(a)
}
...
}

Concurrency
DerivingTimeout
trait IO[A] {
...
def timeout(duration: Duration): IO[A] = {
val err: IO[Throwable / A] =
IO(-/(Errors.TimeoutException(duration)))
IO.absolve(self.attempt.race(err.delay(duration)))
}
...
}

Concurrency
DerivingPar
trait IO[A] {
...
def par[B](that: IO[B]): IO[(A, B)] =
attempt.raceWith(that.attempt) {
case -/ ((-/ (e), fiberb)) => fiberb.interrupt(e).flatMap(_ => IO.fail(e))
case -/ (( /-(a), fiberb)) => IO.absolve(fiberb.join).map(b => (a, b))
case /-((-/ (e), fibera)) => fibera.interrupt(e).flatMap(_ => IO.fail(e))
case /-(( /-(b), fibera)) => IO.absolve(fibera.join).map(a => (a, b))
}
...
}

Concurrency
DerivingRetry
trait IO[A] {
...
def retry: IO[A] = this orElse retry
def retryN(n: Int): IO[A] =
if (n <= 1) this
else this orElse (retryN(n - 1))
def retryFor(duration: Duration): IO[A] =
IO.absolve(
this.retry.attempt race
(IO.sleep(duration) *>
IO(-/(Errors.TimeoutException(duration)))))
...
}

Concurrency
MVar
trait MVar[A] {
def peek: IO[Maybe[A]] = ???
def take: IO[A] = ???
def read: IO[A] = ???
def put(v: A): IO[Unit] = ???
def tryPut(v: A): IO[Boolean] = ???
def tryTake: IO[Maybe[A]] = ???
}

Concurrency
MVarExample
val action =
for {
mvar <- MVar.empty // Fiber 1
_ <- mvar.putVar(r).fork // Fiber 2
result <- mvar.takeVar // Fiber 1
} yield result

ResourceSafety
Uninterruptible
trait IO[A] {
...
def uninterruptibly: IO[A] = ???
...
}

ResourceSafety
UninterruptibleExample
val action2 = action.uninterruptibly

ResourceSafety
Bracket
trait IO[A] {
...
def bracket[B](
release: A => IO[Unit])(
use: A => IO[B]): IO[B] = ???
...
}

ResourceSafety
BracketExample
def openFile(name: String): IO[File] = ???
def closeFile(file: File): IO[Unit] = ???
openFile("data.json").bracket(closeFile(_)) { file =>
...
// Use file
...
}

ResourceSafety
Deriving'Finally'
trait IO[A] {
def ensuring(finalizer: IO[Unit]): IO[A] =
IO.unit.bracket(_ => finalizer)(_ => this)
}

ResourceSafety
BrokenErrorModel
try {
try {
try {
throw new Error("e1")
}
finally {
}
}
finally {
}
}
catch { case e4 : Throwable => println(e4.toString()) }

ResourceSafety
FixedErrorModel
IO.fail(new Error("e1")).ensuring(
IO.fail(new Error("e2"))).ensuring(
IO.fail(new Error("e3"))).catchAll(e => putStrLn(e.toString()))

ResourceSafety
Supervision
object IO {
...
def supervise[A](io: IO[A]): IO[A] = ???
...
}

ResourceSafety
SupervisionExample
val action = IO.supervise {
for {
a <- doX.fork
b <- doY.fork
...
} yield z
}

Principles
AlgebraicLaws
fork >=> join = id
let fiber = fork never
in interrupt e fiber >* join fiber = fail e
And many more!

Versus:Performance
SCALAZ 8 IO FUTURE CATS IO MONIX TASK
Le! Associated flatMap 5061.380 39.088 0.807 3548.260
Narrow flatMap 7131.227 36.504 2204.571 6411.355
Repeated map 63482.647 4599.431 752.771 47235.85
Deep flatMap 1885.480 14.843 131.242 1623.601
Shallow attempt 769.958 CRASHED 643.147 CRASHED
Deep attempt 16066.976 CRASHED 16061.906 12207.417
Scalaz 8 IO is up to 6300x faster than Cats (0.4), 195x faster than
Future (2.12.4), and consistently faster than Monix Task (3.0.0-RC1).

Versus:Safety
Sync Stack Safety ✓ ✓ ✓ ✓
Async Stack
Safety
✓ ✓ ! ✓
Bracket Primitive ✓ ! ! !
No Implicit
Executors
✓ ! ! ✓
No Mutable
Implicits
✓ ! ! ✓

Versus:Expressiveness
Synchronicity ✓ ! ✓ ✓
Asynchronicity ✓ ✓ ✓ ✓
Concurrency Primitives ✓ ! ! ✓
Async Var ✓ ! ! ✓
Non-Leaky Race ✓ ! ! ✓4
Non-Leaky Timeout ✓ ! ! ✓4
Non-Leaky Parallel ✓ ! ! ✓4
Thread Supervision ✓ ! ! !
4
Cancellation only occurs at async boundaries.

Versus
WhatAboutFS2?
IO is not a stream!

Versus
FS2:MissingFoundations
· Mini-actor library
· Mini-FRP library
· MVar implementation — Ref
· Concurrency primitives
· race, bracket, fork, join

Versus
FS2:LeakyFoundations
package object async {
...
def race[F[_]: Effect, A, B](fa: F[A], fb: F[B])(
implicit ec: ExecutionContext): F[Either[A, B]] =
ref[F, Either[A,B]].flatMap { ref =>
ref.race(fa.map(Left.apply), fb.map(Right.apply)) >> ref.get
}
def start[F[_], A](f: F[A])(implicit F: Effect[F], ec: ExecutionContext): F[F[A]] =
ref[F, A].flatMap { ref => ref.setAsync(F.shift(ec) >> f).as(ref.get) }
def fork[F[_], A](f: F[A])(implicit F: Effect[F], ec: ExecutionContext): F[Unit] =
F.liftIO(F.runAsync(F.shift >> f) { _ => IO.unit })
...
}

Versus
FS2: Non-Compositional Timeout
class Ref[A] {
...
def timedGet(timeout: FiniteDuration, scheduler: Scheduler): F[Option[A]] = ???
...
}
Scalaz 8: Compositional Timeout
mvar.takeVar.timeout(t)
mvar.putVar(2).timeout(t)
...
what.ev.uh.timeout(t)

ThankYou
Special thanks to Alexy Khrabrov, Twitter, and
the wonderful attendees of Scale By The Bay!

The Design of the Scalaz 8 Effect System

More Related Content

What's hot

Viewers also liked

Similar to The Design of the Scalaz 8 Effect System

More from John De Goes

Recently uploaded

The Design of the Scalaz 8 Effect System