Composable and streamable Play apps

at

Composable and streamable Play apps

About me

Part of the Service Infrastructure team, which is bringing
the Play Framework to LinkedIn engineers.

At LinkedIn, we’ve been running Play apps
in production for over a year

More than 60 apps on Play now, including:

Many internal tools (e.g. REST search)

Plus many backends services that don’t
have sexy screenshots

We love Play… but we’ve had a
couple tiny problems:

1. Unmanageable complexity

1. Unmanageable complexity
2. Terrible performance

It’s probably not what you think.

This talk is about a couple techniques to
deal with complexity and performance
problems.

These techniques are experimental.

You can find the code from this presentation at:
https://github.com/brikis98/ping-play

Outline
1. Complexity
2. Composition
3. HTTP
4. Performance
5. Enumerators
6. Streaming

For example, consider the LinkedIn home page

Almost every part of the page is dynamic, highly
customized for the user, and interactive.

Trying to cram this all into one controller and
one template is completely unmaintainable

“Managing complexity is the most important
technical topic in software development.”
Steve McConnell, Code Complete

“Abstraction is the ability to engage with a
concept while safely ignoring some of its details.”
Steve McConnell, Code Complete

Abstraction allows you to take a simpler view of a
complex concept

Composition: assemble complex behavior by
combining simpler behavior

Abstraction: structure your app so you
can focus on one part while safely
ignoring the rest

Abstraction: structure your app so you
can focus on one part while safely
ignoring the rest

Composition: structure your app so
you can easily combine the simpler
parts into more complicated parts

We’ll start by building a completely standalone “Who’s
Viewed Your Profile” (WVYP) module

We need:
1. Data: WVYP count, search count
2. Template: to render the HTML
3. Controller: to tie it all together

Backend
Server
Frontend
Server
Internet

Load
Balancer

Backend
Server

Frontend
Server

Data
Store

Backend
Server

Frontend
Server

Backend
Server
Backend
Server

Data
Store

Data
Store

Data
Store

For data, LinkedIn uses a Service Oriented Architecture

For this talk, we’re going to simulate service
calls by calling a mock endpoint

object Mock extends Controller {
def mock(serviceName: String) = Action.async {
serviceName match {
case "wvyp" => respond(data = "56", delay = 10)
case "search" => respond(data = "10", delay = 5)
case "likes" => respond(data = "150", delay = 40)
case "comments" => respond(data = "14", delay = 20)
}
}

private def respond(data: String, delay: Long) = Promise.timeout(Ok(data), delay)
}

app/mock/Mock.scala

For each “service”, this endpoint returns mock data after a
fixed delay. In the real world, the data might be JSON.

GET

/mock/:serviceName

controllers.Mock.mock(serviceName: String)

conf/routes

The routes entry for the mock endpoint

object ServiceClient {
def makeServiceCall(serviceName: String): Future[String] = {
WS.url(s"http://localhost:9000/mock/$serviceName").get().map(_.body)
}
}

app/data/ServiceClient.scala

A simple client to make a remote call to the mock endpoint
and return a Future with the body of the HTTP response

@(wvypCount: Int, searchCount: Int)
<html>
<head>
<link rel="stylesheet" href="/assets/stylesheets/wvyp.css"/>
</head>
<body>
<div class="wvyp">
<h2>Who's Viewed Your Profile</h2>
@views.html.wvyp.wvypCount(wvypCount)
@views.html.wvyp.searchCount(searchCount)
</div>
</body>
</html>

app/views/wvyp/wvyp.scala.html

For the template, we use Play’s Scala templates (.scala.
html). This template uses two partials for the body.

@(wvypCount: Int)

<p class="wvyp-count">
<span class="large-number">@wvypCount</span>
<span>Your profile has been viewed by <b>@wvypCount</b> people in the past 3 days</span>
</p>

app/views/wvyp/wvypCount.scala.html
@(searchCount: Int)

<p class="search-count">
<span class="large-number">@searchCount</span>
<span>Your have shown up in search results <b>@searchCount</b> times in the past 3 days</span>
</p>

app/views/wvyp/searchCount.scala.html

The markup for the two partials that show the counts

object WVYP extends Controller {
def index = Action.async {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp")
val searchCountFuture = ServiceClient.makeServiceCall("search")

}
}

app/controllers/WVYP.scala

Next, the WVYP controller. First, we make two service calls in
parallel to fetch the WVYP count and search count.

def index = Action.async {

for {
wvypCount <- wvypCountFuture
searchCount <- searchCountFuture
} yield {
Ok(views.html.wvyp.wvyp(wvypCount.toInt, searchCount.toInt))
}
}
}


Next, we compose the two Futures into a single Future and
render the WVYP template.

GET

/mock/:serviceName


GET

/wvyp

controllers.WVYP.index

conf/routes

Add a routes entry for the WVYP controller

Now, imagine we also have another standalone module
called “Who’s Viewed Your Updates” (WVYU)

GET

/mock/:serviceName


GET

/wvyp

controllers.WVYP.index

GET

/wvyu

controllers.WVYU.index

conf/routes

It has a controller, template, and routes entry just like WVYP

How can we combine the two modules?

trait Action[A] extends EssentialAction {
def apply(request: Request[A]): Future[SimpleResult]
}

play/api/mvc/Action.scala

In Play, an Action is a function

Request[A] => Future[SimpleResult]

The actions in each standalone module are just functions
from Request to Result. Functions can be composed!

@(wvypCount: Int, searchCount: Int)

<html>
<head>
</head>
<body>
@views.html.wvyp.wvypBody(wvypCount, searchCount)
</body>
</html>

app/views/wvyp/wvyp.scala.html

We need one change to each module: move the body into a
partial. Scala templates are functions, so they also compose!

def index(embed: Boolean) = Action.async {

for {
wvypCount <- wvypCountFuture
searchCount <- searchCountFuture
} yield {
if (embed) Ok(views.html.wvyp.wvypBody(wvypCount.toInt, searchCount.toInt))
else Ok(views.html.wvyp.wvyp(wvypCount.toInt, searchCount.toInt))
}
}


Update each module’s controller to accept an “embed”
parameter: if it’s set to true, render only the body partial.

GET

/mock/:serviceName


GET

/wvyp

controllers.WVYP.index(embed: Boolean ?= false)

GET

/wvyu

controllers.WVYU.index(embed: Boolean ?= false)

conf/routes

Update the routes file too

object Pagelet {

def readBody(result: SimpleResult)(implicit codec: Codec): Future[Html] = {
result.body.run(Iteratee.consume()).map(bytes => Html(new String(bytes, codec.charset)))
}
}

app/ui/Pagelet.scala

Add a helper that can take a SimpleResult and return its body
as Future[Html]. We’ll talk more about Iteratees later.

object Aggregator extends Controller {
def index = Action.async { request =>
val wvypFuture = Wvyp.index(embed = true)(request)
val wvyuFuture = Wvyu.index(embed = true)(request)

}
}

app/controllers/Aggregator.scala

Now for the aggregator controller. First, we call the two
submodules. Each returns a Future[SimpleResult].

for {
wvyp <- wvypFuture
wvyu <- wvyuFuture
wvypBody <- Pagelet.readBody(wvyp)
wvyuBody <- Pagelet.readBody(wvyu)
} yield {

}
}
}


Read the body of each Future[SimpleResult] as Html using
the Pagelet helper we just added

for {
wvyp <- wvypFuture
wvyu <- wvyuFuture
} yield {
Ok(views.html.aggregator.aggregator(wvypBody, wvyuBody))
}
}
}


Pass the Html to the aggregator template

GET
@(wvypBody:/mock/:serviceName controllers.Mock.mock(serviceName: String)
Html, wvyuBody: Html)
GET

/wvyp


GET
<html>

/wvyu


GET
<head>

/aggregate

controllers.Aggregator.index

<link rel="stylesheet" href="/assets/stylesheets/wvyu.css"/>
</head>
<body>
@wvypBody
@wvyuBody
</body>
</html>

app/views/aggregator/aggregator.scala

Add the aggregator to the routes file

The result. We’ve composed two Play modules!

Wins
1.

Abstraction: we can focus on just one small part of the page while safely
ignoring all the rest.

2.

Composition: we can build complicated pages by putting together simpler
parts. We can get lots of reuse from common pieces.

3.

Testing and iterating on a small, standalone unit is much easier and faster.

Caveats
1.

Standalone modules may be inefficient in terms of duplicated service calls.
However, de-duping is straightforward.

2.

Have to merge and dedupe static content.

The composition code glosses over a few details

Some questions:
1. How do we set cookies?
2. How do we handle errors?
3. How do we aggregate static content?

It turns out HTTP is an elegant way to answer
these questions.

def index(embed: Boolean) = Action {
// [snip]
Ok(views.html.wvyp.wvyp(wvypCount.toInt, searchCount.toInt)).withCookies(Cookie(“foo”, “bar”))
}
}


For example, imagine a standalone module sets a Cookie


for {
wvyp <- wvypFuture
wvypHeaders = wvyp.header.headers // The Cookie header here will contain the “foo” cookie!
} yield {
Ok(views.html.aggregator.aggregator(wvypBody))
}
}
}


The aggregator will see that as a Cookie header!

object Pagelet {

def mergeCookies(results: SimpleResult*): Seq[Cookie] = {
results.flatMap { result =>
result.header.headers.get(HeaderNames.SET_COOKIE).map(Cookies.decode).getOrElse(Seq.empty)
}
}
}


We can add a helper to merge the cookies from multiple
SimpleResult objects

for {
wvyp <- wvypFuture
wvyu <- wvyuFuture
} yield {
Ok(views.html.aggregator.aggregator(wvypBody, wvyuBody))
.withCookies(Pagelet.mergeCookies(wvyp, wvyu):_*)
}
}


Update the aggregator to write out the merged cookies

We can see the aggregate endpoint is now setting cookies

HTTP headers are a good way to handle error
cases too.

A quick review of HTTP status codes

For example, if a module has no content, return a 204

Invalid module request? Return 404.

Plus other important status codes

for {
wvyp <- wvypFuture
} yield {
if (wvyp.status == 200) {
Ok(views.html.aggregator.aggregator(wvypBody))
} else {
// Handle errors
}
}
}


The aggregator can read the status codes and react
accordingly

CSS and JS dependencies can be set as a
header and aggregated too.

object StaticContent {
val cssHeaderName = "X-CSS"
val jsHeaderName = "X-JS"

def asHeaders(css: Seq[String], js: Seq[String]): Seq[(String, String)] = {
Seq(cssHeaderName -> css.mkString(","), jsHeaderName -> js.mkString(","))
}
}

app/ui/StaticContent.scala

Add a helper to create the headers

def index(embed: Boolean) = Action {
// [...] all other code is the same as before [...]

val css = Vector("/assets/stylesheets/wvyp.css")
val js = Vector.empty[String]

if (embed) {
Ok(views.html.wvyp.wvypBody(wvypCount.toInt, searchCount.toInt))
.withHeaders(StaticContent.asHeaders(css, js):_*)
} else {
Ok(views.html.wvyp.wvyp(wvypCount.toInt, searchCount.toInt, css, js))
}
}


In embed mode, return CSS/JS dependencies as headers. In
standalone mode, render them in the template.

object StaticContent {
def mergeCssHeaders(results: SimpleResult*): Seq[String] = mergeHeaderValues(cssHeaderName, results:
_*)

def mergeJsHeaders(results: SimpleResult*): Seq[String] = mergeHeaderValues(jsHeaderName, results:_*)

private def mergeHeaderValues(headerName: String, results: SimpleResult*): Seq[String] = {
results.flatMap { result =>
result.header.headers.get(headerName).map(_.split(",").toSeq).getOrElse(Seq.empty)
}.distinct
}
}

app/ui/StaticContent.scala

Helpers to merge CSS and JS headers from multiple
SimpleResult objects

for {
wvyp <- wvypFuture
wvyu <- wvyuFuture
} yield {
val css = StaticContent.mergeCssHeaders(wvyp, wvyu)
val js = StaticContent.mergeCssHeaders(wvyp, wvyu)
Ok(views.html.aggregator.aggregator(wvypBody, wvyuBody, css, js))
}
}


The aggregator can merge and de-dupe the static content
and pass it to its template for rendering

Wins
1.

Modules are truly standalone

2.

Can dynamically compose modules using Play’s router

3.

Can compose modules from remote endpoints

4.

Can reuse endpoints from the browser via AJAX

5.

Static content dependencies explicitly defined

Caveats
1.

Managing static content in a controller, instead of a view, feels clunky.

So far, we’ve been using a mock endpoint to
simulate remote service calls

What happens if one of the remote calls is slow?

serviceName match {
case "wvyp" => respond(data = "56", delay = 10)
case "search" => respond(data = "10", delay = 2000) // SLOW!
}
}

}

app/mock/Mock.scala

As an extreme example, let’s make the search service take
two seconds to respond

Time to first byte is 2 seconds! Everything has to wait for
the one slow service, including static content.

Is there a way to start sending the response
before all the data is available?

Facebook solves this problem with BigPipe
https://www.facebook.com/note.php?note_id=389414033919

Can we build BigPipe with Play?

(Well, yea, or I wouldn’t have made this talk)

Enumerators are part of the Play Iteratees library.

Quick Review
1.

An Enumerator is a Producer. It pumps out chunks of data.

2.

An Iteratee is a Consumer. It reactively consumes chunks of data.

3.

An Enumeratee is an Adapter. You can attach them in front of Iteratees
and Enumerators to filter the chunks of data.

Producer, consumer, and adapter

These are all composable abstractions for
working with streams of data

object EnumeratorExample extends Controller {

def index = Action {
Ok.chunked(/* We need an Enumerator here */)
}
}

app/controllers/EnumeratorExamples.scala

Play has an Ok.chunked method which can stream out the
contents of an Enumerator


Ok.chunked(Enumerator("Created", " using", " Enumerator", ".apply()nn"))
}
}


The Enumerator object has several factory methods. For
example, Enumerator.apply creates one from a fixed list.

Basic “Hello World” example using Enumerator.apply.


Ok.chunked(Enumerator.repeatM(Promise.timeout("Hellon", 500)))
}
}


You can also create an Enumerator that repeats a value
generated from a Future

The word “Hello” is pumped out every 500ms. Note: when
testing streaming, use “curl -N” so curl doesn’t buffer.

val helloEnumerator = Enumerator("hello ")
val goodbyeEnumerator = Enumerator("goodbyenn")

val helloGoodbyeEnumerator = helloEnumerator.andThen(goodbyeEnumerator)

Ok.chunked(helloGoodbyeEnumerator)
}


Most importantly, you can combine Enumerators. Here is an
example using the andThen method.

With andThen, we see all the data from the first enumerator
and then all the data from the second one

val helloEnumerator = Enumerator.repeatM(Promise.timeout("Hellon", 500))
val goodbyeEnumerator = Enumerator.repeatM(Promise.timeout("Goodbyen", 1000))

val helloGoodbyeEnumerator = Enumerator.interleave(helloEnumerator, goodbyeEnumerator)

Ok.chunked(helloGoodbyeEnumerator)
}


We can also combine Enumerators using Enumerator.
interleave

With interleave, data can come in any order. Above, we see
“Hello” every 500ms and “Goodbye” every 1000ms.

Let’s use Enumerators to stream the results
of our remote service calls

object WVYPEnumerator extends Controller {

}
}

app/controllers/WVYPEnumerator.scala

Create the new WVYP controller. Again, we start with two
service calls.


val wvypCountEnum = Enumerator.flatten(wvypCountFuture.map(str => Enumerator(str + "n")))
val searchCountEnum = Enumerator.flatten(searchCountFuture.map(str => Enumerator(str + "n")))

}
}


Next, convert each Future[String] into an Enumerator[String]


val wvypCountEnum = Enumerator.flatten(wvypCountFuture.map(str => Enumerator(str + "n")))
val searchCountEnum = Enumerator.flatten(searchCountFuture.map(str => Enumerator(str + "n")))

val body = wvypCountEnum.andThen(searchCountEnum)

Ok.chunked(body)
}
}


Finally, compose the two Enumerators and use Ok.chunked
to stream them out

GET
GET

/wvyp


GET
<html>

/wvyu


GET
<head>

/aggregate


GET <link rel="stylesheet" href="/assets/stylesheets/wvyp.css"/>
/wvyp/enumerator
controllers.WVYPEnumerator.index
<link rel="stylesheet" href="/assets/stylesheets/wvyu.css"/>
</head>
<body>
@wvypBody
@wvyuBody
</body>
</html>


Add a routes entry for the enumerator-based WVYP controller

Almost immediately, we see “56”, from the wvyp service.

2 seconds later, we see “10”, from the search service.

However, we’re only streaming plain text.
How do we stream a template?

play.Keys.templatesTypes ++= Map("stream" -> "ui.HtmlStreamFormat")
play.Keys.templatesImport ++= Vector("ui.HtmlStream", "ui.HtmlStream._", "ui.StaticContent")

build.sbt

Play allows you to define a new template type. This will allow
us to create .scala.stream files instead of .scala.html.

The new template type must define an
“Appendable” and a “Format” for it

trait Appendable[T] {
def +=(other: T): T
}

play/api/templates/Templates.scala

The Appendable trait is simple: it’s anything with an append
(+=) operator.

class Html(buffer: StringBuilder) extends Appendable[Html](buffer) {
def +=(other: Html) = {
buffer.append(other.buffer)
this
}
}


The default Appendable built into Play is Html, which just
wraps a StringBuilder.

case class HtmlStream(enumerator: Enumerator[Html]) extends Appendable[HtmlStream] {
def +=(other: HtmlStream): HtmlStream = andThen(other)

def andThen(other: HtmlStream): HtmlStream = HtmlStream(enumerator.andThen(other.enumerator))
}

app/ui/HtmlStream.scala

For streaming templates, we create an HtmlStream
Appendable that wraps an Enumerator[Html].

object HtmlStream {

def interleave(streams: HtmlStream*): HtmlStream = {
HtmlStream(Enumerator.interleave(streams.map(_.enumerator)))
}

def flatten(eventuallyStream: Future[HtmlStream]): HtmlStream = {
HtmlStream(Enumerator.flatten(eventuallyStream.map(_.enumerator)))
}
}


We add an HtmlStream companion object with helper
methods to combine HtmlStreams just like Enumerators

object HtmlStream {

def apply(text: String): HtmlStream = {
apply(Html(text))
}

def apply(html: Html): HtmlStream = {
HtmlStream(Enumerator(html))
}

def apply(eventuallyHtml: Future[Html]): HtmlStream = {
flatten(eventuallyHtml.map(apply))
}
}


Also, we add several methods to the HtmlStream companion
object to help create HtmlStream instances.

trait Format[T <: Appendable[T]] {
type Appendable = T

def raw(text: String): T

def escape(text: String): T
}


The Format trait is what Play will use to create your
Appendable from a String

object HtmlStreamFormat extends Format[HtmlStream] {

def raw(text: String): HtmlStream = {
HtmlStream(text)
}

def escape(text: String): HtmlStream = {
raw(HtmlFormat.escape(text).body)
}
}


Here’s the HtmlStreamFormat implementation

object HtmlStreamImplicits {

// Implicit conversion so HtmlStream can be passed directly to Ok.feed and Ok.chunked
implicit def toEnumerator(stream: HtmlStream): Enumerator[Html] = {
// Skip empty chunks, as these mean EOF in chunked encoding
stream.enumerator.through(Enumeratee.filter(!_.body.isEmpty))
}
}


We also include an implicit conversion so HtmlStream can be
passed directly to Ok.chunked

@(body:ui.HtmlStream)
<html>
<head>
</head>
<body>
<div class="wvyp">
@body
</div>
</body>
</html>

app/views/wvyp/wvyp.scala.stream

We can now create a .scala.stream template that has markup
mixed with HtmlStream elements.

object WVYPStream extends Controller {

}
}

app/controllers/WVYPStream.scala

Now for the streaming controller. As usual, start with the
service calls.


val wvypCountHtmlFuture = wvypCountFuture.map(str => views.html.wvyp.wvypCount(str.toInt))
val searchCountHtmlFuture = searchCountFuture.map(str => views.html.wvyp.searchCount(str.toInt))

}
}


Next, render the data in each Future as Html



val wvypCountStream = HtmlStream(wvypCountHtmlFuture)
val searchCountStream = HtmlStream(searchCountHtmlFuture)

}
}


Convert each Future[Html] to an HtmlStream using the factory
methods we created earlier



val wvypCountStream = HtmlStream(wvypCountHtmlFuture)
val searchCountStream = HtmlStream(searchCountHtmlFuture)

val body = wvypCountStream.andThen(searchCountStream)
Ok.chunked(views.stream.wvypStreaming(body))
}
}


Finally, combine the streams using andThen and render the
streaming template

GET
GET

/wvyp


GET
<html>

/wvyu


GET
<head>

/aggregate


GET <link rel="stylesheet" href="/assets/stylesheets/wvyp.css"/>
/wvyp/enumerator
controllers.WVYPEnumerator.index
GET <link rel="stylesheet" href="/assets/stylesheets/wvyu.css"/>
/wvyp/stream
controllers.WVYPStream.index
</head>
<body>
@wvypBody
@wvyuBody
</body>
</html>


Add a routes entry for the stream-based WVYP controller

The page loads almost immediately and we see the WVYP
count right away.

2 seconds later, the search count appears.

Also, the CSS starts downloading immediately!

This is a huge win if the stuff at the top of the
page loads quickly. But what if it’s slow?

serviceName match {
case "wvyp" => respond(data = "56", delay = 2000) // SLOW!
case "search" => respond(data = "10", delay = 20)
}
}

}

app/mock/Mock.scala

Modify the mock endpoint so wvyp is slow and search is fast

Again, the page loads almost immediately, but this time,
neither count is visible.

2 seconds later, both counts appear.

Fortunately, the CSS still loads right at the beginning.

So it’s still a big win, even if the top of the
page is slow, but not by as much.

Is there a way to stream the data out of
order but still render it in order?

@(contents: Html, id: String)

<script type="text/html-stream" id="@id-contents">
@contents
</script>

<script type="text/javascript">
document.getElementById("@id").innerHTML = document.getElementById("@id-contents").innerHTML;
</script>

app/views/ui/pagelet.scala.html

Create a “pagelet” template that will take a snippet of HTML
and use JS to inject it into the proper spot on the page.

object Pagelet {
def render(html: Html, id: String): Html = {
views.html.ui.pagelet(html, id)
}

def renderStream(html: Html, id: String): HtmlStream = {
HtmlStream(render(html, id))
}

def renderStream(htmlFuture: Future[Html], id: String): HtmlStream = {
HtmlStream.flatten(htmlFuture.map(html => renderStream(html, id)))
}
}


Add a Pagelet class with helper methods to wrap Html and
Future[Html] in the pagelet template.

@(body: ui.HtmlStream)
<html>
<head>
</head>
<body>
<div class="wvyp">
<div id="wvypCount"></div>
<div id="searchCount"></div>
</div>
@body
</body>
</html>

app/views/wvyp/wvyp.scala.stream

Update the WVYP streaming template to include placholder
divs for the WVYP and search counts



}


Finally, update the WVYPStream controller. We still make two
service calls and render each as HTML.



val wvypCountStream = Pagelet.renderStream(wvypCountHtmlFuture, "wvypCount")
val searchCountStream = Pagelet.renderStream(searchCountHtmlFuture, "searchCount")

}


This time, we convert the Future[Html] into an HtmlStream
using the Pagelet.renderStream helper method.



val wvypCountStream = Pagelet.renderStream(wvypCountHtmlFuture, "wvypCount")
val searchCountStream = Pagelet.renderStream(searchCountHtmlFuture, "searchCount")

val body = HtmlStream.interleave(wvypCountStream, searchCountStream)

Ok.chunked(views.stream.wvypStreaming(body))
}


Instead of using andThen, we compose the streams using
interleave. This way, the Html will stream as soon as it’s ready.

The page loads almost immediately, but now we see the
search count first

2 second later, the WVYP count appears.

We now have the basics for out-of-order,
BigPipe style rendering!

Wins
1.

Much faster time to first byte

2.

Static content can start loading much earlier

3.

User can see and start interacting with the page almost immediately

4.

Out of order rendering with JavaScript allows even deeper optimizations

Caveats
1.

Once you’ve streamed out the HTTP headers, you can’t change them. This
means cookies and error handling may have to be done client-side.

2.

Have to be careful with “pop in” and redraws. Client side code should
intelligently choose when to insert items into the DOM.

3.

Need to handle CSS and JS dependencies differently

4.

Testing is harder

5.

May need to disable JavaScript rendering for SEO

Composable and streamable Play apps

More Related Content

What's hot

Viewers also liked

Similar to Composable and streamable Play apps

More from Yevgeniy Brikman

Recently uploaded

Composable and streamable Play apps