Using Language Oriented Programming to Execute Computations on the GPU

Using Language Oriented
Programming to Execute
Computations on the GPU
Robert Pickering, ALTI

About the Presenter
• Using F# for about 6 years
• Oldest F# user outside of Microsoft
• Written a book about F#
(now in its second edition)
• Spend at least 2 years as a professional
functional programmer
• I have 3 cats

Contact me:
robert@strangelights.com
http://strangelights.com/blog

2

ALTI
• Large French services company
• Consulting, engineering and
training
• Believe in helping clients
discover upcoming niches

• Already offer F# training and
consulting

What is Language Oriented
Programming ?
• Creating programs that are an abstract
description of the a problem

• This description can then either be
interpreted, compiled, or analyzed in another
way

LOP & Combinators
• F# has two main approaches to Language
Oriented Programming:
– Reinterpreting the F# syntax though the
quotations system
– Using combinators create a new syntax

… both approaches are very similar

What is a Combinator?

A combinator is a higher-order function that
uses only function application and earlier
defined combinators to define a result from its
arguments.

Source: Wikipedia, http://en.wikipedia.org/wiki/Combinatory_Logic

Combinatory Logic in Computing

In computer science, combinatory logic is used
as a simplified model of computation, used in
computability theory and proof theory. Despite
its simplicity, combinatory logic captures many
essential features of computation.

Source: Wikipedia, http://en.wikipedia.org/wiki/Combinatory_Logic

Combinator Library
"A combinator library offers functions (the
combinators) that combine functions together to make
bigger functions"[1]. These kinds of libraries are
particularly useful for allowing domain-specific
programming languages to be easily embedded into a
general purpose language by defining a few primitive
functions for the given domain.

Souce: Wikipedia http://en.wikipedia.org/wiki/Combinator_library

[1] “A History of Haskell” Hudak, Hughes, Peyton Jones, Wadler

History of Haskell: Combinator Libraries

What is a combinator library? The reader will
search in vain for a definition of this heavily
used term, but the key idea is this: a combinator
library offers functions (the combinators) that
combine functions together to make bigger
functions.

History of Haskell: Combinator Libraries

Another productive way to think of a
combinator library is as a domain-specific
language (DSL) for describing values of a
particular type.

What is a Domain Specific Language?

A programming language tailored for a particular application
domain, which captures precisely the semantics of the
application domain -- no more, no less.
A DSL allows one to develop software for a particular
application domain quickly, and effectively, yielding
programs that are easy to understand, reason about, and
maintain.
Hudak

Combinators vs DSLs
• Combinartor libraries are a special case of DSLs
– Sometimes called DSELs (Domain Specific Embed languages)

• DSELs have several advantages:
― Inherit non-domain-specific parts of the design.
― Inherit compilers and tools.
― Uniform “look and feel” across many DSLs
― DSLs integrated with full programming language, and with each other.

• DSELs one disadvantage:
― Constrained by host language syntax and type system

What Makes F# a Suitable for DSLs ?
• Union Types / Active Patterns
– type Option<'a> = Some x | None

• Lambda functions
– fun x -> x + 1

• Define and redefine operators
– let (++) x = x + 1

• Define custom numeric literals
– let x : Expression = 1.0N

Union Types – The Option Type
// The pre-defined option type
type Option<'a> =
| Some of 'a
| None

// constructing options
let someValue = Some 1
let noValue = None

// pattern matching over options
let convert value =
match value with
| Some x -> Printf.sprintf "Value: %i" x
| None -> "No value"

Union Types - Trees
// a binary tree definition
type BinaryTree<'a> =
| Node of BinaryTree<'a> * BinaryTree<'a>
| Leaf of 'a

// walk the tree collection values
let rec collectValues acc tree =
match tree with
| Node(ltree, rtree) ->
// recursively walk the left tree
let acc = collectValues acc ltree
// recursively walk the right tree
collectValues acc rtree
| Leaf value -> value :: acc
// add value to accumulator

Using the Tree
// define a tree
let tree =
Node(
Node(Leaf 1, Leaf 2),
Node(Leaf 3, Leaf 4))

// recover all values from the leaves
let values = collectValues [] tree

Union Types
Union types play a key role in language oriented
programming, as they allow the user to easily
define a tree that will form the abstract syntax
tree of the language

Active Patterns
// definition of the active pattern
let (|Bool|Int|Float|String|) input =
// attempt to parse a bool
let sucess, res = Boolean.TryParse input
if sucess then Bool(res)
else
// attempt to parse an int
let sucess, res = Int32.TryParse input
if sucess then Int(res)
else
// attempt to parse a float (Double)
let sucess, res = Double.TryParse input
if sucess then Float(res)
else String(input)

Active Patterns
// function to print the results by pattern
// matching over the active pattern
let printInputWithType input =
match input with
| Bool b -> printfn "Boolean: %b" b
| Int i -> printfn "Integer: %i" i
| Float f -> printfn "Floating point: %f" f
| String s -> printfn "String: %s" s

// print the results
printInputWithType "true"
printInputWithType "12"

Active Patterns
Active patterns play a supporting role in
language oriented programming in F#. They
allow the programmer to create abstractions of
complex operations on the abstract syntax tree

Lambda Functions

fun () ->
lovin'()

Lambda Functions
Lambda functions are important for language
oriented programming. They allow the users of
the language to embed actions within other
language elements

Custom Operators

let (++) x = x + 1

Custom Operators
Custom operators play a supporting role in
language oriented programming. They allow the
language designer to have a more flexible
syntax.

Custom Numeric Literals
type Expression =
| Constant of int

module NumericLiteralN =
let FromZero() = Constant 0
let FromOne() = Constant 1
let FromInt32 = Constant

let expr = 1N

Custom Numeric Literals
Numeric literals play a supporting role in
language oriented programming. They allow
numeric literals be treated in a more natural
within an embedded language

The Anatomy of a DSL
type Expression = Syntax Tree
| Add of Expression * Expression
| Subtract of Expression * Expression
| Multiply of Expression * Expression
| Constant of int Combinators
| Parameter of string
with
static member (+) (x, y) = Add(x, y)
static member (-) (x, y) = Subtract(x, y)
static member (*) (x, y) = Multiply(x, y)

module NumericLiteralN =
let FromZero() = Constant 0
let FromOne() = Constant 1
let FromInt32 = Constant

let param = Parameter


let expr = (1N + 2N) * (5N - 2N)

val expr : Expression =
Multiply (Add (Constant 1,Constant 2),
Subtract (Constant 5,Constant 2))

• Expressions now have an abstract tree like
representation:
― Multiply
― Add
― Constant 1
― Constant 2
― Subtract
― Constant 5
― Constant 2

• This can then be evaluated
• Or we can preform more advanced analysis

let evaluateExpression parameters =
let rec innerEval tree =
match tree with
| Multiply (x, y) -> innerEval x * innerEval y
| Add (x, y) -> innerEval x + innerEval y
| Subtract (x, y) -> innerEval x - innerEval y
| Constant value -> value
| Parameter key -> Map.find key parameters
innerEval

let expr = (1N + 2N) * (5N - 2N)

evaluateExpression Map.empty expr

let rec simplifyExpression exp =
let simpIfPoss op exp1 exp2 =
let exp' = op (simplifyExpression exp1,
simplifyExpression exp2)
if exp' = exp then exp' else simplifyExpression exp'
match exp with
| Multiply(Constant 0, Constant _) -> Constant 0
| Multiply(Constant _, Constant 0) -> Constant 0
| Multiply(Constant n1, Constant n2) -> Constant (n1 * n2)
| Add(Constant n1, Constant n2) -> Constant (n1 + n2)
| Subtract(Constant n1, Constant n2) -> Constant (n1 - n2)
| Multiply(exp1, exp2) -> simpIfPoss Multiply exp1 exp2
| Add(exp1, exp2) -> simpIfPoss Add exp1 exp2
| Subtract(exp1, exp2) -> simpIfPoss Add exp1 exp2
| Constant _ | Parameter _ -> exp

Quotations

let quotation = <@ (1 + 2) * (5 - 2) @>

val quotation : Quotations.Expr<int> =
Call (None, Int32 op_Multiply[Int32,Int32,Int32](Int32, Int32),
[Call (None, Int32 op_Addition[Int32,Int32,Int32](Int32, Int32),
[Value (1), Value (2)]),
Call (None, Int32 op_Subtraction[Int32,Int32,Int32](Int32, Int32),
[Value (5), Value (2)])])

Quotations
• Allow you to grab a tree structure that
represents a piece of F# code

• This tree structure can then be:
– Interpreted
– Compiled
– Converted to instructions understood by another
system or runtime

Microsoft Accelerator
• A project from MRS, now available on
Microsoft Connect [1]

• Automatically parallelize code for execution
on the GPU or x64 multicore

• Implemented as an unmanaged library with
.NET wrapper
[1] http://connect.microsoft.com/acceleratorv2

• Based on “Parallel Array”

• You define a set of operations to process the
content of the array

• The Accelerator runtime will then process the
operations in parallel


User Program – Define Operations

Accelerator

Input data Accelerated Program Input data

DX9Target X64Target

Microsoft Accelerator – Code!

let nums = [| 6; 1; 5; 5; 3 |]
let input = new FloatParallelArray(nums);
let sum = ParallelArrays.Shift(input, 1) + input +
ParallelArrays.Shift(input, -1);
let output = sum / 3.0f;

let target = new DX9Target();
let res = target.ToArray1D(output);

Game of Life

• Green - When an existing cell (green in the middle) has three
or two neighbours it survives to the next round

• Red - When an existing cell has less than two neighbours it
dies, because it is lonely (first red case), when it has more
than three neighbours it dies of overcrowding (second red
case)

• Blue - Finally, a new cell is born in an empty grid location if
there are exactly three neighbours .

The implementation
1 1

1 1

initial rotation 1

Sum of all
etc. ... neighbours

Game of Life

CPU GPU

Game of Life
/// Evaluate next generation of the life game state

let nextGeneration (grid: Matrix<float32>) =

// Shift in each direction, to count the neighbours
let sum = shiftAndSum grid offsets

// Check to see if we're born or remain alive
(sum =. threeAlive) ||. ((sum =. twoAlive) &&. grid)

CPU GPU

Game of Life
/// Evaluate next generation of the life game state
[<ReflectedDefinition>]
let nextGeneration (grid: Matrix<float32>) =

// Shift in each direction, to count the neighbours
let sum = shiftAndSum grid offsets

// Check to see if we're born or remain alive
(sum =. threeAlive) ||. ((sum =. twoAlive) &&. grid)

CPU GPU

Game of life in F# with Microsoft Accelerator

DEMO

Thanks!
• A big thanks to Tomáš Petříček!

• More info on his blog:
– http://tomasp.net/articles/accelerator-intro.aspx

Shameless Plug!
• Robert Pickering’s Beginning F# Workshop:
– Thursday 9th Sept – 10th September 2010

http://skillsmatter.com/course/open-source-
dot-net/robert-pickerings-beginning-f-workshop

Using Language Oriented Programming to Execute Computations on the GPU

More Related Content

What's hot

Viewers also liked

Similar to Using Language Oriented Programming to Execute Computations on the GPU

More from Skills Matter

Recently uploaded

Using Language Oriented Programming to Execute Computations on the GPU