How to exploit any parallelism in my haskell parallel code?

Question

I've just stated working in haskell semi-explicit parallelism with GHC 6.12. I've write the following haskell code to compute in parallel the map of the fibonnaci function upon 4 elements on a list, and in the same time the map of the function sumEuler upon two elements.

import Control.Parallel
import Control.Parallel.Strategies

fib :: Int -> Int
fib 0 = 0
fib 1 = 1
fib n = fib (n-1) + fib (n-2)

mkList :: Int -> [Int]
mkList n = [1..n-1]

relprime :: Int -> Int -> Bool
relprime x y = gcd x y == 1

euler :: Int -> Int
euler n = length (filter (relprime n) (mkList n))

sumEuler :: Int -> Int
sumEuler = sum . (map euler) . mkList

-- parallel initiation of list walk                                                                                                                                    
mapFib :: [Int]
mapFib = map fib [37, 38, 39, 40]

mapEuler :: [Int]
mapEuler = map sumEuler [7600, 7600]

parMapFibEuler :: Int
parMapFibEuler = (forceList mapFib) `par` (forceList mapEuler `pseq` (sum mapFib + sum mapEuler))

-- how to evaluate in whnf form by forcing                                                                                                                                
forceList :: [a] -> ()
forceList [] = ()
forceList (x:xs) = x `pseq` (forceList xs)


main = do putStrLn (" sum : " ++ show parMapFibEuler)

to improve my program in parallel I rewrote it with par and pseq and a forcing function to force whnf evaluation. My problem is that by looking in the threadscope it appear that i didn't gain any parallelism. Things are worse because i didn't gain any speedup.

Threadscope observation

That why I have theses two questions

Question 1 How could I modify my code to exploit any parallelism ?

Question 2 How could I write my program in order to use Strategies (parMap, parList, rdeepseq and so on ...) ?

First improvement with Strategies

according to his contribution

parMapFibEuler = (mapFib, mapEuler) `using` s `seq` (sum mapFib + sum mapEuler) where
    s = parTuple2 (seqList rseq) (seqList rseq)

the parallelism appears in the threadscope but not enough to have a significant speedup

enter image description here

The parallel package is greatly improved in GHC 7, so you might also consider upgrading. — Don Stewart
– Don Stewart, Commented Mar 17, 2011 at 23:09
You could memoize your fib functions in order to gain some speed up... — Hai
– Hai, Commented Mar 18, 2011 at 11:02

Simon Marlow · Accepted Answer · 2011-03-18 14:33:15Z

The reason you aren't seeing any parallelism here is because your spark has been garbage collected. Run the program with +RTS -s and note this line:

  SPARKS: 1 (0 converted, 1 pruned)

the spark has been "pruned", which means removed by the garbage collector. In GHC 7 we made a change to the semantics of sparks, such that a spark is now garbage collected (GC'd) if it is not referred to by the rest of the program; the details are in the "Seq no more" paper.

Why is the spark GC'd in your case? Look at the code:

parMapFibEuler :: Int
parMapFibEuler = (forceList mapFib) `par` (forceList mapEuler `pseq` (sum mapFib + sum mapEuler))

the spark here is the expression forkList mapFib. Note that the value of this expression is not required by the rest of the program; it only appears as an argument to par. GHC knows that it isn't required, so it gets garbage collected.

The whole point of the recent changes to the parallel package were to let you easily avoid this bear trap. A good Rule of Thumb is to use Control.Parallel.Strategies rather than par and pseq directly. My preferred way to write this would be

parMapFibEuler :: Int
parMapFibEuler = runEval $ do
  a <- rpar $ sum mapFib
  b <- rseq $ sum mapEuler
  return (a+b)

but sadly this doesn't work with GHC 7.0.2, because the spark sum mapFib is floated out as a static expression (a CAF), and the runtime doesn't think sparks that point to static expressions are worth keeping (I'll fix this). This wouldn't happen in a real program, of course! So let's make the program a bit more realistic and defeat the CAF optimisation:

parMapFibEuler :: Int -> Int
parMapFibEuler n = runEval $ do
  a <- rpar $ sum (take n mapFib)
  b <- rseq $ sum (take n mapEuler)
  return (a+b)

main = do [n] <- fmap (fmap read) getArgs
          putStrLn (" sum : " ++ show (parMapFibEuler n))

Now I get good parallelism with GHC 7.0.2. However, note that @John's comments also apply: generally you want to look for more fine-grained parallelism so as to let GHC use all your processors.

Thanks very much for this; it explains some behavior I was wondering about while looking at this problem.

John L · Accepted Answer · 2011-03-18 10:43:39Z

Your parallelism is far too course-grained to have much beneficial effect. The largest chunks of work that can be done in parallel efficiently are in sumEuler, so that's where you should add your par annotations. Try changing sumEuler to:

sumEuler :: Int -> Int
sumEuler = sum . (parMap rseq euler) . mkList

parMap is from Control.Parallel.Strategies; it expresses a map that can be done in parallel. The first argument, rseq having type Strategy a, is used to force the computation to a specific point, otherwise no work would be done, due to laziness. rseq is fine for most numeric types.

It's not useful to add parallelism to fib here, below about fib 40 there isn't enough work to make it worthwhile.

In addition to threadscope, it's useful to run your program with the -s flag. Look for a line like:

SPARKS: 15202 (15195 converted, 0 pruned)

in the output. Each spark is an entry in a work queue to possibly be performed in parallel. Converted sparks are actually done in parallel, while pruned sparks mean that the main thread got to them before a worker thread had the chance to do so. If the pruned number is high, it means your parallel expressions are too fine-grained. If the total number of sparks is low, you aren't trying to do enough in parallel.

Finally, I think parMapFibEuler is better written as:

parMapFibEuler :: Int
parMapFibEuler = sum (mapFib `using` parList rseq) + sum mapEuler

mapEuler is simply too short to have any parallelism usefully expressed here, especially as euler is already performed in parallel. I'm doubtful that it makes a substantial difference for mapFib either. If the lists mapFib and mapEuler were longer, parallelism here would be more useful. Instead of parList you may be able to use parBuffer, which tends to work well for list consumers.

Making these two changes cuts the runtime from 12s to 8s for me, with GHC 7.0.2.

ony · Accepted Answer · 2011-03-17 22:56:07Z

1

Hmmm... Maybe?

((forceList mapFib) `par` (forceList mapEuler)) `pseq` (sum mapFib + sum mapEuler)

I.e. spawn mapFib in background and calculate mapEuler and only after it (mapEuler) do (+) of their sums. Actually I guess you can do something like:

parMapFibEuler = a `par` b `pseq` (a+b) where
     a = sum mapFib
     b = sum mapEuler

About Q2: As I know strategies - is the "strategies" to combine data-structures with those par and seq.
You can write your forceList = withStrategy (seqList rseq)
As well you can write your code like:

parMapFibEuler = (mapFib, mapEuler) `using` s `seq` (sum mapFib + sum mapEuler) where
    s = parTuple2 (seqList rseq) (seqList rseq)

I.e. strategy applied to tuple of two lists will force their evaulation in parallel, but each list will be forced to be evaluated sequentially.

edited Mar 17, 2011 at 22:56

answered Mar 17, 2011 at 22:37

ony

13.4k1 gold badge36 silver badges42 bronze badges

2 Comments

Fopa Léon Constantin Over a year ago

thanks to reply ony, but the code you proposed is similar to the one have wrote in my question, I've tested you proposition and threadscope plot the same as before

Fopa Léon Constantin Over a year ago

Just a litte modification in order to make it work parMapFibEuler = ((mapFib, mapEuler) using s) seq (sum mapFib + sum mapEuler) where s = parTuple2 (seqList rseq) (seqList rseq)

Thomas M. DuBuisson · Accepted Answer · 2011-03-18 00:03:34Z

First off, I assume you know your fib definition is awful and you're just doing this to play with the parallel package.

You seem to be going for parallelism at the wrong level. Parallelizing mapFib and mapEuler won't give a good speed-up because there is more work to compute mapFib. What you should do is compute each of these very expensive elements in parallel, which is slightly finer grain but not overly so:

mapFib :: [Int]
mapFib = parMap rdeepseq fib [37, 38, 39, 40]

mapEuler :: [Int]
mapEuler = parMap  rdeepseq sumEuler [7600, 7600, 7600,7600]

parMapFibEuler :: Int
parMapFibEuler = sum a + sum b
  where
  a = mapFib
  b = mapEuler

Also, I originally fought using Control.Parallel.Strategies over Control.Parallel but have come to like it as it is more readable and avoids issues like yours where one would expect parallelism and have to squint at it to figure out why you aren't getting any.

Finally, you should always post how you compile and how you run code you're expecting to be parallelized. For example:

$ ghc --make -rtsopts -O2 -threaded so.hs -eventlog -fforce-recomp
[1 of 1] Compiling Main             ( so.hs, so.o )
Linking so ...
$ ./so +RTS -ls -N2
 sum : 299045675

Yields: threadscope run with reasonable parallelism

Collectives™ on Stack Overflow

How to exploit any parallelism in my haskell parallel code?

4 Answers 4

1 Comment

Comments

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related