Cost optimal algorithm

Cost optimal algorithm

• 1.
  AssociateProfessor,KGC-DehradunBy Dr. HemanPathak
• 2.
  Definition 2.1: Cost ofan Algorithm The cost of a PRAM computation is the product of the parallel time complexity and the number of processors used. For example, a PRAM algorithm that has time complexity (log p) using p processors has cost (p log p).
• 3.
  Definition 2.2: Cost OptimalAlgorithm A cost optimal parallel algorithm is an algorithm for which the cost is in the same complexity class as an optimal sequential algorithm.
• 4.
  Definition: Cost Optimal Problem Sequential Cost ParallelAlgorithm Cost Optimal No. of PEs Complexity Cost Sum of n Numbers n n/2 log(n) nlog(n) No Prefix Sum (n) n n-1 log(n) nlog(n) No List Ranking n n log(n) nlog(n) No Pre-order Tree Traversal n 2(n-1) log(n) nlog(n) No Merging two sorted list n n log(n) nlog(n) No Graph Coloring (n) Polynomial Cn n2 Cn X n2 No
• 5.
  Cost Optimal: ReductionAlgorithm
• 6.
  Cost Optimal: ReductionAlgorithm  The total number of operations performed by the parallel algorithm is of the same complexity class as an optimal sequential algorithm.  The parallel reduction algorithm performs about n/2 additions in the first step, n/4 additions in the second step, n/8 additions in the third step, and so on, executing a total of n -1 additions over the log⁡( 𝑛) iterations.  Since both the sequential and the parallel algorithms perform n-1 additions, a cost-optimal variant of the parallel reduction algorithm may exists.
• 7.
  Cost Optimal: ReductionAlgorithm Cost = P * Time Complexity = nlog(n) Cost optimal = (n-1) n-1 = P * log(n) P = (n-1)/ log(n) TC = log(n) Cost = (n-1) same as Sequential Once we have determined the appropriate number of processors, we need to verify that there is indeed a cost-optimal parallel-reduction algorithm with logarithmic time complexity.
• 8.
  Brent’s Theorem(2.2)(1974)
• 9.
  Brent’s Theorem Proof
• 11.
  Application to ParallelReduction Parallel Algorithm: Sum of n Numbers • No. of Operations (M) = n-1 • Time Complexity (T) = log⁡( 𝑛) • No. of PEs = 𝑛/2 Cost Optimal Parallel Algorithm • No. of Operations (M) = n-1 • Time Complexity (T) = log⁡( 𝑛) • No. of PEs (P) = (n-1)/ log(n) =⁡ 𝑛/log⁡( 𝑛)
• 12.
  Application to ParallelReduction
• 13.
  Application to ParallelReduction n = 16 p = (n/log n) = 4
• 14.
  Application to ParallelReduction Steps No. of Op. General 1 9 n-1 2 8 n-2 3 6 n-4 4 2 n-8 : : : i n-2i Sequential Algorithm for i = 1 to n-1 a[i] = a[i] + a[i-1] endfor
• 15.
  Application to ParallelReduction