Download as PDF, PPTX
![TYPICAL CV APPLICATIONS: SLIDING
WINDOW APPLICATIONS
Consider a 2D Sliding Window with 16-bit grayscale image inputs
Applies window function against a window from image and the kernel
“Slides” the window to get the next input
Repeats for every possible window
45x45 kernel on 1080p 30-FPS video = 120 billion memory accesses/second
Window KernelWindow 0
Window Function
Output Pixel
Window W-1
Window 1Input: image of size x×y, kernel of size n×m
for (row=0; row < x-n; row++) {
for (col=0; col < y-m; col++) {
// get n*m pixels (i.e., windows
// starting from current row and col)
window=image[row:row+n-1][col:col+m-1]
output[row][col]=f(window,kernel)
}
}
7/16/2018
17](https://image.slidesharecdn.com/1-e3mvsundance-180725124009/75/E3MV-Embedded-Vision-Sundance-17-2048.jpg)
![VCS-1 (EMC2) HARDWARE FEATURES
Connectivity:
FM191-R; FMC-LPC to:
15x Digital I/Os [DB9]
12x Analogue Inputs [DB9]
8x Analogue Outputs [DB9]
1x Expansion [SEIC]
FM191-U; SEIC to:
4x USB3.0 [USB-c]
28x GPIO [40-pin GPIO]
FM191-A1; 40-pin GPIO
28x GPIO [DB9]
7/16/2018
27](https://image.slidesharecdn.com/1-e3mvsundance-180725124009/75/E3MV-Embedded-Vision-Sundance-27-2048.jpg)
![TYPICAL CV APPLICATIONS: SLIDING
WINDOW APPLICATIONS
Consider a 2D Sliding Window with 16-bit grayscale image inputs
Applies window function against a window from image and the kernel
“Slides” the window to get the next input
Repeats for every possible window
45x45 kernel on 1080p 30-FPS video = 120 billion memory accesses/second
Window KernelWindow 0
Window Function
Output Pixel
Window W-1
Window 1Input: image of size x×y, kernel of size n×m
for (row=0; row < x-n; row++) {
for (col=0; col < y-m; col++) {
// get n*m pixels (i.e., windows
// starting from current row and col)
window=image[row:row+n-1][col:col+m-1]
output[row][col]=f(window,kernel)
}
}
7/16/2018
17](https://crownmelresort.com/image.slidesharecdn.com/1-e3mvsundance-180725124009/75/E3MV-Embedded-Vision-Sundance-17-2048.jpg)
![VCS-1 (EMC2) HARDWARE FEATURES
Connectivity:
FM191-R; FMC-LPC to:
15x Digital I/Os [DB9]
12x Analogue Inputs [DB9]
8x Analogue Outputs [DB9]
1x Expansion [SEIC]
FM191-U; SEIC to:
4x USB3.0 [USB-c]
28x GPIO [40-pin GPIO]
FM191-A1; 40-pin GPIO
28x GPIO [DB9]
7/16/2018
27](https://crownmelresort.com/image.slidesharecdn.com/1-e3mvsundance-180725124009/75/E3MV-Embedded-Vision-Sundance-27-2048.jpg)
Uploaded bySundance Multiprocessor Technology Ltd.
E3MV - Embedded Vision - Sundance
The document provides an overview of Sundance Multiprocessor Technology Ltd. and their EM3V - Embedded Vision product. Some key points: - Sundance is an employee-owned company with over 300 years of experience designing and building their own products. - Their VCS-1 (EMC2) system is a modular and reconfigurable hardware platform compatible with Zynq UltraScale+ MPSoC devices and a wide range of sensors. - The system includes open source software, firmware and documentation and is compatible with popular frameworks like ROS, OpenCV and deep learning stacks for running neural networks.
More Related Content
Similar to E3MV - Embedded Vision - Sundance
![[Harvard CS264] 03 - Introduction to GPU Computing, CUDA Basics](https://cdn.slidesharecdn.com/ss_thumbnails/cs264201103-cudabasicsshare-110209024624-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
More from Sundance Multiprocessor Technology Ltd.
![Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day3-cfd-251120170304-ddef8112-thumbnail.jpg?width=640&height=640&fit=bounds)
E3MV - Embedded Vision - Sundance
- 1. EM3V - EmbeddedVision Sundance Multiprocessor Technology, Ltd. Pedro Machado pedro.m@sundance.com Fatima Kishwar fatima.k@sundance.com Flemming Christensen flemming.c@sundance.com 7/16/2018 1
- 2. OVERVIEW • Company profile •Introduction • Technologies • VCS-1 (EMC2) System • Discussion • Future Work 7/16/2018 2
- 3. SERVICE AND EXCELLENCE! Establishedin 1989 by Flemming CHRISTENSEN • Employee Owned and a ’Life-Style’ company • 10x people with 300+ years experince • 4x with accredited Xilinx FPGA training • Always designed and built our own products • BSI - ISO9001-2015 certified, since 2003 Techology Focus • Acceleration,Vision, Sensor & Robotics 7/16/2018 3
- 4. MODULAR, RECONFIGURABLE -LIKE LEGO® • Reduced time-to-market • Rapid Prototyping • Reconfigurable • Flexible • Modular • Scalable • Reliable Year 1989 7/16/2018 4
- 5. 5 TURN-KEY SYSTEM SOLUTIONSFROM A-Z • Custom Bespoke Systems • Hazardous Environment • Custom Integration Service • Design of Enclosures • Commercial-of-the-Shelves Systems (COTS) • Flexible and Upgradeable • Design for Excellence • Maintenance with ease
- 6. INTRODUCTION Increasing demandfor High Performance Computing Everyone wants more compute-power Finer time-steps; larger data-sets; better models Decreasing single-threaded performance Emphasis on multi-core CPUs and parallelism Do computational biologists need to learn PThreads? Increasing focus on power and space Boxes are cheap: 16 node clusters are very affordable Where do you put them? Who is paying for power? How can we use hardware acceleration to help? 7/16/2018 6
- 7. TYPES OF HARDWAREACCELERATOR GPU : Graphics Processing Unit Many-core - 30 SIMD processors per device High bandwidth, low complexity memory – no caches MPPA : Massively Parallel Processor Array Grid of simple processors – 300 tiny RISC CPUs Point-to-point connections on 2-D grid FPGA : Field Programmable Gate Array Fine-grained grid of logic and small RAMs Build whatever you want 7/16/2018 7
- 8. HARDWARE ADVANTAGES: PERFORMANCE A Comparisonof CPUs, GPUs, FPGAs, and MPPAs for Random Number Generation, D. Thomas, L. Howes, and W. Luk , In Proc. of FPGA, pgs. 22-24, 2009 1 10 1 31 0 10 20 30 40 C P U G P U MP P A F P G A S peed-up • More parallelism - more performance • GPU: 30 cores, 16-way SIMD • MPPA: 300 tiny RISC cores • FPGA: hundreds of parallel functional units 7/16/2018 8
- 9. HARDWARE ADVANTAGES: POWER AComparison of CPUs, GPUs, FPGAs, and MPPAs for Random Number Generation, D. Thomas, L. Howes, and W. Luk , In Proc. of FPGA, pgs. 22-24, 2009 1 10 1 31 1 9 18 175 0 50 100 150 200 C P U G P U MP P A F P G A S peed-up E fficiency • GPU: 1.2GHz - same power as CPU • MPPA: 300MHz - Same performance as CPU, but 18x less power • FPGA: 300MHz - faster and less power 7/16/2018 9
- 10. FPGA ACCELERATED APPLICATIONS Finance 2006: Option pricing: 30x CPU 2007: Multivariate Value-at-Risk: 33x Quad CPU 2008: Credit-risk analysis: 60x Quad CPU Bioinformatics 2007: Protein Graph Labelling: 100x Quad CPU Neural Networks 2008: Spiking Neural Networks: 4x Quad CPU 1.1x GPU All with less than a fifth of the power 7/16/2018 10
- 11. PROBLEM: DESIGN EFFORT Researchers love scripting languages: Matlab, Python, Perl Simple to use and understand, lots of libraries Easy to experiment and develop promising prototype Eventually prototype is ready: need to scale to large problems Need to rewrite prototype to improve performance: e.g. Matlab to C Simplicity of prototype is hidden by layers of optimisation Hour Day Week Month 0.25 1 Year 4 16 64 256 Scripted Compiled Multi-threaded Vectorised Relative Performance Design-time CPU 7/16/2018 11
- 12. PROBLEMS: DESIGN EFFORT GPUs provide a somewhat gentle learning curve CUDA and OpenCL almost allow compilation of ordinary C code User must understand GPU architecture to maximise speed-up Code must be radically altered to maximise use of functional units Memory structures and accesses must map onto physical RAM banks We are asking the user to learn about things they don’t care about Hour Day Week Month 0.25 1 Year 4 16 64 256 Compiled C to GPU Reorganisation Memory Opt. Relative Performance Design-time CPU GPU 7/16/2018 12
- 13. PROBLEMS: DESIGN EFFORT FPGAs provide large speed-up and power savings – at a price! Days or weeks to get an initial version working Multiple optimisation and verification cycles to get high performance Too risky and too specialised for most users Months of retraining for an uncertain speed-up Currently only used in large projects, with dedicated FPGA engineer Hour Day Week Month 0.25 1 Year 4 16 64 256 Initial Design Parallelisation Clock Rate Relative Performance Design-time CPU GPU FPGA 7/16/2018 13
- 14. GOAL: FPGAS FORTHE MASSES Accelerate niche applications with limited user-base Don’t have to wait for traditional “heroic” optimisation Single-source description The prototype code is the final code Encourage experimentation Give users freedom to tweak and modify Target platforms at multiple scales Individual user; Research group; Enterprise Use domain specific knowledge about applications Identify bottlenecks: optimise them Identify design patterns: automate them Don’t try to do general purpose “C to hardware” 7/16/2018 14
- 15. WHICH TECHNOLOGY? Cleararchitectural trend of parallelism and heterogeneity Heterogeneous devices have many tradeoffs Usage cases also affect best device choice Problem: huge design space Problem Solution CPU CPU CPU CPU CPU Problem Solution Execution time: 10 sec Execution time: 2.5 sec CPU FPGAPCI CPU GPUPCI Orders of Magnitude Improvement CPU CPU CPU GPU FPGA Clock Rate Base Complexity Base Parallelism Base Power Base ExecutionTime Task Size Sequential CPU Multicore CPU GPU FPGA Huge Design Space •Which Accelerator? •Which Brand? •Number of cores? •Which Device? •Device Cost? •Design time? •Which algorithm? •Use case optimization? 7/16/2018 15
- 16. TYPICAL CV APPLICATIONS:SLIDING WINDOW Contribution: thorough analysis of devices and use cases for sliding window applications Sliding window used in many domains, including image processing and embedded CPU CPU CPU CPU CPU GPU FPGA Devices Algorithms Use Cases KernelSize Image SizeCorrentropyConvolution Sum of Absolute Differences 7/16/2018 16
- 17. TYPICAL CV APPLICATIONS:SLIDING WINDOW APPLICATIONS Consider a 2D Sliding Window with 16-bit grayscale image inputs Applies window function against a window from image and the kernel “Slides” the window to get the next input Repeats for every possible window 45x45 kernel on 1080p 30-FPS video = 120 billion memory accesses/second Window KernelWindow 0 Window Function Output Pixel Window W-1 Window 1Input: image of size x×y, kernel of size n×m for (row=0; row < x-n; row++) { for (col=0; col < y-m; col++) { // get n*m pixels (i.e., windows // starting from current row and col) window=image[row:row+n-1][col:col+m-1] output[row][col]=f(window,kernel) } } 7/16/2018 17
- 18. APP 1: SUMOF ABSOLUTE DIFFERENCES (SAD) Used for: H.264 encoding, object identification Window function: point-wise absolute difference, followed by summation Kernel W1 WindowX W2 W3 W4 k1 k2 k3 k4 W1 W2 W3 W4 k1 k2 k3 k4 d1 d2 d3 d4 a1 a2 a3 a4 Ox Output Pixel Absolute Value() Ox = 0; For pixel in window: Ox += abs(pixeli – kerneli) 7/16/2018 18
- 19. APP 2: 2DCONVOLUTION Used for: filtering, edge detection Window function: point-wise product followed by summation Kernel W1 WindowX W2 W3 W4 k1 k2 k3 k4 W1 W2 W3 W4 k4 k3 k2 k1 p1 p2 p3 p4 Ox Output Pixel Ox = 0; For pixel in window: Ox += (pixeli x kerneln-i) 7/16/2018 19
- 20. APP 3: CORRENTROPY Used for: optical flow, obstacle avoidance Window function: Gaussian of point-wise absolute difference, followed by summation W1 W2 W3 W4 k1 k2 k3 k4 a1 a2 a3 a4 Ox Output Pixel Absolute Difference Ox = 0; For pixel in window: Ox += Gauss(abs(pixeli – kerneli)) g1 g2 g3 g4 Gaussian Function() 7/16/2018 20
- 21. HETEROGENEOUS COMPUTING SYSTEM INTOP500 LIST Reason: Significant performance/energy-efficiency boost from GPU/CPU 1 8 7 17 39 62 53 75 102 94 0 20 40 60 80 100 120 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 (June) Heterogeneous computing system in Top500 7/16/2018 21
- 22. Specialized acceleratorfor data-parallel applications Optimized for processing massive data Give up unrelated goal and features Give up optimizing latency for processing single data Give up branch prediction, out-of-order execution Give up large traditional cache hierarchy More resource for parallel are processing More cores, more ALU GPU: SPECIALIZED ACCELERATOR FOR A SET OF APPLICATIONS Data DataCore Data DataCore Data DataCore Data DataCore 7/16/2018 22
- 23. Only providesprimitive building blocks for computation Register, addition/multiplication , memories, programmable Boolean operations and connections Build application-specific accelerator from primitives building blocks Interconnection between primitive functional units Timing of data movement between primitive functional units Opportunities for optimizations for a specific application! Maximizing efficiency while throwing away redundancy CREATING APPLICATION-SPECIFIC ACCELERATOR WITH FPGA Virtex®-7 FPGA Precise, Low Jitter Clocking MMCMs Logic Fabric LUT-6 CLB DSP Engines DSP48E1 Slices On-Chip Memory 36Kbit/18Kbit Block RAM Enhanced Connectivity PCIe® Interface Blocks Hi-perf. Parallel I/O Connectivity SelectIO™ Technology Hi-performance Serial I//O Connectivity Transceiver Technology 7/16/2018 23
- 24. Require tremendousefforts Extensive knowledge of digital circuit design The potential of FPGAs is not easily accessible by common software engineers THE CHALLENGES OF PROMOTING FPGAS AMONG SOFTWARE ENGINEERS AXI Master Burst inference DSP48 Timing Closure Stable interface Loop rewind 247/16/2018 24
- 25. • Zynq-7000 devicesare equipped with dual-core ARM Cortex-A9 processors integrated with 28nm Artix-7 or Kintex®- 7 based programmable logic for excellent performance-per-watt and maximum design flexibility. • Sundance’s EMC2 carrier board is Compatible with all the Zynq-7000 Series. HYBRID ARQUITECTURE 7/16/2018 25
- 26. • Zynq® UltraScale+™MPSoC devices provide 64-bit processor scalability while combining real-time control with soft and hard engines for graphics, video, waveform, and packet processing. • Sundance’s EMC2 carrier board is compatible with all the Zynq® UltraScale+™ MPSoC. Our focus is now on the XCZU4EV device (automotive grade). WHAT IS THE BEST SOLUTION? 7/16/2018 26
- 27. VCS-1 (EMC2) HARDWAREFEATURES Connectivity: FM191-R; FMC-LPC to: 15x Digital I/Os [DB9] 12x Analogue Inputs [DB9] 8x Analogue Outputs [DB9] 1x Expansion [SEIC] FM191-U; SEIC to: 4x USB3.0 [USB-c] 28x GPIO [40-pin GPIO] FM191-A1; 40-pin GPIO 28x GPIO [DB9] 7/16/2018 27
- 28. The ZU4EV MPSoCis compatible with a wide range of sensors. Depth sensor (up to 20m). Interface USB3.0 Thermal imaging Interface: Eth Movidius Neural Compute Stick Interface: USB3.0 VCS-1 (EMC2) SENSORS COMPATIBILITY 7/16/2018 28JAI AD-130GE Interface: Eth
- 29. VCS-1 (EMC2) COMPATIBILITY VCS-1features: Raspberry PI and Arduino compatible; Compatible with most of the Arduino/RPI sensors and actuators; 4x USB3.0 ports for interfacing with a wide range of sensors; MQTT and OpenCV compatible ROS compatible ROS2 ready HIPPEROS ready 7/16/2018 29
- 30. The VCS-1 willbe fully compatible with the Xilinx reVision stack. Includes support for the most popular neural networks including AlexNet, GoogLeNet, VGG, SSD, and FCN. Optimized implementations for CNN network layers, required to build custom neural networks (DNN/CNN) 7/16/2018 30 Xilinx reVISION stack DEEP LEARNING ON THE VCS-1 (EMC2)
- 31. VCS-1 (EMC2) OPENSOURCE SOFTWARE AND FIRMWARE Open Source Hardware/software and online documentation: Open Hardware Repository https://www.ohwr.org/projects/emc2-dp Microsoft Windows/Linux 64-bit SDK https://github.com/SundanceMultiprocessorTechnology/V CS-1_SDK FM191 ARM SDK https://github.com/SundanceMultiprocessorTechnology/V CS-1_FM191_SDK FM191 Firmware https://github.com/SundanceMultiprocessorTechnology/V CS-1_FM191_FW EMC2 ROS https://github.com/SundanceMultiprocessorTechnology /VCS-1_emc2_ros 7/16/2018 31
- 32. A custom enclosurewas specially designed for accommodating the VCS-1 system. 7/16/2018 32 VCS-1(EMC2) ENCLOSURE
- 33. DISCUSSION The VCS-1 hasthe following characteristics: 1. High performance (24V@0.595A) 2. Low power consumption 3. Highly compatible with a wide range of commercially available sensors and actuators 4. Highly optimised for computer vision applications 5. Fully reconfigurable 7/16/2018 33
- 34. WHAT NEXT? Bepart of our customers family by ordering our products or hiring our services. Sundance University Program (SUP) Access to hardware prototypes Advisory Board Members BSC/MSc/PhD Internships Funding capture H2020 InnovateUK EPSRC Know more about SUP and on-going projects https://www.sundance.com/sundance-in-eu-projects- programs/ 7/16/2018 34
- 35.
- 36.
- 37. QUESTIONS? 7/16/2018 37 Sundance Multiprocessor Technology,Ltd. Pedro Machado pedro.m@sundance.com Fatima Kishwar fatima.k@sundance.com Flemming Christensen flemming.c@sundance.com EM3V - Embedded Vision