Downloaded 189 times
Uploaded byEsri
Enterprise GIS
This document provides an overview of enterprise GIS system architecture design. It discusses the system design process, GIS technology, data administration, network communications, and Esri architecture. Key aspects of system performance including software, hardware, and virtualization are also covered. The document concludes with an example of using Esri's Capacity Planning Tool to help with system design.
More Related Content
![[Day 3] Assessment Grid By Chris Nicholas](https://cdn.slidesharecdn.com/ss_thumbnails/4-1assessmentgridbychrisnicholas-090415140948-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
![Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day3-cfd-251120170304-ddef8112-thumbnail.jpg?width=640&height=640&fit=bounds)
Enterprise GIS
- 1.
- 2. Enterprise GIS: SystemArchitecture Design Isaac Medel Francesc Puigvert Esri Spain
- 3. Agenda: System ArchitectureDesing • System Design Process • GIS Technology • Data Administration • Network Communications • Esri Architecture • Performance - Software - Hardware • Virtualization • Example of CPT
- 4. System Architecture Design User needs Infrastructure specifications Data resources Network design Hardware People (information products) Applications
- 5. Plannning Identify business requirements User Workflows / Services Peak system design loads Data center hardware specifications Network infrastructure needs Define project requirements Project budget Project schedule System performance needs Return on investment/funding Reduce Implementation Risk Identify key performance parameters Establish system performance targets Define system performance validation plan
- 6. Why is SystemArchitecture Design Important? Reduce cost Improve productivity Balanced system design Servers Network DBMS Geodatabase Storage Clients ArcSDE Application Hardware infrastructure Database design User workflow System architecture design Framework for productive operations
- 7. Stages Support asuccessful implementation Application development Prototype test Data Construction needs Design System design Implementation Requirements System delivery Building a GIS User needs is an Incremental Process
- 8. Capacity Planning Tool:System Design • Design tab User workflow requirements Network Display suitability performance Platform Selection Platform sizing and utilization
- 9. Agenda: System ArchitectureDesing • System Design Process • GIS Technology • Data Administration • Network Communications • Esri Architecture • Performance - Software - Hardware • Virtualization • Example of CPT
- 10. GIS Software Evolution What are the lessons learned? Platform Performance Change >> 10x Faster Hardware >> 10x Faster Hardware >> 10x Faster Hardware > 2x Process Load > 2x Process Load >> 10x Faster Technology Change >> 10x Faster Technology Change Service Oriented Architecture Tightly Scripted Software Code Object Relational Software Hardware Virtualization ArcGIS Server AMI 1982–1999 Cloud Computing GIS iPhone Apps ArcGIS Desktop ArcGIS 2008 -2009 ArcGIS 2010 ARC/INFO 1982 - ArcInfo 1999 - REST API -ArcGIS.com - ArcEditor 2000 - WEB 2.0 Interoperability -Online Community Basemap ArcView GIS 1992 - ArcView - User Publishing / Collaboration -Intelligent Maps, Web Mapping MapObjects 1996 - ArcGIS Server 2004 - ArcGIS Online Resources -Packaging toolset (layers, maps) - Rich Internet Clients -Mosaic Dataset, Image Processing ArcIMS 1997 - ArcGIS Engine - Javascript, Flex API, Silverlight Javascript, -Crowd Computing Coverages ArcSDE 1999 ArcGIS Server 2009 New search tools Librarian - Geodatabase Versioning -Faster Maps New parcel fabric Shapefiles (ArcSDE) -Map Cache Temporal Analysis - Distributed Geodatabase -Map Service Description (MSD) ArcStorm -Optimized Mapping tools Replication Spatial Database Engine (SDE) - Simple non-versioned file types non- Scripts Objects Services
- 11. GIS Is Deployedin Many Ways Departmental Enterprise Traditional Emerging
- 12. Esri Product Family Desktop GIS ArcInfo ArcGIS Extensions ArcEditor Mobile GIS ArcView GIS Web Services ArcPad ArcReader ArcGIS.com ArcGIS Mobile Online Resource Centers ArcGIS Engine Templates, Forums, Collaboration Global Map Cache, Community Basemap, 3D Viewers Intelligent maps, Web mapping Google Virtual Earth ArcGIS Explorer Lightweight Network Developer GIS Viewers ESRI Developer Network (EDN) Web Browsers XML/SOAP/OGC REST (Javascript, FLASH, Silverlight) Server GIS ArcGIS Server ArcSDE Image Server ArcIMS Blending into a single server product line GIS Data Files Geodatabase Imagery Map Cache Distributed Geodatabase Management
- 13. Desktop Operations Stand aloneDesktop Connected Desktop Centralized Desktop Management ICA RDP ArcGIS Desktop ArcGIS Desktop Direct Connect* SDE Citrix Microsoft Personal ArcSDE DBMS Client Windows Terminal Server ArcGIS Desktop File GDB Map Cache Direct Connect* SDE ArcGIS DBMS Client Server ArcGIS.com World Basemaps Map Cache Community Basemaps Geodatabase Files Imagery Map Cache Web Maps and Services Physical or Virtual Servers
- 14. Esri Mobile Operations Standalone Desktop ArcGIS Mobile ArcGIS Desktop ArcPad (smart clients) Protected Data Cache Personal ArcSDE 4 or 10 GB SDE GDB Data Synchronization Exchange File GDB Map Cache 1 TB Geodatabase iPhone Replication application Web Server Web Services ArcGIS.com ArcGIS Server World Basemaps Map Cache Community Basemaps Web Maps and Services Files Geodatabase Imagery Map Cache Physical or Virtual Servers
- 15. ArcGIS Server WebOperations Google Microsoft HTML Browser ArcGIS Desktop ArcGIS Explorer ADF Client 3D Analyst Online ArcGIS Desktop ArcGIS Explorer Web Maps HTML Browser ArcGIS Engine JavaScript 2D and 3D Services Data Services Rich Browser Clients Geoprocessing Services Flash / Silverlight Amazon Cloud ArcGIS iPhone / Android (soon) Server Enterprise Data Center ArcGIS.com .NET/JAVA WAS SOAP REST Open App Dev Framework Standards ArcGIS Server World Basemaps Map Cache Community Basemaps Web Maps and Services Files Geodatabase Imagery Map Cache Physical or Virtual Servers
- 16. Software Technology Selection GIS Data Source Remote Data Sources Amazon Cloud Internal Data Sources ArcGIS Server Files Geodatabase Imagery Map Cache Internet Service Physical or Virtual Servers WTS (Citrix) Network Services LAN Web Server Published Network Services ArcIMS ArcGIS Distributed Batch Processing Server Web Services Published Map Products Workstations Published Geoprocessing Services Browser WAN Web Editing / Crowd Computing Desktop Applications Integrated Business Workflows Professional GIS Users Database Maintenance Heavy Business Workflows GIS Project Efforts Data Conversion Terminals Browser Mobile Applications Disconnected Operations Loosely Connected Workflows
- 17. Agenda: System ArchitectureDesing • System Design Process • GIS Technology • Data Administration • Network Communications • Esri Architecture • Performance - Software - Hardware • Virtualization • Example of CPT
- 18. Geodatabase • Geographic Data Store - Scalable, multi-user • Core ArcGIS Data model Networks Survey Addresses Vectors Annotations • Transactional model for 3D Objects Attributes GIS workflows Dimensions Topology • COM Components for Terrain Parcels data accesibility Cartography CAD Raster
- 19. Data management approach • Geodatabase built on a table structure - Spatial Data Types & SQL - Relational Integrity - Scalability, Reliability, Flexibility - Support for continous, large datasets • Based on the simple feature model - Points, Lines, Polygons - Open Access
- 20. Complex behavior inthe Geodatabase • Validation Rules - Attribute, connectivity and relationships • Domains - Ranged or coded values • Subtypes - Defined as the class level • Relationship classes - Association between objects on different classes
- 21. Complex behavior inthe Geodatabase • Geometric Networks - Connectivity rules • Network datasets - Multi-modal networks - Turns, restrictions, costs • Topology - Set of rules that define spatial behavior - Ensure data integrity • Parcel fabrics
- 22. Versioning • Technology that allows multiple users to edit and view data at the same time - Appears to users as if they have their own copy of a table - Does not apply locks or duplicate data Default Readers Version1 Version2 Editor 1 Editor 2
- 23. Replicas ArcSDE GDB • Geodatabase replication extends DFLT the concept of versioning to Parent VR1 VR2 multiple databases. Child check out check in DFLT ArcSDE GDB V1 V2 File GDB DFLT Parent Child ArcSDE GDB VR1 VR2 Two One way way - Versions in parent and child geodatabases are used as replica versions. File GDB DFLT DFLT - Changes in this replica versions are transferred during synchronization V1 V2 ArcSDE GDB ArcSDE GDB
- 24. Replication types • Checkout/Check-in replication: One time - Edits in child replica can be syncronized one single time Parent Child with parent replica. SDE - Uses: Disconected editing; File GDB as a requisite. SDE PGDB FGDB • One-way Replication: - Allows data changes to be sent multiple times in a single Development Production direction (parent-to-child or child-to-parent) - Data in origin GDB are editable; data in destination GDB are Persist read-only. Edit Read - Changes in destination GDB are overwritten during SDE (child-to-parent) syncronization if edits in origin GDB exist. SDE PGDB - Uses: development-staging-production models; data FGDB publishing • Two-way replication: Office A Office B - Allows data changes to be sent multiple times, both ways. Persist - If the same row is edited in both replica geodatabases, it is Edit A Edit B detected as a conflict when the replicas are syncronized. SDE SDE Conflict resolution
- 25. Historical versions • Geodatabase archiving. - Built on Geodatabase versions. - Enables recording and accessing changes made to a subset of data. - No need for storing snapshots for the entire database. - Archiving lives as long as the Version is alive. - Introduces a historical version. - Creation of historical markers possible.
- 26. Geodatabase compress • Maintain database performance and health. - Delete all states that do not participate within a version's lineage. - Collapse any candidate lineage of states into one state. - Move rows from the delta tables into the business tables. Before After 0 0 v2 v2 v1 DEFAULT v1 v4 v4 DEFAULT
- 27. Geodatabase administration in10.1 • New Geodatabase administration dialog box - Manage versions - View and remove connections - View and manage locks - Block new connections • For system tables: - Rebuild indexes - Update statistics
- 28. Managing data inArcGIS 10.1 • View and manage privileges • View locks on a version or specific dataset • Rebuild indexes on multiple datasets • Update database statistics on multiple datasets • Enable editor tracking • One-click spatial table registering
- 29. Enterprise Geodata Management Providingoptions for distributing geodata management activities Geodatabase Versions •Direct editing •Basis for replication Production Publication Geodatabase Geodatabase Connected Replicas •Workgroups •Two-way exchange One-Way Replica Disconnected Replicas •Ideal for outsourcing work •XML-based transfer •Two-way exchange •Read-Write Access •Read-Only Access •Tuned for Editing •Tuned for Services Check Out Replicas •Authoritative Content •Approved Content •File-based GDB •Inhouse or Outsourced •One-time exchange
- 30. Multi-User ArcSDE PostGIS Francesc Puigvert
- 31. Agenda: System ArchitectureDesing • System Design Process • GIS Technology • Data Administration • Network Communications • Esri Architecture • Performance - Software - Hardware • Virtualization • Example of CPT
- 32. GIS Network Impact What GIS Does ... - Graphic Data Representation (Maps) - Large Quantity of Data Analysis - Lots of Network Traffic Dagwood Sandwich Network Communications Distributed Clients Central Data Repository
- 33.
- 34. Network Latency canmake a difference Network latency is minimum communication packet travel time between client and server platform Rule of thumb: GIS application with 20 layers makes 200 round trips to the ArcSDE Server.
- 35. Network Latency Example: 200 trips to server for single map display (i.e. 10 sequential round trips for each map layer – 40 display layers) Local User Local Network (LAN) Latency Transport Time CPU Time 0.001 sec (10 Mb / 10 Mbps) CPU Time 200 trips DBMS 0.66 sec 0.2 sec 1.0 sec 0.05 sec GIS User 2.00 sec per display, maximum 5.00 Mbps traffic (10 Mb/2.00 sec) Remote User 500 miles (WAN) Latency Transport Time CPU Time 0.03 sec (10 Mb / 10 Mbps) DBMS CPU Time 0.66 sec 200 trips 0.05 sec 6.0 sec 1.0 sec GIS User 7.80 sec per display, maximum 1.28 Mbps traffic (10 Mb/7.80 sec) (4 times slower than local environment) Latency reduces peak single user network utilization. Note: Message protocols (minimum chatter) perform best over high latency.
- 36. Network Traffic Adjustments Outputformat adjustments
- 37. Network Traffic Adjustments Data source adjustments SDE and File GDB about the same Shapefile traffic can be significant
- 38. Network Suitability Analysis 1. Identify user workflows Identify average display traffic (Mbpd) Utilization 100% 60% 2. Identify user locations 80% 3. Compute workflow display traffic 60% Traffic = productivity (DPM) x Mbpd / 60 seconds 40% 4. Compute site traffic 20% Site Traffic = Sum (workflow traffic for that site) 0% Traffic 5. Identify Site Bandwidth 6. Compute network utilization Utilization = traffic / bandwidth
- 39. Network Suitability Analysis 7.Calculate Workflow response times Response time Calculate network queue times Transport time Queue time Calculate network transport times Network response time = transport + queue times + latency Calculate workflow display response times (Platform + Network response times) Validate user productivity Workflow Performance Summary 1.6 Client (Adjust productivity as needed) 1.4 Latency NWQ 1.2 Network Performance (sec) 1.0 WTSQ Response time Cycle time Computed Think time WTS 0.8 WebQ Margin Minimum 0.6 Web think time MapQ 0.4 Map 0.2 DBMSQ 0.0 DBMS 8. Show Workflow Performance Summary
- 40. Application Types andNetwork Load •Rich Client applications – Heavy Load (10 Mb per display) •RCA via Citrix(image) – Medium (5 Mb) •RCA via Citrix(Vector) – Light (3 Mb) •Web Apps (Dynamic) – Medium (2 Mb) •Web Apps (Cached) – Medium (1 Mb) •Services (REST) – Medium (2 Mb) •Services (SOAP) – Medium (3 Mb) •Mobile Apps – Light (0.05 Mb)
- 41. Agenda: System ArchitectureDesing • System Design Process • GIS Technology • Data Administration • Network Communications • Esri Architecture • Performance - Software - Hardware • Virtualization • Example of CPT
- 42. ArcGIS System ------- ArcGIS Desktop ------- ArcGIS Web ArcGIS Terminals ArcInfo ArcEditor ArcView --------- ArcGIS Engine --------- Explorer Browser Mobile Terminal Services Web Services ArcGIS Desktop ArcGIS Server Web GIS ArcGIS Online Files Geodatabase Imagery Map Cache
- 43. Single-tier Platform Configuration Standard Configuration High-Availability Configuration Network Load Balancing GIS Web Server GIS Web Server GIS Web Server WAS WAS1 WAS2 SOM SOM1 SOM1 Load Balancing SOM2 SOC1 SOC1 SOC1 SOC2 SOC1 SOC2 SDE DC SDE DC SDE DC SDE DC SDE DC SDE DC DBMS Client DBMS Client DBMS Client DBMS DBMS1 DBMS2 DATA DATA DATA Workgroup Server with SQL Express Enterprise Server with SQL Express or other Supported DBMS Platforms
- 44. ArcGIS Server ImageExtension. Single-tier Traditional Configuration High-Availability Configuration Single Server Multiple Servers Network Load Balancing AGS Image Service AGS Image Service AGS Image Service AGS SOM AGS SOM AGS SOM AGS SOC AGS SOC AGS SOC Mosaic Data Set Mosaic Data Set Mosaic Data Set Imagery Imagery Imagery AGS Image Extension AGS Image Extension AGS Image Extension Data Source Data Source Data Source
- 45. Two-tier Platform Configuration StandardConfiguration High-Availability Configuration GIS Web Server Network Load Balancing WAS SOM1 GIS Web Server GIS Web Server WAS1 SOM WAS2 SOC1 SOC1 SOM1 Load Balancing SOM2 SDE DC SDE DC DBMS Client SOC1 SOC2 SOC1 SOC2 SDE DC SDE DC SDE DC SDE DC DBMS Client DBMS Client Cluster Address DBMS DATA DBMS DBMS Geodatabase Server DATA Primary Data Server Failover Data Server Common Storage Array Enterprise Server with Supported Database
- 46. Three-tier Platform Configurations Standard Configuration High-Availability Configuration Network Load Balancing GIS Web Server GIS Web Server GIS Web Server WAS1 WAS1 WAS2 SOM1 SOM1 SOM2 SOM Load Balancing SOM Load Balancing SOC1 SOC1 SOC1 SOC1 SOC2 SOC1 SOC2 SOC1 SOC2 SDE DC SDE DC SDE DC SDE DC SDE DC SDE DC SDE DC SDE DC SDE DC DBMS Client DBMS Client DBMS Client DBMS Client DBMS Client DBMS Client Container Machine Container Machine Container Machine Container Machine Container Machine Container Machine Enterprise Database Server DS Enterprise Database Server DS
- 47. Votos y CifrasExperience Francesc Puigvert
- 48. Votos y Cifras •Elections & data Web site -Elections Results -Elections historic results -Population Data (economic + demographic) •Cloud Amazon Server •Esri technology -ArcGIS Server workstation -JavaScript API -Tiled Maps
- 49. 1TV 5 Newspapers
- 50. votos y cifras:Results •331.065 Esri España brand impacts -Direct presence in 18 regional newspapers website (Heraldo, Provincias, Verdad, Ideal,….) -More than 10.000 social networks impacts (Twitter, Facebook, eskup, Nolesvotes,… -Daily record: 107.920 visits the 22-M. •An average of 4 minutes remaining in the application •Direct presence in the website of 4 national media (El Mundo, ABC, Publico y Telecinco) •321.085 different visitors to our website from 29-04 to 23- 05
- 52. Architecture Description Instance m1.xlarge Extra Large Instance JavaScript Web App 15 GB memory 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each) Map Service 64-bit platform ArcGIS Server + FGDB Tiled Map Service Internet AWS Esri Instance Basic Unit …
- 53. Elastic Load balancing Elastic Load balancing 1 2 4 2
- 54. Agenda: System ArchitectureDesing • System Design Process • GIS Technology • Data Administration • Network Communications • Esri Architecture • Performance - Software - Hardware • Virtualization • Example of CPT
- 55. User Expectations –a closer look 4.0 Web IMS – ArcIMS ADF – ArcGIS Server ADF SS/SOC 3.5 REST – ArcGIS Server REST MXD ArcSDE MSD – ArcGIS Server REST MSD DBMS 3.0 Light complexity display Display Performance (sec) 2.5 Medium complexity display 2.0 Minimum user performance expectations. 1.5 When is the software fast enough? 1.0 0.5 0.0 IMS IMS IMS IMS ADF IMS ADF IMS ADF IMS ADF IMS ADF IMS ADF REST IMS ADF REST MSD IMS ADF REST MSD ...2001 ...2002 ...2003 ...2004 ...2005 ...2006 ...2007 ...2008 ...2009 ...2010 ...2011 Calender Year
- 56. Map Display Complexity • Only show relevant data - Start simple - Use field visibility • Use scale dependencies - Appropriate data for given scale AGD10 wkstn MXD 100%Dyn 10x7 Feature - Same number of features at all scales Local FGDB Data Source • Select the right point representation Use appropriate text and labeling - Use single layer simple or character markers Use annotation instead of labels - Use EMF instead of bitmaps Use indexed fields Use label and feature conflict weights - Use integer fields for symbol values sparingly Avoid special effects (fill patterns, halos, - Avoid halos, complex shapes, masking callouts, backgrounds) Avoid very large text size (60+ pts) • Select the right lines and polygons Avoid Maplex for dynamic labeling (avoid - Use ESRI Optimized style overuse) - Avoid cartographic lines and polygon outline Things to do to keep the map service time within performance budgets
- 57. Classic dynamic mappingtrade-off Quality vs. Speed Shaded relief Low-resolution relief Transparent layers Solid colors Maplex labeling Annotation Expensive functions Good performance
- 58. ESRI Optimized linesand polygons • Outlines for all fills are simple lines instead of cartographic lines • Picture fills are EMF-based instead of BMP-based • ESRI Optimized improves drawing performance by 50%+ ESRI Optimized improves performance
- 59. Use ArcGIS Optimizetool • Analyze map directly from ArcMap (ArcGIS 9.3.1+) - Analyze function included in Map Service Publishing tools - Provides performance tuning recommendations - Preview allows for visual analysis of performance and graphic quality - Provides optimized display rendering time
- 60. Display complexity (Mxdperfstat) • Measuring MXD Desktop display rendering time (http://arcscripts.esri.com/details.asp?dbid=16931) Key performance metrics Display refresh times at multiple scales Layer refresh times for each map scale Performance tuning recommendations
- 61. Design Phase—Performance Factors PerformanceTest Cache vs. MSD vs. MXD 2D Cache vs. Dynamic Data Response Time Comparison 6 When possible, use 5 Optimized Services for 4 dynamic data. Single user Response Time (sec) response times 3 are similar. 2 If data is static, use 1 cache map Services. 0 1 6 11 16 21 26 31 36 41 46 51 User Count 2D Cache 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 MSD 0.38 0.39 0.41 0.45 0.48 0.54 0.62 0.76 1.17 2.12 2.79 Dynamic 0.57 0.58 0.62 0.67 0.78 0.87 1.22 1.74 2.72 3.8 4.98 Cache map services use the least of hardware resources. 61
- 62. Web Mapping outputimage size Smaller maps travel faster PNG24 = 49 KB Traffic (56Kbps / 1.5Mbps) 0.5 Mbpd (10 sec / 0.6 sec) PNG24 = 267 KB Traffic (56Kbps / 1.5Mbps) • Based on REST MXD service 2.7 Mbpd (151 sec / 2.3 sec) - 600 x 400 resolution - 1280 x 1024 resolution (same display) • Recommendation: Use reasonable output image size to support application while minimizing impact to network
- 63. Web Mapping outputimage type Output Selection Response Image Size Traffic 56Kbps 1.5Mbps RASTER JPEG 60 KB 0.6 Mbpd 13 sec 0.8 sec PNG24 130 KB 1.3 Mbpd 36 sec 1.4 sec PDF 180 KB 1.8 Mbpd 59 sec 1.9 sec Output Selection Response VECTOR Image Size Traffic 56Kbps 1.5Mbps JPEG 60 KB 0.6 Mbpd 13 sec 0.8 sec PNG24 50 KB 0.5 Mbpd 11 sec 0.7 sec PNG8 30 KB 0.3 Mbpd 6 sec 0.5 sec PDF 180 KB 1.8 Mbpd 59 sec 1.9 sec High network traffic Based on REST MXD Service reduces performance 600 x 400 resolution Recommendation: Use appropriate output type to support application while minimizing network traffic
- 64. Concurrent User Loads • Service Instance Configuration (concurrent users) - Random arrival distribution reduces peak throughput - 3-5 instances per core provides optimum throughput - Display response time increases with max instance configuration Random arrival queuing delays Note: 4 core platform – 16 (4x4) instances provide peak throughput What is optimum capacity setting?
- 65. Batch Process Loads ServiceInstance Configuration (batch processing) Batch process consumes a processor core N+1 Instances provide optimum throughput Batch run time increases with max instance configuration Workflow Performance Summary 1.20 DPM/TPM 621 622 622 622 622 622 Client Latency 546 NWQ Network 1.00 100% 100% 100% 100% 100% 100% Performance (sec) Tier 01Q Tier 01 88% Tier 02Q Tier 02 0.80 410 Tier 03Q Tier 03 0.60 66% Tier 04Q Tier 04 273 No random arrival queuing Tier05Q Tier05 0.40 Tier06Q Tier06 44% 137 Tier07Q Tier07 0.20 Tier08Q Tier08 22% Tier09Q Tier09 0.00 Tier10Q Tier10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 Batch Processes Note: 4 core platform – 5 (4+1) instances provide peak throughput What is optimum capacity setting?
- 66. Memory recommendations • Too many instances per server can exhaust memory - Increased paging when not enough memory - Slower processing due to shared compute resources - Maximum of 3-5 instances per core recommended • Too few instances per server - Can limit utilization of host hardware - Minimum of 3 instances per core recommended • Provide sufficient memory to support optimum performance - Minimum of 3 GB memory per core recommended - More memory may be required when using large file data sources
- 67. Why should Icare about map caches? • Performance, performance, performance • Scalability: Industry standard • Cartographic quality • ArcGIS Explorer Online
- 68. What should youcache? • Base maps (always) • Operational layers that satisfy one of the following: - High volumes of traffic - Don’t change often - Cover small scales only
- 69. What about optimizedmap services? • Optimized drawing format - Based on MSD file - Enhanced map drawing engine - Use to generated map tiles faster • Does NOT replace cache map service - Requires dynamic rendering - Web services are optimized for cached tiles
- 70. Tile size • Pixel dimensions of each image • 256x256 is the web standard - 512 X 512 : legacy ArcGIS Online 256 • Larger dimensions are faster to build, but tiles take longer to download 256
- 71. Choosing an imageformat • Image format effects - Tile storage space requirements - Web application performance (speed and supported browsers) - Tile image quality and transparency .JPG • JPEG - Great compression for many colors but not transparency • PNG - Best compression for less colors .PNG • Mixed
- 72. Cached imagery workflow • Prepare imagery Prepare - Build raster pyramids - Create mosaic dataset with overviews • Author map document Author - Add imagery to map - Save as MSD • Publish as a map service Publish • Create and manage the map cache - Configure cache definition Create - Generate cache tiles - Update cached tiles
- 73. Cache processing profile • 500 total processing hours Single instance 5 instances 10 instances (4 core server) (4 core server) (2 x 4 core servers) ArcSOC.exe ArcSOC.exe ArcSOC.exe ArcSOC.exe ArcSOC.exe ArcSOC.exe ArcSOC.exe Thread Thread Thread Thread Thread Thread Thread Thread ArcSOC.exe ArcSOC.exe ArcSOC.exe Thread Thread Thread ArcSOC.exe ArcSOC.exe ArcSOC.exe Thread Thread Thread ArcSOC.exe ArcSOC.exe ArcSOC.exe 500 hours Thread Thread Thread Thread Thread Thread 125 hours 65 hours Take full advantage of available hardware resources J8338
- 74. Standard Performance EvaluationCorporation How do we measure relative performance? SPEC mission: To develop technically credible and objective benchmarks so that both computer designers and purchasers can make decisions on the basis of realistic workloads ESRI system design history 1992–1996, SPEC92 benchmark results 1996–2000, SPEC95 benchmark results 2000–2006, SPEC2000 benchmark results 2006+, SPEC2006 benchmark results SPEC benchmarks have served ESRI customers well
- 75.
- 76.
- 77. Publishing services Francesc Puigvert
- 78. Agenda: System ArchitectureDesing • System Design Process • GIS Technology • Data Administration • Network Communications • Esri Architecture • Performance - Software - Hardware • Virtualization • Example of CPT
- 79. Desktop Virtualization Customer Experiencieand Motivations • Simplified/Streamlined Administration • Release control; Managing Upgrades • WAN bandwidth and latency tolerance • High availability for ArcGIS Desktop solutions • Reduced client side requirements • Specialized skill requirements for server administration and configurations to support printing, working with local data, etc.
- 80. Enhancing Desktopo Deployment- Infraestructure • App/Session Virtualization + High Availability + Simplified Administration + Network efficiency / WAN - 3D - System admin skill level ~ User Experience
- 81. Server Virtualization Customer Experiencieand Motivations • IT Standards • Availability • Disaster Recovery • Simplified/Streamlined Administration • Other
- 82. Enhancing Server Deployment-- Infrastructure • Hardware Virtualization + High Availability + Disaster Recovery + Right sizing + Staging environment - Performance risk
- 83. Example: Recommendations forVMWare Physical Machine: Physical Machine: 4 CPU/16GB RAM 4 CPU/16GB RAM VM: 1 CPU/4GB RAM VM: 1 CPU/4GB RAM VM: 4 CPU/16GB RAM SOM SOC WS SOC SOM SOC WS VM: 1 CPU/4GB RAM VM: 1 CPU/4GB RAM Physical Machine: SOC SOC 4 CPU/16GB RAM VM: 2 CPU/8GB RAM Physical Machine: 4 CPU/16GB RAM SOM SOC WS VM: 1 CPU/2GB RAM VM: 1 CPU/2GB RAM VM: 2 CPU/8GB RAM SOM SOC WS SOC SOC VM: 1 CPU/2GB RAM VM: 1 CPU/2GB RAM Optimal Configuration SOC SOC 4 VMs, 1 CPU/VM, 2GB RAM/VM
- 84. Recommendations for VMWare Penaltyof virtualization (Physical vs. various VM configs) 45000,00 32% Degradation 11% Degradation 40000,00 35000,00 Throughput (Tr/Hr) 30000,00 25000,00 20000,00 15000,00 10000,00 5000,00 0,00 1VM,4C,16R 2VM,2C,8R 4VM,1C,4R 4VM,1C,2R 1P,4C,16R Virtual Machine Configuration
- 85. Example: Virtual Serverpeaks of Works Web Browser Internet Intranet SOM SOM SOC SOC SOC SOC SOC Direct Connects Direct Connects Virtual Server Physical Servers DBMS
- 86. Agenda: System ArchitectureDesing • System Design Process • GIS Technology • Data Administration • Network Communications • Esri Architecture • Performance - Software - Hardware • Virtualization • Example of CPT
- 87. Capacity Planning Tool Francesc Puigvert
- 88. City of Portland- Solution Architecture City Hall Internet T-1 15 Mbps 1 Gbps LAN T-1 15 Mbps Sandy River Maintenance Facility T-1 T-1 15 Mbps 15 Mbps
- 89. City of Mini-Portland- Solution Architecture City Hall Internet T-1 15 Mbps 1 Gbps LAN T-1 15 Mbps Maintenance Facility T-1 15 Mbps City of Minnie-Portland
- 90. User Requirements ArcGIS Desktop(Power, Editor, Viewer) Desktop (LAN), Citrix (remote sites) ArcGIS Server (Internal and Public web services)
- 91. User Workflow Requirements • Suggested DESIGN procedure to build the city of Mini- Portland capacity planning model. - ArcGIS Desktop ArcInfo (AI) Power users - AGD10 wkstn MXD Med 100%Dyn 10x7 Feature - AGD10 WTS Citrix MXD Med 100%Dyn 10x7 V ICA - ArcGIS Desktop Editors - AGD10 wkstn MXD Lite100%Dyn 10x7 Feature - AGD10 WTS Citrix MXD Lite 100%Dyn 10x7 ICA - ArcGIS Desktop Viewer - AGD10 wkstn MXD Lite100%Dyn 10x7 Feature - AGD10 WTS Citrix MXD Lite 100%Dyn 10x7 R ICA - ArcGIS Server internal Web mapping application - AGS10 REST MXD Med 100%Dyn 10x7 R JPEG - ArcGIS Server REST MSD Public Web mapping application - AGS10 REST MSD Lite 100%Dyn 10x7 V PNG8
- 92. User Workflow Requirements - ArcGIS Desktop ArcInfo (AI) Power users - AGD10 wkstn MXD Med 100%Dyn 10x7 Feature - AGD10 WTS Citrix MXD Med 100%Dyn 10x7 V ICA - ArcGIS Desktop Editors - AGD10 wkstn MXD Lite100%Dyn 10x7 Feature - AGD10 WTS Citrix MXD Lite 100%Dyn 10x7 ICA - ArcGIS Desktop Viewer - AGD10 wkstn MXD Lite100%Dyn 10x7 Feature - AGD10 WTS Citrix MXD Lite 100%Dyn 10x7 R ICA ArcGIS Server internal and public Web mapping application
- 93. User Workflow Requirements AGS10 REST MXD Med 100%Dyn 10x7 R JPEG and... Where is AGS10 REST MSD Lite 100%Dyn 10x7 V PNG8?
- 94.
- 95.
- 96. City of Mini-Portland- Solution Architecture City Hall Internet T-1 15 Mbps 1 Gbps LAN T-1 15 Mbps Maintenance Facility T-1 15 Mbps City of Minnie-Portland
- 97. Additional Resources • System Design Process - http://wiki.gis.com/wiki/index.php/System_Design_Process • Esri Press. Building a GIS - http://esripress.esri.com/display/index.cfm?fuseaction=display &websiteID=141 • On-line resource center - http://esripress.esri.com/display/index.cfm?fuseaction=display &websiteID=141&moduleID=27
- 98. Additional Resources • ESRI project methodology - www.esri.com/services/professional- services/methodology.html • Business case resources - The Business Benefits of GIS: an ROI Approach—Outlines case studies and general methodology for doing cost-benefit analysis - Thinking About GIS—Roger Tomlinson - www.esri.com/getting_started/executives/success.html • Project Management Body of Knowledge (PMBOK) - www.pmi.org
- 99.
- 100.
- 101.