Downloaded 269 times
More Related Content
Similar to Chapter 4: Data Architecture Management
![[Redis Released]- FalkorDB - Redis + Graph Agentic Memory’s Secret Sauce](https://cdn.slidesharecdn.com/ss_thumbnails/redisreleased-falkordbslidedeck-1125-251115194922-e1c0046b-thumbnail.jpg?width=640&height=640&fit=bounds)
Chapter 4: Data Architecture Management
- 1.
- 2. Chapters Objectives: • Provideoverviews of Data management second function which is “Data Architecture management”. Through: • Introduction about Data Architecture Management and the Diagram that provides inputs, suppliers, activities…etc. • Concepts and Activities used with Data Architecture management. Beginning with starch “Architecture overview” and then describe enterprise Architecture, Architectural Frameworks, and finally, describe the Zachman Framework for enterprise architecture and all the methodologies and techniques used. • Then, provide the descriptions of each activities of Data Architecture management. • Final part, summary including Guiding principles, processes, Organizational and cultural issues.
- 3. 4 Data ArchitectureManagement • The Second Data Management function in Data Management Framework. • The first Data management function interact and influenced by the Data governance function. • It is The process of defining and maintaining specifications: • Provide a standard common business vocabulary • Express strategic data requirements • Outline high level integrated designs to meet these requirements • Align with enterprise strategy and related business architecture
- 4. 4.1 Introduction “Whatis Data Architecture ?” • Data architecture is • an integrated set of specification artifacts used to define data Requirements, guide integration and control of data assets, and align data investments with business strategy. • An integrated collection of master blueprints at different levels of abstraction. • Include formal data names, comprehensive data definitions, effective data structures, precise data integrity rules, and robust data documentation. • Most valuable when it supports the information needs of the entire enterprise. • Enables data standardization and integration across the enterprise.
- 5. 4.1 Introduction “ThePosition of Data Architecture” • Enterprise Data Architecture is: • part of the larger enterprise architecture. • integrated with other business and technology architecture • Enterprise Architecture integrated data, process, organization, application, and technology architecture. • Enterprise Architecture helps organizations manage change and improve effectiveness, agility, and accountability.
- 6. 4.1 Introduction “TheDiagram of Data Architecture”
- 7. Data Architecture ManagementDiagram • Enterprise data Architecture is an integrated set of specifications and documents. • Include 3 major categories of specifications: • Enterprise data Model: The heart and soul of enterprise data architecture. • Information value chain analysis: Aligns Data with Business processes and other enterprise architecture components. • Related data delivery architecture: including database architecture, data integration architecture, data WH /BI architecture, document Content architecture, meta-data architecture. • Enterprise data architecture defines standard terms for the Business entities that are important to the organization • Establishing a common business vocabulary of business entities and data attributes. • Defines the semantics of an enterprise.
- 8. 4.2 Concepts andActivities • Data architecture management is the function of defining a blueprint for managing data assets. • Data architects play a key role in the critical function of data architecture management. • The concepts and activities related to Data architecture management and the roles of data architects are presented in this section. • 4.2.1 Architecture overview • 4.2.1.1 Enterprise Architecture • 4.2.1.2 Architectural Framework • 4.2.1.3 The Zachman Framework for Enterprise Architecture • 4.2.1.4 The Zachman Framework and Enterprise Data Architecture
- 9. 4.2.1 Architecture Overview •The word “Architecture” • Organized arrangement of component elements • Optimize the function, performance, feasibility, cost, and/or aesthetics of the overall structure or system. • Integrated set of closely related views reflecting the issues and perspectives of different stakeholders. • Used with different fields and areas of sciences. • Help to understand something complex, “Natural(Living organisms, mathematics, geological formation)” or “human-made(Building, Machines, database…etc)” • Include standards and application to specific design needs. • Existed at different levels, from the main planning into the small parts of the specific product.
- 10. 4.2.1 Architecture Overview“In Technology & information systems” • Architecture of Information systems “the design of any complex technical object or system” • Architecture in IT include both “closed” design standards specific vendors, and “open” standards available to any vendor. • Architecture is all about integration. • “IS” is very complex, “Isolated applications, building tactical approaches make IS more complex” • Therefore, restructuring applications and database to overall architecture become more attractive.
- 11. 4.2.1.1 Enterprise Architecture •Enterprise Architecture: an integrated set of business and IT specification models and artifacts reflecting enterprise integration and standardization requirements. • Define business integration of data, process, organization, and technology. • Encompasses both business architecture and IS architecture. • Provide systematic approach to managing information and systems assets, addressing strategic Business requirements, enable informed portfolio management of organization’s projects. • Supports strategic decision-making by helping manage, change, tracing the impact of organizational change of system. • Tools for planning IT governance, and portfolio management.
- 12. 4.2.1.1 Enterprise Architecture •Enterprise Architecture includes many related models and artifacts: • Information architecture: Business entities, relationships, attributes, definitions, reference values. • Process architecture: function, activities, workflow, events, cycles, products, procedures. • Business architecture: Goals, strategies, roles, organization structure, locations. • System architecture: Applications, software components, interfaces, projects. • Technology architecture: Networks, hardware, software platforms, standards, protocols. • Information value chain analysis artifacts: Mapping the relationship between data, process, business, systems, and technology. • Enterprise models generate artifacts (graphical diagrams, tables, analysis matrices and textual documents)
- 13. 4.2.1.1 Enterprise Architecture •Enterprise Architecture distinguishes between the current “as-is” and the target “to be ” perspectives. Often include intermediate stages and migration plans. • The investment in enterprise architecture based on their understanding of business need and business risk. • Benefits of enterprise architecture: • Enable integration of data, process, technologies, and efforts. • Align information systems with business strategy. • Enable effective use and coordination of resources. • Improve communication and understanding across the organization. • Reduce the cost of managing the IT infrastructure. • Guide business process improvement. • Organizations responded effectively to changing market opportunities, industry challenges, and technological advances. Helps evaluate business rik, manage change, and improve business effectiveness, agility, and accountability. • Methods for defining enterprise architecture: • IMB’s Business Systems planning (BSP) • Information Systems Planning (ISP) • James Martin’s agility, and accountability.
- 14. 4.2.1.2 Architectural Frameworks •(Architecture for Architecture), • Provide way to understand architecture and the structures or systems requiring architecture. • Two different kinds of architectural framework: • Classification framework organize the structure and views that encompasses enterprise architecture. Define the standard for the artifacts describing these views and the relationship between these vies. Most artifacts are diagrams, tables, and matrices. • Process framework specify methods for business and system planning, analysis, and design processes. Some IT planning and software development lifecycle(SDLC) methods include their own composite classifications. Not all process frameworks specify the same set of things, and some are highly specialized.
- 15. 4.2.1.2 Architectural Frameworks •The scope of Architectural framework is more than IS architecture. • Guide the solution design for specific Information systems. • Many framework are in existence, such as: • TOGAF: the Open Group Architectural Framework (process framework and standard ) • ANSI/IEEE 1471 -2000 • Some consulting firms, governments and defense departments develop architectural framework, including: • (FEA): Federal Enterprise Architecture • (GEA) Government Enterprise Architecture • DODAF: The US Department of Defense Architecture Framework. • MODAF: The UK Ministry of Defense Architecture Framework. • AGATE: The France DGA Architecture Framework
- 16. 4.2.1.3 The ZachmanFramework for Enterprise Architecture
- 17. 4.2.1.3 The ZachmanFramework for Enterprise Architecture • Most widely known and adopted architectural framework. • First Published in 1987. • Figure 4.2 can be used by data and Information Systems communities. • Provide “Models "used to mange enterprise and develop systems. • 6 by 6 matrix representing the intersection of two classification schemas-two dimensions of systems architecture. Description Labels position Contributors (The Architecture participants) Right hand row labels The contents (results of contributor's work) Left hand row labels Generic answers to each question Column footer labels
- 18. 4.2.1.3 The ZachmanFramework for Enterprise Architecture • The first dimension “Stakeholders”: ( the planner, owner, builder, implementer, and participant) • Each have different issues to identify, understand, and resolve. • The planner (scope Contexts): Lists of business elements defining scope identified by strategists as theorists. • The Owner (Business Concepts): Semantic models of the business relationships between business elements defined by executive leaders as Owners. • The designer (System Logic): Logical models detailing system requirements and unconstrained design represented by Architects as Designers. • The builder (Technology physics): Physical models optimizing the design for implementation for specific use under the constraints of specific technology, people, costs, and timeframes specified by Engineers as Builders. • The implementer (Component Assemblies): a technology-specific, how components are assembled and operate configured by technicians as implementers. • The participant (Operations Classes): Actual functioning system instances used by workers as Participants.
- 19. 4.2.1.3 The ZachmanFramework for Enterprise Architecture • The Second dimension “Questions and way to answers” : (who, what, why, when, where, and how) • The revised labels for each column are in parentheses: • What (the data column): Materials used to build the system ( Inventory sets) • How ( the function column): Activities performed ( process Transformations) • Where ( then network column) : Locations, topography, and technology ( network nodes). • Who ( the people column) : Roles and organizations ( organization Groups). • When ( the time column): Events, Cycles, and schedules ( Time periods). • Why ( the goal column): Goals, strategies, and initiatives ( motivation Reasons). • Each cell of the framework represents a unique type of design artifact, defined by the intersection of its row and column. • Guarantee to achieve enterprise owners and subsequent decisions.
- 20. 4.2.1.3 The ZachmanFramework for Enterprise Architecture • There are several reasons why the Zachman Framework widely adopted: • Simple since it has only two dimensions and easy to understand. • Address both individual divisions and departments in comprehensive manner, and manages architecture. • Use non-technical language which make people think and communicate more precisely. • Can be used to frame and help understand a wide array of issues. • Help solve design problems, focusing on details without losing track of the whole. • Helps teach many different information systems topics. • Tool for planning, provide the context to guide better decisions. • Independent of specific tools or methods. Any design tool or method can map to the framework to see what the tool or method does and does NOT do.
- 21. 4.2.1.4 The ZachmanFramework and Enterprise Data Architecture • The enterprise data Architecture is part of the large enterprise architecture. • Includes process, business, systems and technology architecture. • Enterprise data architecture consists of three major sets of design components: • Enterprise data model, identifying subject areas, business entities, the business rules governing the relationships between business entities, and at least some of the essential business data attributes • The information value chain analysis, aligning data model components ( subject areas and/or business entities) with business processes and other enterprise architecture components, include organizations, roles, applications, goals, strategics, projects. • Related data delivery architecture, including data technology architecture, data integration, data WH/BI architecture, enterprise taxonomies for content management, meta-data architecture.
- 22. 4.2.1.4 The ZachmanFramework and Enterprise Data Architecture • The first cell “data” column-no known as “Inventory sets”, represent familiar data modeling and database design artifact ( more detail in chapter 5): • Planner view: a list of subject areas and business entities. • Owner view: conceptual data models showing the relationship between entities. • Designer view: fully attributed and normalized logical data model. • Builder view: physical data models optimized for constraining technology. • Implementer view: detailed representations of data structures, typically in SQL (DDL) • Functioning enterprise: actual implemented instances. • Finally, The Zachman framework help Data Architects building their design and architecture piece by piece. “big picture”
- 23. 4.2.2 Activities • Thedata Architecture management function have several activities to defining the blueprint for managing data assets. • 4.2.2.1 Understanding Enterprise Information Needs • 4.2.2.2 Develop and Maintain the Enterprise Data Model • 4.2.2.3 Analyze and Align with Other Business Models • 4.2.2.5 Define and Maintain the Data Integration Architecture • 4.2.2.6 Define and Maintain the DW/BI Architecture • 4.2.2.7 Define and Maintain Enterprise Taxonomies and Namespaces • 4.2.2.8 Define and Maintain the Meta-data Architecture
- 24. 4.2.2.1 Understanding EnterpriseInformation Needs • Evaluate the current inputs and outputs by organization, both from internal and external targets. • Use actual system documentation and reports and interview the participants. • The results are a list of important data entities, data attributes, and calculations. • Organize these items by business unit and subject area. • Review the list with the participants to ensure categorization and completeness. • The list then become the basic requirements for an enterprise data model.
- 25. 4.2.2.1 Understanding EnterpriseInformation Needs • Data models define business entities and data attributes. • Data modeling is an analysis and design method used to: • Define and analyze data requirement • Design logical and physical data structures that support these requirements. • (EDM) enterprise data model is an integrated, subject-oriented data model defining the essential data produced and consumed across an entire organization. • Integrated: all data in this model is produced once, and fit together seamlessly, as CEO see the enterprise. • Subject –oriented: the model is divided into commonly recognized subject areas. • Essential: means the data critical to the effective operation and decision- making of the organization.
- 26. 4.2.2.1 Develop andMaintain the Enterprise Data Model • Data Development implements data Architecture, extending and adapting enterprise data models to meet business application needs and project requirements. • In this sections present The EDM, and its layers and levels as follows: • 4.2.2.2.1 The Enterprise Data Model • 4.2.2.2.2 The Subject Area Model • 4.2.2.2.3 The Conceptual Data Model • 4.2.2.2.4 Enterprise Logical Data Models • 4.2.2.2.5 Other enterprise Data Model Components • Figure 4.3 Represent Data Model Layers
- 27. Figure 4.3 EnterpriseData Model Layers
- 28. 4.2.2.2.1 The EnterpriseData Model • It is an integrated set of closely related deliverables. Most of these deliverables are generated using a data modeling tool. • An enterprise data model is a significant investment in defining and documenting an organization’s vocabulary, business rules, and business knowledge. • Organization can purchase an enterprise data model or build it from starch. Large database vendors include them as additional. However, customization is required. • Enterprise data models effected by the time and effort consumed to build. “Most Successful are built incrementally and iteratively” • Enterprise Data Model build in Layers, initial on the most critical business subject areas. The higher layers are the most fundamental, with lower layers dependent on the higher layers.
- 29. 4.2.2.2.2 The SubjectArea Model • The highest Layer in an enterprise data model known as (SAM). • List of major subject areas that collectively express the essential scope of the enterprise. (Row1, Column 1) in Zachman framework. (Business entities). • Two main ways to communicate a subject area model: • An outline, which organizes smaller subject areas within larger subject areas. • A diagram that presents and organizes the subject areas visually for easy reference. • Selection and naming SAMs are important to the success of the entire enterprise data model • Subject areas typically share the same name as central business entity. • Are also important tools for data stewardship and governance. • Define the scope of responsibilities for subject area-oriented data stewardship teams.
- 30. 4.2.2.2.3 The ConceptualData Model • The next lower level of the enterprise data model “The set of conceptual data model diagrams for each subject area” • Defines business entities and the relationships between business entities. • Business entities are the concepts and classes of things, people and palaces that familiar and interest to the enterprise. • For data governance and stewardship, every business entities should have one primary subject area which ‘owns’ the master version of that entity. • Conceptual data model diagrams do not depict the data attributes of business entities. ( may include Many-to-Many business relationships between entities). • The enterprise conceptual data model must include a glossary containing the business definitions and other meta-data associated with all business entities and their relationships. • Can improved business understanding and semantic reconciliation.
- 31. 4.2.2.2.4 Enterprise LogicalData Model • Some enterprise data models also include logical data model diagrams for each subject are, adding a level of detail below the conceptual data model. • Through, depicting the essential data attributes for each entity. • Enterprise Logical data model identifies the data needed about each instance of a business entity. • Essential data attributes are those data attributes without which the enterprise cannot function. • Determining which data attributes to include in the enterprise data model is a very subjective decision. • Reflecting the enterprise perspective. Independent from any particular need, usage, and application context. • Only Partially attributed. ( can not identify all possible data entities) • Should include a glossary of all business definitions associated meta-data.
- 32. 4.2.2.2.5 Other EnterpriseData Model • Other components included in enterprise data models (Optional): • Individual data steward responsibility assignments for subject areas, entities, attributes, data value sets. (Data Governance) • Valid reference data values: controlled value sets for codes or labels or both. Sometime cross-referenced with departmental, divisional or regional equivalents. ( reference and master) • Additional data quality specifications and rules for essential data attributes ( accuracy / precision requirements..etc.) ( Data Quality) • Entity life cycles transition diagrams ( lifecycles states of the most important entities and the trigger events that change an entity from one state to another, and very useful in determining a rational set of status values ( codes and/or labels) for business entity.
- 33. 4.2.2.3 Analyze andAlign with other Business Models • Information value-chain analysis maps the relationships between enterprise model elements and other business models. • The term derives from the business value chain. • Information value-chain matrices are composite models, while information value-chain analysis is an output of data architecture. • Information value-chain analysis is the glue binding together the various form of “Primitive models” in enterprise architecture. • Data architects, data stewards, and other enterprise architects and subject matter experts share responsibility for each matrix’s content.
- 34. Figure 4.4: exampleInsurance Business Value Chain
- 35. 4.2.2.4 Define andMaintain the data Technology Architecture • Data Technology Architecture defines standard tool categories, preferred tools in each category and technology standards and protocols for technology integration. • Guide the selection and integration of data-related technology. • Both part of the enterprise overall technology architecture, as well as part of its data architecture. • Technology categories in the data technology architecture include: • DBMS • Database management utilities • Data modeling and model management tools • BI software for reporting and analysis • Extract-transform-load (ETL) , changed data capture (CDC), and other data integration tools. • Data Quality analysis and Data Cleansing tools. • Meta-data Management software, including meta-data repositories.
- 36. 4.2.2.4 Define andMaintain the data Technology Architecture, continue. • Technology Architecture Components are included in different Categories: • Current: Products Currently Supported and used • Deployment Period: Products deployed for use in the next 1-2 years. • Strategic Period: Products expected to be available for use in the next 2+ years. • Retirement: Products the organization has retired or intends to retire this year. • Preferred: Products preferred for use by most applications. • Containment: Products limited to use by certain applications. • Emerging: Products being researched and piloted for possible future deployment. • Chapter 6 has more about managing data technologies.
- 37. 4.2.2.5 Define andMaintain the Data Integration Architecture • Data Integration Architecture defines how data flows through all system. • From beginning to end • Considers both data architecture and application architecture, because it includes both databases and applications that control the flow into the system. • Matrices can define the relationships to other aspects of the enterprise architecture besides business processes, such as: • Data related to business roles, which roles have responsibility for CRUD.“create, Read. Etc..” • Data related to specific business organizations with these responsibilities. • Data related to applications that may cross business functions. • Data related to locations where local differences occur. • Practices such as (BSP) and (ISP) methods could be useful with this architecture.
- 38. 4.2.2.5 Define andMaintain the Data Integration Architecture. Continuo • The Corporate information factory (CIF) concept is an example of data integration architecture. • Data integration Architecture generally divides data warehouses, staging database, and data marts supporting BI from the source databases, Operational data stores (ODS), Master data management, and reference data/ code management systems supporting online transaction processing and operational reporting. • Data/ Process relationship matrices can have different levels of detail. Subject areas, business entities, or even essential data attributes can all represent data at different levels. • High-level functions, Mid-level activities, or low-level tasks all represent business processes.
- 39. 4.2.2.6 Define andMaintain DW/BI Architecture • Focuses on how data change and snapshots are stored in data warehouse systems for maximum usefulness and performance. • Data integration architecture shows how data moves from source systems through staging databases into data warehouses and data marts. • BI Architecture defines how decision support makes data available, including the selection and use of BI tools. • Discussed in more detail in Chapter 9 “Data warehousing and business intelligence Management”
- 40. 4.2.2.7 Define andMaintain Enterprise Taxonomies and Namespaces • Taxonomy is the hierarchical structure used for outlining topics. • Formal taxonomies are class hierarchies, while informal taxonomies are topical outlines that may not imply inheritance of characteristics from super-types. • Organizations develop their own taxonomies to organize collective thinking about topics. • Enterprise data architecture includes organizational taxonomies. • Terms used in taxonomies should be consistent with enterprise data model, as well as other modes and ontologies.
- 41. 4.2.2.8 Define andMaintain The Meta-data Architecture • Meta-data architecture defines the managed flow of meta-data. • Define how meta-data is created, integrated, controlled, and accessed. • The meta-data repository is the core of any meta-data architecture. • Meta-data architecture design for integration of meta-data across software tools, repositories, directories, glossaries, and data dictionaries. • The focus of data integration architecture is to ensure the quality, integration, and effective use of reference, master, and BI data. • The focus of meta-data architecture is to ensure the quality, integration, and effective use of meta-data. • More detail (Chapter 11 on Meta-data Management Covers this Topics)
- 42. 4.3 Summary • Theimport to defining Data Architecture is shared responsibilities among different Organizations members start from the business stewards along with Data architect and Data Analytes and more. • Data Architecture should be able to change and update in order business change • The value of data architecture is limited until data stewards participate, review and refine. Data Governance Council is the sponsor and approving body for enterprise data architecture. • Data architecture is one part of overall enterprise architecture. Serves as a guide for integration, when: • Defining and evaluating new information system projects. “Long-term integration IS” • Defining Project data requirements. • Reviewing project data designs: ensure conceptual, logical, and physical data models are consistent and contribute to the long-term implementation of enterprise data architecture.
- 43. 4.3.1 Guiding Principles •8 Guiding Principles to implement Data architecture Management in Organization: 1. DA is integrated set of specification artifacts (master blueprint) to define data requirements, guide data integration, control data assets, and align data investment with business strategy. 2. Enterprise data architecture “EDA” is part of the overall enterprise Architecture, along with process, business, systems, and technology architecture. 3. EDA include 3 major categories of specifications: The enterprise data model, information value chain analysis, and data delivery architecture. 4. EDA is about more than just data. Help to establish the semantic of an enterprise. 5. Enterprise data model is an integrated subject-oriented data model defining the essential data used across an entire organization. Build in layers: a subject area overview, conceptual views of entities and relationship for each subject are, and more detailed, partially attributed views of these same subject areas. 6. Information value-chain analysis define the critical relationship between data, process, roles and organization, and other enterprise elements. 7. Data delivery architecture defines the master blueprint for how data flows across databases and application. Ensuring data quality and integrity to support transactional Business process and BI reporting and analysis 8. Architectural framework like “TOGAF, The Zachman” help organize collective thinking about architecture.
- 44. 4.3.3 Organizational andCultural issues Q1: Are there any ramifications to implementing an enterprise data architecture? A1: many ramifications to organization such as: • Discover of redundant systems and processes , which need to changes roles and responsibilities of some organization teams and individuals with degree removing fears of people for losing their jobs. • Change of organization’s culture. Application-centric IT shops should become more data-aware. And attract to their product through the IT more knowledgeable about business needs and practices, rather than just a service provider.