Download as PDF, PPTX
Uploaded byManta Tools
Agile BI Development Through Automation
The document outlines various services offered by Data to Value, specializing in helping organizations maximize the value of their data through consulting and managed services. It emphasizes a lean approach to data management, promoting prototypes and iterative solutions, while also detailing the founders' extensive experience in data strategy and architecture. Additionally, it touches on the importance of data warehouse automation and effective communication between modelers and business professionals.
More Related Content
Similar to Agile BI Development Through Automation
Agile BI Development Through Automation
- 1. Tomas Kratky CEO Manta Tools NigelHiggs Co-founder Data To Value
- 2.
- 3.
- 4. We help organisationsget more value from their Data Architecture 5 Core practice areas covering both business and architecture aspects of data managment Service delivery through: Onsite consulting Onsite / Offsite Managed Services Expert users of technology accelerators for bridging technology & business Data gap
- 5. Lean Data Management Focuson reducing waste & minimising TCO Unification of unstructured information / knowledge management & structured data management Key mantra is minimising time spent on building solutions customers do not want Lean Information Managemen t Shorter iterations Prototyping & Minimum Viable Products Build- Measure- Learn cycle Early adopters Cross functional teams Actionable metrics
- 6. Unique Value Proposition Experience& expertise Founders have over 40 years experience working with data Skilled in defining data strategy and implementing data architecture, governance and analytics solutions Focussed on delivering business value Align data strategy with client’s strategic goals Work packages based on business case and ROI Lean, agile & iterative approach Hybrid consultancy model to scale to meet demand Partner with innovative vendors of data tools software
- 7. We have anumber of industry partnerships that allow us to hit the ground running We also use industry leading platforms such as AWS for hosting and Tableau for Data Visualisation Our focus is on providing customers with the most appropriate tooling to continue to make progress after initial projects have completed Tooling & Partners Data to Value industry partners & platforms:
- 8. Lean Approach –Iterative Process Maturity benchmarking Data Profiling & Data Discovery Harvest key metadata (apps, lineage, processes etc.) Test rules & capture metrics Generate risk & cost metrics Capture quality, governance & modelling notes Review issues using visualisations & dashboards Prototype data solutions Implement practical
- 9. Integrated Approach data quality& governance metrics data models, glossaries & dictionaries disparate data & metadata data profiling & metadata discovery ontologies controlled vocabularies
- 10. Typical Outputs DQ Issuelists & KPIs to guide decision making Powerful, interactive visualisations Models, Knowledge Graphs & Glossaries to understand what data assets you have Dashboards articulating the data quality issues that are holding you back Clean, actionable & well structured data in a variety of formats Ready to use Prototypes & POCs
- 11. Passionate, Innovative, Lean LeanInformation Management specialists Data to Value Ltd. 2nd Floor Elizabeth House, Waterloo, London SE1 7NQ United Kingdom T +44 (0) 208 278 7351 www.datatovalue.co.uk info@datatovalue.co.uk
- 25.
- 26.
- 27.
- 28.
- 30. Enterprise vs BusinessDriven BI Raju Sonawane
- 31. About Me TwentySix years’ experience in Business Intelligence, enterprise architecture, strategy/roadmap, design, development and project/people management. Roles played - Head of Business Intelligence, Data Architect, BI Architect, Solution Architect, Agile Scrum Master, Project Manager and onsite/offshore Business Development Manager Domain/Industry - Fund/Investment Management, Lloyds of London Specialist Insurance/ Reinsurance, Life Insurance and Consultancy The content in this presentation is my opinion/view of BI. My current/previous employers may have different views.
- 32. BI Maturity inmy view
- 33.
- 34.
- 35.
- 36. The side effect– Homegrown UDAs
- 37.
- 38. Ways to deliverit Strategy •Define the Enterprise BI Strategy •Set the BI products roadmap Enable the Platform •Open the platform to the users •Empower the users with self-service Rapid Prototypes •Promote “BI as a Service” •Identify the high value low size use cases •Build the rapid prototypes along with the users Leverage User-driven BI •Govern the user-driven BI •Leverage the popular user-driven BI to build Enterprise BI
- 39. Ways to deliverit Demo the existing BI capabilities Conduct Workshops for the new use cases Deliver the rapid prototypes with the users
- 40.
- 41.
- 42.
- 43.
- 44. Oliver Cramer The Modelis the Foundation for Data Warehouse Automation Agile BI Development Through Automation London
- 45. DWH42 Agenda • About me •Understanding • The gap • The model • Data Warehouse Automation
- 46. DWH42 About me • DataWarehouse Architect • 13 years working in the Business Intelligence area • Since 2003 working with elementary building blocks for the Data Warehouse • Blog www.dwh42.de Data Warehouse Automation • Interested in the exchange of knowledge about Core Data Warehouse modeling styles
- 47. DWH42 About me • TDWIEurope Fellow • ANCHOR CERTIFIED MODELER Version 2014 • Certified Data Vault 2.0 Practitioner • Coautor of „Neue Wege in der Datenmodellierung - Data Vault heißt die moderne Antwort“ in BI-Spektrum 03-2014 • Member of the Boulder BI Brain Trust • Member of the BI-Podium Advisory Board Germany • Responsible editor of the TDWI Germany Online Special „Data Vault“ • Organizer Data Vault Modeling and Certification, Hannover with Genesee Academy (CDVDM course)
- 48. DWH42 Agenda • About me •Understanding • The gap • The model • Data Warehouse Automation
- 49. DWH42 The maturity pathof understanding • Multiple perspectives on the facts = Data Warehouse as an enabler to make your own picture of the world from existing data and information! • One version of the facts = Data Warehouse as a recording device • One version of the truth = Data Warehouse delivers the truth
- 50. DWH42 Data: consistency vs.availability There is a fundamental choice to be made when data is to be 'processed': • a choice between consistency vs. availability or • a choice between work upstream vs. work downstream or • a choice between a sustainable (long term) view vs. an opportunistic (short term) view on data Ronald Damhof http://prudenza.typepad.com/dwh/2015/11/there-is-a-fundamental-choice-to-be-made-when-data-is-to-be-processed-a-choice-betweenconsistency-vs-availability-or-a.html
- 51. DWH42 The confusion solution LarsRönnback: "When working with information, confusion is sometimes unavoidable. To be more precise, when the process of identification cannot give unambiguous results, such confusion arises. ... Push that problem into the future, to solve it when you find the missing pieces, while still retaining analytic capabilities. Simply store all the possible outcomes in advance, with different reliabilities, or store the most likely scenario and correct it later if it was wrong. http://www.anchormodeling.com/?page_id=360
- 52. DWH42 Main model requirements •The model must be capable to absorb multiple perspectives on the facts! • The model must be capable of corrections!
- 53. DWH42 Our problem ->rendering knowledge Dave Snowden: 7 Principles of Knowledge Management / Rendering Knowledge: 1. Knowledge can only be volunteered, it cannot be conscripted. 2. We only know what we know when we need to know it. 3. In the context of real need few people will withhold their knowledge. 4. Everything is fragmented. 5. Tolerated failure imprints learning better than success. 6. The way we know things is not the way we report we know things. 7. We always know more than we can say, and we always say more than we can write down. http://cognitive-edge.com/blog/rendering-knowledge/
- 54. DWH42 Agenda • About me •Understanding • The gap • The model • Data Warehouse Automation
- 55. DWH42 The gap The biggap between modelers and business people is the language we speak! The modelers mantra: • We have to close the gap! • They will never close the gap! • They will not move in our direction!
- 56. DWH42 The logic • Weaspire to be logical modelers, to create the best logical model! • Are the business people logical? Are they like Spock from the Starship Enterprise? Are they from the planet Vulcan? • No, they are humans from the planet earth like we are!
- 57. DWH42 The advancement • Themodel must have a fully communication orientation (in this case business speech) (is that logical modeling?) • For this reason the model must support homonyms and synonyms! • A synonym is a word or phrase that means exactly or nearly the same as another word or phrase in the same language. • In linguistics, a homonym is one of a group of words that share the same pronunciation but have different meanings, whether spelled the same or not.
- 58. DWH42 Fully communication orientation ->Business model From Quipu: • This business model does not normally exist in any source system: it must be developed in close cooperation with the business to reflect the terms and definition of the data that the business chooses to work with. It identifies the business keys that identify the various business entities and their inter- relations. It also specifies all relevant attributes and facts related to these business entities that are required for management reporting, (predictive) analysis, etc. http://www.datawarehousemanagement.org/
- 59. DWH42 Agenda • About me •Understanding • The gap • The model • Data Warehouse Automation
- 60. DWH42 The model Model requirements: •It must have integration points. • It must support identification. • It must support relationships / Unit of Work relationships! • It must support dynamic relationships. • It must support storing attributes from different origins / integration of attributes is not necessary! • It must categorize attributes for identification. • It must be a model with historization capabilities. And history of history?
- 61. DWH42 The model Model requirements: •Support of data provenance! Data provenance refers to the ability to trace and verify the creation of data, how it has been used or moved among different databases, as well as altered throughout its lifecycle. • It must follow standards. • It must follow naming conventions. • It must follow patterns.
- 62. DWH42 The model Model requirements: •It must be scalable. • It must be readable and understandable! • It must be searchable. Crawler! • It must be partition able. • It must be extendable. • The possibility to model extensions without destruction of current entities! • It must be version able.
- 63. DWH42 The model Model requirements: •It must support the separation of concerns! • It must have a raw data area. • It must have a integration area. • It must have a rule area. • It must have a area for sensible data. • It must have a business area. • It must support temporal and business perspectives.
- 64. DWH42 Agenda • About me •Understanding • The gap • The model • Data Warehouse Automation
- 65. DWH42 Data Warehouse Automation Thebig picture All these details might make it hard to understand how this has anything to do with automation of Data Warehouse. • The only two steps that can’t get automated are the information modeling process and the semantic mapping exercise. • Is that statement subject to change? • Today the rest, before applying of rules, is the domain of data warehouse automation! • And what can be done ...!
- 66. DWH42 Data Warehouse Automation Baselineis that the model must separate keys/identifiers, relationships and attributes / group of attributes for Data Warehouse Automation. It must be fully communication oriented, so we can close the gap to the business. In the end we can focus on asking better questions. This is the next generation of Data Warehouse Automation.
- 67. DWH42 End Thanks for theattention!
- 69. • Models • BusinessGlossary • Relations • … Implementation • Marts • Reports • ETLs • … • Enhancement • Consolidation • Restructuralization
- 71.
- 75.
- 81.
- 82. Connected Data London2016 The leading conference bringing together the Linked and Graph Data communities 12th of July – Central London www.connected-data.london @Connected_Data
- 83. Connected Data London MeetUp Joinus at our informal MeetUp event. Listen to short talks delivered by Connected dData experts and share your ideas with like minded Connected Data fans! 7th of June – Central London http://www.meetup.com/Connected-Data-London/ @Connected_Data
- 84. Thank you forcoming! Please fill out our survey before leaving @DataToValue @Manta_tools https://www.linkedin.com/company/data-to-value-ltd https://www.linkedin.com/company/manta-tools