For many years, a very large Enterprise Data Warehouse (EDW) has been at the heart of the Enterprise Architecture (EA), mostly driven by the Information Technology (IT) engineering team. Many of these EDW projects were expensive and often took years to put in place by a highly specialized IT team (often outsourced consultants). Although many of these EDWs evolved through extensions, most of the tools surrounding this traditional EDW rely upon and leverage the stability of the architecture. Though complex, the Extract Transform and Load (ETL) jobs were generally stable, and the Business Intelligence (BI) design paradigm is based upon a DW schema driven reusable business layer (e.g. Business Objects universes), on which thousands of canned reports have been custom build by IT.

Big data initiatives and on demand data are changing this , providing direct access to the business users who now need self-service BI technology to analyze the new deluge of data. Even Data Integration (DI) technology is evolving from the more traditional ETL to a new generation of big data or Extract Load and Transform (ELT) driven DI solutions. Furthermore, the new generation of DI and BI tools are usually more affordable than their predecessors. This new business model and associated technologies are often not replacing the traditional EDW, but rather complementing the EDW initiative, or even integrating with it.

With all that going on, we have Metadata Management (MM) and Data Governance (DG) initiatives, which have played a key role in managing and analyzing the classic EDW based Enterprise Architecture (EA) through more and more powerful tools providing, for example, resuse, data lineage and impact analysis. MM is even more critical in the new agile Enterprise Architectures. Indeed, powerful data lineage and impact analysis are even more critical yet in a multi-vendor and multi-technology Data Integration (big data, ETL, ELT) environment. Self-service BI on big data or on demand data requires that we provide even more metadata (Business Glossary, Data Models) to the business users to understand, and often to design BI reports.

In this session we will present this evolving “Big Picture” in a comprehensive and very understandable manner, highlighting the critical importance of metadata management and data governance.