Overview
Business-oriented Conceptual modeling is a fundamental part of D♯ Methodology. The model elements are complemented with appropriate metadata (including source system mappings), after which the D♯ Engine tool can generate the entire technical implementation of the model, that is, the Data Warehouse. Not quite magic, but rather fully transparent deterministic automation based on a carefully selected minimal set of complementary information superimposed on the model. For all practical purposes, it’s about achieving more by doing less, in as many situations as possible.
Conceptual Modeling as part of D# Toolbox
Modeling is done with a dedicated modeling tool such as Visual Paradigm (UML Object Model modeling style) or Ellie (Entity-Relationship modeling style). In the picture above, a Conceptual model has been created using Ellie and then imported to D♯ Engine. The modeling tools also support the additional metadata needed to automatically generate the Data Vault.
D# Engine Generates all Data Vault Structures
Fed with the Conceptual model and source data mappings, D♯ Engine generates a Data Vault 2.0 structure that can be populated by the source data. Stored procedures are generated that perform hashing, loading and other operations that are needed to keep the data in order. Additionally, the (possibly multisource) data is automatically collected, merged and presented through a set of views built on top of the Data Vault structure. Each view perfectly represents a class/entity in the Conceptual model, so the developer can navigate the view structure using the original Conceptual model as a map. So, for those who don’t specifically need it, the Data Vault structure is conveniently hidden behind the view structure.
D# Engine Deploys the DW to Production
All changes and additions are safely and incrementally installed. No manual editing is needed. After installation, data can be loaded into the DW. All code will run natively on the target platform, so the presence of D♯ Engine is not needed for this. A set of default source system specific orchestration procedures that execute the individual hash and load procedures are automatically provided by D♯ Engine, but you may also build your own using whatever technology suits your situation.
D# Engine Excels in Everyday DW Maintenance
While the deployed solution will operate standalone within its target platform, D♯ Engine can be used for daily maintenance and debugging tasks with its ad hoc SQL-generating features. The Conceptual model operates as the “user interface” to the underlying Data Vault table structure, so you can work with the original classes instead of the individual tables. In fact, you don’t even need to know about the actual table structure in order to get to the data.
