Overview

Business-oriented Conceptual modeling is a fundamental part of D♯. The model elements are complemented with appropriate metadata (including source system mappings), after which the D♯ SmartEngine tool can generate the entire technical implementation of the model, that is, the Data Warehouse. Not quite magic, but rather fully transparent deterministic automation based on a carefully selected minimal set of complementary information superimposed on the model. For all practical purposes, it’s about achieving more by doing less, in as many situations as possible.

Conceptual Modeling

Modeling is done with a dedicated modeling tool such as Visual Paradigm (UML Object Model modeling style) or Ellie (Entity-Relationship modeling style). In the picture above, a Conceptual model has been created using Ellie and then imported to D♯ SmartEngine. The modeling tools also support the additional metadata needed to automatically generate the Data Vault.

The Automation


Fed with the Conceptual model and source data mappings, D♯ SmartEngine generates a Data Vault 2.0 structure that can be populated by the source data. Stored procedures are generated that perform hashing, loading and other operations that are needed to keep the data in order. Additionally, the (possibly multisource) data is automatically collected, merged and presented through a set of views built on top of the Data Vault structure. Each view perfectly represents a class/entity in the Conceptual model, so the developer can navigate the view structure using the original Conceptual model as a map. So, for those who don’t specifically need it, the Data Vault structure is conveniently hidden behind the view structure.

Deployment

All changes and additions are safely and incrementally installed. No manual editing is needed. After installation, data can be loaded into the DW. All code will run natively on the target platform, so the presence of SmartEngine is not needed for this. A set of default source system specific orchestration procedures that execute the individual hash and load procedures are automatically provided by SmartEngine, but you may also build your own using whatever technology suits your situation.

Day-ta-Day Life

While the deployed solution will operate standalone within its target platform, D♯ SmartEngine can be used for daily maintenance and debugging tasks with its ad hoc SQL-generating features. The Conceptual model operates as the “user interface” to the underlying Data Vault table structure, so you can work with the original classes instead of the individual tables. In fact, you don’t even need to know about the actual table structure in order to get to the data.

Scripting

Nobody likes black boxes. The main principle of D♯ is maximal openness. The entire user interface as well as user commands are scriptable (and are indeed scripted) using the C♯ programming language, and all of the functionality can be tailored by the user to replace default functionality. SQL code generation is based on editable templates, and the templates are populated by editable C♯ code. It doesn’t get more flexible than this. This immense flexibility is also a testament to the power of desktop solutions.