The D♯ Command set consists of all commands available in the menu structure, toolbars (buttons) and popup menus.
The command structure is highly dynamic. The placement of the command in the menu structure, toolbars and popup menus is defined in the command itself. The source code for the command exists in a file in the Workspace/Scripting/DSharp/UI/Commands-subdirectory of the D# Engine installation, and it is always recompiled at runtime. The placement of the command may vary a bit between versions, and it may not always be perfectly in sync with this page due to occasional restructuring. To find the current location of the command in the menu structure at any moment, please run the About.Command reference command in D# Engine.
Commands may at any time be added by the user to perform any imaginable functionality, which also adds to the dynamic nature of the command structure.
File
Open Visual Paradigm Model
Opens a file dialog for importing the contents of a Visual Paradigm Model.
The following actions are performed:
- the contents of the Visual Paradigm export file (project.xml) is read and the internal SmartEngine Object Model is built from it
- each mapping file located in the same directory as or a subdirectory of the export file will be read and processed
- Staging Area schemas and tables are generated from the mapping rows
- Columns are mapped to Attributes and Roles in the Object Model
- The imported model as well as the mappings are analyzed. If modeling or mapping errors that affect the correct creation of the DW solution are found, they are listed in a text window. In this case the DW objects will not be generated. If no errors are found, the DW objects will be generated.
- The model is searched for unmapped properties. These are not considered as errors, but are reported in a separate text window. Deploying objects with unmapped properties will result in a deployment error.
Open Ellie Model
Opens a file dialog for opening export files containing an Ellie model.
The following actions are performed:
- the contents of the the Ellie export files (project.xml) are read and the internal SmartEngine Object Model is built from it. The files are:
- ellie-report-MODELNAME-attributes: contains attribute details
- ellie-report-MODELNAME-entity-metadata: contains class/entity details
- ellie-report-MODELNAME-relations: contains relation metadata
- each mapping file located in the same directory as or a subdirectory of the export files will be read and processed
- Staging Area schemas and tables are generated from the mapping rows
- Columns are mapped to Attributes and Roles in the Object Model
- The imported model as well as the mappings are analyzed. If modeling or mapping errors that affect the correct creation of the DW solution are found, they are listed in a text window. In this case the DW objects will not be generated. If no errors are found, the DW objects will be generated.
- The model is searched for unmapped properties. These are not considered as errors, but are reported in a separate text window. Deploying objects with unmapped properties will result in a deployment error.
New Mapping File
Asks for the source name and creates a new mapping file using the next available sequence number as a prefix for the file name. Opens the empty file in the Mapping Editor.
Open Mapping
Contains each mapping file as a submenu element. Clicking on the mapping file name opens the file in the Mapping Editor.
Refresh Project
Closes and reinitializes the application:
- all scripts will be recompiled
- if a model was open when the command was run, it will be reopened
- if any export files have been changed after the previous app execution or refresh, SmartEngine will reflect these changes
Exit
Closes the SmartEngine application.
View
Console
Toggles the Console visibility.
Log
Toggles the Log visibility.
Conceptual Model
Toggles the Conceptual Model tool window showing the Conceptual Model hierarchy.
Class Implementation
Show a tree view with alphabetic list of classes, grouped to Implemented and Non-implemented classes
- For Raw Model classes, shows
- Staging Area objects (tables)
- Hashing objects (work tables and corresponding hashing procedure)
- Raw Vault objects (tables and corresponding load procedures)
- Business Vault objects (class view/table with corresponding materialization procedure)
- For Business Model classes, shows
- Business Vault objects (generated tables as well as class view/table with corresponding materialization procedure)
Primary use is to quickly find a class and its implementation. Use together with the SQL pane to verify the implementation.
Server Model
Toggles the Server Model tool window showing the generated database server structure.
Servers
Opens the Servers hierarchy, which allows you to browse the contents of any database server that has been configured for use with D♯ Engine.
Languages
Toggles the Languages tool window showing the datatypes that are available for attributes as well as platform specific transformations.
Properties
Toggles the Properties tool, which shows details of the selected object.
Class mappings
Toggles the Class Mappings window. Shows the current mappings of the selected class.
SQL Code
Toggles the SQL viewer, which displays the SQL code of the extended selection.
Data Grid
Toggles the Data Grid tool window. The command is available if there are configured database servers to connect to.
Diagram
Related Classes
Opens a Diagram for the selected class which shows the selected class as well as all other classes that are connected to it through an association.
Table Structure
Opens an ER diagram showing the table structure of the selected classes implemented in the Raw Vault or Business Vault.
Data Flow
Opens an ER diagram showing the full incoming and outgoing data flows for the selected class or table/view.
Incoming Data Flow
Opens an ER diagram showing the full incoming data flows for the selected class or table/view.
Outgoing Data Flow
Opens an ER diagram showing the full outgoing data flows for the selected class or table/view.
Open in Default Diagram
Opens the elements in the extended selection in a diagram appropriate for the the type of the selection.
SQL
Conversion Errors
Creates SQL queries that identify conversion errors for each class or table in the extended selection. A conversion error is identified when a data column contains a null value but the corresponding _orig column contains a non-null value. This situation has arisen when the conversion of the value in the _orig column has failed to produce a valid value of the data type of the target data column.
Referential Integrity
Creates SQL queries that identify referencing hash values that are missing from the referenced table. These are looked for in:
- satellites referencing hubs
- links referencing hubs
- PITs referencing hubs / links
Invalid Time Periods
Classes that have attribute pairs of data type startdate and enddate are considered to be time periods. The command generates SQL queries for each class in the extended selection that is identified as a time period. The query will return all rows where the startdate value is larger than the enddate value, ie where the time period’s dates are illogical.
Duplicate Hash Values
The command generates SQL queries for identifying rows with identical hashes. The hash column being queried is the “main” hash column of the table, so the usability of the result is dependent on what you are looking for. For example, querying a satellite for duplicate hashes returns technically valid rows, as the hash column is not the only primary key column. So for the satellite you get rows that have “changed” more than once, whereas for a work_ table you get rows that will cause problems in the load process going forward. And, obviously, for a hub and a link, you never get any rows, as the hash column is unique by definition.
Query Tables
Creates select statements for all the tables implementing the classes in the extended selection.
Query Expanded Tables
For the classes in the extended selection, creates select statements that also apply each outgoing join and this way includes “neighbouring” classes in the query.
Query View
Creates select statements for the class views implementing the classes in the extended selection.
Search Tables for Hash Value
Asks the user to input a hash value, then generates select statements to find that hash value in all the tables and views for the classes in the extended selection. The submitted hash value is compared to the primary key hash column of these objects.
Select Max Column Lengths
creates SQL code that extracts the maximum string length of actual data in all columns of all tables of all classes/tables in the extended selection.
Trim to Distinct
Generates SQL statements to trim the contents of the selected table so that only distinct rows remain.
ETL
Hash
Creates a SQL script that performs the hashing of the classes in the extended selection.
Load
Creates a SQL script that performs the loading of the classes in the extended selection. Additionally, these post-load operations are performed:
- state updates
- PIT updates
- materialization
Hash And Load
Performs the commands Hash and Load sequentially. The content of the DW is updated with the data available in the staging area.
Reload Last Failed Rows
Creates a SQL script that
- copies the latest rows from the target error tables to their corresponding work tables
- runs the hash procedure for those work tables
- runs the corresponding load procedures and post-load routines
Use this when you have identified the correct rows among all rows moved to the error table and removed the incorrect ones. The command will result in the correct rows being loaded in to the DW, after having failed to do so in a previous load attempt.
Batch Load History Data
Generates a SQL template for loading versioned data in loops. Depending on the mappings of the source data, you may or may not need to manually specify the version column name used to sort the batches by.
Materialize
Will execute the materialization procedure for the classes in the extended selection that have materialization activated.
Set Hash Duplicate Handling On
Creates a SQL procedure call which sets the parameter value for the Drop duplicate hash rows to true for the classes in the extended selection. Executing the procedure call will activate the error handling procedure for the relevant work tables resulting in all rows containing duplicate hashes being moved to the corresponding error table after hashing has been completed. This is done in order not to crash any subsequent insert in the loading stage.
Set Hash Duplicate Handling Off
Creates a SQL procedure call which sets the parameter value for the Drop duplicate hash rows to false for the classes in the extended selection. Executing the procedure call will deactivate the error handling procedure for the relevant work tables. If the result of the hashing leads to duplicate hash values, the subsequent loads will crash due to a primary key error. This may or may not be what you want.
Set Partial Load Deletion On
Creates a SQL procedure call which sets the parameter value for the Partial load deletion detection to true for the classes in the extended selection. Executing the procedure call will activate the deletion detection algorithm that marks hashes not detected in the incoming data as deleted using a state table designed for this purpose. The algorithm checks the work tables for the current load and “turns off” hashes that are not present in the incoming data from any sources that they have been loaded from previously. This way, not all sources need to have been loaded in the same batch, and the algorithm can still safely marks some hashes as deleted.
Set Partial Load Deletion Off
Creates a SQL procedure call which sets the parameter value for the Partial load deletion detection to false for the classes in the extended selection. Executing the procedure call will deactivate the deletion detection algorithm that marks hashes not detected in the incoming data as deleted based on the sources they have previously been loaded from. Now hashes will be deactivated only if the current batch contains data loaded from all sources for the class.
Set Soft Delete On
Creates a SQL procedure call which sets the parameter value for the Soft delete to true for the classes in the extended selection. Executing the procedure call will activate the soft delete rule for the class view, meaning that the view will hide all rows that have been marked as deleted in their corresponding state satellite. This command is applicable only for classes with the meta parameter State tracking method value set to Simple. The view with the parameter handling must also have been deployed.
Note: for a materialized view the content changes are not immediate; the materialization has to be re-executed.
Set Soft Delete Off
Creates a SQL procedure call which sets the parameter value for the Soft delete to false for the classes in the extended selection. Executing the procedure call will deactivate the soft delete rule for the class view, leading the view to show the data for all hashes that exist in the main hub / link, regardless of the contents in their state table.
Note: for a materialized view the content changes are not immediate; the materialization has to be re-executed.
Set Query For Determining Deleted State
Creates a SQL procedure call which sets the algorithm for the Ignore when setting deleted state deletion detection mechanising for the classes in the extended selection. Executing the procedure call defines which hashes will not be set as deleted when determining the deleted state.
Clear Query For Determining Deleted State
Creates a SQL procedure call which deletes the algorithm for the Ignore when setting deleted state parameter for the classes in the extended selection. After clearing this algorithm, for the deletion state updates to work correctly, the loads of this class must be full loads, enabling loads to be fully incremental but still having the possibility to identify hashes that should be marked as deleted.
Install
Full DW Install
Creates SQL code for installing the database objects needed for a DW implementation, plus all objects generated from the model.
Create Empty DW
Creates an empty DW with logging mechanisms and other relevant base functionality, but no model-based structures or ETL mechanisms.
Create Installation Package
Creates an incremental deployment structure for all classes, table and procedures in the extended selection.
The resulting SQL code will
- create new tables
- add new columns to existing tables
- alter existing columns (sizes, datatypes)
- drop all indexes and recreate model-based indexes
- create or update procedures and views
It will not delete any existing table columns, tables, procedures or views.
Create System Specific Installation Package
Creates an incremental deployment structure for all classes, tables and procedures in the extended selection for a specific source system.
It will create the same objects as the full incremental deployment package, but only the ones that are relevant for the selected source system:
- Staging area and hash procedures for all the tables in the source system’s schema
- The corresponding satellites and their load procedures
- hub and link processing for the selected source system only
- Views, PITs, states, orchestration procedure and other common elements updated
Fix Hashes
Creates a skeleton SQL solution for handling the situation where existing hashes need to be recalculated, for example due to a change in the business key structure.
Install Ghost Row
Creates an SQL insert clause that adds a ghost row to the tables in the extended selection (classes or tables). This is useful in case a table has been truncated.
Clear Tables
Creates SQL code for clearing the tables for the class/tables in the extended selection.
Drop Tables
Creates SQL code for dropping the tables for the class/tables in the extended selection.
De-installation Candidates
Generates a list of database objects that may no longer be needed, based on comparing the contents of the current contents SmartEngine object model and the information_schema tables. That is, the list of database objects that SmartEngine currently generates is compared to the actual contents of the database.
Objects listed as potentially deletable can be deleted manually, but only after careful consideration. Some objects may could considered to be “archived”, which SmartEngine would not know about.
Delete Last Batch
Runs a query that determines the last used load date among the classes in the extended selection, then generates delete statements that delete any rows from the tables having that load date. Effectively, this “undoes” the last load for the selected classes.
Create Full HTML Documentation
Creates a full set of HTML documentation containing the conceptual model and all generated ER models. The HTML documents and linked images are saved in the Workspace/Output/$ModelName$ directory.
Develop
Class As CSV
Generates a list of semi-colon separated values for all classes in the extended selection, containing the class name, property name and property type (either Attribute or Reference. The output is printed as text in the Console window, and can, for example, be used to populate the mapping file’s class details instead of writing them by hand.
Create Same-As Insert
Find Unmapped Properties
Generates a list of all properties (class attributes and small cardinality associations) that do not have a mapping to any source data. For classes in the Raw model, these mappings should be defined in mapping files. For classes in the Business model, the missing mappings should be defined in class-to-class mappings in the modeling tool.
Build Data Vault
Restarts the DW build process if it has been halted due to validation error. Before building, all classes that have failed the validation will be excluded from the build.
Import Mappings
When database tables are selected in the Servers view, use this command to import the table metadata into a mapping file.
Compare Model With DW
Generates a SQL query which compares the current model content with the content of the database where the query is run. The query uses the information_schema tables to find the differences, and list elements missing from the database that exist in the model and vice versa.
Scripting
Clear Console
Clears the Console pane.
Debug to App.Message
Sets the target for debug messages. In scripts, the call ScriptSettings.Debug(pMessage) will result in the text pMessage being shown in a message box.
Debug to Console
Sets the target for debug messages. In scripts, the call ScriptSettings.Debug(pMessage) will result in the text pMessage to be printed to the Console.
Debug to Log
Sets the target for debug messages. In scripts, the call ScriptSettings.Debug(pMessage) will result in the text pMessage to be written to the Log.
Debug Off
Sets the target for debug messages. In scripts, the call ScriptSettings.Debug(pMessage) will result in no debug message showing up anywhere. This is the default mode.
Export Classes to Visual Paradigm
An experimental feature. You can export the classes in the extended selection to a Visual Paradigm xml file, which can then be imported to VP for finalizing.
Use this in situations where you have converted an existing table structure to a class structure using the Generate Classes command. The generated mapping information is also exported, and it will be visible in Visual Paradigm after import.
Create Mappings
Experimental Feature. Outputs to the Console a string that can be pasted into the mapping file. This command can be used
- after having generated new classes, and therefore also mappings, using the Generate Classes command, or
- after having opened a Visual Paradigm model that contains classes first imported to Visual Paradigm using the Export to Visual Paradigm command (containing generated mappings in a tagged value)
Generate Classes
Experimental Feature. Operates on table elements. The command creates a class structure that is identical to the table structure in the selection. Tables will be converted to classes, columns to attributes and foreign key references to associations. Rudimentary mapping information will be created and stored as a Tagged Value for the generated class.
Settings
Application Settings
Opens the Application Settings window.
Set Mode: Clipboard
Sets the SQL Mode to Clipboard. All generated SQL code will be placed on the Clipboard. The SQL code will not be visually displayed anywhere, but a message confirming the clipboard operation is displayed in the Console.
Set Mode: Run File
Sets the SQL Mode to Run File. All generated SQL code will be saved as a file using an -sql file extension, after which the file is executed. It depends on the system settings which program is used to execute the file. You may want to define the best suited program for you to handle the .sql extension.
Help
About D♯ Engine
Opens the About Box showing version and license details.
Command Reference
Generates a real-time list of commands registered in SmartEngine and displays them as an HTML document, showing the command name, a short description as well as the location of the command in the menu structure. The content is extracted from the commands themselves.
Error Log
Opens the newest error log file that SmartEngine has created. Can be used after a crash for debugging purposes.
Release Notes
Opens the Release Notes page on this site.
Upgrades
Higher level menu item. Contains commands to be run after upgrading SmartEngine to the latest version. The commands analyze the current model and identify possible actions to be performed is conjunction with the version upgrade.
Tool Support
Higher level menu item. Contains commands for opening relevant modeling tool resources on this site,
Demo
Higher level menu item. Contains commands to be run when demoing SmartEngine.
Tutorials
Higher level menu item. Contains commands to be run when preparing for and working with the tutorials.