Data Warehouse concepts and data models
Each of us needs information in our work. Information is acquired, studied, asked for, and collected. In organizations, this often means many different source systems where the required information resides. Scattered information needs to be managed. DSharp’s model-driven approach highlights not only the information needs but also its gaps.
Just ten years ago, building a data warehouse had a reputation for being labor-intensive, slow, and outdated by the time it was completed. At the same time, the increasingly popular Data Vault method changed the entire way of building data warehouses, but it also brought new challenges. The Data Vault method is based on breaking data into smaller parts, and it is infinitely and flexibly expandable without needing to change what has already been built. One of its challenging aspects is the specific expertise required due to the complex structure and implementation method, as well as the increased workload in both building the solution and accessing the data.
The traditional solution for accelerating difficult or slow processes is automation, and building a Data Vault solution certainly benefits from automation.
DSharp’s solution for better information management and data-driven leadership is DSharp Studio, a low-code tool for building data warehouses. DSharp Studio approaches the task through conceptual modeling: in conceptual modeling, the key phenomena of the business are modeled based on needs, and the relationships between them are defined. The resulting conceptual model is completely independent of the source systems. It does not replicate the structure of any specific source system, and the terminology used directly reflects the business operations. For the modeler, Data Vault is conspicuously absent.
However, DSharp Studio still leverages the flexibility and change resilience of Data Vault. In the background, based on the conceptual model, the tool automatically creates a fully functional Data Vault solution. To implement the model, it is enough to map the data from various source systems to the content of the conceptual model, and the tool takes care of the rest. This way, data from different source systems integrate seamlessly into one cohesive whole without any additional efforts. Also, the data scattered in the Data Vault structures is published through a layer that completely mirrors the original conceptual model. So in short, for anyone using the data, the data looks exactly like the conceptual model. For most purposes, there is no need for anyone to directly access, or even know about, the more complex Data Vault layer, but it is there if it’s needed.
We asked DSharp Studio’s creator Kim Johnsson how DSharp’s tools are changing the everyday work of data professionals:
“For us, technology is just a tool for acquiring the necessary information, but our focus is on the content, not the technology itself. It doesn’t really matter where the data lives or how it is transferred, that’s just automatable logistics. The most important thing is that the data serves the organization in a business-driven way, and conceptual modeling helps find a common language for the issues. It makes sense to use the same terms when talking about the same things.”
Not only does automation greatly benefit end users by giving them the data and reports that they need much quicker, it does so by eliminating entire categories of manual work for the developer. When time and nerves are no longer spent struggling with cumbersome technology, bandwidth unexpectedly frees up for other things: thinking.
“The Data Vault approach to doing things is future-proof, meaning it can be expanded indefinitely. However, it is both complex and labor-intensive. Our DSharp Studio tool solves this problem cost-effectively by automatically creating the entire Data Vault solution directly from the conceptual model, completely eliminating the risks of traditional development work in terms of scheduling, workload, and costs.”
From Decentralized Data to a Comprehensive Overview
Beauty may be fleeting, but a well-designed conceptual model is eternal. It lives, grows, and expands according to the organization’s needs. The number of source systems is unlimited, the platform used doesn’t matter, and the number of concepts can always be increased and integrated into the existing model. Then just update the Data Vault. This is not only cost-effective but also smart business development.
Model-driven thinking shifts the focus from technology to content. It requires data users across different teams to think together about what information is handled in the organization, how it is described and used, and what additional information is needed. Johnsson believes that in the future, organizations with smarter data-driven leadership will stand out from the rest:
“A conceptual model can include things that we know are not available in any system. The traditional data-driven approach does not support this as clearly. The fact that something exists in the conceptual model demonstrates the need for that information. When the company makes new system acquisitions or changes, the conceptual model can be checked to see what data the business still needs.”
In the best case, new information is created by creatively combining existing data, which provides comparable and reliable information for business development, as well as new perspectives. For example, financial figures can be analyzed in relation to numbers from the personnel system, helping to identify future areas for development.
“To gain a comprehensive view of the business, information from all available source systems is needed—everything the business has. The conceptual model identifies the key phenomena of the business, names them, links them together, and brings the data into one place. The conceptual model always starts from a real need that is modeled. The conceptual model itself is the goal, and it’s fantastic when it is realized.”
