Even Beyond the Logical Data Warehouse

What is a Logical Data Warehouse? There is still much uncertainty and ambiguity around this subject. Or, perhaps I should say, there should be.

Instead of trying to lock down a definition, let’s take advantage of the opportunity to think about what it CAN be. It is the role, if not the obligation, of Experts to describe the essence of any new discipline. However, in the case of LDW, a premature assessment is likely to sell short the potential reach and extensibility of the contribution of Data Virtualization (DV) and Federation to the entire universe of data and application integration and management.

Certainly, the players with the biggest marketing budgets are likely to spread a limited, but compelling, definition and set of case studies, which could become the de facto discipline of Logical Data Warehouse. While these definitions may represent a significant step forward for data management, they would be limiting the full potential of what these new models could bring to the marketplace.

I fear, however, a repeat of the biggest historical impediment to realizing a universal data management framework. Each new “wave” of innovation has been blindly adopted and touted as the single best approach ever. ETL only went so far, then EAI came along as a separate technology (to save the world), Data Warehouse (to store the world) then SOA (to serve the world), and now Data Virtualization and Logical DataWarehouses (to access data faster and with more agility). In this case of Data Virtualization and Logical Data Warehouse, we owe it to our fellow technology implementers to leverage every aspect possible, to advance the cause of the ultimate data integration and management platform.

If we look at all of the data integration patterns, don’t we see that there is a tremendous amount of functionality that overlaps all of these patterns? Why do we even have these distinctions?

What if we seize this DV/LDW revolution as the opportunity to reinvent how we think about data integration and management altogether? Consider the possibility of a platform where:

LDW is a collection of managed virtual models 

  

• These can be queried as needed by authorized users.
• The same logic of each virtual model is reusable for physical data movement
• Virtual data models incorporate data validation and business logic
• Staging of data is eliminated except caching for performance
• Virtual data models federate data live for ETL
• Virtual data models and accompanying logic can be designated, or “sanctioned” as Master Data definitions
• Master Data Management eliminates the need for maintaining copies of the data
• Golden Records are auto-updated, and in many cases, become unnecessary
• With the “write-back” capabilities, data can be updated or corrected in either end user applications/dashboards or by executing embedded logic
• Write-back capabilities mean that anytime a source is updated, all of the relevant sources can be synchronized immediately also. (Imagine that eventually, the sync  process as we know it today simply disappears.)
• Complex data workflows allow the use of virtual models and in-process logic to be incorporated into the LDW definitions.
• These logic workflows handle preventive and predictive analytics as well as application and process logic
• Data Lineage is easily traced based on traversing the metadata that describes each virtual model. 
• Every possible source: applications, databases, instruments, IoT, Big Data, live streaming data, all play seamlessly together.

         Oh, and LDW is pretty cool for preparing data for BI/BA also!

We at Stone Bond Technologies have been leaders in Data Federation and Virtualization for more than ten years. We believe it is our responsibility to remove all obstacles, allow data to flow freely, but securely wherever and whenever it is needed. Our vision has always been a single, intimately connected, organic platform with pervasive knowledge of all of the data flowing throughout the organization, whether cloud, on premise, or cross-business; applications, databases, data lakes.. any information anywhere.

Being too quick, individually or collectively, to take a stand on the definition of Logical Data Warehouse is likely to abort the thought process that is still ripe with the opportunity to take it way beyond the benefits that are commonly extolled today.