The Virtual Cycle: Bi-Directional Data Virtualization

With all the buzz about data virtualization (DV), it surprises me how little bi-directional data virtualization is discussed.  Without the ability to write back to sources, the use of DV is limited to Business Intelligence, and other reporting.  Of course, it’s hugely important for that, but when you add write-back to the sources, you are opening up a whole new world of possibilities for a new dimension of interaction with data. 

Suddenly all those dashboards become consoles where business operations can be performed, with end users not just viewing data, but correcting it, updating it, and taking action on decisions. Any application can leverage bi-directional DV to access federated data and to write back to the sources without having to know where it came from, treating it as a single entity. This capability goes a long way to reducing the time to value of many IT initiatives beyond reporting and analytics.

For those skeptics who are not already familiar with bi-directional data virtualization, the first questions are typically, “How do you handle the security to make sure users only write back when they have permission to do so?” and “What happens if  there’s a failure writing back to one of multiple sources?” 

The short answers are that end user security is handled for full CRUD capabilities using SSS or other models, and transaction rollback is managed using two phase commit or other modes.

Now, we can move on to the cool things you can do with this.  Suppose your training is frustrating and time consuming for new employees to learn how to navigate and use multiple systems that are necessary for them to handle their responsibilities. They need to log in to SAP, then the CRM, and then a scheduling system, plus a spreadsheet, all just to perform one task. You could build a browser based app that presents the relevant data from all these systems in one screen, aligned and meaningfully presented. This is the standard data virtualization, which is essentially a reporting tool. Now, turn on write capability for appropriate fields, and voila! That browser screen is a full-service, role-based application, interacting directly with backend systems and data stores just as if you were logged in to all of them. This, my friends, is the virtuous virtual cycle of bi-directional data virtualization.

Using an Agile Integration Software like Enterprise EnablerÒ you can leverage all the federated data services not just for BI, but also for Business Operations. These light weight rapidly deployed nuggets enable this third generation of data virtualization to make agile business a reality.

Agile Integration: Foundation for All That Hype

By definition, Agile Integration Software (AIS) is charged with accommodating all data sources, standards, etc. and moving data agilely throughout the enterprise, adjusting to changes over time. As it does, it captures information about all of the participating endpoints and modalities of data movement. The more an AIS like Enterprise EnablerÒ is used, the more it learns and the more metadata it maintains about the information that is involved in the company’s activities.  Perhaps it’s time to think about AIS as a central core of actionable metadata and a useful engine that can be leveraged for use in new initiatives as they are defined, as opposed to continuing to use AIS to adapt to whatever the new initiative independently demands.  

Consider Master Data Management (MDM) for example. Like all other hype waves, the need is there, and the initiative is fueled by the analysts in concert with a surge of new tech companies as well as the Big Players.  Your company decides it better get on the bandwagon; MDM is imperative to maintaining competitive advantage.  One of the top project architects is charged with establishing MDM, so he studies, attends symposia, and learns what it’s all about.

Next comes a technology selection phase, looking at the emerging companies, but mostly looking at the Big Guys, since that’s always safe, and besides the architect already knows the “Rep” really well. It seems the Big Guy has just bought one of the up-and-coming MDM companies, so they are off to the races.  After a couple of years, the architects begin contemplating questions like, “How does this MDM solution relate to the last decade’s SOA path?”   Of course, at this point it’s obvious that SOA is pretty closely related, or could be, had the two initiatives not each been addressed in its own blissful vacuum.  How does the MDM metadata relate to SOA, and how does it relate to the metadata that your multiple integration platforms require?  Some, with wishful thinking, fall for the Big Guys’ claims of interoperability across all the products of the companies they bought.  In the Quadrant or Wave, they cover all the requisite features, but Alas, poor Yorick! Those mighty features are compartmentalized and each discipline is a separate product with separate underpinnings unable to work cleanly together.  

Let’s look at this from a different perspective. With Agile Integration Software comes a complete flexible integrated metadata stack for use and reusability across all the historic and forward-looking integration schema and models.  Instead of integration adapting to the stand-alone fragmented hype solutions, leverage the power of an existing AIS platform  that brings together the disciplines of Data and Application Integration, Application Development, as well as all the special initiatives of SOA, MDM, B2B, Middleware, Virtualization, Federation, Cloud, Change Management and Big Data, all leveraging the AIS integrated metadata stack.

That means eliminating lots of steps to accomplish any task.  With visibility via the metadata stack, a Master Data definition can be combined with all the associated sanctioned sources and all the related business rules and security. Auto-generated bi-directional web services can handle security and rollback to federated sources. SOA is just another mechanism to make an integration or process available for consumption. Different Data Masters and integrations are chosen at run time based on the current state of any other activity throughout the system.

And finally, AIS monitors for change throughout the metadata stack, validating against the actual endpoints and determining the potential impact and remediating and reconciling as necessary across all those hype wave initiatives. This means stability with agility in your IT infrastructure.