Gartner’s
Market Guide to Data Virtualization (DV) that was published a few months ago
was really a “coming of age” milestone for that relatively unknown data
integration pattern. With the data explosion on all fronts, the traditional
tools and patterns such as ETL, EAI, ESB, or Data Warehouse are mostly
obsolete. To download and read full Gartner
Market Guide for Data Virtualization click here.
Unfortunately, it looks like we’re entering another déjà vu scene, where
the next "best way" to handle integration problems is hyped as one more
stand-alone category of integration. Remember how we had to decide, before
initiating a new project, whether the problem required ETL, ESB, or SOA? Bear
in mind that it was never that cut-and-dry; every project needed a little of
each, so you just picked one. Then you realized you had to have three different
tools and vendors, not to mention plenty of custom coding and timelines,
counted in years, to get to the desired end, if at all. In my experience, no architecture can rely
solely on a single integration pattern. Most DV tools focus exclusively on Data
Virtualization. There may be a vendor that offers tools in each category, but
those are typically separate tools that don’t share objects and functionality.
Stone Bond Technologies has always considered integration as a
continuum. There is a huge body of capabilities that are necessary for every single
pattern. You always have to access all
manner of disparate data sources; you always have to align them to make sense
of them together; you always need to apply business rules and validations. You
need to make sure the formats and units of measure are aligned … and on and on.
Then you need data workflow, notifications, and events. You need security
at every turn. That’s where Enterprise Enabler started – as a technological foundation that handles these requirements without staging the data anywhere, and that
virtually eliminates programming. With that, delivering as DV, ETL, EAI, ESB, or
SOAP is not so difficult. Most integration software, on the other hand, starts
with a particular pattern and ends up adding tools or custom coding to figure
out "The Hard Part."
It turns out that Data Virtualization demands that multiple disparate
data sources be logically aligned in such a way that together they comprise a
virtual data model that can be queried directly back to the sources.
I like the diagram that Gartner included in the Guide (To view Gartner's
diagram and read the full Market guide, click here). Below is a similar
image depicting Stone Bond’s Enterprise Enabler® (EE) integration
platform in particular. Note, the single agile Integrated Development
Environment (IDE) covers all integration patterns, and is 100% metadata driven.
The only time data is stored is when it is cached temporarily for performance
or for time-slice persistence.
Enterprise Enabler®
Refer to the above diagram for a few additional
things you should know about Enterprise Enabler:
- As you can see, all arrows depicting data flow are bi-directional in this diagram. EE federates across any disparate sources, and also can write back to those sources with end-user awareness and security.
- IoT is also included as part of the source list. Anything that emits a discernible signal can be a source or destination
- AppComms™ are Stone Bond’s proprietary connectivity layer. An AppComm knows how to intimately communicate with a particular class of sources (e.g., SAP, Salesforce, DB2, XML, and hundreds of others) including leveraging application-specific features. It also knows how to take instructions from the Transformation Engine as it orchestrates the federation of data lives from the sources.
- The Transformation engine manages the resolution of relationships across sources and the validation and business rules.
- EE auto-generates and hosts the DV services
- Data Virtualizations and associated logic can be re-used as Agile ETL with a couple of clicks. Agile ETL leverages the federation capabilities of DV without staging any data.
- EE includes a full data workflow engine for use with Agile ETL or seamlessly inserted as part of the overall DV requirements.
- EE has a Self-Serve Portal which allows BI users to find and query appropriate virtual data models
- EE monitors endpoints for schema changes at touch-points where data is used in any of the DV services or Agile ETL. You’ll be immediately notified with detailed import analysis. (patented Integration Integrity Manager)