2 Keys to Identifying Agile Integration Software

For years I’ve been talking about Agile Integration Software (AIS) as a class of products that enable very flexible and agile data flows across the enterprise, eliminating the clunky, expensive, time consuming infrastructure of the past. This paradigm is agnostic to integration patterns and shares metadata for all uses, such as Data Virtualization (DV), ETL, EAI, SOA, etc.

Over these years, I have come to understand that AIS is not a class of products. As it turns out, Enterprise Enabler® is the only product that has all of the characteristics necessary to fulfill the AIS vision in scope and flexibility. So much for a product-agnostic concept-oriented blog! Many of the features can be found in other products, but somehow there is always at least one critical feature missing that negates the possibility of agility for the solution.

As I think about what constitutes agility in this space, a few things come to mind that are imperatives in such a platform. The two most telling indicators lead the list:

1. The product must have a transformation engine that aligns and transforms data a) from multiple disparate sources at once and b) in their native formats. Without this, complex integrations get little assistance from the platform itself, but are accomplished by extensive custom coding. A streamlined, high-performance data virtualization solution is impossible. Writing back to the sources becomes cumbersome at best, and live, real-time end user interaction with the endpoint simply cannot be effective. Older one-to-one transformation engines do not satisfy the needs; XSLT transformation engines also do not meet the two criteria, because all data must be converted to XML before it is transformed, and the XML output must then be converted to the destination format. Each of those conversions: to and from XML are effectively additional full transformations that generally must be accomplished with custom coding.
2.  There must be a single Integrated Development Environment (IDE) that crosses the entire scope of functionality for all integration patterns. This IDE must incorporate the run-time engines in order to be able to design, develop, test, deploy, and monitor in the same environment. Leaving the environment for anything dramatically reduces the speed of implementation and of change, which is essential for agility, time to value, and minimizing tech debt.

Some things are only possible with AIS

The most recent feature that has been touted by all is in the Data Virtualization space, where data from multiple sources is brought together, cleaned, aligned and transformed without actually moving, staging, or copying the data anywhere, and then delivering it virtually as a “view” to an application or dashboard on demand, again without ever moving it. The most powerful functionality that simply cannot be done successfully with other technologies is data virtualization with write-back to the sources. Enterprise Enabler automatically generates data virtualization, including the ability to write back securely to the sources. These integrations are published for consumption in multiple formats for consumption, such as web services, ADO.Net driver, SharePoint External List, and others.

                          (Click picture to see full chart)

Bottom line value of Enterprise Enabler (AKA Agile Integration Software)

• Time to Value is reduced by up to 90%
• Tech debt, the cost of maintenance and change over time is similarly reduced
• No need for expensive, specialized skill sets particular to a specific tool
• Streamlined architecture inherently enables high performance
• Lower security risks: with Data Virtualization data remains securely in the sources of record, without copies being made
• With write-back to sources in Data Virtualization patterns, dashboards become interactive consoles instead of simply reporting tools
• Without copies being made, a multitude of application specific databases become unnecessary, and synchronization activity is reduced
• Latency issues are removed since applications and end users have always the most current (“live”) information.
• Maintenance and change over time is no longer an overwhelming problem.

 We have observed that while Enterprise Enabler automatically generates bidirectional services, competing product companies tend to proliferate bi-directional lip service.