Agile Integration Software Rescues the Dead Side of Bimodal IT Architecture

I’m sure you’ve heard of the recent high profile discussions about how to bring much-needed innovation to mature companies that are carrying the comforting ballast of old fashioned infrastructure.  That infrastructure is the greatest impediment to agility and innovation (unless you count the people and culture that go along with it). I first heard about “bimodal” at a local Gartner program a few weeks ago and found the concept both thrilling and disturbing.

The idea of bimodel divorces the reliable and stable back-office (Mode 1 “Core”) from all that is innovative (Mode 2 “Innovation”). This means that innovation is explicitly separate, with new, presumably agile, infrastructure to create new lines of business for generating new revenue streams, and to provide more contemporary modes of interacting with consumers and employees.

I’m a little concerned that a bi-modal declaration promotes an easy way out for Mode 1 laggards. Their management no longer have to worry about modernizing or even interacting with Mode 2. In approaching the problem this way, we are continuing to be enablers of the infrastructure and its management who are addicted and afraid to even try to wean from their brittle, ancient technologies and methodologies.  I suspect that part of the reason that we got to this point is their continued abdication of decisions and recommendations to vendors and consultants on the dead side of bimodal. With advancement generally limited to creative marketing and re-messaging the same 20-year-old technologies and ideas, the name-brand consultants in enterprise IT nominally grab the buzz but only deliver it on the periphery.

I definitely agree that relying on the Mode 1 (reliable, stable) IT is highly unlikely to bring  significant innovation, and I also believe that the best way to get real innovation underway is to completely separate it out, with different people, skill sets, management, and objectives. But if we imagine how this will play out, there is likely to be a complete bifurcation where the innovative side never is able to leverage the back-office functions. They will inevitably invent their own (less reliable and less stable) versions of back-office. What happens then? Mode 1 eventually dies on the vine? We regress to pre-1980 basics?  Business in general accepts worse performance on the backend functions?

(You didn't really try clicking that button, did you?)

Makes more sense to contact Stone Bond Technologies.

It’s probably obvious that my take is that BOTH modes should advance aggressively, ‘though I do believe the innovative side should be unencumbered by the Old World. Mode 1 management should take this as a gauntlet to push hard to replace their integration infrastructure with Agile Integration Software, such as Stone Bond’s Enterprise Enabler®, which is a proven enterprise-ready framework that boosts agility, interacts with both Mode 1 and Mode 2 applications and data, and offers up to 90% reduction in time to production along with a huge reduction in tech debt. That is what is needed for companies to survive and enjoy a competitive advantage in the coming years.

You Mode 1 people do have a choice. You can continue as is and sit by waiting for your inevitable demise, or you can be the hero that solidly bridges Mode 2 and Mode 1. We are seeing this successfully implemented by forward-looking CIOs. So, find a leader and press GO!

Agile Big Data is Coming

Agility with Big Data?

Shiny but not Agile, perhaps destined to never be Agile.

Just look at ETL. Never, never agile.

So Clunky. After all these years, why is it so Clunky? So Clunky it’s fragile.

Big Data follows in the footsteps.

Big feet. Big hype, Big opportunity

for discoveries otherwise unfathomed.

Great programmers performing great feats,

Coding, coding new horizons… Open Source assist.

No hope for Agility.

Where are the trailblazers of Agility?

Ah, yes!

They’ve conquered ETL, EAI.

They’ve conquered Data Virtualization.

Agile Big Data is coming.

Are great programmers actually an impediment to Agility in complex software problems?  I‘m not talking about Agile development, but rather Agile solutions. Great programmers like to be on the leading edge of the latest new shiny trend. I remember the rush of programmers to work on the Y2K-driven Enterprise Application Integration and ERP rage. Now we’re all over Big Data.

It takes time for the market, business, and technologists to evolve the thinking about what the Shiny thing really is for, what it means, and what the technology requirements are. So, who jumps in first? The really good programmers who want to blaze the trails and solve the problems first-hand. Unfortunately, the early players can’t benefit from the evolution and maturing of the Shiny and what the requirements will really be. By the time it’s clear what it takes to genericize the problem, the Great programmers have moved on to the next trend, and are certainly not going to step back and solve the problem again in a tool or platform that hides all the repetitive work behind the scenes. 

Product companies then step in to harvest the rest of the hype curve. Leveraging their Big Name, they make plenty of revenues on hard-coded, specialized solutions where custom coding is accepted as the only way to solve the problem. In my view, a timeless benchmark for Agility is the minimization of custom code. The more coding involved in generating a solution, the farther away that solution is from Agile.

The growing love affair with Open Source code is a huge step backward for the cause of Agility. Programmers love coding. I, for one, love coding, too. But I HATE having to code essentially the same thing over and over, with just a couple of tweaks difference each time. This is exactly what integration is all about: lots of small (very important) tweaks to the same code over and over.

That is why Enterprise Enabler exists. That is why we hide all the technical details behind the scenes of our agile integration software. Any time a programmer has to do essentially the same thing more than a handful of times, it is automated with tweaks configurable in a UI. Programmers should be programming exciting, innovative new things, not laboring over repetitive, boring scripts and code changes, and maintenance.

We’re applying the same philosophy to automate Big Data integration and analysis.

2 Keys to Identifying Agile Integration Software

For years I’ve been talking about Agile Integration Software (AIS) as a class of products that enable very flexible and agile data flows across the enterprise, eliminating the clunky, expensive, time consuming infrastructure of the past. This paradigm is agnostic to integration patterns and shares metadata for all uses, such as Data Virtualization (DV), ETL, EAI, SOA, etc.

Over these years, I have come to understand that AIS is not a class of products. As it turns out, Enterprise Enabler® is the only product that has all of the characteristics necessary to fulfill the AIS vision in scope and flexibility. So much for a product-agnostic concept-oriented blog! Many of the features can be found in other products, but somehow there is always at least one critical feature missing that negates the possibility of agility for the solution.

As I think about what constitutes agility in this space, a few things come to mind that are imperatives in such a platform. The two most telling indicators lead the list:

1. The product must have a transformation engine that aligns and transforms data a) from multiple disparate sources at once and b) in their native formats. Without this, complex integrations get little assistance from the platform itself, but are accomplished by extensive custom coding. A streamlined, high-performance data virtualization solution is impossible. Writing back to the sources becomes cumbersome at best, and live, real-time end user interaction with the endpoint simply cannot be effective. Older one-to-one transformation engines do not satisfy the needs; XSLT transformation engines also do not meet the two criteria, because all data must be converted to XML before it is transformed, and the XML output must then be converted to the destination format. Each of those conversions: to and from XML are effectively additional full transformations that generally must be accomplished with custom coding.
2.  There must be a single Integrated Development Environment (IDE) that crosses the entire scope of functionality for all integration patterns. This IDE must incorporate the run-time engines in order to be able to design, develop, test, deploy, and monitor in the same environment. Leaving the environment for anything dramatically reduces the speed of implementation and of change, which is essential for agility, time to value, and minimizing tech debt.

Some things are only possible with AIS

The most recent feature that has been touted by all is in the Data Virtualization space, where data from multiple sources is brought together, cleaned, aligned and transformed without actually moving, staging, or copying the data anywhere, and then delivering it virtually as a “view” to an application or dashboard on demand, again without ever moving it. The most powerful functionality that simply cannot be done successfully with other technologies is data virtualization with write-back to the sources. Enterprise Enabler automatically generates data virtualization, including the ability to write back securely to the sources. These integrations are published for consumption in multiple formats for consumption, such as web services, ADO.Net driver, SharePoint External List, and others.

                          (Click picture to see full chart)

Bottom line value of Enterprise Enabler (AKA Agile Integration Software)

• Time to Value is reduced by up to 90%
• Tech debt, the cost of maintenance and change over time is similarly reduced
• No need for expensive, specialized skill sets particular to a specific tool
• Streamlined architecture inherently enables high performance
• Lower security risks: with Data Virtualization data remains securely in the sources of record, without copies being made
• With write-back to sources in Data Virtualization patterns, dashboards become interactive consoles instead of simply reporting tools
• Without copies being made, a multitude of application specific databases become unnecessary, and synchronization activity is reduced
• Latency issues are removed since applications and end users have always the most current (“live”) information.
• Maintenance and change over time is no longer an overwhelming problem.

 We have observed that while Enterprise Enabler automatically generates bidirectional services, competing product companies tend to proliferate bi-directional lip service.