SaaS Vendors Take Note: You Can Operate in the Cloud but Not in a Vacuum

This week I watched a webinar that proposed that more responsibility of integration with SaaS applications needs to be carried by the software provider. Analyst Dana Gardner of Interarbor discussed this topic in the webinar and on his blog http://tinyurl.com/4fgyw5j. Workday recently announced its cloud-based integration services as part of its SaaS ERP offering, stepping up to the plate to provide tools that will ensure that customers can integrate to their own software. They very well may be setting a trend that other ISVs will need to keep up with.

It is clear that, while the convenience and specialization of cloud-based solutions opens a new opportunity for businesses to trim down their infrastructure and the associated maintenance costs and effort, it doesn't eliminate the need for integration. Instead, it calls for a new paradigm for data integration.

Unfortunately, the heavy middleware in enterprises today will have trouble keeping up with the increasing business demands for agility. Change is tough when no one wants to touch the integration for fear of breaking something.

With the growing dependence on cloud-based software, your customers need a new generation of integration that can streamline data flows as their business processes move freely across legacy, clouds, and collaboration portals. If you are a SaaS vendor you need to seriously think about how you can rise to the occasion for your customers. The better you do it, the happier your customers will be.

Workday addressed the challenge by buying an integration software company! That's not only an expensive way to go, but it means that now, in addition to domain expertise and software development teams for their ERP, they must also maintain expertise and developers in the rapidly-changing integration space. That's probably not a prudent business approach for most SaaS vendors.

The alternative is to embed, rebrand, and/or offer Agile Integration Software, such as Enterprise Enabler® in your offerings www.enterpriseenabler.com. That way you get all the benefits without the headaches. I don't know if Workday's integration fits the Agile Integration Software model (see http://tinyurl.com/3yv85bw), but with AIS, even a SaaS software vendor is able to offer integration across cloud apps, and also incorporate on-premise backend legacy systems as well as pass data to and from your customer's SharePoint installations, on-premise or hosted.

Crossing the Chasm Between Consumer and Business Technology

How can it be that consumer technology manages always to deliver on ease of use, compatibility, and basic human appeal? Why do we have to deal with ugly, difficult, clunky software in the business world? How is it that there are standards that have been pervasively accepted and implemented for cell phones and all manner of electronic gadgets? Don't you ever wonder why it is that consumer technology hits the spot and continues to do so?

A huge contributor to the difference is the nature of the products themselves. Consumer electronics, phones, and even software are basically "throw-away." They are low cost and are expected to be completely replaced every couple of years, so consumers are forgiving of bugs and don't even think about upgrades. They simply get a new one. Maintaining the history of the world part 2 or part 1 is not the responsibility of these devices and software. The closest they come is the need to accommodate a few years of contacts' phone numbers and addresses. And even that.. only the business users of these things care.

For consumer software, the imperatives of interoperability with existing and old technology are very minimal, limited to internet and communications standards, which are discreet and maturing. The consumer market is huge, and with the constant replacement by customers, it's a path of enviable "recurring revenue" streams. Besides that, marketing to consumers is much more intuitive and pervasive than targeting the specific individuals who might want/need the business solution and also have the authority and the budget to buy it. With business systems, there's no walking by a store, seeing a flashing display, and buying on impulse.

For business software product companies, it's a whole different world: more complexity, expectations for longevity, necessarily higher price tags, and requirements of "upward compatibility" for new releases. There is always the imperative of having to work in tandem with everything else that's in place or being invented by competitors.

So our challenge is to figure out how to close the gap and move business technology closer to consumer technology. Of course, it's SaaS that offers the greatest potential to shake up the software product market over the next few years, but I think that Agile Integration will come into the mix in a big way, dramatically simplifying the underlying corporate IT infrastructure.

The Power and Flexibility of Data Federation and Virtualization Together

Finally, the world seems ready to take data federation and virtualization seriously. The power of each is, and has historically been, severely limited by the absence of the other in the same product. Using data virtualization means that you can switch out the underlying data and where it comes from without changing the actual integration layer, but rather by just redefining the metadata. That definitely adds flexibility to a company’s underlying infrastructure. Federating data means that you can bring data from multiple sources, combine them into a single data set or view, and let users and applications consume it without having to stage it. That eliminates the overhead of defining, implementing, and maintaining a data store. Combining the two, along with a robust transformation engine, in a single execution, adds orders of magnitude improvement in the agility of the overall IT infrastructure.

It seems to me that this is a very good time for IT decision makers to start aggressively implementing these combined capabilities throughout their organizations. Data warehouses have come of age, maybe have even reached "a certain age" status, and are carrying the ballast of years of data, maintenance, and accommodation of change. Streamlining and speeding up can only done by more computing power and better access algorithms and tools: new forklifts. Meanwhile, to remain competitive, the company must be able to rapidly deliver information quickly and appropriately wherever it is needed.

It's essential that the core IT department management and corporate executives not leave the flexibility to the "Shadow IT" that comes of desperation from the business and operations people who cannot wait and deal with the front lines of delivering the company's activity and daily business decisions.

With Agile Integration Software, the IT department could quickly deliver flexible, maintainable layer that interacts with everything that is already in place, but that steps up to the plate with agility. Please check out my white paper on Data Federation and Virtualization.  http://tinyurl.com/6a8zbxv

Federated Data for SharePoint 2010 using AIS

If you're not switching to SharePoint 2010, you are missing a huge opportunity. The new features actually position it so that it can become the ONLY application that end users need to log into. All kinds of useful SharePoint applications can be easily built to bring data from multiple backend systems together, aligned in a single window, for user viewing and updating.

You probably don't think of SharePoint as an MDM product, either, but maybe you don't really need the multi-million-dollar, huge upkeep, of the MDM products from the (other) Big Guys.

You may know that SharePoint 2010 advanced the concept and usability of virtual data definitions from the earlier Microsoft Office SharePoint Services (MOSS) to include full CRUD (Create, Read, Update, Delete) and other features like SSO (security). This "external entity" catalog, called BDC, or Business Data Catalog, functions as an MDM of sorts that contains the metadata describing corporate business entities along with the metadata describing how to get the data and where it comes from. When a user opens a window that is bound to one of the entities, s/he can interact with the live data without knowing or caring where it came from. The interaction is with virtual data accessed live from backend systems.

SharePoint Designer is a nice tool for building the metadata when it comes from a limited range of single sources. With some amount of programming, you can extend its use a little. Unfortunately, without the ability to update an external entity's metadata definition, SharePoint Designer must create a whole new definition, which then needs to be switched out for all the apps that use it.

SharePoint 2010 really flies, though, when you combine it with an Agile Integration Software (AIS) like Enterprise Enabler http://www.enterpriseenabler.com/. Agile integration brings the ability to build complex data mappings, from multiple sources (without programming) and generate the desired external entities in SharePoint. Federating data on the fly from multiple sources is powerful.

Now, suppose the backend systems changes. Maybe your entity combines data from SAP and Dynamics CRM. If you switch from Dynamics CRM to Salesforce, an AIS can apply the changes behind the scenes to connect the same external entity definition in SharePoint without any disruption.

There are a couple of things you might be concerned about, like security and transaction rollback on updates to multiple back ends. SharePoint supports SSO, so AIS will pick up the end user security from SharePoint and pass it to the backend systems. If the user does not have permission to read from a backend system, s/he will not see the data. If the user does not have write access to one of the backends, the full transaction will be rolled back. This is generally the only time data is actually persisted in an AIS.

This should definitely give you something to think about. You could use all the savings for a company trip to Colorado, or Hawaii, or anywhere else. Or maybe give a fat raise to everyone in IT.

----------------------------------
Video of building a SharePoint entity in AIS http://tinyurl.com/5vxce55

Agile Integration Serving End Users Directly

Social media is beyond ubiquitous, putting the power of communication and availability of information in the hands of everyone individually and collectively. It has moved from being a thrill for the eager to being a fact of life even for the reluctant. Social sharing of unstructured data is proliferating, and its management is the subject of much conversation, posting, tweeting, and blogging.

But what does this mean and how does it relate to individuals who need structured data? Isn't it time for an end user to be able to access data without calling the boss, who calls the IT department, who calls the consultant, who plans the project, that gets a team, that does an analysis, that includes everything but the kitchen sink, that has a garbage disposer, that's the only thing that works, with which the project finally gets euthanized?

It's time to think about what this means to data integration: lean, mean, or otherwise. These days end users are empowered by SharePoint and BI tools and such to determine for themselves what information they need in order to do their job and make decisions. If an end user can sign in to SAP and to Salesforce and to JD Edwards, why can't they use their SharePoint portal or a dashboard to access the same data from those applications? Of course, one of the key reasons is the issues around security. Historically integration has been behind the scenes, working with no visibility to the actual end user. ETL, of course, has none, and EAI is also not end-user aware. Dashboards are designed for specific users who have permission to see the data displayed. Although the dashboard app may have its own login for security, the integration behind the scenes does what it's told without knowing anything about the user.

Agile Integration Software http://tinyurl.com/3yv85bw supports SSS and other layers of security so that an end user of SharePoint, for example, can see live data with updates passing back to the applications, only if the user has privileges to do so. That same security is available out of box in AIS for any vendor software, BPM, or BI that is designed to read and write data to an ADO.Net entity via an ADO.Net driver, or alternatively by web service calls.

For example, Enterprise Enabler® http://www.enterpriseenabler.com/ is an AIS that can be configured in minutes to merge and align data from multiple applications and generate the entity definition and custom connector for SharePoint (BDC) or generate the entity for the ADO.Net driver that is called from WSS or any other ADO.Net Application. If preferred, it can generate a web service just as easily. The SSS end user security is honored in all these models.

So, in response to the expectations of the "me and everybody else now" surge, integration needs to step up to the plate.

Two Laws of War, IT, and Human Nature

There are two things that I have considered to be fundamental rules of life from as far back as I can remember. In my early twenties, it dawned on me that I actually thought of them with greater conviction than I ever did any of the laws of physics I learned. I decided that these rules held equivalent un-challengeable level of "truth" as laws of physics. Of course I did, in fact challenge every "truth" handed out in my first physics class. How in the world could everyone in the class just sit there and believe that there's some Force pressing up on the floor to keep us from falling through? Because the textbook said so? Hmm. I grew up in Virginia, and everyone knew that the State published the history books, and it was in Virginia's best interest to publish that every President except Lincoln was from Virginia! So, how can you believe there's an equal and opposite arrow somewhere that you can't see?

Now for my two laws, in the order of discovery.

Pamela's Law of Set Points. This may not be a very good name for it, and since I never named it before this moment, I reserve the right to change the name some time. The law is something like this: More things than you can imagine try to obey a set point. The set point may gradually shift over a long time, especially in response fundamental change due to extraordinary events, but in general nature gravitates to that set point. I remember thinking about this law and human nature when I was "into" change management and a person's or institution's capacity for change ("resilience"). While each person or organization has a different level of tolerance, that maximum is essentially constant. The same can be applied to a person's good humor. I believe there's a set point for each of us that is fulfilled by our reactions and attitudes no matter what is actually happening in the real world outside us. Worriers manage to find a certain level of worry, easily filling the void with a new subject of worry as another gets resolved. I'd love to be one of those eternally happy people that have a set point way on the happy side. I guess it's kind of the "glass half full" conundrum, which somehow leads to:

Pamela's Law of Constants. Again a name of the moment. I grew up in the Washington, D.C. area and from the time I was six or so, through high school, I ritualistically read the front page of the Washington Post every single morning (then I read the comics if my siblings were done). The most obvious conclusion to me was that there was always a war somewhere, often in some tiny country I never heard of before. As one subsided (from the headlines) another popped up somewhere else, like the cartoons where the guy with the big hammer pounds down one bump and it pops up somewhere else. That's when I began formulating the theory that unrelated happenings like this really were fulfilling some constant amount of world-wide war. Over the years, I discovered that this same law applies in many, many areas. I could elaborate on the law of constants in the IT world, but that's for another day. You can see, though that if there are constants to be balanced by seemingly unrelated events, that means the events really are related… my crude contribution to the evolution of Chaos theory.

Five Ways to Get Immediate ROI from Agile Integration Software

One of the great things about Agile Integration Software (AIS) is that you can install it and start reaping the benefits immediately.  You don't have to change your whole company's philosophy about middleware. Think about AIS as an adjunct to what's already there, and just solve a couple of important challenges quickly.

Don't know what AIS is? Take a look at this: http://tinyurl.com/3yv85bw

Here are a few ideas that can prove the value of AIS and bring real value to your company before the rest of the team wakes up.

ONE. Get all manner of data into SharePoint. Empower your SharePoint developers and users with secure access to data from all your backend systems. That means that the people that need to see and interact with data that comes from SAP, Salesforce, and Excel, or a custom application, for example, can have that without even declaring a project. Line of business (LOB) applications built in SharePoint invariably need data from multiple systems, and it all needs to be aligned somehow to make sense together. Try AIS for this. Build a couple of interfaces to SharePoint External Lists - BDC (business data catalog) or to SharePoint Lists and see how it goes. The data is secure, and you can even use the same interfaces to handle write-backs to those systems.

TWO. Rescue a project that is way behind schedule. Chances are that the part of the project that's lagging behind is the integration with other systems. Don't displace whatever technology is in being used, but try a rapid parallel path with Agile Integration. Skip the staging database and get the data live from the systems align it on the fly and send it where it needs to go. This is what AIS is designed to do really well. If you find that the easiest way to get started is to go with a staging database that's in place, then use AIS to get the missing data into the repository quickly. Or build the integration and expose it as a web service in two clicks.. Voila! Then sit back and reap the benefits of success (and be a hero).

THREE. Fast-track your BI dashboard project. Let's say you have a dashboard project that has lots of promise. Maybe you just bought a very cool analytics software package that will allow you to understand your business and the market better so you can maintain your competitive advantage. The vendor delivered the software and gave a two-day training on how to configure and use the cool graphs and such. But now they've gone home, and it's up to you to figure out how to get all the data you need into the tool. The more data that's available, the greater the value of this investment. Big problem. Big project, fraught with all the perils of any integration project. Before you spend a year creating the database, see if the tool is just making a call that could be tied to an AIS that could grab the necessary data, align it, and pass it live to the dashboard. That's definitely faster and a much better solution. Otherwise, at least reduce your population of the staging database by 70% by using AIS.

FOUR. Close your books in half the time. If your company is like most, there are multiple ERP or financial systems as well as inventory systems and even spreadsheets involved in the process of consolidating all of the information required to close the books every month. With a changing environment, new entities must be accommodated or eliminated from the process on a regular basis. Using AIS you can automate the capture and alignment of the information from the multiple sources, but even better than that, you can update the integration in a matter of minutes to reflect the changes in your business from month to month or quarter -to-quarter.

FIVE. Improve the productivity of your IT team. Empower your Data Analysts to configure integration without programming. Free up your programmers from doing repetitive integration activities. AIS can eliminate a tremendous amount of activity that simply does not need to be done anymore. You owe it to your company to take advantage of the value of AIS. Start with new data flows that need to be done rather than initiating a lifelong switchover project.