Mainframe nearly to the cloud…

Most people know that Salesforce.com is one of the first and certainly most successful SaaS (Software-as-a-Service) applications on the market.  One good thing is that Salesforce stores all the data in the cloud and manages it, eliminating the need for their customers to have the skills and the hardware, software, and maintenance costs  to keep it on-premise.  That good thing is also the biggest  downside of SaaS: the concern that the data is stored in the cloud.   Unfortunately,  companies worry about having their data stored off-premise with very little control over its management, security, and perhaps even accessibility. 

 

Nevertheless, Salesforce.com has a huge customer base and offers business functionality important to every business I can think of. While business sectors like financial institutions and healthcare could easily make valuable use of the  functionality of Salesforce and other cloud apps, the risk and regulatory restrictions make storing their data in the cloud impossible.  

 

These institutions simply cannot make copies of their data or move it to the cloud. The data that is inherent to the functionality of Salesforce may not necessarily be the concern, but often it must be presented to users side-by-side with ancillary data that must come from the company's backend, on-premise systems.

But all is not lost!  Agile Integration Software (AIS) naturally solves this problem by creating federated views from multiple sources and making them available to any application, complete with end user access authentication. Here is the crux of the solution with Salesforce as the example:

1.     Salesforce.com offers the capability of modifying the screens, so anyone who is conversant in doing that can modify a screen to populate the data from an external source. One option would be to configure it to call a web service when the screen is presented or refreshed.

2.    Within a few minutes, an Agile Integration Software, such as Stone Bond's Enterprise Enabler Virtuoso, can be configured, generating metadata that virtualizes and aligns backend data with Salesforce data, and packages it as a web service compliant with Salesforce. Optionally, this would be a bi-directional (Read/write) connection.

3.    When an end user brings up the Salesforce page, Salesforce calls the web service, and Enterprise Enabler Virtuoso accesses the on-premise data live, aligns it with the relevant Salesforce data, and sends it to Salesforce screen. With the bi-directional option,  data can be entered or corrected on the screen to automatically update not only the Salesforce data, but also the on-premise data, assuming the end user has proper permissions to write back to those systems.

Companies have spent millions of dollars over the last few years trying to do this, and with the Agile Integration Software as the basis, Enterprise Enabler Virtuoso was configured in three weeks to incorporate this Salesforce connectivity. Now it is available off-the-shelf so that anyone can implement it in a few minutes or at most a day. 

The diagrams below depict the data residence and flow where on-premise data is required in a Salesforce.com implementation. The first is the common solution where a copy of the on-premise data is made and resides on the Salesforce cloud. I don’t need to tell you the overhead and pervasive concern with doing this.  The second shows the on-premise remaining on-premise, where it belongs, and AIS accessing, federating, and delivering a data view virtually to the Salesforce page.

 

http://tiny.cc/id5h0

MDM - Making It Actionable/Transactional as You Define It

How useful is your MDM… really? Does it just sit there in a repository, waiting for your MDM team to update it?

One of the common criticisms of MDM projects is the magnitude of the project and the low ROI. More than likely, you are in the middle of a project with great expectations of value.

Metadata and MDM

When most people think of metadata, the scope is limited. It's a schema that defines a virtual data set, for example. It may include a cross-reference in a lookup table. And maybe it includes definitions of what each element means and what unit of measure it is in.

Then what?

Then you have to add references to where the data ought to come from.

But then what?

You've spent quite a lot of resources defining this. Are you any better off than with the ancient
"Corporate Dictionary?" How do you actually use it?

The most common ways to implement Master Data definitions are indicative of Big Projects:

1)    Define a data warehouse to store the data in, so that it is accessible in the form defined in the Data Master. Once the data warehouse is designed, corresponding integration must be built to populate it from the appropriate sources, aggregating and transforming as needed, as often as necessary for minimal latency.

2)    Write web services to access the data from the sources and make them available as Master Data sets.

When I talk about metadata, I think in terms of representing not only the data schemas but also the metadata that describes where the data is, what part of it is relevant, how it aligns with other data of interest, how you or the real or virtual destination (master) needs to see it, and how it must be converted, or mapped, to be meaningful to the destination.

Then there are the events that trigger data flows, and all the surrounding logic notifications, security, and a host of other things. If you can capture all of this information as metadata, in reusable, separable "layers," you will have a highly flexible and "actionable" collection of metadata.

If you define a metadata Master, say, "Customer," for use corporate-wide, you will have several different sources that are in play to ensure that the various parts of the virtual "Customer" definition has the best information from the most appropriate sources. Part may come from your ERP, part from Salesforce.com, and another part from an Oracle database.

· Does your Master definition encapsulate everything you

  need to use the data?

· Can your metadata be pumped onto a message bus?

· Can it be packaged as a web service?

· As an ADO.net object?

· As a SharePoint external content type?

· Does it incorporate the capabilities to perform CRUD

  (Create, Read, Update and Delete) operations at the

  endpoints?

· If one of the sources schemas changes, do you have to do

  anything to accommodate it?

· Do you even need to know a source changed?

If I'm a programmer, I want to leverage the corporate Master Data for my programs and the users of my programs. I can look up the data definitions, sources, etc., and use them, but that still requires a lot of work. When the Master Data includes a full set of metadata, then all I have to do is invoke the web service or External Content Type in SharePoint, or ADO.net and so on. I simply select the Master I need and indicate how I want to use it. I don't need to know what the various sources even are, and if the source changes, I won't need to make any changes, since the metadata will reflect what it needs to. And I can pass that selection process on tot the end user of my application or dashboard.

The diagram above shows the scope of metadata captured for MDM by Agile Integration Software. The metadata is generated from a GUI and has an atomic structure so that a change to any metadata can be made without impacting the whole hierarchy of metadata. Using this type of metadata infrastructure, changes are absorbed without creating waves. Data is accessed directly from the original source, eliminating the need for a costly data warehouse to resolve virtual relationships across sources.

MDM - Making it Actionable/Transactional as you Define it

How useful is your MDM, really? Does it just sit there in a repository, waiting for your MDM team to update it?  One of the common criticisms of MDM projects is the magnitude of the project and the low ROI.  More than likely, you are in the middle of a project with great expectations of value.

When most people think of metadata, the scope is limited. It's a schema that defines  a virtual data set, for example. And a Data Master may include meaningful keywords and tags for identification. It may include a cross-reference in a lookup table. And maybe it includes definitions of what each element means and what unit of measure it is in. Then what? Then you have to add references to where the data ought to come from. But then what? You've spent quite a lot of  resources  defining this. Are you any better off than with  the ancient "Corporate Dictionary?"  How do you actually use it?

The most common ways to implement Master Data definitions are indicative of Big Projects:

1.       Define a data warehouse to store the data in, so that it is accessible in the form defined in the Data Master. Once the data warehouse is designed,  corresponding integration must be built to populate it from the appropriate sources, aggregating and transforming as needed, as often as necessary for minimal latency.

2.       Write web services to access the data from the sources and make them available as  Master Data sets.

When I talk about metadata, I think in terms of representing not only the data schemas but also the metadata that describes  where the data is,  what part of it is relevant, how it aligns with other data of interest, how you or the real or virtual destination (master)  needs to see it, and  how it must be converted, or mapped, to be meaningful to the destination.  Then there are the events that trigger data flows, and all the surrounding logic notifications, security, and a host of other things.  If you can capture all of this information as metadata, in reusable, separable  "layers,"  you will have a highly flexible and "actionable"  collection of metadata.

If you define a metadata  Master, say,  "Customer,"  for use corporate-wide, you will have several different sources that are in play to ensure that the various parts of the virtual "Customer" definition has the best information from the most appropriate sources.  Part may come from your ERP, part from Salesforce.com, and another part from an Oracle database. Does your Master definition encapsulate everything you need to use the data? That is, can your metadata be pumped onto a message bus?  Can it be  packaged as a web service?  As an ADO.net object? As a SharePoint external content type?  Does it incorporate the capabilities to perform CRUD (Create, Read, Update and Delete) operations at the endpoints? If one of the sources schemas changes, do you have to do anything to accommodate it? Do you even  need to know a source changed?

If I'm a programmer, I want to leverage the corporate Master Data for my programs and the users of my programs.  I can look up the data definitions, sources, etc., and use them, but that still requires a lot of work.  When the Master Data includes a full set of metadata, then all I have to do is invoke the web service or External Content Type in SharePoint, or  ADO.net and so on.  I simply select the Master I need and indicate how I want to use it.  I don't need to know what the various sources even are, and if the source changes, I won't need to make any changes, since the metadata will reflect what it needs to. And I can pass that selection process on tot the end user of my application or dashboard.

The diagram above shows the scope of metadata captured for MDM by Agile Integration Software. The metadata is generated from a GUI and has an atomic structure so that a change to any metadata can be made without impacting  the whole hierarchy of metadata. Using this type of metadata infrastructure, changes are absorbed without creating waves. Data is accessed directly from the original source , eliminating the need for a costly data warehouse to resolve virtual relationships across sources.