Historically, Big Projects have been the only way to solve IT problems for Big companies. I've watched "generations" of IT management fall for the "next, next Big Project" promoted by BiG hardware companies, Big Systems Integrators, Big-time analysts. After all, who are you going to trust to set the direction for your Big company? The Big waves always are very well sold, and for the newbies, there is an air of doing something really new and really Big. Of course there's also Big money involved, enough to keep the economy healthy, maybe. As soon as one Big wave of Big Projects are several years in progress, the next Big begins to emerge and put the last one out of business before most are completed. Many stall, are pared way down to the only working prototype, or are abandoned altogether to be replaced with a fresh new Big approach.Big Projects started long ago, but in the last twenty or so years they have included:
○ Defining a single corporate database, planning for all the applications to share that same db
○ ERP - A single comprehensive application means that you don't have to rewrite all the apps to use that db
○ EAI - to address the reality that the above two Big Projects can't be realized
○ Business Process Re-engineering (BPR) - Shifting focus from data to processes- Big Consulting Projects with no need to know much of anything about technology
○ Change Management - because radical BPR created lots of employee issues and confusion
○ Data Warehouse (DW) - in spite of the intentions, smaller projects and best of breed applications were more successful than Big Projects, and businesses came to rely heavily on those systems. Data Warehouses were supposed to bring all the data together for reporting.
○Business Intelligence (BI) - analyze the data in the DW.
○ MDM - the modern Big Project for a corporate dictionary.
There were others, of course, but you get the idea. Finally wedges are putting crevices in the Big Project and opening it for solutions that are more atomic and less global. Some of the wedges are being driven by:
○ SOA
○ SaaS
○ Agile Integration Software (AIS)
○ Social Media
○ The economy and the imperative for improved time-to-value on projects
These factors open the floodgates for a bifurcation of approaches to enterprise technologies. As Mark Twain said, "If there's a fork in the road, take it." Traditionally Big technologies, like BI and ERP, are now offered as a cloud based service and for single users without the overhead of Big.
These wedges are all eroding the cornerstone of Monolithic solutions. For example, SOA is inherently atomic, with an enterprise solution being a collection of SOAP objects. While the initial SOA initiatives were envisioned as enterprise-wide, in the end even the prototype projects were Big, long, and difficult. If the technology were not built on reusable components, the ongoing work that continues to be done would likely have been abandoned. Similarly, while data warehouses continue to be expanded for Business Intelligence, we are seeing a huge number of BI tools coming on the market for specific use or end user-centric implementation. Cloud computing is also whittling away at Big Projects, with significant cost and time reductions as well as shorter time to value.
One of the interesting things is that this split is creating an environment where emerging technology waves now may have two completely different interpretations, one the old Big approach and the other a more agile and atomic approach.
Take data federation and virtualization, for example. The Big approach is to define a complete (or at least really Big) virtual enterprise data model for federation that acts like a staging database would, and then to implement the integration across and through the virtual staging model. Of course, at some point, it's necessary to define those integrations based on what the end result datasets or use happen to be for the consumers.
The new fork in the road (which I would take) requires no data model, virtual or not. An Agile Integration Software addresses federation and virtualization in an atomic manner, with the end use the initial driving force. Entities that describe , for example, "customer" are defined, the source of record for each piece of the Customer data is identified, and metadata is auto-generated and packaged to grab the data from the sources, federated it "on the fly" and deliver it to the calling program, end user or data workflow on demand or in an event-driven manner. An atomic approach to MDM naturally follows.
Hooray for the fork in the road!