Tuesday, September 25, 2018

Ask not what AI can do for Data Virtualization. Ask what Data Virtualization can do for AI.

Artificial Intelligence (AI) can perform tasks that are considered “smart. Combining AI with machine learning, it reduces programming and algorithm learning. AI has been exploding recently and taken a big leap in the last few years.

But what about Data Virtualization (DV) with AI? The first thought is usually AI optimizing queries on virtual data models. However, how can DV help AI? 

Why not leverage the virtues of Data Virtualization to streamline the AI data pipeline and process?

Example 1: Expedite the Machine Learning process when you need data from multiple sources

Suppose you want to train your Machine Learning (ML) scenario with composite data sets? Federating disparate data “on-the-fly” is a core competency of Data virtualization. Without data virtualization, of course, you could always add custom code here and there to do the federating and cleansing, but that complicates and certainly slows down the configuration of the AI as well as its execution. Even with huge amounts of data to deal with, Data Virtualization can quickly align and produce appropriate data for ML without incurring the time and risk of hand coding.

 Example 2: Define and Implement Algorithms

 With Data Virtualization a model or database is driving the iterations, DV secure write-back capability can continuously update the data sources to reflect the latest optimum values in real time.  

The learning occurs with many iterations through the logical model, usually with a huge data set. Each iteration brings more data (information) that is then used to adjust and fine tune the result set. That newest result becomes the source values of the next iteration, eventually converging on an acceptable solution.  That processing loop may be a set of algorithms that are defined in an automated workflow to calculate multiple steps and decisions. The workflow is configured in the Enterprise Enabler Process engine, eliminating the need for programming.

Example 3: DV + AI + Data Lake
Many companies are choosing to store their massive Big Data in Data Lakes. A data lake is usually treated as a dumping zone to catch any data (To learn more read - 5 Reasons to leverage EE BigDataNOW™ for your Big DataChallenges). Architects are still experimenting as to the best way to handle this paradigm, but the mentality is, “If you think we’ll be able to use it one day, throw it in.” With Data Virtualization, information can be fed in with a logical connection and can be defined across multiple data sources.

Enterprise Enabler® (EE) goes much further since it is not designed solely for Data Virtualization. The discovery features of EE can help after the fact to determine what’s been thrown in. EE can ensure that the data cleansing and management process executes flawlessly. Agile ETL™ moves data anywhere it is needed, without staging.

Since EE is 100% metadata-driven and fully extensible, it can morph to handle any integration and data management task that is needed. Enterprise Enabler is a single, secure platform that has pro-active configuration suggestions, maintains user access controls, versioning, monitoring, and logs.

From data access to product overview to data prep for business orders and reporting Enterprise Enabler is the single data integration platform that supports it all.


  1. This concept is a good way to enhance the knowledge.thanks for sharing. please keep it up selenium Online Training

  2. This comment has been removed by a blog administrator.

  3. This comment has been removed by a blog administrator.

  4. I feel satisfied to read your blog, you have been delivering a useful & unique information to our vision even you have explained the concept as deep clean without having any uncertainty, keep blogging.

    Digital Marketing Training in Chennai

    Digital Marketing Course in Chennai

  5. I truly appreciate the time and work you put into sharing your knowledge. I found this topic to be quite effective and beneficial to me. Thank you very much for sharing. Continue to blog.

    Data Engineering Services 

    AI & ML Solutions

    Data Analytics Services

    Data Modernization Services