5 Reasons you should leverage EE BigDataNOW™ for your Big Data

Big Data has been swooping the BI and Analytics world for a while now. It’s touted as the better way of Data Warehousing for Business Intelligence (BI) and Analytics (BA) projects. It has removed hardware limitations on storage and data processing. Not to mention, it has broken the barriers of schema and query definitions. All of these advancements have sprung the industry in a forward direction.

Literally, you can dump any data in any format and start building analytics on the records. We mean any data whether it’s a file, table, object, or in any schema into Hadoop.

1. EE BigDataNOW™ will organize your Big Data repositories no matter the source

Ok, so everything is good until you realize all your data is sitting in your Hadoop clusters or Data Lakes with no way out; how are you supposed to understand or access your data? Can you even trust the data that is in there? How can you ensure everyone who needs access has a secure way of retrieving the data? How do you know if the data is easy to explore and understand for the average user?

Most importantly, how do you start exposing your Big Data store with API’s that are easy to use and create? These are some of the questions you are faced with when you want to make sense of you Big Data repositories.

Stone Bond’s EE BigDataNOW™ helps you achieve the “last-mile” of your Big Data journey. It helps you organize your Big Data repositories, whether in a Lake, in the cloud or on-premise, EE helps make sense of all the data for your end users to access. Users will be able to browse the data with ease and expose it through APIs. EE BigDataNOW™ lets you organize the chaos and madness that the data loading individuals uploaded.

2. Everyone is viewing and referencing the same data

For easy access to the data, Stone Bond provides a Data Virtualization Layer for your Big Data repository that organizes the data into logic models and APIs. It lets you provide a mechanism for administrators to build logical views with secure access to sensitive data. Now everyone is seeing the same data and not different versions of it. This reduces the confusion by providing a clear set of Master Data Models and trusted data sets that are sanctioned to have the accurate data for their needs. It auto-generates APIs for the models on the fly so users can access the data through SOAP/REST or OData and be able to build dashboards and run analytics on the data. It also provides a clean queryable SQL interface, so users are not learning new languages or writing many lines of code. It finally brings a sense of calmness and sureness that is needed for true Agile BI development.

3. It’s swift … did we mention you access & federate your data in real-time?

EE BigDataNOW™ can be a valuable component on the ingestion side of the Big Data store too; it will federate, apply transformations and organize the data to be loaded into the Data Lake using its unique Agile ETL capabilities, making your overall Big Data experience responsive from end to end. EE BigDataNOW™ has a fully UI driven, data workflow engine that loads data into Hadoop whether its source is streaming data or stored data. It can federate real-time data with historical data on demand for better analysis.

4. Take the load off your developers

One of the major complexities that Big-Data developers run into is building and executing the Map-Reduce jobs as part of the data workflow. EE BigDataNOW™ can create and execute Map-Reduce jobs through its Agile ETL Data Workflow Nodes; this will help run Map-Reduce jobs and store results in a meaningful, easy way for end users to be able to access the Map-Reduce jobs.

5. EE BigDataNOW™ talks to your other non-Hadoop Big Data sources

EE BigDataNOW™ includes non-Hadoop sources such as Google Big Query, Amazon Redshift, SAP HANA, etc. EE BigDataNOW™ can also connect to these nontraditional Big Data sources, and populate or federate data from these sources for all your Big Data needs.

To read more about Big Data, don’t forget to check out Stone Bond’s Big Data page. What are you waiting for? Break through your Big Data barriers today!


This is a guest blog post written by,

9 Questions to Help Uncover Your Big Data Requirements

The whole concept of Big Data projects can be overwhelming, 'though the promise is compelling. Whether you are analyzing Social Media Data or digging through corporate data,  it's not just about processing huge amounts of data. Just like any other new technology project, it is easy get caught up in the vortex of the hype and lose the bigger picture of what’s involved.  You don't want to find out after the swirling starts that you may be swimming in unwelcome growing tech debt. If you understand the type of functions your solution will need to handle, you will be better equipped to select the most appropriate tools to solve it in ways that incur the least tech debt.

 Here is a quick methodology that will help you develop your own perspective on the Big Data opportunity at hand. Of course, you do need to understand what you really want to accomplish with your BD project, but let's assume you already know the objectives. This will help reveal how complex it will be to handle the data capture, manipulation, and analysis and put the Big Data part of it into perspective of the overall project.

Print this out. Cut out the nine Big Data game cards. Now put them all on a flat surface, turn off your ipod, close the door, and consider each one carefully. Pick "blue" or "red" for each, whichever best describes the data you will be dealing with. Set aside any that you really want to answer "both" or "purple."

 The Big Data Game

As you handle and shuffle the cards, you will see some interdependencies across the cards, and perhaps you start lining them up in the order of processing. If you have the inclination to throw one out completely, set it aside to think about again.

 Now, when you're done, if your answers are a loud and clear "Blue!" on every front, you have the most straight-forward Big Data situation - one that is just about Big Data without the noise of most realities that magnify the project dramatically. Does your table look like this, with all the blues marked?

"Blue!"

Most likely not.  Hopefully you have identified lots of ancillary tasks that will be necessary and that make this look like a data integration project as much as a Big Data project.  You will have to deal with other issues like:

·         Data security

·         Data transformation

·         Data federation

·         Data cleansing

·         Data capture

·         Data migration

·         Data updates

·         Data latency

These are all known problems, with solutions, of sorts. All of these requirements incur additional steps, and are often solved via staging of the data.  More than likely, with this exercise, you are contemplating that you will either need to have multiple staging of the Big Data (3 times Big Data is Big Big Big Data).  This is a huge driver for your company to adopt agile integration software (AIS), an imperative to such projects. Complementing Hadoop, AIS handles federation, inline cleansing and analytics, transformation and other processing without multiple steps along the way. Its transformation engine works directly across multiple sources, orchestrating and merging in their native modes as opposed to requiring intermediate conversion to XML, as XSLT engines do. Secure write-back to sources offers more degrees of freedom to the way you can think about Big Data problems.

 Enterprise Enabler® represents a new paradigm of integration, tremendously streamlining the creation of a Big Data processing environment, eliminating separate steps along the way. Enterprise Enabler is a leading edge federation and virtualization technology, combining EAI, ETL, ESB, and data orchestration to keep up with a constantly changing Big Data environment.

You've Got Big Data! Use it in the Cloud… Virtualized!

I get why Big Data is capitalized. It's even getting to the point where it could rate all caps, "BIG DATA." The part I don't get is why this is such a new big deal. Big Data has been around for a long time. Just think about the huge bodies of data that scientists have been gathering and analyzing just fine over the last who-knows-how many years.  The explosion of social media certainly brings a new dimension on rapidly growing, potentially valuable data, and poses the challenge of making it as valuable as the Big Data you already have.

The most valuable information in your company is probably not social media, but rather your corporate data. What about all that Big Data lurking in your corporate systems, data warehouses and across your multiple divisions? Your corporate Big Data is not like the scientific nor the Social Media Big Data:

o It's not in one place like the scientific data or the Social Media Cloud.
o It's across different organizations and geographies
o It looks different each place

I think of the Big Data focus as being on how to make it useful, not just for business analytics, but more importantly, to make it actionable to your elastic and agile enterprise. This means your solutions needs to be something that can be configured in very short time frames, which means eliminating custom coding. 

In order to be effective at leveraging your Big Data assets, integrating Big Data sources must be treated with streamlined transformation, federation, and virtualization, in an extremely unified architecture. This is necessary to provide the high performance requirement and is achieved by eliminating the classic steps through multiple components for extraction, transformation, staging to federate, and transforming again to align to the destination. It also requires the ability to address complex data manipulation with inline quality checks and error management.

o Access data from multiple disparate systems
o Leverage premises based and cloud based data virtually together in your own cloud
o Federate as data is being accessed from multiple sources, lookup tables
o Virtualize your data. Make it consumable by applications on premise or in the cloud live, virtually. This means that there is never a copy made of your data. (Of course, if you should want to actually send data, that's fine, too!)

Now, about that cloud…

Leveraging the cloud brings a host of opportunities not usually found within the walls of a corporate enterprise. The cloud brings an elasticity to try new architectures, explore new applications and meld your legacy data with Social and New Media for breakthrough business tools such as Predictive Analytics.

The trouble with the cloud is that we are concerned about maintaining the security of our data.  We feel this new technology should bring a tectonic shift in the way we think about integration to the cloud and in the value it can bring, without paying the generally accepted price.

To the best of my knowledge, there is only one product on the market that actually can do this: Stone Bond Technologies' Enterprise Enabler. With Enterprise Enabler you have the solution for Federating, Virtualizing and Leveraging your Big Data in the cloud.

o Your Data remains your data. Your data is never physically copied or placed to the cloud or anywhere along the way. It is accessed, transformed, aligned and virtualized in a single execution
o The data integration is end-user aware. The end user of the cloud application will see only the data he has authority to see.
o Your data is immediately actionable. Users of the cloud applications can update and add data on their screens, and the data is passed back to the source for updating, provided the user has authorization to do that.

A use-case example is a financial institution using this technology to supplement the Salesforce.com data with on-premise information that needs to be seen by the loan personnel alongside the cloud data.