Denodo Architecture I Complete Guide On Denodo Architecture


Denodo is a data virtualization tool that helps organizations to integrate all their business data. In today’s world, it’s a complex task to manage business data. Denodo makes jobs easier as it offers a unified view of the data to the various business applications. It also helps to guarantee the governance rules and policies across the data delivery chain. Denodo also helps to enrich the core master data from MDM (master data management) with real-time data from various sources such as Salesforce, and web service and it also provides a complete view of enterprise data.

                              Interested in learning Denodo Join HKR and Learn more on Denodo Certification Course!

Why use Denodo?

Till now we have just learned that Denodo is a data virtualization tool. Now let’s know the reasons why Denodo is so much in demand?

Here we would like to mention a few points ;

Integrated with cost-based optimizer 

Based on data value estimation and the cost of these particular operations, the CBO (cost based optimizer) can decide to move all parts of the execution tree to the MPP query accelerations.

Offers partial aggregation push down

  • It will maximize the source processing.
  • Dramatically reduces network traffics.

Fast parallel execution

Denodo supports spark, presto, Hive, and Impala for fast analytical processing in in-expensive Hadoop-based solutions.

On-demand data transfer

Denodo automatically generates and uploads parquet files.

Agile business intelligence

  • A logical data warehouse.
  • Virtual data marts.
  • Self-services.

Denodo-a single view application

  • single customer view
  • single product view
  • Single inventory view
  • Vertical specific

Big data and cloud integration

  • Advanced analytics
  • Data warehouse offloading
  • Big data for enterprise
  • Cloud and SaaS product integration

Denodo Training

  • Master Your Craft
  • Lifetime LMS & Faculty Access
  • 24/7 online expert support
  • Real-world & Project Based Learning

Data services

  • The unified data services layer.
  • Logical data abstraction.
  • Agile application development.
  • Linked data services

An Architecture is essential to explain the overall workflow as well as the components used. In this section, we will be explaining the architecture of Dendodo and its components;

These three components help to deliver a holistic view of enterprise information to the business users across all of the underlying source systems. The Denodo data virtualization tool connects to all types of data sources. The data sources can be databases, data warehouses, cloud applications, big data repositories, and even excel files. In the data virtualization, the platform also combines the related data into the business views irrespective of their data formats such as relational databases, no SQL, Hadoop, Webservices, Cloud APIs(application programming interfaces), and files, etc. Data virtualization enables business users to consume data through reports, dashboards, portals, mobile applications, and web applications.

The consumer layer enables a single point of access and interaction with the underlying data sources as well as abstracted data views in a standard delivery format. The consumer layer offers the broadcast data delivery options to suit business user needs via JDBC, ODBC, ADO.Net, SOAP web services and RESTful API services, and data widgets.

Get ahead in your career with our Denodo Tutorial!

HKR Trainings Logo

Subscribe to our YouTube channel to get new updates..!

Denodo is a unified, virtual data layer with which enterprise applications and users can access any enterprise information regardless of its location, format, or protocol, using the methods that best suit their work needs such as data discovery, and search                                                                   

Key features of Denodo

  • The following are the key features of Denodo;
  • Enhanced, unified, and web-based user experience.
  • Performance acceleration for complex analytical queries in Logical data warehouse and data fabric scenarios.
  • PaaS support for cloud and hybrid environments.
  • Enhanced data services APIs with graph-like access to Denodo views.
  • Going beyond the logical data warehouse: new support for data science and machine learning.
  • Enhanced user experience in the data catalog.

Common scenarios where Denodo is required

Here we are going to discuss the problems where we need the Denodo tool:

Let’s make a list of them;

  • You are working in a company that has information about the customers (CRM) which is stored in MySQL databases.
  • A Denodo is needed to maintain the organization’s data and ensure that the data is saved to perform various business operations.
  • A Denodo is required to maintain the complete billing information for any customer is exposed in an internal web service.
  • When end users decided not to use different applications to restore the available information of customers (CRM, sales applications, etc).
  • The IT companies don’t like the data of creating a specific application for this business need and it would like to reuse this customer global view, if possible in any other current applications.

Installation and Bootstrapping: A pre-installation requirement

Once you are done with learning the Architectural overview of the Denodo, you might be thinking about the Denodo installation.
Here we are going to guide you on how to install and bootstrap teh Denodo;

Steps required to configure the Denodo:

Product name

Denodo platform 8.0 -> go to the website https://community.denodo.com/express then download the install, and accept the license.

Product name

MySql Version 5.0 or higher -> here you will need MySql to store some of the data used in the tutorial. You can download MySql from the website http://dev.mysql.com/downloads/mysql/.

Denodo Training

Weekday / Weekend Batches

 

Product name

MySQL connector/j version 5.0 or higher -> this is the JDBC driver that will enable Denodo to connect to MySQL.
http://dev.mysql.com/downloads/connector/j/ use this link to download the MySQL connector/j.

Product name

MySQL Workbench version 5.0 or higher -> these are the useful tools you might want to use with MySQL but are not strictly necessary for the tutorial: http://dev.mysql.com/downloads/tools/workbench/.

Product name

Denodo Tutorial files -> Download and extract the contents of the compressed tutorial files into a conventional local directory, which will be referred to as throughout this guide.

Final words

Maintaining the business data in any organization is a challenging task. Nowadays most of the top companies are looking for an innovative tool to manage the data transactions across the organization. Denodo is one of the popular data virtualization platforms that help to achieve the vision of logical data fibers. This serves as a unified data delivery platform that bridges the gap between the IT infrastructure and consuming business applications. We have designed the Denodo architecture post to help data analysts, and data scientists enhance their skill sets in the data virtualization platforms. This article also answers your question on why use Denodo? and what are the features that make it different from the rest? Stay tuned to our website for more blogs and we will keep you updated. Happy learning!

Related Articles:

  1. Denodo Data Catalog
  2. Denodo certification



Source link

Leave a Reply

Subscribe to Our Newsletter

Get our latest articles delivered straight to your inbox. No spam, we promise.

Recent Reviews


What are Data Catalogs?

A Data Catalog seems to be an accumulation of metadata especially in data planning and search tools that assists experts as well as other data consumers in locating the data they require, acts as a current asset of data available, and offers criteria to assess strength and conditioning data for potential purposes.

This succinct process made several locations regarding data catalogs—data management, looking, data inventory, and interpretation of data; they all rely on the central capacity to deliver a catalogue of metadata.

    Start learning Denodo Online Course from hkrtrainings to make a bright career in the world of Denodo!

What is the Denodo Data Catalog?

The Data Catalog seems to be a web-based self-service device that is included in the Denodo System that would allow technology and commercial consumers to query, lookup, and explore details and metadata saved in a Virtual DataPort server. Consumers could use this device to create knowledge and open the way for best choices.

If you want to explore the denodo server metadata, then use the dendo data catalog. Here I am going to explain how all these things happen.

Launching the data catalog:

The Data Catalog seems to be a web based application distributed as part of Denodo 8.0 that allows data analysts, enterprise customers, and app developers to search and browse data and metadata in a business-friendly way for personal exploration and predictive analysis.

To use this web tool, open the Denodo Platform Control Center and launch the Data Catalog. When the status changes to “Running,” click the Data Catalog link to launch the Web tool (by default, https://127.0.0.1:9090/denodo-data-catalog).

Login into the denodo platform with your login details.If you are logging for the first time, you will see a pop up window showing the synchronize metadata option.This must be run the first time you start the Data Catalog to make sure that it reflects the most recent state of the Denodo 8.0 server to which you are connected.

The VDP Synchronization should be performed as follows:

  • Click the Synchronize metadata now button.
  • On each Synchronization step, click Continue.
  • The views have now been synchronized, and you can begin exploring!

Using the metadata search:

The first example comes from the Data Catalog’s home page.

Let’s use the Business Analyst’s scenario to look at a simple use case: searching for clients by typing in client and pressing enter.

denodo data catalog

Here are the outcomes of our search. Starting with Data Catalog 8.0, this search will look for views or web services that include the query terms in the element’s metadata, such as:

  • It has a name.
  • It is described.
  • The names of the fields in which it operates.
  • The descriptions of its various fields.
  • The values of any custom properties that have been assigned to it.

denodo data catalog 1

For instance, let’s click on the view client to go to the summary of the selected view:

denodo data catalog 2

For the time being, we have conducted a search in the Virtual DataPort metadata. In the following section, we’ll look into the Data Catalog’s more advanced features!

Denodo Training

  • Master Your Craft
  • Lifetime LMS & Faculty Access
  • 24/7 online expert support
  • Real-world & Project Based Learning

Exploring the features of Data Catalog:

We’ll now look at the features in the Data Catalog that allow for more in-depth interrogation of a view. This includes the following:

  • Filtering and querying results from a view
  • Results are being saved to a file.
  • Developing new fields
  • Queries saved
  • Investigating viewpoint relationships
  • Investigating data ancestry
  • Views with related fields can be queried

Exploration of Data Catalog Views:

We chose our client view from the previous section. We can now look through the contents of this view.

Summary Tab:

We could see a summary of the selected view under the Summary Tab. It will display the metadata of the selected view, such as the database name, the list of categories, the list of tags, and collaboration information provided by the user, such as Endorsement and Warnings. You can edit the view’s description by clicking the Edit button next to the Description option. If the view is deprecated, an indication will appear at the top of the summary tab.

Furthermore, the Summary tab includes buttons such as Add Tags/Categories , Collaborative effort possibilities further to create custom the view, and Connection URLs, Tableau to display different opportunities to link to the view/datasource.

denodo data catalog 3

Schema tab:

Under the Schema Tab, we can see the view’s schema, which includes the view description as well as all of the fields and types. We can add a field description by clicking the Edit button next to the column. We can also use the search option at the top of each section to look for fields, data types, and descriptions.

denodo data catalog 4

Query Tab:

The Query Tab is the following tab. Ad-hoc queries can be run against the view here (the query is created graphically).

Business Intelligence & Analytics, denodo-data-catalog-description-3, Business Intelligence & Analytics, denodo-data-catalog-description-6

Subscribe to our YouTube channel to get new updates..!

Select all of the following fields for our view and drag them into the Output columns area.

client_id

name

surname

Client_type

denodo data catalog 5

Click on the execute to get the results.

denodo data catalog 6

Editing of data catalog metadata:

In this segment, we would then look at the metadata characteristics of the Data Catalog. Users could use Denodo Data Catalog to append tags and categories to views, and also keep updating the view as well as field descriptions, with such a function.

In our instance, we will: (1) add explanations to the customer fields in addition to allowing for more particular discovery of such a view, (2) add tags and categories, and (3) implement those to our client view.

Data catalog Metadata:

The capacity to exhibit view metadata, such as the View Description and Field Descriptions, is a good feature of the Data Catalog. Then see how we can make that data more modifiable.

Editing view and field descriptions:

  • Browse to the Client View’s Summary page and click the Edit option beside Description.
  • Add the necessary descriptions to the View and then click Ok.
  • Similarly, you can add a description to fields by going to the Schema tab and clicking on the Edit button next to each one.
  • The new descriptions are now visible in the view. These descriptions are saved as metadata in the Data Catalog.

Adding of tags and categories in data catalog metadata:

Tags and Categories are useful for allowing users to search the Data Catalog more precisely. While the number of Data Sources and Views in our tutorial is small, maintaining good Categorization and Tagging habits will pay off in the long run by allowing users to navigate the Data Catalog more easily.

In order to add categories follow the below mentioned points.

  • Navigate to Administration > Configuration and Management.
  • Click the Categories option in the Administration window’s Catalog Management section.
  • Select the + Add Category option.
  • Make a category with the following information:
    • Customer’s name
    • Customer data sources
  • Make a new category with the following information:
    • CRM (Customer Relationship Management)
    • Customer is the parent of the Acme crm System.
  • Make a final category with the following information:
    • Name: Billing 
    • Description: Billing
    • Parent:customer

Denodo Training

Weekday / Weekend Batches

Adding tags:

  • Navigate to Administration > Configuration and Management.
  • Click the Tags option in the Administration window’s Catalog Management section.
  • Click the + Add Tag icon to add a new Tag with the following information:
    • Name:JDBC 
    • Description: JDBC data sources
  • Creating another tag with the following data.
    • Name:SOAP
    • Description:SOAP data sources

We can easily modify the views for adding tags and categories as well.

                            Get ahead in your career with our Denodo Tutorial!

Recommendation and collaboration in data catalog:

Recommendations in data catalog:

The AI Feature Package includes Automatic recommendation of datasets in the Data Catalog to assist you in discovering new elements among your company’s data resources.

This feature displays individualized recommendations based on earlier activity in the Data Catalog, like datasets which are most utilised, lately used, suggested, and so on.

Go to the Data Catalog’s homepage to see the recommendations.

The homepage displays a collection of products provided by various topics, such as one titled Recommended to you. This dataset recommendation is really only accessible only with the AI Feature Pack.

Collaboration in data catalog:

In the collaboration there are 3 options such as endorsements, warnings and decrepation notes.

Endorsements seem to be comments made by users on a view or a webservice to express their support. A user can only endorse a perspective or web service once, which means whenever a new comment is added, the prior endorsement is removed.

Warnings have been used by customers to write and exhibit “advise against” texts on opinions and web services. A consumer could only add one warning to a view as well as web service.

Deprecations have been used to notify people that a feature has become outdated and should no longer be used. A consumer could only write one deprecation for a perspective as well as web application.

Conclusion:

In the above blog post we had clearly discussed the dendo data catalog, data catalog metadata, adding of tags and categories, recommendations, collaboration etc in a more detailed way. If you have any doubts please drop your query in the comments section to get them clarified.

Other Related Articles:



Source link