Integration framework of Dynamics 365 for Operations

 

Integration framework: In D365 the integration framework is a framework that enables data flow between different application. It starts with Data Entity. Data Entity is a new concept in Microsoft Dynamics 365 for Finance and Operations. A major part of the integrations is based on Data entity that is deeply embedded at the kernel level.

Data Entities: According to the MSDN, Data entities provide conceptual abstraction and encapsulation of underlying table schema that represents data concepts and functionalities.

Data Entities are updatable views. They have replaced previous concepts in Microsoft Dynamics AX, including AXD, the Data/Import Export Framework (DIXF), and aggregate queries. Data Entities provide a single pack of records that capture business logic and enables scenarios like import and export, integration and programmability. It can also be exposed as OData services for some other third-party integration. 

In D365, when we create a new customer through a Customer Form we not only provide basic information about the customer, but also other important additional information, like addresses, credit limit, tax details etc. All the records internally don’t get stored in a single table. It is scattered among a small set of related tables (LogisticPostalAddress, LogisticElectronicAddress, DirParty etc.). During a customer record creation from an external application we must take care of all the mandatory fields across all those related tables. A data entity encapsulates all these tables and exposes the fields that we want to update. The customer data entity is an updatable view in which each row contains all the data from the customer table and its related tables.

There are few highlights of Data Entities -

- Microsoft shipped 1800+ out of the box Data entities and counting.

 - Customer/ISVs can extend all out of the box existing entities or add new entities based on their business requirement.

         --  Out of the box Sales Order entities can be used and consumed from any third party  application for further processing. 

         --  ISVs can create new Data entities and use those across their integration.

          For example, an ISV is developing an interface or add-ins where separate configuration and master tables need to be updated from other .NET based application. In this scenario the ISV needs to create new Data Entity as per the requirement

 - Composite entity is another useful and interesting feature of D365 that I mentioned above.

For example, a Sales Order document consists of several tables in D365. In this process, there are some mandatory fields and some optional fields. A composite entity takes care of that. A composite entity contains only the fields that you want to update from an outside application. If you are using Data Entities then missing on a mandatory field will not return any run time error. It will be handled internally by D365.

There are a couple of ways Data entities can be created.

a.     Using out of the box Wizard

b.     Programmatically using API

(Ref : https://docs.microsoft.com/en-us/dynamics365/unified-operations/dev-itpro/data-entities/build-consuming-data-entities)

Data entities are exposed to the outer world through Data Management Platform and OData Services. For an Asynchronous Integration Data Management Platform is used and Synchronous integration uses OData service.

Data Management Platform:

Data Management Platform is a platform that facilitates data exchange between systems. Any data that goes through Data Management Platform goes through staging tables.

This platform is used for File-based integration and Application lifecycle management.

The data management framework allows us to:

·        Move data between two similar systems. For example, between D365FO Development and Production.

·        Maintain a reusable library of data templates and datasets.

·        Use data packages to create incremental data entities. Data entities can be sequenced inside the packages. You can name data packages, which can be easily identifiable during import or export. When building data packages, data entities can be mapped to staging tables in grids or by using a visual mapping tool.

·        Validate and compare view data during imports.

File based Integration – Like any other ERP applications, a file import and export are an important feature in D365. This feature can be used for various purposes. For example, any hourly machine rate updates, sales tax rate updates, exchange rate update etc. There are generally two types of integration.

a.  File Import/export - This is an interactive and attendant integration. A rate table, sales update, employee timesheet are some typical examples of that. Initial data migration, copy configuration are some examples of it.

b.  Recurring integration – This is an unattended asynchronous integration, end of the day kind of batch process. Daily export of sales data, weekly GL export in a file and folder are some examples. 

Application lifecycle management – In an implementation, we often need to copy the master data and other configurations related to settings. Also, data migration between UAT environment and production environment. Application lifecycle management takes care of all these.

a.   Copy integration – Data Migration Framework can be used to copy configurations data          across companies or environments. It also can configure processes or modules using              Microsoft Dynamics Lifecycle Services (LCS).

b.   Data migration between systems – This is a data migration or initial data load scenario.          After a successful UAT testing of D365, the environment will need to roll over to the              production.

Microsoft Dynamics Lifecycle Services (LCS) is used for both the above scenarios.

OData Service : OData is a standard protocol for creating and consuming data. OData uses web technologies such as HTTP and JSON to provide access to information from various mobile or third-party programs. A Data entity is automatically exposed as OData service. Once done Office integration and mobile integration can talk with D365 through OData REST based endpoints. This is a synchronized integration between D365 and any real-time mobile or office apps.

OData also worked with Power BI. Power BI Desktop connects to the OData feed and accesses the available tables and other data elements.

Custom Service :  The last bit of integration is the custom services from the old flavor of AX 2012. Custom service features are still supported in D365. Out of the box or custom X++classes can be exposed as custom service to cater with any custom integration requirement. These custom services are consumed from outside applications to read and update D365 tables based on custom business scenarios. In D365 while creating a custom service and exposing it, it automatically creates two service endpoints.

·       JASON/REST endpoint

·       SOAP/XML endpoint

Custom services are used to integrate with third-party applications and add-ons. In a specific business scenario where an integration is needed with other ERP systems like SAP, custom services are generally used for handshaking.  Based on business needs and technology preference D365 integration is possible across the environments.

Web Service solution for Microsoft Dynamics family

Microsoft Dynamics has come over a long way with respect to technology. It is an amazing experience to witness the journey of a beautiful product range from all proprietary languages to Microsoft .NET compatible platform. Now we, the developers and architects have the ability to connect all the products and establish a business logic in the service tire. We can access all the native ERP code base, forms, and tables and update as and when needed.

In this article, I want to put forward a concept that I have been cultivating for some time. Playing around with Dynamics 365 has made me more curious to explore the concept, I foresee in near future that all Dynamics products will be coming into a single platform. This could be helpful for any Microsoft Dynamics service company that has the same integration for different Dynamics products. In this discussion, I am going to include MS Dynamics AX, MS Dynamics CRM (365), Dynamics NAV and Dynamics GP.

Let us take an example of a service organization that works on common Microsoft Dynamics add-on. For example exchange rate. Exchange rate and taxes are important factors for any organization with Global business. All Microsoft Dynamics products support these feature extensively. However, both the exchange rate and taxes need a continuous update and run-time determination.

In our discussion let us consider that an organization developed connectors for all the following Microsoft Dynamics products. All the products work on different programming languages. –
 1. Microsoft Dynamics AX – native programming language is X++. However, with Dynamics 360 we use the Visual studio for programming.
        a. Retail and Point of sales – MPOS – Dot Net.
        b. Mobile application
2. Microsoft Dynamics CRM – Dot Net based CRM application.
3. Microsoft Dynamics NAV – native programming language CAL.
4. Microsoft Dynamics GP – Dexterity is used to customize the application. Architecture:

Dot NET Web Services:

Let us assume that the organization built up a robust .NET tax engine and database. That is exposed as a web service. The web service is very generic and based on the parameters. It queries the master database and returns the result either for tax rate or exchange rate. We call this the Master Web Service, which is hosted in the service provider’s cloud.

Another secure web service is going to take care of product specific parameters as input, validation of the transaction based on the parameters and fetching the result from the above web service and return it back to the respective Microsoft dynamics product. This web service can be hosted in the same cloud or can be shipped as a package that customer can implement to their own IT infrastructure. That intelligent web service determines the source product and based on that it will interact will the respective databases and codebases of that particular product. For example –

  Source = parameter that will be passed from the respective Dynamics products when it will call the service from the respective codes.

if (source = ) 
{ 
        // work with AX.svc 
        // update the tables and classes. 
} else if (source = ) 
{ 
       // work with CRM.svc 
       // update the tables and classes. 
} else if (source = ) 
{ 
       // work with GP.svc 
       // update the tables and classes. 
}

//The generic section that will cater with common business logic and data validation.

 Microsoft Dynamics: 

All Microsoft Dynamics products provide us an opportunity to work with web services from the respective native codes. To consume services over the internet, we must host services on Internet Information Services (IIS). Services that are hosted on IIS use the WCF message routing service. Through the service, we can expose a page/query/class/tables as a SOAP web service. We can create custom service with our custom code to cater to our solution requirements.

In this service, there will be three sets of actions. 
       • First one will validate the data that come from the EPRs or CRM. 
       • Second one will calculate the rates for all valid transactions. 
       • Third one will push back the result to the respective ERPs or CRM and let the product handle rest. 

From Microsoft Dynamics point we are going to work extensively with the inbuilt feature to work with web services. We want to modify native Microsoft Dynamics products as less as possible. We do not want to add, modify or delete a single native table field of the Dynamics product. All will modify are codes (classes/methods/functions etc.). 

Dynamics AX (AV.svc) :

Microsoft Dynamics AX supports Custom services or Document Services that expose AX functionality as web services. Microsoft Dynamics AX still runs on X++. In Dynamics 360 we have the opportunity to code X++ in Developer Studio. Services that are hosted on IIS use the WCF message routing service. These services are consumed from any dot NET application. Custom Services are used to expose very simple business entities as a service or to expose custom logic from Microsoft Dynamics AX. For example, address validation functionality for a customer or vendor. Document Services are used to expose business entities with varying degrees of complexity or to support medium-to-complex integration scenario involving business entities. For example, exposing any existing AX class or query to update tax detail. During a transaction process currency converter service be consumed from outside application or service and update the latest rate the table/class of AX. 

Dynamics AX Retail POS (POS.svc): 

AX POS and MPOS are mainly developed in .NET. From a .NET application is it easier to work with a Web Service. During a POS invoice transaction, the .NET code can interact with the service at the run time and fetch the tax rate or the exchange rate. This will use the same Web Service that is developed for the core AX Sales Order or Invoice. The service can update the tables directly or particular class based on the requirement. 

Dynamics NAV (NAV.svc): 

Microsoft Dynamics NAV 2016 supports creation and publishing of Microsoft Dynamics NAV functionality as web services. We can expose pages, code units, or queries as web services, and even enhance a page web service by using an extension of code unit. When we publish Microsoft Dynamics NAV objects as web services, they are immediately available on the network. A code unit web services provide us control and flexibility. When a code unit is exposed as a web service, all functions defined in the code unit are exposed as operations. We will use this service to integrate our business logic and accordingly update the tax rate or exchange rate at runtime in the service so that it updates the code units appropriately. 

 Dynamics CRM (CRM.svc) :

Web service is an integral part of Microsoft Dynamics CRM. Microsoft Dynamics CRM provides two important web services that are used to access CRM from an external application and invoke web methods to perform common business data operations like create, delete, update, find etc. in CRM. IDiscoveryService and IOrganizationService Web Service are used to work with external applications. IOrganizationService web service is the primary web service used for accessing data and metadata in CRM. 

Dynamics GP (GP.svc) :

Like other Microsoft Dynamics products, GP comes with Web Services Integration. GP comes with few out of the box web services that we can expose to consume from a .NET application. Some of the examples are Sales Documents, Update Customers etc. We can create new web services from GP as per our requirement. In our example, we can expose Sales Documents web service and create a method to cater to the exchange rate. The master web service can be consumed from the .NET application and based on the parameters it will update GP tables and processes. 


This is a draft concept of the solution. I know lot need to be explored and a lot can be done. I am excited that Microsoft is offering us to explore a lot in terms of technology and flexibility. I welcome your feedback and wisdom on this concept.

AX 7.0 Technical enhancement - Development and Deployment.

I am sure that we all are excited about AX 7.0. I have traveled a long way with AX. My journey began with AX 4.0. At that time, we were totally dependent on X++ and MorphX. Now the situation has changed. Microsoft already made our lives very easy with the new development interface. The biggest advantage being is to play with services. Handshaking between two applications is now possible and you can create your own .NET based application and incorporate it into your rich AX application.

Recently I started exploring AX 7.0. Today I am going to share some of the key excitements I felt related to the AX7.0 within this very small span. Let’s discuss a few areas of AX 7.0 deployment which I found interesting.

In AX 2012 Microsoft introduced a great concept called Model. Model is basically a container for metadata elements. Metadata in AX is basically the objects, like class, form, labels etc. You can import and export a Model very easily. Model is widely used in AX world and especially by ISVs.

In AX 7.0 there is no change in the concept of Model. In AX 7.0 a new deployment concept has been introduced by Microsoft was known as Package.

AX 7.0 key deployment concepts:

Project - simple Visual Studio project that stores AOT elements. A Project always belongs to a particular Model. AX 7 Projects are smaller than AX 7.0 Models in size. Therefore, Projects can be moved easily between development environments. A complete Model compilation is not needed for a simple development activity.

Model is a group of elements (metadata and source files) that typically constitute a distributable software solution (including customization of an existing solution). For example, an Inventory module, Accounts Receivable module with tax and shipping customization.

Package is simply a collection of compiled binary versions of models that can be deployed on the cloud. A package can be created in Visual Studio. Single or multiple AX 7 packages can be incorporated into a big deployment package. Mainly used for a big deployment, like moving an environment from staging to test and then to production cloud.

Packages are folders located in the model store folder of the AX 7.0. Typically the default model store folder is “c:\packages\”. Model folders are sub-folders in their package folder. Every sub-folder represents an element type: a table, a form, an EDT etc. For example, all forms in a model belong to the sub-folder Ax Form.

A sales tax model folder belongs to the Sales Tax add-on package folder.

Let us discuss a scenario over here. In an implementation, there are four ISV add-ons along with some additional customization involved.

Model 1: contains project 1 and project 2.

Project 1 contains all customization related to HR (ISV).

Project 2 contains all customization related to Finance.

Model 2: contains Project 3 that is related to an ISV for sales tax for Accounts Receivable and Project Accounting.

Package 1: model 1 + model 2. A package is basically an association of assembly and associated runtime files.

Model 3: contains two projects related to customization of inventory and CRM module.

                Project 3 contains a bunch of custom services related to Microsoft Dynamics CRM real-time integration (ISV).

                Project 4 contains customization related to inventory.

Package 2: contains model 3.

There is a dependency of Package 2 on Package 1. In that way, you can deploy multiple packages to Azure establishing a dependency.