Developing and Testing Plug-ins in Dynamics 365 - Part 3 - DataMigration Utility

In the last part of this series we looked at using an option set (drop down) to store the list of relevant search engines, it turns out that our customer wants to have both the name of the Search  Engine and its URL.

1. Using a Lookup instead

We could add two fields but that's a bit clunky so we will add a new Entity called Search Engine and a new look up from the account form.

The entity has two fields:

Name (new_name)
URL (new_url)

I have also added a new 1:N relationship to the account entity

Search Engine (new_searchengineid);

The Dynamics 365 solution can be found here.

The plug-in has been modified like this

We have landed ourselves in some trouble when it comes to unit testing again with this code as GetSearchEngine depends on the IOrganizationService, which we can't mock as discussed previously.

I've refactored the code here to remove this dependency and with unit tests.

2. Test induced Damage

Let's say that we really wanted to use mocks, so I have refactored again to allow this

The solution is here, with account plug-in and test

In order to be able to mock the data (Search Engines) I have created a ISearchEngineRetriever that has the GetSearchEngines method and inject this interface on a new class SearchEngineFinder that has our old trusty GetCorrespondantSearchEngine method.

This allows to mock away the GetSearchEngines method.

This is silly, don't do it.

You could argue that this would allow you to, in the future, inject a different ISearchEngineRetriever  if you wanted to get the data from somewhere else, and that would be true but why worry about that eventuality when it might never happen and if it does happen it's unlikely to happen in the way you've anticipated.

If you do know that this data will come from another source in the future, then may be something along these lines would be reasonable, maybe.

3. Data

There is a problem in the approach that we have taken, namely adding a new entity as we now need to have that data (Search Engines) in the production environment (as well as test, etc..)

Luckily we have tools, see this post for more details

We will use the Data Migration Utility to export from our dev environment into our other environments

1. Fire up Data Migration Utility (DataMigrationUtility.exe)
2. Select Create Schema

3. We will need to Log In
4. Select Solution
5. Select Entity
6. Add Fields to be exported, new_name and new_url have been selected here.

7. Click Save and Export
8. Select appropriate files for the schema and data

9. Finished !!!

The advantage of this method is that the Ids (Guids) of the entities will be preserved across environments, which means that Workflows will continue to work seamlessly, more on this here.

A Guid has 2 ^ 122 possibilities (there are six bits that are reserved) so it's extremely unlikely that duplicate Guids will happen.

To import

1. Fire up Data Migration Utility (DataMigrationUtility.exe)
2. Select Import Data
3. Select data file and Click Import Data

Developing and Testing Plug-ins in Dynamics 365 - Part 2

In part 1, we made a start into the fabulous world of plug-in development on Dynamics 365. In this post we modify our plug-in by first using an option set and then using lookup.

1. Add Search Engine Option Set

It turns out that we need to store the related Search Engine according to our company's magic sauce for determining the company's Search Engine(namely does the first letter of the company name match a known search engine) and not the url.

So we create a new field to store this

1. Navigate to an account and click Form

2. Click New Field

3. Save And Close
4. Refresh Form
5. Add New Field to Form
6. Save
7. Publish

2. Alert Code to use New Option Set

The code has been now changed see full solution here, plugin here and unit test here but there is a major flaw, which is that any new search engine that might need adding will require code changes, which is clearly not an acceptable state of affairs.

We can see the results here:



3. Metadata FTW

Instead of hard-coding the values, we will retrieve the values from Dynamics, which will ensure that we have an up to date list.

This does present us with a bit of a conundrum, in that we will not be able to unit test this any more to the same degree as before. To be clear, we can create unit tests but ultimately if there is an edge case that our algorithm misses then it might be missed by our unit tests.

Let's say that if the first letter of the company is D, then it our magic sauce, should also return no selection, at the moment, see code here, this has not been implemented and thus will not be tested for,

Ultimately the only way to solve this is to have integration tests, where we actually query our integration test Dynamics instance.  The problem is that this will make the tests slow and if you want to run a lot of them, and you generally do, then that is a problem, so we have to compromise

This has been implemented here, with tests here and the full solution here.

4. Mocking Metadata

Let's say that we are not happy with the approach that we've taken previously and that we want to mock our dependencies.

We can do something similar to this and this is how it would be tested (full solution) Note that it's not working as I cannot mock RetrieveAttributeResponse object easily (AttributeMetadata is read only) so reflection is called for.

In reality this is somewhat pointless as there aren't likely scenarios that would require this level of complication for little or no benefit.

Furthermore, somebody has already done the hard work, which will be discussed on part 5.

An alternative would be to inject an interface, say ICRMRepository to GetSearchEngineOptionSet, so that ICRMRepository.GetSearchEngines could be mocked to return the desired data, but I think that this is a completely unnecessary complication that adds no benefit.

If you are reading this and thinking:

but if you had an interface in the future  you'd be able to do ...

I would counter that it makes little sense do extra work now to potentially save work later as we are sure to never get that time back but we're not sure whether we will need the extra work.

Developing and Testing Plug-ins in Dynamics 365 - Part 1

In this post I will introduce what a plug-in is, how to develop, deploy and test plug-ins.

Pre-Requisites

• Visual Studio 2017 Installed
• Admin Access to a Dynamics 365 instance

What is a Plug-in?

A plug-in is custom business logic (code) that you can integrate with Microsoft Dynamics 365 (online & on-premises) to modify or augment the standard behavior of the platform. Another way to think about plug-ins is that they are handlers for events fired by Microsoft Dynamics 365. You can subscribe, or register, a plug-in to a known set of events to have your code run when the event occurs.

So there we have it, it's definitely not the most user friendly name but it's here to stay, I guess better than callouts (Dynamics CRM 3.0 FTW)

1A. Tooling

In order to work with Plug-ins we will need to install the Plugin registration tool

1. Open Powershell
2. Navigate to the folder you want to install the tools to
3. Copy and paste the following PowerShell script into the PowerShell window and press Enter.

A Tools folder with four sub folders will be created:

• ConfigurationMigration
• CoreTools
• PackageDeployment
• PluginRegistration

The last one is the one we are interested in.

1B. Tooling

In order to be able to register plug-ins and do a large set of operations, we need to use the PluginRegistration tool

1. Open the tool from ..\Tools\PluginRegistration\PluginRegistration.exe
2. Click Create New Connection 
3. Tick Show Advanced
4. We enter user account details
• Online Region
• User Name
• Password
5. Click Login

The connections details will be saved so that next time they will be available.

2A. Project Preparation

A standard library (dll) project is needed for Plug-ins, additionally the project will need to be signed.

1. Create new Class Library (.NET Framework) Project called Plugins in Visual Studio and set Framework version to 4.5.2. 
2. Right Click Plugins project and go to Properties (Alt + Enter might work)
3. Click on Signing
4. Tick Sign the assembly
5. On the drop down, select <new>
6. Give the key a name and untick Protect my key file with a password
7. Click OK

2B. References

We are going to need some references to the SDK in order to create our first plugin

1. Right Click on References
2. Select Manage Nu-Get Packages
3. Click Browse
4. Search for Microsoft.CrmSdk.CoreAssemblies
5. Click Install to install Latest version (9.0.2.4 at the time of writing)

3. Development

The project is stored here and the plug-in code can be found here

The plug-in is really simple, it will set a search engine as the url for a new account if the first letter of the account name matches the first letter of a known search engine, empty otherwise.

We will register this in the pre-create operation so that it only fires on account creation.

One thing to note is that as the plug-in as it's currently written cannot be unit tested very easily or at all really. We will explore this in more detail on section 6.

4. Register Plug-in

Registering a plugin has two phases:

1. Register Library
2. Register Steps

The first step refers to installing the library on the server, while the seconds configures the events for which the plug-in will be triggered.

Start the Plugin Registration Tool

1. Click Register and then Register New Assembly


 
2. Select the plugins assembly (Plugins.dll) and click Register Selected Plugins



3. Locate Plugins Assembly, right Click and then select Register New Step



4. We fill in the details as shown below and Click on Register New Step

Note how the plug-in has been registered to trigger on the Create event (message in CRM parlance) of the account Entity and it will happen synchronously before the write operation. This is needed as the plug-in modifies the value of the entity and then the modified value is saved.

5. Test Plug-in

We create a record and save it. The website field is populated.

The problem is that the current solution is hard to unit test so let's try to improve that

6. Unit Testing

In order to make unit testing easier, we refactor the plug-in. The whole project can be found here and the changes can be found here.

Note how nothing really has changed, except that we can now easily unit test the plug-in, by extracting out the method.

The sole unit test is failing at this stage as we failed to take into account capitalisation when matching the first letter, hurrah for unit tests!!.

The fixed project is here and the fixed code is here.

The last comment I will make in this section is the Test Driven tests that I have used, see GetSearchEngineTest test method here.

This is a very powerful feature as it allows us to write a single unit test and then pass different inputs into it.

This is what it looks like on Test Explorer, which allows you to see which one is failing if any.

In part 2, we will look at how we can remove the hard-coded search engines to improve the code and also introduce more Dynamics 365 functionality

Azure Cognitive Services and Dynamics 365 - Part 2

In part 1, we looked at how we could integrate Azure Cognitive Services with Dynamics 365 to provide keyword search suggestions for Knowledge Based articles.

In this second part we discuss the Similar Records Suggestions Feature

Important

At the time of writing, the integration of Cognitive Services with Dynamics 365 is in preview mode and the preview is only available for North American instances.


In order to configure the integration, see sections 1 to 3 of part 1


1A. Add Advanced Similarity Rules

1. Navigate to Settings | Data Management | Similar Records Suggestions
2. Click New
3. Fill in details, ensuring that Use Text Analytics for Target Match is set to Yes and entity is set to Case
4. Click Save

1B. Add Match Fields

On the same record as section 1A, we click the plus sign as shown below and we add some Match Fields.

We set the same fields as in part 1, namely Subject of any regarding activity, title of any regarding note and description field.

Once this has been done, the similarity rule can be activated.

2. Testing It

We create a few cases with similar description and/or subject for tasks, title for notes.

When we click on Similar Cases: Find

This appears to have picked up the word fault on the description and used it to suggest it, not exactly what we were after but it highlights the posibilities

These are a complement to the duplicate detection rules although I have to admit that I've not managed to get them to trigger but the API in Azure is being invoked ...

Azure Cognitive Services and Dynamics 365 - Part 1

Microsoft Cognitive Services is a collection of APIs, which can be used to add artificial intelligence capabilities into applications. You can see see the full list here and play with the demos by clicking on the links therein.

Important

At the time of writing, the integration of Cognitive Services with Dynamics 365 is in preview mode and the preview is only available for North American instances.

1. Activate Text Analytics

In order to enable the integration follow these steps:

1. Navigate to Settings | Administration | System Settings
2. Go to Preview tab
3. Select Yes on Enable the Dynamics 365 Text Analytics Preview
4. Click OK

2. Create Text Analytics Endpoint in Azure

We need to create a Text Analytics service in Azure:

1. Login to Azure Portal
2. Click Create a resource
3. Search for Text Analytics
4. Click Create, which will take you to the wizard (of sorts)
5. Fill in:
• Name
• Subscription
• Location
• Pricing Tier
• Resource Group

Once the resource has been created we will need endpoint URL and an access key to access it from Dynamics 365, which we can get from the resource itself on the azure portal, as shown below:

3. Configure Dynamics 365 to use Text Analytics

The final configuration step is to add the details from section 2 to Dynamics 365.

1. Navigate to Settings | Administration | Azure Machine Learning Text Analytics Service Configuration
2. Fill in with details from the previous section: 
• Azure Service Url
• Azure Account Key
3. Click Test Connection
4. Click Activate
5. Tick Activate existing text analytics models and Click Activate

Note that the results of Test Connection will be displayed on the Connection Test Information section.

4A. Create Knowledge Search Model

1. Navigate to Settings | Service Management | Knowledge Search Field Settings
2. Click New
3. Fill in details
4. Click Save

4B. Create Keyword or Key phrase Determination Field

On the same record as section 4A, we click the plus sign as shown below and we add some rules

The rules determine which fields will be used to generate the default search strings for Knowledge Based articles.

In the example below we  have set up, perhaps somewhat redundantly the subject of regarding Tasks, Activities, the case description field and the title of any regarding notes as relevant keywords/phrases.





5. Add Knowledge Base Search Suggestions to Case

1. Navigate to Settings | Customizations | Customize the System
2. Expand Entities | Case | Form
3. Click on the Default Form: Case
4. Double Click on the Conversation Tab
5. Go to Web Client Properties | Knowledge Base Search
6. On Additional Options, tick Turn on Automatic Suggestions 
7. On Give knowledge base (KB) suggestions using, ensure you select Text Analytics
8. Save Form
9. Publish Entity.

6. Testing it

The feature will use Text Analytics to extract meaningful keywords to search for from the fields set up on section 4B

As mentioned on section 4B, in this example we  have set up,  the subject of Tasks, Activities, the case description field and the title of any notes as relevant keywords/phrases.

So in an example case, where we have a description and a task like this:



If we click on KB Records this is the result



It's used the fields to suggest the search words

I have to admit that I expected it to search the KB articles, but this is not what the feature is about.


7. Enable Knowledge Management on Other Entity

It is possible to do the same for pretty much any entity

1. Navigate to Settings | Customizations | Customize the System
2. Expand Entities | Account 
3. Tick Knowledge Management
4. Click on the Default Form: Account
5. Add Knowledge Base Search
6. On Additional Options, tick Turn on Automatic Suggestions 
7. On Give knowledge base (KB) suggestions using, ensure you select Text Analytics
8. Click Ok
9. Save Form
10. Publish Entity.

We will then need to create a Knowledge Search Model as per section 4.

Visual Studio 2015, Git, Aurelia and case Sensitiveness

The support team was complaining today that they couldn't reset user passwords anymore, which was surprising.

After a bit of digging there was this really useful error

Unhandled rejection (SystemJS) Template markup must be wrapped in a <template> element e.g. <template> <!-- markup here --> </template> Error: Template markup must be wrapped in a <template> element e.g. <template> <!-- markup here --> </template>

and a more useful part

Error loading https://....../dist/main/user/resetPasswordDialog.html!template-registry-entry

The issue is that there is discrepancy in case for the first letter of the view, so that it's looking for this:

main/user/resetPasswordDialog.html

but we actually have this:

main/user/ResetPasswordDialog.html


Simple error to fix, right?

This is where things get interesting.

Git's client on Visual Studio 2015 would not recognise the change of case on the file name as a change and thus I had a bit of a problem.

I tried making changes to the file to force the change but to no avail, in the end and to cut a long story short, it turns out that I could change the case directly on Visual Studio Team Services or should I say Azure DevOps?

In any case, I suppose this is a Windows thing  but it was very annoying as I thought I might need to get creative, e.g. spin up a Linux VM and clone the repo ...

Validation Travails with Aurelia-Validation

One of our testers finally got a chance to have a look at our Aurelia App, which is extremely rare and she found an issue where validation would not apply under certain conditions.

After a little bit of investigating, the conditions turned out to be editing ... sigh

This is the code:

import { ValidationController, validateTrigger, ValidationRules, ValidationError } from 'aurelia-validation';
import { BootstrapFormRenderer } from '../../validation/bootstrapFormRenderer'
import { stuff here } from ...
import { inject, NewInstance, computedFrom, BindingEngine } from 'aurelia-framework';
import { Router } from 'aurelia-router';
import { AuthService } from 'aurelia-authentication'


@inject(BindingEngine, NewInstance.of(ValidationController), Router, AuthService)
export class AddEditUser {
    private User this.user;

    constructor(bindingEngine: BindingEngine, validationController: ValidationController, router: Router, authService: AuthService) {
        this.bindingEngine = bindingEngine;        
        this.router = router;
        this.authService = authService;
        this.validationController = validationController;
        this.validationController.validateTrigger = validateTrigger.changeOrBlur;
        this.validationController.addRenderer(new BootstrapFormRenderer());
        this.user = new User();
    }

    activate(params, navigationInstruction) {
        this.editMode = Object.keys(params).length > 0;

        if (this.editMode) {
            this.service.find(params.id).then(user => {
                this.user = new User(user.Id,
                    user.FirstName,
                    user.MiddleName,
                    user.LastName,
                    user.Email,
                    user.UserName,
                    user.JobRole,
                    user.PhoneNumber,
                    user.UserRole,
                    user.ContactPreference,
                    null,
                    null,
                    user.Status);

            });
        } 
    }

    bind() {       

        ValidationRules
            .ensure("userName").required().maxLength(256).matches(/^[a-z0-9@\.\-\_]+$/i).withMessage("User Name can be up to 256 characters long. Only alphanumeric characters and . _ - @ are allowed.")
            .ensure("firstName").required().maxLength(64).matches(/^([a-zA-Z\'\-\s])+$/i).withMessage("First Name can be up to 64 characters long. Only letters, apostrophes, hyphens and spaces are allowed.")
            .ensure("middleName").maxLength(64).matches(/^([a-zA-Z\'\-\s])+$/i).withMessage("Middle Name can be up to 64 characters long. Only letters, apostrophes, hyphens and spaces are allowed.")
            .ensure("lastName").required().maxLength(64).matches(/^([a-zA-Z\'\-\s])+$/i).withMessage("Last Name can be up to 64 characters long. Only letters, apostrophes, hyphens and spaces are allowed.")            
            .ensure("email").email().withMessage("Provide a valid Email.").maxLength(256)
            .ensure("email").required().when((user: User) => user.contactPreference === ContactPreferenceType.Email).withMessage("Email is required when it's your contact preference")
            .ensure("phoneNumber").minLength(10).maxLength(12).matches(/^\d+$/).withMessage("Provide a valid Phone number. Only numbers allowed")
            .ensure("phoneNumber").required().when((user: User) => user.contactPreference === ContactPreferenceType.SMS).withMessage("Phone number is required when it's your ontact preference")
            .ensure("status").required()
            .ensure("role").required()
            .on(this.user);
    }

    public saveUser() {

        this.validationController.validate()

            .then((errors: ValidationError[]) => {
                if (errors.length === 0) {

                    this.service.update(this.user.userName, this.user.firstName, this.user.middleName, this.user.lastName,
                        this.user.email, this.user.phoneNumber, this.user.contactPreference, this.selectedTeam.id, this.selectedOrganization.id,
                        this.user.jobRole, this.user.status, this.user.role).then(result => {
                            this.userUpdated = true;
                            this.navigateBack();
                        }).catch(error => {
                            console.log(error);
                            this.failedToUpdateUser = true;
                        });
                }
            });
    }

    public addUser() {
        this.validationController.validate()
            .then((errors: ValidationError[]) => {
                if (errors.length === 0) {
                    this.service.register(this.user.userName, this.user.firstName, this.user.middleName, this.user.lastName,
                        this.user.email, this.user.phoneNumber, this.user.contactPreference, this.selectedTeam.id, this.selectedOrganization.id,
                        this.user.jobRole, this.user.status, this.user.role).then(result => {
                            this.userAdded = true;
                            this.navigateBack();
                        }).catch(error => {
                            console.log(error);
                            this.failedToAddUser = true;
                        });
                }
            });
    }

    public submit() {

        if (this.editMode) {
            return this.saveUser();
        }

        return this.addUser();
    }

}

And this is what I did to get it working, namely add the rules to the object after the object (user) was created.

activate(params, navigationInstruction) {
        this.editMode = Object.keys(params).length > 0;

        if (this.editMode) {
            this.service.find(params.id).then(user => {
                this.user = new User(user.Id,
                    user.FirstName,
                    user.MiddleName,
                    user.LastName,
                    user.Email,
                    user.UserName,
                    user.JobRole,
                    user.PhoneNumber,
                    user.UserRole,
                    user.ContactPreference,
                    null,
                    null,
                    user.Status);
                this.bind()
            });
        } 
    }

I tried this too but it made no difference, on the saveUser method.

this.validationController.addObject(this.user);