Issues with large solutions in Ms Dynamics CRM 2011.

We have a Ms Dynamics CRM 2011 in box Test environment, i.e. CRM and SQL on the same box, don't ask why, and our main solution is a good 10+ MB zipped up, which means that sometimes it takes a few attempts to get the solution imported.

As of late, my lack of commitment to scientific inquiry and betterment of the world continues to show as I really haven't tested this thoroughly enough, but here it goes anyway.

The problem seemed to be that the W3WP process was using almost all the available memory on the server, which resulted in timeouts when running various SQL queries, at least that's what the trace log said, it's hard to trust a log that seems surprised that the are no errors, but I digress. 

The solution was to set upper and lower limits of memory on SQL server, to be fair I think the problem was the lower limit, but it makes sense to limit memory usage at the high end as well, lest SQL server thinks all the memory it's for itself.

EXEC sys.sp_configure N'show advanced options', N'1'  RECONFIGURE WITH OVERRIDE

GO

EXEC sys.sp_configure N'min server memory (MB)', N'512'

GO

EXEC sys.sp_configure N'max server memory (MB)', N'2048'

GO

RECONFIGURE WITH OVERRIDE

GO

EXEC sys.sp_configure N'show advanced options', N'0'  RECONFIGURE WITH OVERRIDE

GO

For the record the server had 4 GB of RAM, which could well be the source of the issue in the first place, i.e. this might not happen in a server with 8 GB of RAM.

We've not had any of these issues on our OAT environment, which features separated CRM and SQL boxes, each with 8 GB of RAM, so hopefully setting limits to the memory used by SQL server was the solution to the problem.

Updates are currently disallowed on GET requests. To allow updates on a GET, set the 'AllowUnsafeUpdates' property on SPWeb.

So today I hit a limit on SharePoint when listing from a library

The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator

The solution is simple, just increase the number of items, so from the Central Administration Site:

1. Application Management -> Manage Web Application and select your web application
2. In the Ribbon, click on General Settings drop-down and choose “Resource Throttling”.
3. In the “List View Threshold”, increase the value

The problem I was having was that when I tried to do this I would get the following error:

Updates are currently disallowed on GET requests.  To allow updates on a GET, set the 'AllowUnsafeUpdates' property on SPWeb.

The solution, from the SharePoint PowerShell console:

$sp = get-spwebapplication https://myapp
$sp.HttpThrottleSettings
$sp.Update()

The problem seems to be related to the web application not having a value for HttpThrottleSettings, which will be set by running the above commands.

Issues with Word Automation Services in SharePoint 2010

This week we had an issue with Word Automation Services, in one of our test servers, where our custom code (really boiler plate code, see below) would fail on the second line:

var context = SPServiceContext.GetContext(SPsite);
var wsaProxy = (WordServiceApplicationProxy)context.GetDefaultProxy(typeof(WordServiceApplicationProxy));

Since the same code was working fine in our development environment, it was clear that it was not the code that was at fault but our SharePoint configuration that was at fault.

The issue was that the Word Automation Services Application had not been configured to be added to the default proxy list, see screenshot below, and thus the code was failing to get the proxy.

Note that this adding Word Automation services is done from the Central Administration website:
Central Administration -> Manage Service Applications -> New Word Automation Service

Add HTTPS/SSL Binding to website in IIS from powershell

Edit:

I've crearted a powershell module to create and remove websites that includes this function, see it here post.

Since Wix seems to be frowned upon at work, I have been looking at PowerShell as a replacement to try to automate deployment of builds.

This little script will set the HTTPS binding. The certificate thumbprint is needed but the rest of the parameters are optional, defaulting to the most common option.

param ([String]$thumbprint, [String]$sitename="Default Web Site", [int]$port=443, [String]$hostheader)

if (-not($thumbprint))
{
  Write-Error "Certificate Thumprint is needed"
  exit
}

Import-Module WebAdministration

If (-not ([Security.Principal.WindowsPrincipal] [Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator"))
{
 Write-Warning "Run this script with elevated permissions"
 exit
}

function AddHTTPSBinding([String]$thumbprint, [String]$sitename, [int]$port, [String]$hostheader)
{
 $cert = Get-ChildItem cert:\LocalMachine\My | ?{$_.Thumbprint -eq $thumbprint}
 
 if( -not($(gci iis:\sslbindings| ? {$_.Port -eq $port})))
 {
  New-Item IIS:\SslBindings\0.0.0.0!$port -Value $cert | out-null
  
  if ($hostheader)
  {
   New-ItemProperty $(join-path iis:\Sites $sitename) -name bindings -value @{protocol="https";bindingInformation="*:$($port):$($hostheader)";certificateStoreName="My";certificateHash=$thumbprint}
  }
  else
  {
   New-ItemProperty $(join-path iis:\Sites $sitename) -name bindings -value @{protocol="https";bindingInformation="*:$($port):";certificateStoreName="My";certificateHash=$thumbprint}
  }
 }
 else
 {
  Write-Warning "SSL binding already exists on port $port"
 }
}

AddHTTPSBinding $thumbprint $sitename $port $hostheader

There is a New-WebBinding cmdlet in the WebAdministration module, but I think it needs to be used in conjunction with the Set-WebBinding to set the certificate and certificate store.

Issues with solutions in MS Dynamics CRM 2011 - Maintaining vs Overwriting Customizations

The ongoing saga of the CRM solutions continues, MS Dynamics CRM 2013 can't come too soon, hopefully there will be improvements around this area.

Having undertaken no rigorous testing whatsoever we have determined that overwriting customizations is slower than maintaining customizations when importing a solution, and so we normally try to avoid overwriting customizations, the problem is what CRM considers a customization.

I haven't really done a serious investigation, but a colleague did a semi serious one and he found that for workflows, the simple action of deactivating it and then activating it, was enough to for CRM to think it had been changed and thus would not be updated if the solution was imported maintaining customizations.

I guess the modifiedon attribute is changed when a workflow is deactivated and that must explain why CRM thinks it has been changed and thus will not modify it.

Like I said, this is all preliminary, so take it with a pinch of salt. Good thing that Ms Dynamics CRM 2011 has just been released, I'm sure this post will save people loads of headaches

ADFS issues - ID3242: The security token could not be authenticated or authorized.

What a load of fun I had yesterday with ADFS. For some unknown reason, working web services stopped working today and after a lot of pleading we managed to get the logs from the ADFS server, which was showing this error:

 ID3242: The security token could not be authenticated or authorized.

After a lot of soul searching and hair pulling, we realized that the issue might be with the encryption certificate as the ADFS server cannot get to the CRL distribution point of the encryption certificate, due to the firewall.

This can be sorted out with these commands:

Add-PSSnapin Microsoft.ADFS.PowerShell (Import-Module ADFS - if using Win2k12 R2)  

Set-ADFSRelyingPartyTrust -TargetName <name> -EncryptionCertificateRevocationCheck None 

Set-ADFSRelyingPartyTrust -TargetName <name> -SigningCertificateRevocationCheck None 

We also set the signing certificate revocation check to none, although I think this is not needed, but there seems to be some reluctance to remove it.

Edit:

I write most of my posts well in advance and I'm not 100% that this is entirely correct. I'd like to say that I will check to make sure, but it's extremely unlikely.

Edit 2:

In our case it seems that this is indeed the solution as the ADFS server cannot get to the CRL Distribution Point, which causes issues :), which is why disabling the Revocation Checks on the certificates works.

Issues with solutions in Ms Dynamics CRM 2011 - Be careful when renaming custom workflow activities

We have a couple of custom workflow activity libraries, and one of them is a bit like a helper library with about 10 custom activities and yesterday I decided to screw it all up had a brilliant idea. One of the custom activities was confusingly named as it had been extended from its original purpose of retrieving an entityreference to a custom entity, to actually create that custom entity if it could not be found, so far so good.

I then thought that a bit of control would be nice, so I added a new argument so that we could choose whether the custom entity would be created if it wasn't found.

I updated the assembly on the server using the pluginregistration tool and nothing happened, the new attribute would not appear, cue bouncing of the CRM services, still nothing. IIS went down and up and still nothing. Server does the same thing and nothing.

So bullet biting time then; I removed the custom workflow activity from the three dialogs where it was being used, thankfully at the very end of the dialogs and then deleted it using the pluginregistration tool, updated the assembly again, bounce the crm services and iis and it starts working.

Incidentally, in theory this should not be needed, i.e. adding a new argument to a workflow should just show in CRM, but this might be only if it's not in use anywhere, I don't really know, every time I have made a change like this, i.e. add or remove attributes I have had to go through the rigmarole, which is really annoying.

I fixed the dialogs back to what they were and think nothing of it, until late this afternoon, when the import of the solution to the test server fails.

We then tried doing a new import but this time we use the overwrite all customizations and still it fails, so same procedure as in the development environment, namely remove the custom workflow activity from the dialogs, remove the custom workflow activity from the server using the pluginregistration tool, then finally we were able to import the solution successfully.

The moral of the story is:

Be Careful When  Renaming Custom Workflow Activities

Editing Active Directory user accounts from PoweShell

For reasons too long to explain I had to make changes to a few accounts (5+) today so rather than do them one by one I thought I would try using PowerShell.

Import-Module ActiveDirectory 

Get-ADUser -Filter 'name -like "*service"' | %{Set-ADUser -PasswordNeverExpires $true -Identity $_.Name}

No prices for guessing what the change needed was.

Using Selenium with Microsoft Dynamics CRM 2011

Earlier this week I was asked to look at the possiblity of using Selenium with Microsoft Dynamics CRM 2011 since it now supports Firefox (nothing like a clued up Test manager). At any rate, I thought I would give it a try.

The problem was that recording tests with the Selenium IDE wasn't not working as I was hitting constant javascript errors, so I decided to use the IDE as some sort of guidance and then modify the code generated to get it to work.

The test test [sic.] was to generate an entity (change) off another (callback) and then check that depending on the change type (field pre_type) various workflows and/or plugins would trigger.

Why this needed doing with Selenium it's beyond me, but there you go.

I won't go into details about the entities, but suffice to say that callback is the main entity in the system, there is a 1:N relationship between callback and change and changes can be created from the callback form.

So, I installed Selenium, downloaded the IE driver and got coding:

using System.Text;

using OpenQA.Selenium;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.IE;
using OpenQA.Selenium.Support.UI;
using System.Runtime.InteropServices;
using System.Security.Cryptography;
using System.Threading.Tasks;
using System.Threading;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            IWebDriver driver = new InternetExplorerDriver(@"C:\Users\john\Downloads\selenium-dotnet-2.32.1\IEDriverServer_Win32_2.32.3\");
            driver.Url = "https://devcrm.dev.local//main.aspx?skipNotification=1";
            driver.FindElement(By.CssSelector("#pre_callback > nobr.ms-crm-NavBar-Subarea-Title")).Click();

            driver.SwitchTo().Frame("contentIFrame");

            driver.FindElement(By.Id("crmGrid_findCriteria")).Clear();
            driver.FindElement(By.Id("crmGrid_findCriteria")).SendKeys("*William*");
            driver.FindElement(By.Id("crmGrid_findCriteriaButton")).Click();

            driver.FindElement(By.Id("gridBodyTable_primaryField_{B2C895DC-DEAD-BEEF-9B08-F05056B2009F}_0")).Click();

            WaitForNewWindow(driver, 2);
            driver.SwitchTo().Window(driver.WindowHandles[1]);
            driver.SwitchTo().Frame("contentIFrame");

            driver.FindElement(By.Id("nav_pre_pre_callback_pre_change")).Click();
            driver.SwitchTo().DefaultContent();

            driver.FindElement(By.Id("pre_change|OneToMany|SubGridAssociated|Mscrm.SubGrid.pre_change.AddNewStandard-Large")).Click();

   WaitForNewWindow(driver, 3);

            driver.SwitchTo().Window(driver.WindowHandles[2]);
            driver.SwitchTo().Frame("contentIFrame");
            driver.FindElement(By.Id("DateInput")).SendKeys(DateTime.Now.ToString("dd/MM/yyyy"));
            driver.FindElement(By.Id("pre_changedetails")).SendKeys("Selenium Attack");
            
   for(int i=1; i < 6; i++
   {
                 SelectDropDown(driver, "pre_type", i);
                 driver.SwitchTo().DefaultContent();
                 driver.FindElement(By.Id("pre_change|NoRelationship|Form|Mscrm.Form.pre_change.SaveAndClose-Large")).Click();
   }
            driver.Quit();

        }

        private static void WaitForNewWindow(IWebDriver driver, int windowNumber)
        {
            while (driver.WindowHandles.Count != windowNumber)
            {
                Thread.Sleep(133);
            }
        }

        private static void SelectDropDown(IWebDriver driver, string fieldName, int selection)
        {
            IWebElement sourceWeb = driver.FindElement(By.Id(fieldName));
            SelectElement source = new SelectElement(sourceWeb);
            source.SelectByIndex(selection);
        }
    }
}

There is no reason why this could not be done as a unit test, but I thought it would be easier to distribute to the testers as a console app (It does need a lot of work, I know)

I have to say that I found it extremely flaky, in fact it seemed to need two runs, one to warm up and then it would almost always work.

Since, I haven't used Selenium much, I can't say how reliable or otherwise it is, but using the IE driver was not found suitable for testers.

I think it can be used for early morning checks and things like that but not for automated testing, all in all it was a big disappointment.

Sed equivalent in Powershell

A few days back I was trying to fix some issues with our Visual Studio solution, where the hintpaths were all wrong, so I thought it would try to use PowerShell:

ls -recurse -Include *.csproj | % {sp $_ isreadonly $false; (Get-Content $_) -replace "here","there" | Set-Content -Path $_}

A few comments to make:

% is an alias for foreach-object
sp is an alias fro Set-ItemProperty
-replace allows using regular expressions

The only downside was that I still had to check in the projects back into TFS manually, for some reason the TFS powertools would not allow me to checkout more than 2 projects at the same time, but on the plus side this is a good equivalent to Sed in PowerShell.