Using ARR (Reverse Proxy) with Microsoft Dynamics CRM 2015 - IFD

In this post I'll discuss how we have used the Application Request Routing module from IIS to expose Ms Dynamics CRM to the big bad world.

In this project I'm working on at the moment we want to allow authorized third parties to access to our application, which means that there has been a lot of discussions around what's the best way of doing this.

The Problem:

We want to allow third parties access to our MS Dynamics CRM 2015 system but these third parties might be mom and pop operations so federating with them is not an option. In reality, this is a fantasy conjured up by the PM, but we all go along with it, because sometimes it's important to know which battles to fight.

We also don't want to expose our MS Dynamics CRM 2015 servers to the big bad internet.

The Solution:

Use ARR to expose the MS Dynamics CRM 2015 website to the outside world, while keeping the servers happily inside the perimeter.

This is an architecture diagram of what we are trying to achieve.

I'm assuming that you have a working MS Dynamics CRM 2013/2015 installation configured for Internet Facing Deployment and a wildcard certificate for the appropriate domain available.

Install ARR.

Navigate to this page and click on install this extension, which will use the Windows Platform installer to install ARR, just follow the instructions therein. Alternatively you can do it manually, which I haven't tried.

Configure ARR

ARR is configured from the IIS Manager console.

So fire it up (windows key + r -> inetmgr)  and let's get started:

We first create a Server Farm for the MS Dynamics CRM Web Servers. If you have web and application CRM servers then you only need to add the web servers. We have full servers.

Ensure you add all servers, not just one server, unless you only have one server like myself in this environment :)

If you are only using ARR to expose MS Dynamics CRM to the outside world, then it's probably ok to click yes here.

 We can now configure our server farm.

We want to make sure that traffic is only directed to servers that are up and running so I normally set the organization's wsdl page as the health test URL. The server's page (using FQDN) should also work and in fact should be used if you have more than one organization.

The other thing that needs to be done is to set load balancing to be a based on Client affinity, in other words, a client will always hit the same server, unless that server is down. This is an optional step, that's only required, as far as I can tell, for reports.

We now need to check that the routing rule is correct for the current set up.

Pretty simple, as we're sending everything that hits this server onward to the MS Dynamics CRM Farm. However, the rule will, by default, forward it to http rather than https, so this needs to be changed.

The final step is to create the proxy websites. I'm going to assume that a wildcard certificate has been installed already for the server. These website will receive the traffic from the internet and forward it to the MS Dynamics CRM Web Servers.

When using IFD, a website is needed for each organization and another one for the auth website. You can also add one for the discovery service, but as far as my limited testing goes, it works without it.

At this point ARR is configured, now all that remains is to set up name resolution.

I normally use CNAME aliases for auth, disc and the various organizations to point to the ARR server:

stark.dev.local --> ARR Server (Organization)
auth.dev.local --> ARR Server  (auth)
disc.dev.local --> ARR Server   (discovery)

In my next post I will add ADFS to the reverse proxy, note that it's perfectly acceptable to use ARR for this purpose as well, but it also possible to use WAP, which provides pre-authentication.

MS Dynamics CRM - Blocking Direct Access to SharePoint

In this project I'm working on we've had a requirement to block direct access to SharePoint, in other words. Our users can only access their documents through CRM.

There are various ways, in which this can be achieved and today I will be discussing achieving this by leveraging ARR.

We have a similar architecture to this:

We have a Sharepoint site collection and every document is stored in this site collection, so our users would go https://sp.dev.local/sites/spsite/<crm_entity>/ to access documents for <crm_entity>

This is actually a somewhat irritating requirement because IE behaves differently than Firefox and Chrome. We're lucky enough not to have to support Opera and Safari as well, nothing wrong with these browsers, but 5 browsers would drive our testers crazy.

In any case, we've configured two farms, CRM and SHAREPOINT, so we need rules for those.

So for CRM we have:

<rule name="CRM Farm Forwarding Rule" enabled="true" patternSyntax="Wildcard" stopProcessing="false">
    <match url="*" />
    <conditions logicalGrouping="MatchAny" trackAllCaptures="true">
        <add input="{HTTP_HOST}" pattern="*crm.dev.local" />
    </conditions>
    <serverVariables>
    </serverVariables>
    <action type="Rewrite" url="https://CRM/{R:0}" />
</rule>

And for Sharepoint we have:

<rule name="Sharepoint Farm Forwarding Rule" enabled="true" patternSyntax="Wildcard" stopProcessing="true">
    <match url="*" />
    <conditions logicalGrouping="MatchAny" trackAllCaptures="true">
        <add input="{HTTP_HOST}" pattern="*sp.dev.local" />
    </conditions>
    <serverVariables>
    </serverVariables>
    <action type="Rewrite" url="https://SHAREPOINT/{R:0}" />
</rule>

So far so good, this is where things start to get interesting. We want to block direct access to SP for IE browsers, which we can achieve like this:

 
<rule name="IE - Allow Access to SharePoint grid in CRM" enabled="true" patternSyntax="Wildcard" stopProcessing="true">
    <match url="*sites/SPSITE/crmgrid*" />
    <conditions logicalGrouping="MatchAny" trackAllCaptures="false">
    </conditions>
    <serverVariables>
    </serverVariables>
    <action type="Rewrite" url="https://SHAREPOINT/{R:0}" />
</rule>    

and

<rule name="IE - Block Direct Access to SharePoint" stopProcessing="true">
    <match url="sites\/SPSITE" />
    <conditions logicalGrouping="MatchAll" trackAllCaptures="false">
        <add input="{HTTP_REFERER}" pattern="^/?$" negate="true" />
        <add input="{HTTP_USER_AGENT}" pattern="MSIE" />
    </conditions>
    <serverVariables>
    </serverVariables>
    <action type="CustomResponse" statusCode="403" subStatusCode="1" statusReason="IE" statusDescription="Direct Access to SHAREPOINT is not permitted" />
</rule>

The first rules allows traffic if it's trying to access SP through the List component, this allows the SharePoint CRM List component to work.

The second rule will block access for IE user agents ( i.e containing MSIE) and where the referer is not empty. It will stop processing if there is a match.

For some reason, I.E. blanks the referer when accessing documents in SharePoint from the list component but crucially fills it in if accessing documents directly from SharePoint.

Firefox and Chrome will have the correct referer, i.e. sp.dev.local/sites/spsite/crmgrid, so there is a single rule:

<rule name="Other Browsers - Block Direct Access to SharePoint" enabled="true" patternSyntax="Wildcard" stopProcessing="true">
    <match url="*sites/SPSITE*" />
    <conditions logicalGrouping="MatchAll" trackAllCaptures="false">
        <add input="{HTTP_REFERER}" pattern="*dev.local/sites/SPSITE/crmgrid*" negate="true" />
        <add input="{HTTP_USER_AGENT}" pattern="*MSIE*" negate="true" />
    </conditions>
    <serverVariables>
    </serverVariables>
    <action type="CustomResponse" statusCode="403" subStatusCode="2" statusReason="Other" statusDescription="Direct Access to SHAREPOINT is not permitted" />
</rule>

Just need to make sure that the user agent is not from IE. The rule will stop processing if there is a match.

Rules.xml can be found below, with the rules in the correct order.

Thus effectively we do the following:

CRM -> Rewrite to  CRM Farm
SP - crm grid?  -> Rewrite to SP Farm
SP - IE and empty Referer -> Rewrite to SP Farm
SP - Other Browser and correct Referer -> Rewrite to SP Farm
SP -> Rewrite to Farm

The last rule is only needed if there are other sites in SP that the users might need to access.

<?xml version="1.0" encoding="UTF-8"?>
<appcmd>
    <CONFIG CONFIG.SECTION="system.webServer/rewrite/globalRules" path="MACHINE/WEBROOT/APPHOST" overrideMode="Inherit" locked="false">
        <system.webServer-rewrite-globalRules>
            <rule name="CRM Farm Forwarding Rule" enabled="true" patternSyntax="Wildcard" stopProcessing="false">
                <match url="*" />
                <conditions logicalGrouping="MatchAny" trackAllCaptures="true">
                    <add input="{HTTP_HOST}" pattern="*crm.dev.local" />
                </conditions>
                <serverVariables>
                </serverVariables>
                <action type="Rewrite" url="https://CRM/{R:0}" />
            </rule>
            <rule name="IE - Allow Access to SharePoint grid in CRM" enabled="true" patternSyntax="Wildcard" stopProcessing="true">
                <match url="*sites/SPSITE/crmgrid*" />
                <conditions logicalGrouping="MatchAny" trackAllCaptures="false">
                </conditions>
                <serverVariables>
                </serverVariables>
                <action type="Rewrite" url="https://SHAREPOINT/{R:0}" />
            </rule>
            <rule name="IE - Block Direct Access to SharePoint" stopProcessing="true">
                <match url="sites\/SPSITE" />
                <conditions logicalGrouping="MatchAll" trackAllCaptures="false">
                    <add input="{HTTP_REFERER}" pattern="^/?$" negate="true" />
                    <add input="{HTTP_USER_AGENT}" pattern="MSIE" />
                </conditions>
                <serverVariables>
                </serverVariables>
                <action type="CustomResponse" statusCode="403" subStatusCode="1" statusReason="IE" statusDescription="Direct Access to SHAREPOINT is not permitted" />
            </rule>
            <rule name="Other Browsers - Block Direct Access to SharePoint" enabled="true" patternSyntax="Wildcard" stopProcessing="true">
                <match url="*sites/SPSITE*" />
                <conditions logicalGrouping="MatchAll" trackAllCaptures="false">
                    <add input="{HTTP_REFERER}" pattern="*dev.local/sites/SPSITE/crmgrid*" negate="true" />
                    <add input="{HTTP_USER_AGENT}" pattern="*MSIE*" negate="true" />
                </conditions>
                <serverVariables>
                </serverVariables>
                <action type="CustomResponse" statusCode="403" subStatusCode="2" statusReason="Other" statusDescription="Direct Access to SHAREPOINT is not permitted" />
            </rule>
            <rule name="Sharepoint Farm Forwarding Rule" enabled="true" patternSyntax="Wildcard" stopProcessing="true">
                <match url="*" />
                <conditions logicalGrouping="MatchAny" trackAllCaptures="true">
                    <add input="{HTTP_HOST}" pattern="*sp.dev.local" />
                </conditions>
                <serverVariables>
                </serverVariables>
                <action type="Rewrite" url="https://SHAREPOINT/{R:0}" />
            </rule>
        </system.webServer-rewrite-globalRules>
    </CONFIG>
</appcmd>

It can be exported with this command:

appcmd.exe list config -section:system.webServer/rewrite/globalRules -xml > rules.xml 

The imported with this command:

appcmd.exe set config /in < rules.xml

Understanding default values in C#

I was refactoring a bit of code last week and I ended creating an interesting scenario.
In short, I was re-writing a method to see if by using a completely different approach I could make the code simpler easier to maintain and faster, and I ended up with two methods like this (don't even ask, what I was thinking):

public class Point
{
    public int X { get; set; }
    public int Y { get; set; }

    public Point(int x, int y)
    {
        this.X = x;
        this.Y = y;
    }

    public void Move(int distance)
    {
        this.X += distance;
        this.Y += distance;
    }

    public bool Move(int distance, bool dummy = true)
    {
        this.X += distance;
        this.Y += distance;
  
        return true;
    }
}

So with this calling code, which polymorphic operation will be selected:

class Program
{
    static void Main(string[] args)
    {
        var p = new Point(1, 2);
        p.Move(1);
        p.Move(1,true);
    }
}

p.Move(1) invokes the void method p.Move(1,true); invokes the bool method.

The question is how does the compiler know which polymorphic operation to invoke, after all, if the Point class had no void Move method, both statement would invoke the second Move method. 

Time to look at the IL output of Main:

.method private hidebysig static 
 void Main (
  string[] args
 ) cil managed 
{
 // Method begins at RVA 0x2100
 // Code size 27 (0x1b)
 .maxstack 3
 .entrypoint
 .locals init (
  [0] class ConsoleApplication6.Point p
 )

 IL_0000: nop
 IL_0001: ldc.i4.1
 IL_0002: ldc.i4.2
 IL_0003: newobj instance void ConsoleApplication6.Point::.ctor(int32, int32)
 IL_0008: stloc.0
 IL_0009: ldloc.0
 IL_000a: ldc.i4.1
 IL_000b: callvirt instance void ConsoleApplication6.Point::Move(int32)
 IL_0010: nop
 IL_0011: ldloc.0
 IL_0012: ldc.i4.1
 IL_0013: ldc.i4.1
 IL_0014: callvirt instance bool ConsoleApplication6.Point::Move(int32, bool)
 IL_0019: pop
 IL_001a: ret
} // end of method Program::Main

This is not very illuminating, but if we comment out the void method on the Point class and get the IL output again:

.method private hidebysig static 
 void Main (
  string[] args
 ) cil managed 
{
 // Method begins at RVA 0x20e0
 // Code size 28 (0x1c)
 .maxstack 3
 .entrypoint
 .locals init (
  [0] class ConsoleApplication6.Point p
 )

 IL_0000: nop
 IL_0001: ldc.i4.1
 IL_0002: ldc.i4.2
 IL_0003: newobj instance void ConsoleApplication6.Point::.ctor(int32, int32)
 IL_0008: stloc.0
 IL_0009: ldloc.0
 IL_000a: ldc.i4.1
 IL_000b: ldc.i4.1
 IL_000c: callvirt instance bool ConsoleApplication6.Point::Move(int32, bool)
 IL_0011: pop
 IL_0012: ldloc.0
 IL_0013: ldc.i4.1
 IL_0014: ldc.i4.1
 IL_0015: callvirt instance bool ConsoleApplication6.Point::Move(int32, bool)
 IL_001a: pop
 IL_001b: ret
} // end of method Program::Main

As expected both invoke the second method but why this behaviour?

Well, it turns out that this is part of the spec:

Use of named and optional arguments affects overload resolution in the following ways:

• A method, indexer, or constructor is a candidate for execution if each of its parameters either is optional or corresponds, by name or by position, to a single argument in the calling statement, and that argument can be converted to the type of the parameter.
• If more than one candidate is found, overload resolution rules for preferred conversions are applied to the arguments that are explicitly specified. Omitted arguments for optional parameters are ignored.
• If two candidates are judged to be equally good, preference goes to a candidate that does not have optional parameters for which arguments were omitted in the call. This is a consequence of a general preference in overload resolution for candidates that have fewer parameters.

The authentication endpoint Kerberos was not found on the configured Secure Token Service! - Redux

So today I had an interesting moment of Deja vu today, where we started getting this issue again.

This time though, the configuration had already been done so it was a bit mystifying to say the least and to further add to the confusion, it worked on one server but not the other.

After a lot of blood, sweat and tears the issue was tracked down to the affected server being unable to communicate with the ADFS load balancer. In essence, our network guys had not opened the firewall for both servers but just one of them.

So, if you get this issue, it might simply be due to a good old fashioned network issue.

Removing HTTP Headers for an ARR/MS Dynamics CRM/Sharepoint 2013 system

We had a pen test carried out last week and one of the outcomes was that we were leaking information with our HTTP headers and they must be removed.

Our environment consists of a web layer (IIS using ARR) and then an app layer (MS Dynamics CRM and MS SharePoint)

Personally I think this is a bit of security through obscurity but needs must so here we go:

X-AspNet-Version Header:

MS Dynamics CRM 2013

In the web.config, <drive>:Program Files\Microsoft Dynamics CRM\CRMWeb  add enableVersionHeader="false" to the httpRuntime element, normally you'll end up with something like this:

<httpRuntime executionTimeout="300" maxRequestLength="32768" requestValidationMode="3.0" encoderType="Microsoft.Crm.CrmHttpEncoder, Microsoft.Crm" enableVersionHeader="false"/>

MS SharePoint 2013

In the web.config,  <drive>:inetpub\wwwroot\wss\VirtualDirectories\80\ add enableVersionHeader="false" to the httpRuntime element, normally you'll end up with something like this:

<httpRuntime maxRequestLength="51200" requestValidationMode="2.0" enableVersionHeader="False" />

X-Powered-By Header:

From IIS Manager -> Server -> HTTP Response Headers

Server Header:

The simplest way I found is to use URL Rewrite to blank this header, which works very well for our system as we're using ARR already so just need to do this one on the web layer ... from IIS Manager -> Url Rewrite -> Add Rule

Select Blank Outbound Rule

Fill in the details as below

Don't forget to click Apply when you've finished.

It's worth pointing out  that this will simply blank out the value of the Server Header, rather than remove it completely.

If you want to remove it completely you will need to install urlscan.

This approach can be used for all the other headers above I suppose.

X-Powered-By: ARR/2.5 Header

From a powershell console with elevated permissions go to C:\Windows\system32\inetsrv and run this command:

.\appcmd.exe set config -section:webFarms /"[name='serverfarmname'].applicationRequestRouting.protocol.arrResponseHeader:false" /commit:apphost